Elastic recently moved Elastic Cloud on Kubernetes (ECK) into GA. Originally announced as an alpha release in May 2019, Elastic is looking to support the growing number of users leveraging Kubernetes for deploying ElasticSearch. This release includes support for many of Elastic's core features and can run on a number of public cloud Kubernetes offerings.
ECK is built using the Kubernetes Operator pattern and installs directly into a Kubernetes cluster. Elastic has focused this product on streamlining daily operations tasks including: managing and monitoring multiple clusters, upgrades, scaling cluster capacity, adjusting cluster configuration, and dynamically scaling local storage. According to Anurag Gupta, principal product manager at Elastic, "The vision for ECK is to provide an official way to orchestrate Elasticsearch on Kubernetes and provide a SaaS-like experience for Elastic products and solutions on Kubernetes."
All clusters deployed using ECK include the core features available via other means of deploying ElasticSearch. These include APM, Logs, Metrics, SIEM, Canvas, Lens, and index lifecycle management. Additional features are available through an enterprise subscription.
ECK also includes the core Elasticsearch security features that were previously moved into the free core version. This includes TLS certificate management, file and native realm for creating and managing users, and role-based access control for managing access to cluster APIs and indexes. In addition to these core features, ECK is built to be secure by default by having a strong password set at installation time and encryption enabled by default.
Common use cases and patterns are also supported by ECK. This includes having dedicated master and machine learning nodes, and a hot-warm-cold deployment. A hot-warm-cold deployment allows for a better balance between long-term storage needs and performance. Logs can be shuffled into colder storage nodes based on index lifecycle management (ILM) policies. As an example, logs from this week are the most heavily searched logs and would be within the hot cluster. Last week's logs may still be searched but less frequently so can be moved into the warm cluster. Older logs may be needed at some point, so instead of deletion, they can be moved into the cold cluster. Within this grouping, the nodes can be allocated unevenly between the three categories, with more nodes supporting the more heavy search logs.
Included in the alpha release of ECK was an integrated storage driver for Kubernetes, Elastic Local Volume. This project allowed for the creation of a PersistentVolumeClaim (PVC) for local volumes. However, with the move into GA, this project is no longer being actively maintained. Recommend alternatives include TopoLVM, Sig-storage local static provisioner, and OpenEBS.
The release of ECK builds upon previous changes from Elastic to further support container workloads. Previously, Elastic released official Docker images for both ElasticSearch and Kibana. In 2018, Elastic joined the Cloud Native Computing Foundation (CNCF) and released Helm charts for ElasticSearch on Kubernetes.
ECK is able to be run on Google Kubernetes Engine, OpenShift, Azure Kubernetes Service, Amazon Elastic Kubernetes Service, and vanilla Kubernetes. Currently ECK supports kubectl 1.11+, Kubernetes 1.12+, OpenShift 3.11+, and ElasticSearch 6.8+ and 7.1+.
The source code for ECK is available in GitHub under the Elastic License. Elastic has released the core ECK functionality under their free-forever Basic tier. With a paid subscription, additional features are available including machine learning, graph analytics, and the ability to deploy clusters with advanced features such as field- and document-level access control.