Supergiant is a container hosting platform built using Kubernetes for distributed, stateful applications. Supergiant was built by the Qbox.io team which provides hosted Elasticsearch.
Supergiant is a framework that runs Docker containers hosting stateful, clustered applications. It utilizes Kubernetes under the hood and has its own structures and code in cases where Kubernetes does not serve the purpose, like persistent storage and external load balancing. InfoQ got in touch with Mark Brandon, CEO and Co-Founder of Qbox, the creators of Supergiant, to learn more about the technical details behind it.
The QBox.io team started with hosted Elasticsearch. Many of the issues they faced include not being able to scale due to the "noisy neighbor" effect. Docker and Kubernetes seemed like the right options to manage networked containers and volumes, according to their blog.
Storage for applications deployed on Supergiant is allocated on high-availability arrays, however, it does not use Kubernetes' persistent volumes. Currently available on AWS, Supergiant uses Amazon's Elastic Block Store (EBS) so that volumes can move freely around the cluster. EBS volumes behave like hard drives that can be plugged and unplugged (and mounted/unmounted) into virtual machines. Since EBS volumes are persisted across node server reboots, data is not lost when a node goes down.
Kubernetes has native load balancing and HA features. The Supergiant team wrote code specifically for external load balancing, says Brandon, because "while Kubernetes does internal load balancing masterfully, its external load balancing strategy was less than ideal for distributed applications like Elasticsearch."
To run as an application on Supergiant, a piece of software needs to be packaged as a "Component". A component could be a NoSQL database, a web-server or a node.js app. "Any application that can be Dockerized can be made into a component", according to Brandon. Supergiant provides an HTTP based API for defining components.
One of the requirements for applications running on AWS is to be tolerant of availability zone (AZ) failures, with support for cross-region availability being the next level. Supergiant is working on managing clusters across AWS zones, said Brandon.