The Kubernetes Effect

Kubernetes Cookbook (By O’Reilly)

Learn how to use Kubernetes for container orchestration with easy‑to‑use recipes for installation, API access, monitoring, troubleshooting, and more. Download the FREE eBook Now.

 

Sponsored Content

Kubernetes (k8s) has come a long way in a very short time. Only two years ago it had to compete and prove to be better than CoreOS’s Fleet, Docker Swarm, Cloud Foundry Diego, HashiCorp’s Nomad, Kontena, Rancher’s Cattle, Apache Mesos, Amazon ECS, etc. Today it is a completely different landscape. Some of these projects openly announced to be discontinued and in favour of joining efforts with Kubernetes, others didn’t accept defeat openly and tactically announced either partial support or a parallel, full integration with Kubernetes which meant a quiet and slow death for their container orchestrator. In either case, k8s is the last platform standing. In addition, more and more big names continued joining the Kubernetes ecosystem, not just as users or platinum sponsors, but by fully betting their container business on the success of Kubernetes. Google’s Kubernetes Engine, Red Hat’s OpenShift, Microsoft’s Azure Container Service, IBM’s Cloud Container Service, Oracle’s Container Engine, and others are the first to come to mind here.

But what does all that mean? First, it means developers have to master only a single container orchestration platform to be relevant for the 90% of the container related job market. A good reason to invest the time to learn k8s well. It also means that we are all knee deep in k8s. Kubernetes is like Amazon, but for containers, if you don’t want a lock-in, you lock-in with Kubernetes. Designing for, implementing and running applications on Kubernetes gives you the freedom to move your applications between the different cloud providers, kubernetes distributions and service providers. It gives you the opportunity to find Kubernetes certified developers to kickoff a project and support personnel to continue running it afterward. It is not the VM, it is not the JVM, it is Kubernetes that is the new application portability layer. It is the common denominator among everything and everybody.

Knee Deep in Kubernetes

Below is a diagram of a containerized service demonstrating how much it depends on Kubernetes.

[Click on the image to enlarge it]

Figure 1 - Application dependency on Kubernetes

Notice that I didn’t write Kubernetes is the application portability API, but a layer. The diagram here shows only the K8s objects that we have to explicitly create - which can call the k8s API. But in reality, we are much more coupled with the platform. K8s offers a full set of distributed primitives (such as pods, services, controllers) that addresses the requirements and drives the design of our applications. These new primitives and the platform capabilities dictate the guiding design principles and design patterns we use to implement all future services. They, in turn, affect what techniques we will use to address everyday challenges and even affects what we call a "best practice". Hence the way we should look at Kubernetes is more like a fundamental paradigm that has implications in multiple dimensions, rather than an API to interact with.

The Kubernetes Effect

The container and the orchestrator features provide a new set of abstractions and primitives. To get the best value of these new primitives and balance their forces, we need a new set of design principles to guide us. Subsequently, the more we use these new primitives, the more we will end up solving repeating problems and reinventing the wheel. This is where the patterns come into play. Design patterns provide us recipes on how to structure the new primitives to solve repeating problems faster. While principles are more abstract, more fundamental and change less often, the patterns may be affected by a change in the primitive behaviour. A new feature in the platform may make the pattern an anti-pattern or less relevant. Then, there are also practices and techniques we use daily. The techniques range from very small technical tricks for performing a task more efficiently, to more extensive ways of working and practices. We change the techniques and practices as soon as we find a slightly better way of doing things easier and faster. This is how we and the platform evolve. And why we do all that? That is where we get the benefits and values of using this new platform and satisfy our needs. In a sense, "The Kubernetes Effect" is self-enforcing and multifaceted:

Figure 2 - Kubernetes effect on the software development life cycle

For the rest of this article, we will dive deeper and see examples from each category.

Distributed Abstractions & Primitives

In order to explain what I mean by new abstractions and primitives, I will compare them with the well known object-oriented world and Java specifically. In the OOP universe, we have concepts such as class, object, package, inheritance, encapsulation, polymorphism, etc. Then the Java runtime provides certain features and guarantees on how it manages the lifecycle of our objects and the application as a whole. The Java language and the JVM runtime provide local, in-process building blocks for creating applications. Kubernetes adds a completely new dimension to this well-known mindset by offering a new set of distributed primitives and runtime for creating distributed systems that spread across multiple processes and nodes. With Kubernetes at hand, I don't rely only on the Java primitives to implement the whole application behavior. I still need to use the object-oriented building blocks to create the components of the distributed application, but I can also use Kubernetes primitives for some of the application behavior. A few examples of distributed abstractions and primitives from Kubernetes are:

• Pod - the deployment unit for a related collection of containers.
• Service - service discovery and load balancing primitive.
• Job - an atomic unit of work scheduled asynchronously.
• CronJob - an atomic unit of work scheduled at a specific time in the future or periodically.
• ConfigMap - a mechanism for distributing configuration data across service instances.
• Secret - a mechanism for management of sensitive configuration data.
• Deployment - a declarative application release mechanism.
• Namespace - a control unit for isolating resource pools.

For example, I can rely on Kubernetes health checks (e.g. readiness and liveness probes) for some of my application reliability. I can use Kubernetes Services for service discovery rather than doing client-side service discovery from within the application. I can use Kubernetes Jobs to schedule an asynchronous atomic unit of work. I can use ConfigMap for configuration management, and Kubernetes CronJob for periodic task scheduling, rather than using Java-based Quartz library or an implementation of the ExecutorService interface.

Figure 3 - Local and distributed primitives as a part of a distributed system

The in-process primitives and the distributed primitives have commonalities, but they are not directly comparable and replaceable. They operate at different abstraction levels and have different preconditions and guarantees. Some primitives are supposed to be used together. For example, we still have to use classes to create objects and put them into container images. But some other primitives such as CronJob in Kubernetes can replace the ExecutorService behavior in Java completely. Here are few concepts that I've found commonalities between the JVM and Kubernetes, but don't take it too far.

Figure 4 - Local and distributed primitives categorized

I’ve blogged about distributed abstractions and primitives in the past here. The point here is, as a developer you can use a richer set of local and global primitives to design and implement a distributed solution. With time, these new primitives give birth to new ways of solving problems, and some of these repetitive solutions become patterns. This is what we will explore further down.

Container Design Principles

Design principles are fundamental rules and abstract guidelines for writing quality software. The principles do not specify concrete rules, but they represent a language and common wisdom that many developers understand and refer to regularly. Similarly to the SOLID principles that were introduced by Robert C. Martin, which represent guidelines for writing better object-oriented software, there are also design principles for creating better-containerized applications. The SOLID principles use object-oriented primitives and concepts such as classes, interfaces, and inheritance for reasoning about object-oriented designs. In a similar way, the principles for creating containerized applications listed below use the container image as the basic primitive and the container orchestration platform as the target container runtime environment. Principles for Container based application design are as following:

• Build time
  • Single Concern Principle - every container should address a single concern and do it well.
  • Self-Containment Principle - a container should rely only on the presence of the Linux kernel and have any additional libraries added to it at the time the container is built.
  • Image Immutability Principle - containerized applications are meant to be immutable, and once built are not expected to change between different environments.
• Runtime
  • High Observability Principle - every container must implement all necessary APIs to help the platform observe and manage the application in the best way possible.
  • Lifecycle Conformance Principle - a container should have a way to read the events coming from the platform and conform by reacting to those events.
  • Process Disposability Principle - containerized applications need to be as ephemeral as possible and ready to be replaced by another container instance at any point in time.
  • Runtime Confinement Principle - every container should declare its resource requirements and it is also important that the application stay confined to the indicated resource requirements.

Following these principles, we are more likely to create containerized applications that are better suited for cloud-native platforms such as Kubernetes.

[Click on the image to enlarge it]

Figure 5 - Container Design Principles

These principles are well documented and freely available for download as a white paper from here. A sneak preview of the principles is above.

Container Design Patterns

New primitives need new principles that explain the forces between the primitives. The more we use the primitives the more we end up solving repeating problems which leads us to recognize repeating solutions called patterns. Container design patterns are focused on structuring the containers and other distributed primitives in the best possible way for solving the challenges at hand. A short list of container related design patterns is as follows:

• Sidecar Pattern - a sidecar container extends and enhances the functionality of a preexisting container without changing it.
• Ambassador Pattern - this pattern hides complexity and provides a unified view of the world to your container.
• Adapter Pattern - an Adapter is kind of reverse Ambassador and provides a unified interface to a pod from the outside world.
• Initializer Pattern - init containers allow separation of initialization related tasks from the main application logic.
• Work Queue Pattern - a generic work queue pattern based on containers allows taking arbitrary processing code packaged as a container, and arbitrary data, and build a complete work queue system.
• Custom Controller Pattern - with this pattern, a controller watches for changes to objects and act on those changes to drive the cluster to a desired state.This reconciliation pattern can be used to implement custom logic and extend the functionality of the platform.
• Self Awareness Pattern - describes occasions where an application needs to introspect and get metadata about itself and the environment where it is running.

The foundational work in this area is done by Brendan Burns and David Oppenheimer in their container design patterns paper. Since then, Brendan published a book that also covers the design patterns and related topics around distributed systems. Roland Huß and I are also writing a book titled Kubernetes Patterns covering all these design patterns and use cases for container-based applications. Below are few of these patterns visualized.

[Click on the image to enlarge it]

Figure 6 - Container Design Patterns

Primitives need principles and makeup patterns. Let’s see also some of the best practices and overall benefits of using Kubernetes.

Practices & Techniques

In addition to the principles and patterns, creating good containerized applications requires familiarity with other container-related best practices and techniques. Principles and patterns are abstract, fundamental ideas that change less often. Best practices and the related techniques are more concrete and may change more frequently. Here are some of the common container-related best practices:

• Aim for small images - this reduces container size, build time, and networking time when copying container images.
• Support arbitrary user IDs - avoid using the sudo command or requiring a specific userid to run your container.
• Mark important ports - specifying ports using the EXPOSE command makes it easier for both humans and software to use your image.
• Use volumes for persistent data - the data that needs to be preserved after a container is destroyed must be written to a volume.
• Set image metadata - Image metadata in the form of tags, labels, and annotations makes your container images more discoverable.
• Synchronize host and image - some containerized applications require the container to be synchronized with the host on certain attributes such as time and machine ID.
• Log to STDOUT and STDERR - logging to these system streams rather than to a file will ensure container logs are picked up and aggregated properly.

Below are few links to resources with container related best practices:

• Best practices for writing Dockerfiles
• OpenShift Guidelines
• Kubernetes Production Patterns
• Kubernetes By Example

Kubernetes Benefits

As you can see from this article, Kubernetes dictates the design, development and day-2 operations of distributed systems in a fundamental way. The learning curve is not short either and crossing the Kubernetes chasm takes time and patience. That’s why here I’d like finish with a list of benefits that Kubernetes brings to developers. Hopefully, that will help justify why it is worth jumping into the Kubernetes ship and use it to steer your IT strategy.

• Self Service Environments - enables teams and team members to instantly carve isolated environments from the cluster for CI/CD and experimentation purposes.
• Dynamically Placed Applications - allows applications to be placed on the cluster in a predictable manner based on application demands, available resources, and guiding policies.
• Declarative Service Deployments -  this abstraction encapsulates the upgrade and rollback process of a group of containers and makes executing it a repeatable and automatable activity.

Figure 7 - Examples of deployment and release strategies with Kubernetes

• Application Resilience - containers and the management platforms improve the application resiliency in a variety of ways such as:            
  • Infinite loops: CPU shares and quotas
  • Memory leaks: OOM yourself
  • Disk hogs: quotas
  • Fork bombs: process limits                    
  • Circuit breaker, timeout, retry as sidecar
  • Failover and service discovery as sidecar
  • Process bulkheading with containers
  • Hardware bulkheading through the scheduler
  • Auto-scaling & self-healing
• Service Discovery & Load Balancing & Circuit Breaker - the platform allows services to discovery and consume other services without in application agents. Further, the usage of sidecar containers and tools such as Istio framework allow to completely move the networking related responsibilities outside of the application to the platform level.
• Declarative Application Topology - using Kubernetes API objects allow us to describe how our services should be deployed, their dependency on other services and resources prerequisites. And having all this information in an executable format allows us to test the deployment aspects of the application in the early stage of development and treat it as programmable application infrastructure.

[Click on the image to enlarge it]

Figure 8 - Declarative application topology using Kubernetes resource descriptor files

There are many more reasons why IT community is getting so excited about Kubernetes. The one I listed above are the ones I find really useful as somebody coming from a developer background.

Resources

Hopefully, I’ve managed to describe to you how I see Kubernetes affecting the daily life of developers. If you want to read more about this topic i.e. Kubernetes from developer’s point of view, checkout my book and follow me on twitter at @bibryam.

Below are few links to resources that focus on Kubernetes from a developer point of view:

• Design patterns for container-based distributed systems (white paper)
• Principles of container-based application design (white paper)
• Designing Distributed Systems (ebook)
• Kubernetes Patterns (ebook)                                            
• Kubernetes in Action (ebook)                                            

About the Author

Bilgin Ibryam (@bibryam) is a principal architect at Red Hat, committer and member of ASF. He is an  open source evangelist, blogger and the author of Camel Design Patterns and Kubernetes Patterns books. In his day-to-day job, Bilgin enjoys mentoring, coding and leading developers to be successful with building cloud-native solutions. His current work focuses on application integration, distributed systems, messaging, microservices, devops, and cloud-native challenges in general. You can find him on Twitter, Linkedin or his Blog.

Kubernetes Cookbook (By O’Reilly)

Learn how to use Kubernetes for container orchestration with easy‑to‑use recipes for installation, API access, monitoring, troubleshooting, and more. Download the FREE eBook Now.

 

Sponsored Content