Packaging Applications for Docker and Kubernetes: Metaparticle vs Pulumi vs Ballerina

With the emergence of microservice architecture, the software industry is moving towards cloud native application development. Two pizza development teams, agility, repeatability, and CI/CD are playing a major role in fast and productive innovations of current software industry.  

Docker and Kubernetes are the key elements of modern cloud native deployment automations. The common practice is to create reproducible packaging for developed applications with containers. Docker allows developers to produce repeatable runtime environments where they can define the dependencies and configuration of their applications in a simple and repeatable way. Kubernetes is an open source container orchestration platform, allowing deploy these application containers across multiple host machine while providing scalability and high availability. This required writing Dockerfile and Kubernetes YAML deployment descriptors; which can be painful and error prone.  

Metaparticle, Ballerina, and Pulumi are three open source projects that introduce their own approach to solve this problem. I recently came across three tweets discussing these approaches.

The first was Andress Guisado explaining how Metaparticle provides a standard library to create cloud native applications which directly deployable on Kubernetes. Brendan Burns enounced Metapaticle at the KubeCon early in this year, and Andress was thinking this will be a big focus on 2018.

After exposed to Ballerina in the Istio community meetup, Dan Ciruli sent a tweet saying that Ballerina is an interesting language as it can auto generate Kubernetes and Istio YAMLs as part of your build process. He further expressed this is an great idea and he imagines other frameworks will probably adopt this style too.

Third, Ustun Ozgur tweeted saying that the Pulumi effort of defining infrastructure as code as opposed to silly YAML files is going to do to DevOps as what React did to web development.

In this article I am comparing how these three projects are helping to automate application code deployment in container orchestration platform like Kubernetes without having handwritten YAMLs. Read on to find out more details of these approaches. 

Metaparticle

• Website
• Github

Metaparticle/Package simplifies the task of building and deploying container images. It is a collection of libraries that enable programmers to build and deploy containers using code that feels familiar to them. Currently its supports Java, .NET core, Javascript (NodeJS), Go, Python and Ruby programming languages.

Let's look at how deploy your Java code into Kubernetes by using Metaparticle.

Prerequisites: 

1. Docker/ Kubernetes
2. mp-compiler command line tool
3. io.metaparticle:metaparticle-package as a maven dependency.

The following code starts a Kubernetes pod containing an HTTP service which prints “Hello World!” 

package io.metaparticle.tutorial;

import io.metaparticle.annotations.Package;
import io.metaparticle.annotations.Runtime;

import static io.metaparticle.Metaparticle.Containerize;

import java.io.IOException;
import java.io.OutputStream;
import java.net.InetSocketAddress;

import com.sun.net.httpserver.HttpExchange;
import com.sun.net.httpserver.HttpHandler;
import com.sun.net.httpserver.HttpServer;

public class Main {
   private static final int port = 8080;

   @Runtime(ports = {port},
       replicas = 4,
       publicAddress = true,
       executor = "metaparticle"
   )
   @Package(repository = "docker.io/lakwarus",
           jarFile = "target/metaparticle-package-tutorial-0.1-SNAPSHOT-jar-with-dependencies.jar", publish = true)
   public static void main(String[] args) {
       Containerize(() -> {
           try {
               HttpServer server = HttpServer.create(new InetSocketAddress(port), 0);
               server.createContext("/", new HttpHandler() {
                   @Override
                   public void handle(HttpExchange t) throws IOException {
                       String msg = "Hello World!";
                       t.sendResponseHeaders(200, msg.length());
                       OutputStream os = t.getResponseBody();
                       os.write(msg.getBytes());
                       os.close();
                       System.out.println("[" + t.getRequestURI() + "]");
                   }
               });
               server.start();
           } catch (IOException ex) {
               ex.printStackTrace();
           }
       });
   }
}

Few things to note:

• Imported io.metaparticle.annotations.Package and io.metaparticle.annotations.Runtime
• @Package annotation that describes how to package the application.
• @Runtime annotation that describes runtime configs of the application
• Wrapped the main function in the Containerize function which kicks off the Metaparticle code.

Compiling the code:

mvn compile

This will create a jar file with all the dependencies built in.

Running the code:

mvn exec:java -Dexec.mainClass=io.metaparticle.tutorial.Main

This will generate the Dockerfile, Docker image and push it to the given registry. It then starts a Kubernetes deployment with 4 pods in the configured Kubernetes cluster. 

Accessing the service:

You need to create a proxy to access the service.

$ kubectl port-forward deployment/io-metaparticle-tutorial-main 8080:8080

$ curl http://localhost:8080/
Hello World!

Highlights

• No YAMLs/JSON are created
• Fully automated deployment
• Supports multiple languages
• Supports limited Kubernetes functionalities such as services and deployments. Only clusterIP services are supported
• Need to wrap users code with Containerize() block. Then your code can’t run in standalone mode.  

Ballerina

• Website
• Language
• Kubernetes builder extension

Ballerina is a new open source cloud native programming language that is designed to bring code-first agility to the challenge of integrating across endpoints. Ballerina has first-class support for APIs, distributed transactions, circuit-breakers, stream processing, data-access, JSON, XML, gRPC, and many other integration challenges. 

Ballerina understands the architecture around it; the compiler is environment aware with microservices directly deployable into infrastructure like Docker and Kubernetes by auto generating Docker images and YAML’s. 

Let’s look at how to use the Ballerina Kubernetes annotation to deploy your code into Kubernetes.

Prerequisites: 

1. Ballerina
2. Docker/Kubernetes

The following code starts an HTTP service which prints “Hello World!” 

hello_world_k8s.bal

import ballerina/http;
import ballerinax/kubernetes;

@kubernetes:Service {
serviceType: "NodePort",
name: "hello-world" 
}
endpoint http:Listener listener {
 port: 9090
};

@kubernetes:Deployment {
image: "lakwarus/helloworld",
name: "hello-world"
}
@http:ServiceConfig {
  basePath:"/"
}
service<http:Service> helloWorld bind listener {
  @http:ResourceConfig {
     path: "/"
  }
  sayHello(endpoint outboundEP, http:Request request) {
      http:Response response = new;
      response.setTextPayload("Hello World! \n");
      _ = outboundEP->respond(response);
  }
}

Few things to notice:

• Imported ballerinax/kubernetes package
• @kubernetes:Service on top of Ballerina Service

Compiling the code:

Compile the hello_world_k8s.bal file. Command to run Kubernetes artifacts will be printed on success:

$> ballerina build hello_world_k8s.bal

@kubernetes:Docker          - complete 3/3
@kubernetes:Deployment      - complete 1/1
@kubernetes:Service         - complete 1/1

Run following command to deploy kubernetes artifacts: 
kubectl apply -f ./kubernetes/

Ballerina compiler will generate hello_containers_k8s.balx, Dockerfile, Docker image and Kubernetes artifacts in following structure.

$> tree
.
├── hello_world_k8s.bal
├── hello_world_k8s.balx
└── kubernetes
    ├── docker
    │   └── Dockerfile
    ├── hello_world_k8s_svc.yaml
    └── hello_world_k8s_deployment.yaml

Running the code:

`kubectl apply -f ./kubernetes/` will deploy app into Kubernetes and can access via Kubernetes NodePort.

Accessing the service:

$> kubectl get svc
NAME                    TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
hello-world           NodePort    10.96.118.214    <none>        9090:32045/TCP   1m

$> curl http://localhost:<32045>/
Hello, World!

Highlights

• Kubernetes supports is provided as native with Ballerina. 
• Code should be written in Ballerina.
• Generate deployment artifacts depending on annotation defined in the code.
• Deployment is not fully automated. It required to run a kubectl command
• Support many Kubernetes functionalities like; all kind of Kubernetes services, deployment, ingress, secrets, persistent volumes, config maps, liveness probe and horizontal pod autoscaling.
• No need to change or wrap user’s code. Annotations decorate objects in Ballerina code. The Ballerina compiler parses annotations into an AST that can be read and acted upon.

Pulumi

• Website
• Github

Pulumi is a cloud development platform that makes creating cloud programs easy and productive. You can author cloud programs in your favorite language and Pulumi will automatically keep your infrastructure up-to-date: “skip the YAML and just write code”. Pulumi is multi-language, multi-cloud and fully extensible in both its engine and ecosystem of packages.

Currently it supportß JavaScript, TypeScript, Python, and Go programming languages. Amazon Web Services, Microsoft Azure, Google Cloud Platform, and Kubernetes cloud platforms are supported. 

Pulumi is mainly focusing on infrastructure code automation rather than application code automation. You can use your prefered programing language to automate infrastructure deployment.  

Pulumi support out-of-the-box FaaS deployment with public serverless providers like AWS Lambda, but in this article I am focused only on deployment automation on Docker and Kubernetes.

Prerequisites: 

1. Docker/ Kubernetes
2. Install Pulumi (curl -fsSL https://get.pulumi.com/ | sh)
3. Configure Pulumi with Kubernetes cluster 

Let's look at how to use TypeScript and Pulumi to deploy a “helloworld” sample application. 

I have created a “helloworld” sample application (printing “Hello World” for an HTTP request), and corresponding Docker image (lakwarus/helloworld:latest). Now I am going to use TypeScript with Pulumi libraries to write simple code to deploy my app into Kubernetes without creating handwritten YAML artifacts.  

Creating the Pulumi project:

$> pulumi new
Please choose a template: typescript
This command will walk you through creating a new Pulumi project.

Enter a value or leave blank to accept the default, and press <ENTER>.
Press ^C at any time to quit.
project name: (hello) 
project description: hello world
Created project 'hello'.
stack name: (hello-dev) 
Created stack 'hello-dev'.
Installing dependencies...

added 113 packages in 12.549s
Finished installing dependencies.
New project is configured and ready to deploy with 'pulumi update'.

Update the package.json with following dependencies.

{
    "name": "hello",
    "main": "bin/index.js",
    "typings": "bin/index.d.ts",
    "scripts": {
        "build": "tsc"
    },
    "devDependencies": {
        "typescript": "^2.7.2",
        "@types/node": "latest"
    },
    "dependencies": {
        "@pulumi/kubernetes": "^0.14.0",
        "@pulumi/pulumi": "^0.14.0",
        "npm": "^6.1.0"
    }
}

Edit index.ts file with your application and deployment information. Here I just added the pod configuration but you can extend it to other Kubernetes functionalies.

import * as pulumi from "@pulumi/pulumi";
import * as k8s from "@pulumi/kubernetes";

let helloPod = new k8s.core.v1.Pod("hello", {
   metadata: {
       name: "hello",
   },
   spec: {
       containers: [{
       name: "hello",
       image: "lakwarus/helloworld",
       ports: [{
           containerPort: 9090,
       }],
       }],
   },
});

Compiling and running the code:

$> npm update
$> npm run build
$> pulumi update
Previewing update of stack 'hello-dev'
Previewing changes:

     Type                    Name             Plan       Info
 +   pulumi:pulumi:Stack     hello-hello-dev  create     
 +   └─ kubernetes:core:Pod  hello            create     
 
info: 2 changes previewed:
    + 2 resources to create

Do you want to perform this update? yes
Updating stack 'hello-dev'
Performing changes:

     Type                    Name             Status      Info
 +   pulumi:pulumi:Stack     hello-hello-dev  created     
 +   └─ kubernetes:core:Pod  hello            created     
 
info: 2 changes performed:
    + 2 resources created
Update duration: 10.132746709s

Accessing the service:

$ kubectl port-forward pod/hello 9090:9090

$ curl http://localhost:9090/
Hello World!

Highlights

• Mainly focused on infrastructure code automation.
• Able to use prefered programing language to control your infrastructure. 
• Application code should be in as aFunction(like AWS Lambda) or need to be in as a docker images to use in deployment automation.
• Support all-most all the public cloud providers and Kubernetes
• Able to create complex deployment in few line of codes without handwritten YAMLs.
• Fully automated.
• Potential vendor lock-in, as you need to have an account on http://pulumi.io

Summary 

With the emergence of microservice architecture, the software industry is moving towards cloud native application development and deployment. Docker and Kubernetes are the key elements of modern cloud native deployment. However, the current requirement as having to manually create YAML deployment descriptors is painful and error prone.  

One popular way of deploying application on top of Kubernetes is by adopting different tools and frameworks. Draft, Gitkube, Helm, Ksonnet, and Skaffold are popular tools which leads in this front and here is a very interesting article “Draft vs Gitkube vs Helm vs Ksonnet vs Metaparticle vs Skaffold” that compares these tools that help developers build and deploy their apps on Kubernetes. All of these tools has different workflows but they will be solving the same problem, which is increasing agility and productivity of application deployment on Kubernetes.

Metaparticle, Ballerina, and Pulumi have introduced different approaches by empowering developers to handle deployment automation within programing language itself without having handwriting YAMLs. This is becoming a trend and will change the DevOps practice in the software industry.

About the Author

Lakmal Warusawithana works as a Sr. Director and architect at WSO2, the largest Open Source integration provider. Lakmal has a long history of working in open source, cloud, and DevOps technologies and has been Vice President of Apache Stratos PaaS Project. In 2005, Lakmal co-founded the thinkCube, the pioneers in developing the next generation of Collaborative Cloud Computing products that are tailored towards Telecom operators. He oversaw the overall engineering process, giving special attention to scalability and service delivery of thinkCube solutions. Prior to co-founding thinkCube, Lakmal spent four years at ITABS, a company that specialized in Linux based server deployments that came with a custom easy-to-use server management interface. Lakmal has also presented at numerous events, including ApacheCon, CloudOpen, QCon, JaxLondon, Cloud Expo, Cloudstack Collaboration Conference, WSO2Con and many tech meetups. Lakmal holds a BSc (Hons) Special in Computer Science from the University of Colombo, Sri Lanka.