HashiCorp has released Consul 1.2.1, the latest version of their highly available and distributed service discovery and key-value store, which also includes a public beta launch of Consul Connect. Consul Connect provides service-to-service connection authorization and encryption using mutual TLS, and "automatically turns any existing Consul cluster into a service mesh solution." Connect requires only a single binary for all necessary subsystems.
The current Connect use case focuses on deployment best-practices with service-to-service encryption and identity-based authorization. Connect does not yet provide all of the features that engineers may typically expect to find in other service mesh offerings like Linkerd, Cilium or Envoy / Istio, such as traffic shifting, rate limiting, timeouts, retries, circuit breaking and observability. However, in personal correspondance with Nic Jackson, developer advocate at HashiCorp, he stated that:
Connect has been built to be extensible so that it will integrate with existing proxies, for example, Envoy to provide all the expected features. Native integration for Kubernetes will also be available by the time Connect goes GA.
Services or applications can use sidecar proxies to automatically establish TLS connections for inbound and outbound connections without being aware of Connect. Applications can also natively integrate with Connect for environments with higher performance and security demands. In addition, the solo.io team has released "Gloo Connect", a prototype integration between the Envoy Proxy and Consul Connect.
Connect was designed to be easy to use, and can be enabled with one additional option within a typical Consul configuration; adding an extra line, "connect:" (and associated metadata) to the service registration config automatically enables any existing application to accept Connect-based connections. Connect uses standard TLS over TCP/IP, which allows it to work on any network configuration as long as the IP advertised by the destination service is reachable by the underlying operating system.
The core of Connect is based on mutual TLS. Connect provides each service with an identity encoded as a TLS certificate. According to the Connect documentation, this certificate is used to establish and accept connections to and from other services, and the identity is encoded in the TLS certificate in compliance with the SPIFFE X.509 Identity Document. Certificate rotation is automatic and causes no downtime. To generate and distribute certificates, Consul has a built-in CA that requires no other dependencies, and also ships with built-in support for Vault. The PKI system has been designed to be pluggable.
In regards to service-to-service authorization, rather than authorizing host-based access with IP address access rules, Connect uses the registered service identity to enforce access control with "intentions". This use of logical service identity names, rather than IP addresses and ports, can make it easier to reason about access control. Rules are scale independent -- it doesn't matter if there is a single "web server" service or hundreds -- and the use of names can also facilitate the mobility of services. For example, a service name provides location transparency when services are re-scheduled across hosts by a container orchestration framework such as Kubernetes, Nomad or AWS ECS.
Intention enforcement can be done regardless of the underlying network. Therefore, Connect works with physical networks, cloud networks, software-defined networks, and cross-cloud. For HashiCorp Enterprise customers, intentions can also be combined with the Sentinel policy as code framework to provide multiple levels of security policy or additional granularity.
Connect currently only works for service-to-service connections within a single Consul datacenter; Connect may be enabled on multiple Consul datacenters, but only services within the same data centers can establish Connect-based connections. In addition, CA configurations and intentions are both local to their respective data centers; they are not replicated across datacenters. Multi-datacenter support for Connect is under development and will be released as a feature of Consul Enterprise in late 2018. This feature will facilitate intention replication, datacenter constraints on intentions, CA state replication, multi-datacenter certificate rotations, and more.
The documentation states that all APIs required for Connect typically respond in microseconds and "impose minimal overhead to existing services". This is because the Connect-related APIs are all made to the local Consul agent over a loopback interface. The Consul agent locally caches most Connect-related data and sets up background blocking queries against the server to update the cache in the background. This allows most API calls such as retrieving certificates or authorizing connections to use in-memory data and respond very quickly.
The "Agent Caching and Performance" section of the documentation states that with Connect enabled, engineers will most likely see increased memory usage by the local Consul agent. The total memory is dependent on the number of intentions related to the services registered with the agent accepting Connect-based connections.
More information on the Consul 1.2 and Consul Connect release can be found on the HashiCorp blog. HashiCorp co-founder and CTO Armon Dadgar has also presented a whiteboard-based overview of Consul Connect on YouTube, and additional information is available on the feature homepage. Consul 1.2 can be downloaded from https://www.consul.io/
This news item was updated 25th July to include the quote from Nic Jackson.
