Key Takeaways
- Istio helped make the "service mesh" concept more concrete and accessible, and with the recent release of Istio 1.0, we can expect a surge in interest.
- Istio attempts to solve some particularly difficult challenges when running applications in a cloud platform: application networking, reliability, and observability.
- Another challenge Istio addresses is security. With Istio, communication between services in the mesh is secure and encrypted by default.
- Istio can also help with "origin" or "end-user" JWT identity token verification.
- These foundational security features pave the road for building "zero-trust" networks where we assign trust based on identity as well as context and circumstances, not just "the caller happens to be on the same internal network".
Read our ultimate guide to managing service-to-service communications in the era of microservices and cloud.
Read the guide
Istio helped make the "service mesh" concept more concrete and accessible, and with the recent release of Istio 1.0, we can expect a surge in interest. Jasmine Jaksic did a great job introducing Istio and service mesh in a previous InfoQ article, and so I would like to take the opportunity to introduce a particular area of Istio that will bring lots of value to developers and operators of cloud services and applications: security
Istio Use Cases
Istio attempts to solve some particularly difficult challenges when running applications in a cloud platform. Specifically, Istio addresses issues around application networking, reliability, and observability. In the past we've tried to use purpose-built application libraries to solve some of these challenges like circuit breaking, client-side load balancing, metrics collection, and others. Doing this across different languages, frameworks, runtimes, etc creates a prohibitive operational burden that most organizations can't scale.
Additionally, it is hard to get consistency across the implementations found in each language, never mind upgrade them all in lock-step when things need to change or bugs are identified. A lot of challenges around reliability, observability, and policy enforcement are very horizontal concerns and are not business differentiators. Although they are not directly differentiators, neglecting them can cause massive business impact, and so we need to address them. Istio aims to solve these concerns.
The Importance of Network Security
Another one of the horizontal, difficult to get right, concerns for application teams is security. In some cases, security is an afterthought, and we try to shoehorn it into our apps at the last possible moment. Why? Because doing security right is hard. For example, something foundational like "encrypting application traffic" should be commonplace and straightforward, right? Configuring TLS/HTTPS for our services should be straight forward, right? We may have even done it before on past projects. However, getting it right in my experience is not as easy as it sounds. Do we have the right certificates? Are they signed by the CA that the clients will accept? Are we enabling the right cipher suites? Did I import that into my truststore/keystore properly? Wouldn't it just be easy to enable the “--insecure” flag on my TLS/HTTPS configuration?
Mis-configuring this type of thing can be extremely dangerous. Istio provides some help here. Istio deploys sidecar proxies (based on Envoy proxy) alongside each application instance that handles all of the network traffic for the application. When an application tries to talk to http://foo.com it does so via the sidecar proxy (over the loopback network interface) and Istio will redirect that traffic to the other service's sidecar proxy which then proxies the traffic to the actual upstream http://foo.com. By having these proxies in the request path, we can do things like encrypt the transport automatically without the application having to know anything. This way we get a consistently applied traffic encryption without relying on each application development team to "get it right".
One of the issues with setting up and sustaining an implementation of TLS and mutual TLS for your services architecture is certificate management. Istio's Citadel component in the control plane handles getting the certificates and keys onto the application instances. Citadel can generate the certificates and keys needed for each workload to identify itself, as well as rotate certificates periodically so that any compromised certificates have a short life. Using these certificates, Istio-enabled clusters have automatic mutual TLS. You can plug in your own CA provider root certificates as needed as well.
With Istio, communication between services in the mesh is secure and encrypted by default. You no longer have to mess around with certificates and CA certificate chains to get TLS working. Operators no longer have to hope and pray each developer properly implements and configures their TLS/HTTPS settings correctly. It's done for you automatically through some Istio configurations.
Using Istio to enable mTLS
Istio follows the same path to configuration as Kubernetes does. Actually, on Kubernetes, Istio is configured with Kubernetes Custom Resource Definition (CRD) objects. The two main objects for configuring Istio's security policies are the Policy and DestinationRule object. Policy objects are used to configure the security settings of a service (or group of services). For example, to configure all services running in the customer namespace in Kubernetes, we can use the following Policy object:
apiVersion: “authentication.istio.io/v1alpha1”
kind: “Policy”
metadata:
name: “default”
namespace: “customer”
spec:
peers:
When we use this configuration, services running in customer namespace will expect any incoming traffic to use mTLS. But to use mTLS, we also need to tell the clients to use mTLS when they call a service. To do that, we'll use an Istio DestinationRule. A DestinationRule in Istio is always used to configure how clients talk to a service. With a DestinationRule we can specify things like circuit breaking, load balancing and TLS. To enable mTLS, we can use a configuration like this:
apiVersion: “networking.istio.io/v1alpha3”
kind: “DestinationRule”
metadata:
name: “default”
namespace: “foo”
spec:
host: “*.customer.svc.cluster.local”
trafficPolicy:
tls:
mode: ISTIO_MUTUAL
This DestinationRule will require clients use mTLS when making calls to services in the customer namespace. We are also configuring the tls mode to be ISTIO_MUTUAL which means we'll expect Istio to manage the certificates and keys as well as mount them into the services (using Kubernetes secrets in Kubernetes) so that the service proxy can use them to establish TLS. If we wanted to have our own control over the client certificates, we could use a mode of MUTUAL and also supply the location on disk where the client can find its certificates and private keys.
Using Istio to validate origin identity (with JWT)
When we use mTLS as described above, we can not only encrypt the connection, but more importantly know exactly who is calling whom. Istio uses the Secure Production Identity Framework for Everyone (SPIFFE) specification. Identities are encoded into the certificates that are used for mTLS. This way, service A knows that when service B is talking with it, that in fact it is service B. We can write rules around these identities about what rules or policies the service mesh must enforce. For example, if service A is not allowed to talk to service B, we can enforce that with the sidecars that run alongside each applications using the identities used to establish mTLS.
But what happens when service A is making a request to service B on behalf of user X? If service A is allowed to call and ask service B to do something (check account balances), but user X cannot, how can we verify and enforce this? In a services architecture, the typical way for services to communicate end-user or origin identity (the user logged in) is passing identity tokens like JSON Web Tokens. These tokens are given out to represent an authenticated user and the claims that user has.
Istio can help with "origin" or "end-user" JWT identity token verification. This is another area where each application language/framework combination has historically had to rely on libraries to handle verifying and unpacking the JWT tokens. For example, for the popular Keycloak Identity and SSO project, there are language plugins for each popular language to handle some of this responsibility. If we're using Istio then we can get that kind of functionality for free. For example, to configure Istio to both use mTLS and verify the JWT token in a request (and fail the request if it doesn't exist, is invalid, or is expired), we can configure a Policy object. Remember, Policy objects specify the behavior we want from the services:
apiVersion: “authentication.istio.io/v1alpha1”
kind: “Policy”
metadata:
name: “customer-jwt-policy”
spec:
targets:
- name: customer
peers:
- mtls:
origins:
- jwt:
issuer: http://keycloak:8080/auth/realms/istio
jwksUri: http://keycloak:8080/auth/realms/istio/protocol/openid-connect/certs
principalBinding: USE_ORIGIN
With this configuration, if a client tries to connect to the customer service, their request wouldn't make it to the service unless the JWT authentication succeeds. Another good thing about Istio's implementation here is that the request is also protected my mTLS. This helps protect the JWT token from getting leaked and used for some kind of replay attacks.
The Future: Zero-Trust Networks
We've covered a couple of ways Istio can improve your security posture when building cloud-native applications. With strong identity in the services talking with each other as well as origin/end-user, we can write some pretty powerful access-control rules about how the system is allowed to behave. This foundation paves the road for building "zero-trust" networks. In a zero-trust network, we assign trust based on identity as well as context and circumstances, not just "the caller happens to be on the same internal network". As we start to move toward fully connected and hybrid cloud deployment models, we need to rethink how best to build security into our architectures. Istio can solve challenges in today's architectures as well as give you more options in the future.
Istio provides some very powerful capabilities that service teams will have to solve one way or another. It provides nice APIs and configuration objects to accomplish this outside of application services. It's implemented in a highly decentralized way and is intended to be highly resilient to failures. If you're looking to adopt a service mesh, and consider security high on your list, take a look at Istio.
About the Author
Christian Posta (@christianposta) is a Chief Architect of cloud applications at Red Hat and well known in the community for being an author (Introducing Istio Service Mesh, O’Reilly 2018, Microservices for Java Developers, O’Reilly 2016), frequent blogger, speaker, open-source enthusiast and committer on various open-source projects including Istio, Apache ActiveMQ, Fabric8, et.al. Christian has spent time at web-scale companies and now helps companies create and deploy large-scale, resilient, distributed architectures - many of what we now call Microservices. He enjoys mentoring, training and leading teams to be successful with distributed systems concepts, microservices, devops, and cloud-native application design.