Kelsey Hightower on Extending Kubernetes, Event-Driven Architecture, and Learning

Show Notes

Can you recent talk about your recent highlights and what you're working on now? -

• 01:20 I work on Google Cloud, focussed on cloud native; Google have contributed lots to this space, including: Kubernetes, ideas around observability and tracing etc
• 01:30 I work in open source, helping our customers adopt some of theses ideas, tools and practices.
• 01:35 My history goes back as a long time open source contributor and teaching other people everything I know.

Why do you think Kubernetes has become the orchestrator of choice? -

• 01:55 I think it's because of its extensibility.
• 02:00 Kubernetes is probably more complete in terms of its extension APIs; the people we have working on it, from Red Hat, CoreOS, Google, Microsoft - all of these people are contributing to Kubernetes.
• 02:20 We've implemented the extensions in a pragmatic way: custom resource definitions; CNI plugins; the container runtime interface.
• 02:25 All of these extension points allow the existing ecosystem to adopt Kubernetes in a first-class way without necessarily convincing everyone to do it the same way.

Is the Google way forward to build on top of Kubernetes, like Anthos? -

• 02:45 Anthos represents people who want that commercial relationship with Google.
• 02:50 Kubernetes is great, but by itself - you have to add a little bit more to make it a complete thing.
• 03:00 GKE has been wildly popular is that we have a dedicated set of UIs, workflows for managing a full lifecycle of a cluster.
• 03:10 Those are things that you don't find in the GitHub project, because it doesn't make sense for it to live there.
• 03:15 Lots of people have asked "we want GKE in other places", and in order to do that, we decided to create this other brand Anthos.
• 03:25 That brings you GKE in the other environments that we plan to support.

Do you think Kubernetes is where it's at for DevEx [Developer Experience], or does it need to evolve some way to make it easier? -

• 03:35 We have to understand where we're coming from.
• 03:40 For a lot of people, their DevEx experience has involved the command-line, SSH, Puppet and Chef.
• 03:50 If you have been primarily using the command line, you've been copying things around.
• 03:55 You could use Puppet or Ansible, and running these playbooks and watching them complete, and resolving errors as necessary.
• 04:05 For those people, using Kubernetes will probably be seen as a step up, in terms of DevEx.
• 04:10 Instead of crafting all of these scripts to get something running, with infrastructure as code, just to get a deployment done, now you're using "kubectl apply".
• 04:20 Just using that single command, you're going to get something highly available, automated scheduler, metadata on what's running - I can see why people think that Kubernetes is the end game.
• 04:35 If you're coming from the other direction - maybe you have experience with Cloud Foundry or Heroku - you may wonder why it's not more opinionated.
• 04:50 The reason why Kubernetes is popular is that it doesn't force you down a single path of getting things done, and it supports more workload types: crontab, batch jobs, stateless and stateful workloads.
• 05:05 Given that, I think people want to have a little more opinion there, but I don't think they want to give up on their freedom.
• 05:15 For those that want a PaaS-like feel, Kubernetes has become a platform for building platforms, so OpenShift is built on top, knative is built on top, CloudRun has a layer that runs on top of Kubernetes.
• 05:30 I think now you can get any experience you want, without sacrificing the ability to do things that you haven't thought of yet.

What do you think are the core components of a kubernetes platform? -

• 05:50 If you're building a 12-factor application - the classic target point for Cloud Foundry and Heroku - then Kubernetes is going to give you a lot out of the box.
• 06:00 Assuming that you have access to a cluster (like GKE or Amazon Fargate or AKS from Azure) then it is managed for you - there is a lot of things taken care of for you.
• 06:20 At that point you have this Kubernetes API, and your entry point as a developer is going to be to create a container image, then describe how it's going to run.
• 06:30 If you're brand new to Kubernetes, it may take you a while to realise what 20 lines of YAML you need to craft to get Kubernetes to do what you want.
• 06:40 For most people, you can move between containers from Heroku or Cloud Foundry into Kubernetes with very little to no modification.
• 06:50 It's more about understanding the declarative nature of Kubernetes, and then making a go of it.

Do you see Kubernetes evolving into a serverless or function-as-a-service? -

• 07:05 You have to put it into perspective.
• 07:10 If you are building a serverless platform from the ground up, you don't want to start from zero - like a Linux VM - and rebuild the functionality that Kubernetes or Nomad has.
• 07:25 It will probably take you more than 6 years because you won't have the contribution base from scratch.
• 07:30 If you start with Kubernetes, now you can focus on the missing parts - what does your experience need to have?
• 07:35 If you want to build a function-as-a-service, then you can build on top of knative, which has a serving framework, meaning when a request comes in you can scale a workload up and down on demand.
• 07:45 It has an eventing framework, which allows you to define the various event types - if you have experience with Amazon Lambda or Google Cloud Functions, you can create a function and map an event to it.
• 08:00 If you're going to build one yourself, you can build this up on top.
• 08:05 CloudRun is interesting, because it's a bit in-between.
• 08:10 Where I think functions-as-a-service really leans into this eventing nature - that everything will be called from events - the contracts are strong and opinionated.
• 08:15 A lot of people are going to get value from working that way, but that assumes that you have a workflow into a set of functions.
• 08:25 I think the majority of the world want to write in whatever programming language suits them; they're probably OK because there's so many great HTTP frameworks out there, like Ruby on Rails or the Go standard libary.
• 08:40 You can just build any app you want, handling one route or multiple routes - and CloudRun is about taking that container and managing that for you.
• 08:50 It gives you that serverless paradigm, even though you have slightly more responsibility than you would have in a classic function-as-a-service platform.

What do you think about service mesh? -

• 09:10 When you hear the word 'service mesh' alone, you may not be sure what people are talking about - but it's the functionality that people are after.
• 09:15 To be honest, my guess is you can get 90% of service mesh functionality from NGINX.
• 09:20 If you knew how NGINX worked, you can do things like TLS mutual auth, a bit of observability, custom service mesh like features running in Lua scripts inside NGINX.
• 09:40 Many of these things that we would classify as service meshes or API gateways are built on top of nginx.
• 09:45 I think one thing that was new in things like Envoy does is it becomes the data plane with a very nice configuration language.
• 10:05 Envoy was created with the idea that there would be this dynamic service discovery, back-ends would come and go - and it offered an API to manage that.
• 10:15 When you combine that with a data plane, you can do things now to say these two applications can talk to each other.
• 10:25 In a standard NGINX config, we put it on the edge and it was a load balancer - but we are now saying what if we put this load balancer next to every application?
• 10:35 We end up with a service mesh with a bunch of capabilities.

What is the cloud native learning experience like for later adopters? -

• 10:55 These fundamentals have been there for a very long time - an application needs to be packaged and deployed; when it crashes, it needs to move to a new thing to keep running.
• 11:05 These are principles that have been around for 20 or 30 years.
• 11:10 Same with networking; you want to route traffic, based on certain attributes - these are principles and fundamentals.
• 11:20 I think it's easier to adopt these fundamentals than ever before.
• 11:25 If you didn't understand how mutual TLS worked, and I told you that the Envoy open-source project has support for that out of the box ...
• 11:35 If you don't understand that how a CA works, but that Istio has a built in CA that is a flag away from using ...
• 11:40 Now you start to see the floodgates open of opportunity and possibility.
• 11:45 It's not like it's a bunch of new stuff; it's that we've made the existing stuff much easier to adopt.
• 11:55 You can adopt these ideas without becoming an expert yourself.

Are the challenges more tech-focussed or people-focussed? -

• 12:05 Your whole life is people-focussed - this has nothing to do with technology; your whole life, you're a person first.
• 12:10 That's the hard part that people have trouble understanding.
• 12:15 If you're a person first, then your whole life is going to be seen through your own eyes - and the pain where you are is always have those people challenges confronting you.
• 12:25 Your challenges are going to be very unique to your situation.
• 12:30 If you work somewhere that fosters education and new learning, and you work with colleagues that are helpful and respectable, you can adopt technology in a more pragmatic and safe way.
• 12:40 If you're around a group of people that don't share those values, you are going to be pushing against the grain.
• 12:45 I'm one of those people who believe in sharing everything I know, to the point where I hope you are a better person - and I hope you can return the favour and teach me something new.
• 13:00 The technology comes second; if I know have a set of problems, and I can communicate those problems, and I'm in the business of solving problems, it's a matchmaking exercise from tehre.
• 13:10 You get into a listening dynamic, a feedback loop.
• 13:15 The community presenting problems and solving problems is the way it fits together.

Where should a learning journey start for a new technologist? -

• 13:40 I can talk from personal experience - when I was new to the game, I was respectful that everyone has to learn at some point.
• 13:50 Everyone's starting from zero; even if they're an expert in one thing, they may not be an expert in another thing.
• 13:55 I was OK with this idea that I'm not the best or expert in the area - I'm going to start investing in myself.
• 14:05 Everything that you do is a net improvement to your net personal value.
• 14:10 Fifteen years ago, I decided that I wanted to learn at least one programming language - I started with bash, then progressed to Python.
• 14:20 I think if you pick one tech stack that you may gravitate to - go buy the books, study at home, invest in yourself.
• 14:30 The other area to start - whatever your are using in your job - don't fall into the trap of the fear of missing out, using X because company Y is using it too.
• 14:40 You can get the experience you need by really learning the stack you have at your job.
• 14:45 If you have a PHP, MySQL and Apache - you can go really deep in each of those, and retain 80% of the fundamentals to the next thing you learn.
• 14:55 If you're new to the game, respect what's in front of you, take all the opportunity because the experience is more important than the specific technology that you're using.

What do you think of the role of mentors? -

• 15:25 If you consider education and investment in yourself, and you're willing to buy the book and do it yourself, you have to give a lot of credit in someone who is willing to invest in you and accelerate the process.
• 15:40 If they give you a few things they wish they had known in their time, why would you deny someone willing to invest in you?
• 15:50 If you've gone to school, you can consider those teachers as mentors in many ways - that's how humans work.
• 16:05 If you want to block people out from teaching you, that's going to slow down the process - why not take what's a net benefit to you?

How do we encourage the development of healthy communities? -

• 16:30 As much as I value the community, the community isn't your life.
• 16:35 The technology will let you down; building communities around technologies are weird on their own.
• 16:45 It's nice to have a common set of interests that bring people together.
• 16:50 No matter how or what brings you together, you still need to be a good person as an individual.
• 17:00 If you're having a bad day, maybe don't show up - you don't need to bring that negative energy into the gathering.
• 17:05 That's true for all walks of life.
• 17:10 To foster a good community, you need to have some leadership.
• 17:20 Some people consider me as one of the leaders in the Kubernetes community - I try to have good behaviour.
• 17:25 I try to make myself available time-wise; I try to help people out in many ways; teaching, feedback.
• 17:35 When I see the wrong behaviour, calling the wrong behaviour out and doing so in a respectful way trying to make sure the community has one star to look at.
• 17:50 The community isn't your identity - don't go and get tattoos of Kubernetes on your body.
• 18:00 I think there's a healthy balance here: people first, and people come together to form communities.
• 18:05 When we show up, let's treat each other the right way.

What do you think is the biggest challenge for Kubernetes technically over the next year? -

• 18:15 Kubernetes did a good job about attempting to define its identity - what does core look like.
• 18:20 I think the best thing we've been doing over the years, we've been moving things out of the core.
• 18:25 We've moved the client libaries out of the core, we've moved all the cloud provider integrations out of the core.
• 18:35 That means that the core in many ways is getting a little bit smaller over time, and focussing on the core things.
• 18:40 For example, RBAC, operating system support, pluggable APIs for various container runtimes.
• 18:45 Instead of having the container runtimes built into the core, we're starting to move that out to the container runtime interface.
• 18:50 The same is true for networking and storage.
• 18:55 Now that we have a very clean core, the challenge is to preserve the core and be comfortable with an ecosystem evolving around Kubernetes.
• 19:05 Serverless is really hot, but let's not bake that into the core -- and instead let's work on knative.
• 19:10 Similarly for service meshes -- let's have Istio building on top of Kubernetes.
• 19:20 As the ecosystem gets bigger, new vendors show up with different expectations, we'll have to make sure they use the extension APIs.

Do you think Kubernetes needs its Rails moment? -

• 20:00 Everyone's going to have their own flavour of Kubernetes with a few add-ons to give people what their Rails moment will feel like.
• 20:10 The truth is that you're going to have not only Rails, but also React and Machine Learning - the experience will be different for all of them.
• 20:20 In many ways, Kubernetes will start to disappear over time, to the point where maybe we don't talk about Kubernetes as much.
• 20:25 That's going to be OK - it's like Linux, fundamental to everything we do, but people don't talk as much about Linux as they used to.
• 20:35 I think the future of Kubernetes is that it disappears from view a bit.

Do you think Kubernetes and Machine Learning have a future together? -

• 20:50 In the platform-building-as-a-platform regard, KubeFlow has attempted to take TensorFlow and layer it on top of Kubernetes.
• 21:05 That will continue, because you need a distributed system underneath, and Kubernetes is great.
• 21:15 Since Kubernetes produces such rich data, and most ML need data to build models.
• 21:30 Imagine where you have enough data coming out of these systems that we can have more intelligent responses around scheduling.
• 21:35 Predictive autoscaling based on the data that's being generated from Kubernetes.
• 21:40 I expect in the future we will see the ML models being roped into the scheduler, some of the capacity planning.

What are your personal goals for 2020? -

• 22:00 I want to hammer out "configuration as data".
• 22:05 Coming from the community of configuration as code, with if branches and put that on the front end to describe our infrastructure.
• 22:15 I think there's a powerful notion to be able to write code to make things happen, but the tools aren't interchangeable in that world.
• 22:20 If you pick a particular language to do this in, you're locked into that ecosystem and it's hard to interchange the tools.
• 22:30 When you force yourself only to have a data model - YAML files - and you force all the logic into the control loops.
• 22:40 You end up with a set of disciplines which says the config is always data.
• 22:45 The layer above it all should generate the data, and we can have data pipelines.
• 22:55 You can now have static analysis, without parsing a whole language.
• 23:00 You can apply policies to this data.
• 23:05 This means you have have interchangeable tools that we can add at any point in the process without affecting our existing investment.
• 23:10 That's going to be a very powerful concept that I hope to further in 2020.

Will that be open-sourced? -

• 23:20 I think a lot of things in my life, it makes its way into open source.
• 23:25 At Google we encourage people in that direction - you'll see Google Cloud already doing this.
• 23:35 We have the Kubernetes config connector, and the idea is that you can give us a Kubernetes object (a custom resource definition) to create a CloudSQL database, a load balancer etc.
• 23:40 Even things that are not traditionally seen as a Kubernetes resource; we're starting to take the existing Google Cloud resources and giving them their own Kubernetes object equivalent.
• 23:55 Where people were using Terraform before - I still think that's a valid path - but we want to make it easy for the tools to create the right data model - then we can manage those objects in the back-end.
• 24:05 That's happening at Google.

Have you bumped into CrossPlane? -

• 24:15 I knew when they were getting off the ground - I met the founder last year.
• 24:20 They understand this as well - once you start to have a data model across all resource types, you can think about the world differently.
• 24:30 If we assume that we can describe the infrastructure using data, we can start using tools like CrossPlane which think about a federated set of Kubernetes edges, and the databases and CDNs, using the same configuration language.

If you could travel back in time, what would be the most important advice you could give a 20 year old Kelsey? -

• 25:00 I would tell him this whole idea of being debt-free and financial independence is the right one.
• 25:05 It's going to let you focus on the things you want to rather than a specific dollar amount.
• 25:10 That is going to be the key to your success.
• 25:15 Public speaking is going to be more powerful than you know.
• 25:20 The ability to do public speaking works both externally and internally.
• 25:25 I have been an engineering management, a team lead, an individual contributor - and the most valuable is being able to inspire people.
• 25:35 When people feel motivated and inspired, they tend to do their best work, even more than you could ask or demand of them.

What's the best way to follow your work online? -

• 25:45 Twitter is where I spend a lot of the time in terms of sharing the work that I'm doing.
• 25:50 On GitHub, I try to produce all of the tutorials and libaries that I produce for my work.