Tracy Miranda on the Continuous Delivery Foundation, Interoperability, and Open Standards

Introduction

Bryant: Hello. Welcome to "The InfoQ Podcast." I'm Daniel Bryant, the News Manager at InfoQ and Product Architect at Datawire. I recently had the pleasure of sitting down with Tracy Miranda, Director of Open Source Community at CloudBees, and Board Chair of the Continuous Delivery Foundation. I've been lucky enough to have known Tracy for many years now. Last year she ran a superb track at QCon London that focused on innovations within the continuous delivery space. Since then, she has gone on to become an influential member in the relatively new Continuous Delivery Foundation, or CDF.

Goals

One of my goals in this podcast was to learn more about this foundation. I was also keen to explore the list of currently hosted projects within the CDF, and get Tracy's thoughts around the need for open standards within the continuous delivery space. As we've heard in other recent InfoQ podcasts from Bryan Liles and Gareth Rushgrove, there is definitely a potential need for better abstractions, interoperability, and componentization within continuous integration and continuous delivery. I'm also becoming increasingly interested in topics such as progressive delivery, championed by James Governor and the RedMonk team. I'm also curious around things like software supply chain management. These are not necessarily new topics, but they are getting more interest now as many of us are thinking about security, reliability, and availability within all aspects of our software systems. I was definitely keen to pick Tracy's brains around this.

Hello, Tracy. Welcome to "The InfoQ Podcast."

Miranda: Hi, Daniel. Thanks so much for having me.

Bryant: Could you briefly introduce yourself, please, and share a bit about your background?

Background

Miranda: I'm the Director of Open Source at CloudBees and also on the governing board for the Continuous Delivery Foundation. I've come into the CI/CD space recently. My background is actually in electronics, developer tools, but the overarching theme has always been open source.

Bryant: Awesome. This is perfect. That's my next question. How important do you think open source is to the community at large?

Miranda: How important do I think open source is to the community at large? I think it's critical. I think it's great to see that these days that's generally acknowledged, everybody understands and gets open source in a way that maybe they didn't 10 or more years ago. It makes all the difference to innovation, building communities around technology, and pushing things forward.

Bryant: I wanted today to pick your brains around the Continuous Delivery Foundation. Actually, you and I were chatting at the Open Core Summit last year. I've definitely heard many good things from you in the past as well. You're on the governing board of the Continuous Delivery Foundation. If I asked you about getting an elevator pitch for this, what would you tell me?

Miranda: What would I tell you about the Continuous Delivery Foundation? The Continuous Delivery Foundation is an open source foundation that was set up to be a vendor neutral place to drive the future of continuous delivery. That's it in a nutshell. I'm more than happy to go into all the different things that that covers in reality.

Bryant: Yes. I'm definitely keen to cover some of those specific projects later. It'd be nice to get almost a vision statement thing. That's what I was looking for. A lot of folks see these foundations, and they're not sure, is it just tech, is it specs, is it implementations, is it TOC(Technical Oversight Committee)? There's a whole mix of things in there.

Miranda: Is there a whole mix of things inside these foundations? Yes. The way I see it, the main remit of the Continuous Delivery Foundation is to advance continuous delivery adoption. Where a foundation comes in, is to take care of the things that might be neglected if you just had individual products or individual open source communities, pursuing things from a narrow field of view. The Continuous Delivery Foundation is well set up to cover cross-cutting concerns, things like security across the industry, interoperability across tools, promoting standards, or de facto standards. Even things like diversity, making sure as an industry as a whole we tackle problems that would be difficult for any one smaller group to deal with.

Bryant: Interesting. I'm guessing it can be challenging, though, getting all the folks involved to agree sometimes.

Miranda: Is it challenging getting all the folks to agree? Yes. I think herding cats is often the phrase that people use with open source foundations, but part of the challenge is also the fun of it.

Bryant: I certainly hear you. I'm sure you get this question all the time, and I definitely see some interesting stuff on Twitter. Could you briefly outline the similarities and differences between the CDF and the CNCF, the Cloud Native Computing Foundation?

Miranda: Could I outline the similarities and differences between CDF and CNCF? Yes. I think you're putting it kindly there. This is something that comes up for us all the time. We like to think of the CNCF as effectively a sibling foundation. The reality is, when we were looking to set up CDF, we looked at the CNCF and that whole model of how it's set up. The success it's had in evangelizing technology and changing the industry is what we're looking at.

While CNCF is focused on cloud native technologies, CDF has a different scope. We are focused on delivering software, all the processes that go up into that. Also, it's not just cloud native. We talk about continuous delivery, it could be for embedded platforms, the whole IoT space. It could be mobile platforms. The scope of continuous delivery and software delivery in general is just overarching, and that way that's another difference we see between CDF and CNCF.

Bryant: Yes. That's definitely very helpful. I wanted to dive into some of the projects now a little bit if that's ok? Assuming some listeners and myself included, probably, before I started looking into it may not know what projects are included in the Continuous Delivery Foundation. The first thing that jumped out to me as a longtime Java developer was Jenkins and Jenkins X. I have used Jenkins for years. I've been looking at Jenkins X quite a bit. Actually, one of your colleagues, Oscar Medina, got to give a shout out to Oscar, mentioned to me recently that Jenkins X is very different to the original Jenkins. Could you give a bit of a 101 to folks on the differences between Jenkins and Jenkins X, and where they're positioned?

Miranda: Could I give a 101 on the differences between Jenkins and Jenkins X and where they're positioned? Yes, absolutely. Again, this is something we get asked all the time, because as they say, software naming is hard. This is, I think, the prime example of it.

Jenkins and Jenkins X

Jenkins, as everybody knows, is the original platform for automation and CI and to some extent CD. Jenkins X is CI/CD reimagined. We talk about how the Jenkins community could see there were some limitations with this platform, which was developed 15 years ago. With Jenkins X, it was a chance to take a fresh approach to that to say, "If we were looking to do the same thing, how would we start from scratch? How would we build on Kubernetes, which has become the base platform now?" Jenkins X completely reimagined CI/CD building on Kubernetes.

When it comes to the naming, I like to think it's almost like VS Code to Visual Studio. You have VS Code as the new, modern IDE, and Visual Studio is what people are used to and traditionally used, but actually, there's no similarity in codebase. I think that's one of the things people don't see that Jenkins X is completely written in Go for the large part. There is no Java in there. It orchestrates a whole bunch of tools underneath it, which could include Jenkins, Kubernetes, Tekton, all things are under there. You've got Helm, you've got Vault.

Bryant: I know I bumped into Jenkins X, I think, from James Strachan talking about it. He was talking a lot about the ideas in Nicole Forsgren's book "Accelerate" being captured. They believe there are four metrics that correlate with high performing organizations. Jenkins X is focused on enabling that change but from a technological point of view?

Miranda: Is Jenkins X focused on enabling change but from a technological point of view? Yes. No, absolutely. I think they have looked very specifically at, I think they call them capabilities in the "Accelerate" book. We acknowledge that not everything can be fixed by tools, but there are certain things that tools drive. Some of the capabilities include loosely coupled architectures, continuous integration, continuous delivery. There's a lot more we can do for automating these or connecting them up to Git in a very connected way, this whole concept of GitOps. That's all about true automation. That's what Jenkins X is really looking to do. That then ties in directly with those four metrics spelled out very well in the "Accelerate" book.

Bryant: I've used Spinnaker a little bit in a couple of projects, but quite a while ago. I remember when it, originally, I believe, came out of Netflix and Google. I think it's evolved quite a bit since I last used it. Why would folks use Spinnaker these days?

Miranda: Why would folks use Spinnaker these days? Spinnaker, we see a lot of folks really interested in it. Spinnaker comes very directly from the continuous delivery side of things. It was one of the pioneering platforms to really simplify some of those concepts, different types of deployments. We talk a lot about blue-green deployments today, or red-black in Spinnaker. It just made things very easy for people to do in that way.

Bryant: What's the pitch for Tekton? Because that's continuous delivery with Kubernetes, I'm guessing there's some overlap with Jenkins X there?

Miranda: Is there some overlap between Tekton and Jenkins X? Absolutely. Tekton, the folks that are looking at building on Kubernetes, but really establishing some of the basic building blocks you have behind CI/CD. This is a lot building around Kubernetes CRDs, but setting those up in a way that you can use them for CI/CD. One of the sayings that the Tekton team has is that Tekton is the plumbing and Jenkins X is the porcelain. Tekton is a colection. Which then, of course, prompts everybody to go, "What are we saying about delivery and toilet metaphors?" I think Tekton provides all these really standardized blocks, and it allows you to build platforms on top of it. You can have Tekton CI but Jenkins X uses Tekton as the pipeline engine underneath it as a pure cloud native. You can ramp it up. You can scale to whatever size you want. That's one of the ways you can do real, proper cloud native CI/CD.

Bryant: Super. One more technology I wanted to cover and then we'll pull out a little bit and look at that bigger picture. But I bumped into screwdriver.cd. I did not actually hear that I'll confess, until I went onto the website and then I looked around, I believe it's an incubating project or not a graduated project yet. Could you explain to the listeners what the value prop of screwdriver.cd is, please?

Miranda: What is the value prop of screwdriver.cd? This one is a newer project. I think it's a more end-to-end application. It came out with the folks at Yahoo and Verizon. One of the things in CDF is that we're looking to encourage folks who buy into the ethos of interoperability. I think that's one of the places where the Screwdriver team came in and said, "We've done these things, but we want to be part of the community. We want to leverage what other people are doing." Yes, we're interested to work together with them and have them be part of that.

Bryant: I'm guessing a big value of the CDF is that you can provide this platform to allow folks to interoperate not only on a technical level but also on a terminology level as well.

Miranda: Is the big value of the CDF providing a platform to allow interoperability at a technical and terminology level? Yes, exactly. For me, on a personal level, I came into CI/CD a couple of years ago. In fact, it was a bit of a nightmare. It doesn't even happen immediately, but after a while, you realize you've been talking at cross purposes about what a job is, or what a stage is. By virtue of having this community where we then put all these people developing different tools together, it was obvious that this is a problem. We need to have a reference way of talking about it or making it clear that we're using the same word, but it doesn't mean the same thing.

Interoperability

That's been really exciting because we have an interoperability special interest group, and they've come up with exactly what you described, the Rosetta Stone for terms for CI/CD. We started with the terms around a pipeline. We've already got almost a dozen terms, and there's this really interesting table where you've got, "This is what it is in GitHub Actions. This is what it is in Jenkins. This is what it is in Tekton." You only have to look at that table, and you can see, this is where we all agree as an industry. This is where a step, a job, and a task are all different. This is pretty exciting. I'm looking forward to how it evolves. First of all, let's agree there's a problem. Second, how do we solve that? That's going to be pretty interesting to follow.

Bryant: I wanted to pull out at a little bit higher level now. One thing that, I think, I learned from Martin Fowler and folks like that, was separating deployment and release. It took me a while to get there. Definitely my Java days, we used to just deploy the monolith and that was effectively the release at the same time. Now we use the bounded microservice-based systems or service-oriented systems, we can deploy and then use API gateways or service mesh to release separately. What's the CDF's or your thoughts on that within the CDF? Is CDF interested in continuous delivery, deployment, and release?

Miranda: Is CDF interested in continuous delivery, deployment, and release? Yes. I think, again, going back to the naming things, the term continuous delivery, I think has very specific connotations, particularly in certain circles. It's a very specific defined thing, which is as opposed to continuous deployment or release. I think I like to take a bigger picture view, probably more accurately, it should be the software delivery foundation. We talk about in the mission of CDF that we're focused on the software delivery lifecycle. Pretty much everything it takes to ship the software and get it to users in an effective and safe way.

Bryant: I think you and I are fans as well of James Governor's work of RedMonk. He talks about progressive delivery. He's got some fantastic tweets, fantastic blog posts. Is progressive delivery something that the CDF would be interested in too?

Miranda: Is progressive delivery something that the CDF would be interested in? Absolutely. I was privileged enough to head up a track at QCon in the UK last year, and I had James Governor come along. It was almost one of the first times in a big audience that he was talking about progressive delivery concepts. You could see it was instantly resonating with folks. They were almost jumping to, this totally sets out a concept that we've known, but we haven't known how to articulate. Then it tells us, we want this feedback and we want to have this concept of blast radius. I think that is definitely an area of interest. In the CDF's self-appointed role of helping people adopt continuous delivery, and making it easier for them to do, I think that will very much be in its future.

Bryant: One other thing I was curious to ask about is the notion of observability. Because when I've chatted with James, when I've been reading James's stuff, he's talking about continuous delivery but giving that good feedback. You need observability, both from an app level and a platform level to understand what your changes are doing. Then you can get the feedback and hopefully make it better. Is observability something that CDF would be interested in?

Miranda: Is observability something that CDF would be interested in? I would say, I think some of the conversations around observability are happening in other places at the moment, so it's not necessarily something we get to immediately, but maybe as we go down that path of progressive delivery. Probably something we're keeping tabs on. I think there are other places as well doing a lot of that.

Some of the areas we do want to cover tend to be less new, the areas that there's a lot of buzz around, some of the things like progressive delivery and observability. We are also keen to just cover the basics. I think testing is going to be an area I'd love us to focus on, which is pretty boring, and doesn't get the coverage as some of these newer terms. We need a new term for it. No, I didn't just say that. Yes, I think seeing what we can do in spaces like that is going to be super important.

Bryant: That's one of the things I learned. I did some stuff on the Java Community Process. A lot of the benefits of foundations and community processes is you're actually, like you're saying, you're standardizing the boring stuff. You're standardizing the super important, but we haven't all perhaps agreed on it or set it in stone.

Miranda: We haven't agreed on standardization or set it in stone. Yes. For me, if I had to imagine some magical things CDF could do, it would almost be standardizing some things around testing like, why don't we have standardized metadata for how we report testing around continuous delivery pipelines? Could we take JUnit, which is de facto standard, and build on that. Then say, "Everybody, let's all use the metadata in this way. Let's agree to do that, whether you're Jenkins X, or Spinnaker," or whoever. Then all of a sudden downstream consumption of the tests, all the things we could build on top of that, how we lay on observability, all becomes so much simpler and it becomes just more powerful.

Bryant: That message of interoperability you talked about several times already, Tracy, and off-mic we were talking about it. It's such a powerful thing, the interoperability. If you can get that abstraction and the interface correct, I think it unlocks a lot of other innovation.

Miranda: If you can get the abstraction and the interface of interoperability correct, does it unlock a lot of other innovation? Yes, for sure. I've seen it in other industries. My favorite story is NumPy, which before it came along, there were three or four different types of things all doing pretty much the same thing. Then as soon as, I think it was Travis Oliphant went and standardized everything on NumPy, look at where the world is now with all machine learning and data processing. Yes, for me, that's the ultimate direction we want to head.

Bryant: Other than the things you've talked about there. Are there any other projects coming down the pipeline with the CDF? Or if folks are listening and thinking about what projects would they be submitting, any guidance for us there?

Miranda: Are there any other projects coming down the pipeline with the CDF? We'd love to encourage folks to submit their projects. I think we've been doing a lot of bootstrapping, a lot of conversations in the technical oversight committee about different directions. Folks can propose projects to the talk. I think they've got a few coming down the pipeline, but I'm actually not seeing what's coming out lately. There are a couple of areas, if there were some particular ones, I'd love to see something on resiliency engineering, potentially, as well. Anything in delivering software, we'd be interested in.

Bryant: Before moving on to some of the platform and future stuff. If folks are looking to get involved with the CDF, what's the best way for individuals or organizations to contribute?

Miranda: If folks are looking to get involved with the CDF, what's the best way for individuals or organizations to contribute? I think starting off at the website, cd.foundation. That's got links to lots of the different initiatives we have going on. Then from there, you can find your way to the Slack channel, or just to a specific working group. We've got the interoperability one. We've got the machine learning one, so MLOps, or just some of the more general communities.

Bryant: What's your opinion on the state of developer tooling at the moment? Obviously, there's the cloud space, and there are many other spaces. I'm more probably interested in this respect from the cloud era. I know you've done a bunch of work in this space. Do we need to get better at DevX, UX, what are your thoughts there?

Miranda: What's my opinion on the state of developer tooling at the moment, with respect to the cloud? Not to be harsh, but I'm pretty glad I am not a new developer coming into the software industry at this point, because I have no idea how someone doing that is supposed to make heads or tails of what all the different choices are and how to get started. It is a bit of a fragmented nightmare. We need to get better as an industry at solving that.

Bryant: You think that's the best way of all of us trying to come together? Because I've seen the CNCF has put together their landscape, for example. At the start that was super helpful, but now it's got so big that even me trying to use the landscape, I'm like, "It's like an eye chart." Yes, I think they know that, of course. I'm not trying to be negative on them. It's genuinely hard, yes, to make it easy for developers?

Miranda: Is it genuinely hard to make it easy for developers? Absolutely. Software development has become so complex. It's tricky to try to say, what are the ways we could fix things? I'm personally a fan of more opinionated information, which to a certain extent gets me in trouble as well. Because I will say things like, why can't we have one right way of doing specific things just like Git is a given for version control. If you're coming in today, if you don't know that you have special requirements, or you just don't know, everybody will just say, "Start with Git. Don't even think about it, just start with Git. If that doesn't work for you, then you can switch to something else." We don't have that in most other areas, Kubernetes is consolidating things, so at least that's solving that.

A lot of other areas, there's so much choice and there's so much noise. Then sometimes there are people in the space who make it worse, trying to piggyback on terms or piggyback on platforms, so they add to the noise. That's where it gets really horrendous because people stop trusting. They just don't know how to pick things and who to trust.

Bryant: If we now step back a little bit, and you're now talking off-mic about the future of continuous delivery, the future of platforms. I'd love to get your thoughts on where this is going, perhaps from a high level a little then we can dive a bit deeper as we go along?

Miranda: My thoughts on the future of continuous delivery, and the future of platforms? I think at a high level, and again, this ties to what the Continuous Delivery Foundation can help drive, I think it's time for us to really standardize the pipeline layer of CI/CD pipeline. It's ready to be commoditized. If we can do that we can move on just to higher-level things. There are a couple of concepts coming out, if you start looking at, let's say you commoditize the pipeline and you start moving up the stack. Where does that lead to? We have these couple of terms, that certainly CloudBees and a few others are looking at, and it's around software delivery management. How do you see software as something like a factory that's producing software? How do you manage it?

Then the other half of that is software delivery automation. How do you automate it? How do you make all the blocks work together? How do you put machine learning around that so you don't necessarily need humans in the loop for some of it? The software delivery management and software delivery automation, I think is where we're headed as an industry such as that high-level people managing software like you would a factory, and all the efforts we've had around factories, and the processes, and Kanban, and all that kind of thing. We're now having that same set of events play out for software delivery.

Bryant: Very interesting. As you were discussing that, Tracy, I was thinking about something I heard many years ago, but it only frequently pops up now is the notion of a software supply chain. I'm guessing things like the CDF is super in the right place for security, and due diligence, and this stuff. I'm guessing that might be of interest in the future as well.

Miranda: Would the software supply chain be of interest in the future? Yes. Software supply chain, I think you're going to be hearing a lot more about that. There's going to be some pretty big efforts in the space, not just from CDF, but it's generally an area everybody wants to come together to solve and make better.

Bryant: If we're wrapping up now, Tracy, what would be your core message for folks looking to get into open source work and open source community?

Miranda: What would be my core message for folks looking to get into open source work and open source community? I think we've covered a lot of it, but in general, as I say, I'm a big open source advocate. The models around the foundation really work well to promote these. I'd encourage people to get involved, to come get stuck in with the CDF, because we talk about in open source, the communities, the capacity. We're really unlimited with the things we can do, as long as we come together and we're focused on this vision. It's pretty powerful. It feels good to be part of it.

I think we're at a time in the world where the world is changing so much. We're seeing that software delivery and automation is just going to be key to every industry, really. I think this is our one chance to get that right. Open source and open source communities are going to be the way that we do that. Yes, get involved, folks.

Bryant: Well said, Tracy. If folks want to follow you online, what's the best way?

Miranda: Twitter, always happy to connect with people there, so @tracymiranda on Twitter. Please feel free to reach out.

Bryant: Super. Thanks for your time today, Tracy.

Miranda: Perfect. Thanks so much for having me, Daniel.