Andrew Clay Shafer on Three Economies, the Wall of Confusion, and the Origin of DevOps

00:04 Introduction

00:04 Wesley Reisz: Hello, and welcome to the InfoQ podcast. I'm Wes Reisz, one of the co-hosts of the show and chair for QCon Plus, which is the online version of the popular QCon Software Conference that starts in just a few days, November 4th. Featuring 18 tracks, 54 speakers and 11 workshops, QCon Plus's agenda focuses on topics that matter right now in software development and features thoughtfully designed, highly interactive technical sessions with peer sharing built in. The conferences spaced out over three weeks. I said that right? It's over three weeks where you can attend just a couple of hours a day. So it's lunchtime in the Bay area or evening hours in the Central European Time Zone, just two or three days a week. So check us out at plus.qconferences.com. Today on the podcast, we're talking to one of our speakers for QCon Plus, and the show's all about DevOps. The person we're talking to knows a thing about it. He's the person most familiar with having stood up and described what it first means.

01:00 Wesley Reisz: We're talking to Andrew Clay Shafer. Shafer is the VP of Transformation at Red Hat, where he focuses on people, process, and technology. We'll talk later about that a bit. And as I mentioned before, he's perhaps most well-known as the person attributed to the creation of the term DevOps. Today on the show, we'll talk about organizational transformation. We'll talk about DevOps and technology. We'll hear directly from Andrew about things like Three Economies, the Wall of Confusion, and even the origin of the term DevOps. As always, thank you for joining us on your jogs, walks, and commutes. On with the show. Andrew, thanks for joining us.

01:34 Andrew Clay Shafer: Thanks for having me. I don't think you can say the word DevOps without mentioning at least Patrick Debois though, as well. A bit of a story.

01:41 What is the origin of the term DevOps and your involvement?

01:41 Wesley Reisz: Let's start right there. Tell me the story.

01:43 Andrew Clay Shafer: In some ways, and this is something I've kind of put in my talks as well. I don't think DevOps ever really started as a date. And is common in certain circles, the inner circles, there's little sub genres of DevOps now. And some of the incident management people are fond of saying, "There's no root cause, there's all these contributing factors or what have you." So if you look at what's going on at the time, it was kind of what I'll say, a movement in need of a flag or in need of a name. And lots of stuff that had happened in the years leading up to 2009, which is when the first kind of DevOps conference starts, there's a few things. But one is, I met Patrick Debois at an Agile Conference.

02:24 Andrew Clay Shafer: And at that Agile Conference, I was interested in talking about Puppet and talking about bringing Agile practices and software practices to system administration. And he'd already written a bit of a paper and you can find some of this stuff that I did or that he did back in that timeline. And then we were also conspiring to have a conference. So, I'd been giving talks about Agile infrastructure. There's two versions of it. So there's the infrastructure for Agile people, where people didn't really understand anything about infrastructure. And then there's the Agile for the infrastructure people. And I was kind of giving both those talks around that time. And then Patrick's like, "Let's have a conference. I want to do a conference on this topic." And we were kind of leading up to what the logistics would be and planning the speakers and some of the stuff that you do to throw an event, which I'm sure you're familiar with.

03:13 Andrew Clay Shafer: And there's a conference called Velocity, which I would consider, in some ways, Velocity is definitely seminal or foundational to the movement, right? The bringing together of kind of Devin Hobbs and the topics that were discussed at the first Velocity. So the first one's in 2008, are really kind of resonating and you could feel this sort of tribe swelling, right? So a bunch of these things are going on. 2009 Velocity conference, I'm sitting in the audience and the talk is Paul Hammond and John Allspaw talking about Dev and Ops Cooperation at Flickr. And so the Dev and Ops Cooperation at Flickr, which is recorded, you can go watch this, you can go to Velocity talk. And they're kind of going back and forth and bantering and having a good time showing slides and it's like this pretty funny slide with the Star Trek theme and comparing Dev and Ops like what's hard for both sides and it's very fun.

04:05 Andrew Clay Shafer: And I was tweeting out, basically just like quote tweeting, which I used to live tweet a lot more. Now a lot of other people do that more than me, but they were kind of seeing all these things. And I was talking to them Devops, Devops, Devops, because that was the pilot, right? So it's like DevOps, Devops, Devops, a bunch of those threads. It's like, I think June 23rd, 2009, something like that. And then Patrick's reading that tweet stream and then he's like, "We're going to call it Devopsdays." That's sort of the beginning of the use of the term in that context.

04:32 Wesley Reisz: I've actually never heard the origin DevOps story. I'm surprised. I know you've talked about it before, but I've never heard it. It's awesome.

04:39 Andrew Clay Shafer: I make this funny joke in some of my talks or it's like, I walk through a bunch of stuff that was happening from 2006 to 2009, right? So 2006, you have EC2 is born and some of the AWS stuff. The first one is the queuing service. And then you have these interviews with Werner Vogels right? So there's this famous interview from 2006 with Werner Vogels, where he says that "If you build it, you run it." That's like a pretty famous quote that's sort of thrown around in these circles. And then there's the Velocity Conference. There's these blog posts from Jesse Robbins, who was the founder of Velocity Conference, went on to become Chef and like some other startups. And he was running infrastructure at Amazon. And so it's like, some of this is just like bleeding out of all say, Amazon, Google, whatever you want to call now would kind of call them the cloud natives, but it was sort of bleeding out into the water. And then 2009 you get DevOps. So I call everything from before 2006 is BC before cloud, and then everything 2009 is after DevOps A.D.

05:34 What is the focus of your work at Red Hat?

05:34 Wesley Reisz: Yeah, I like it B.C. And A.D. So let's talk about your title at Red Hat is VP Transformation. What type of things are you focused on solving?

05:44 Andrew Clay Shafer: So that's a sufficiently vague title, but the core premise is when you look at what is happening with enterprises who want to be able to do technology projects better, that the rate limiting stuff in their progress and success is really them changing their own behaviors. And it's particularly pointed when you get to these kinds of platforms and things. When you're talking about early proto DevOps type stuff, and it's like, "Oh, here's a better way to do system administration, or here's a better way to do deployment." That kind of stuff. It was maybe more slightly aligned with roles, but the best projects always came when you could bring both sides of the kind of like social dynamic together and sort of work through and negotiate with those interfaces and handoffs and what I call borrowing from kind of sociology. Those are like boundary objects, right? So it's like Puppet became a boundary object in an organization between Dev and Ops in the best cases.

06:40 Andrew Clay Shafer: And then in the cloud platform world, everyone's all hot and bothered about Kubernetes and some of this stuff where it's like, literally you can't unilaterally get the most out of this platform. You kind of need to simultaneously change the behaviors of both operator roles and developer roles, even that's an oversimplification, right? Because we know developers and operators are not a monolith either, but just for the sake of kind of like the dev ops model is oversimplification. You have to bring those people together and be thoughtful about this social-technical system that you're trying to build to deliver these outcomes.

07:15 Wesley Reisz: So are you focused on delivering cloud native outcomes like changing the behavior of teams or getting them to agree on a common behavior?

07:25 Andrew Clay Shafer: I mean, I feel like those are both related and I'll say kind of yes to both in a sense. I think there are certain things that are not necessarily proven, but kind of established, right? So you have the state of DevOps report and some of the stuff that's out there provide some evidence for the efficacy or legitimacy of some of these practices. And for the most part, the data seems to suggest that you're better off if you can, do deployments faster. Not forcing yourself to do the deployments faster, but you actually ended up in a safer, more secure place if you can put your systems into a known state, a known configuration at any point. To me the core concept back when I used to call this Agile infrastructure, what I was really interested in is extending the idea of Agile engineering practices into infrastructure, right?

08:16 Andrew Clay Shafer: So if you think about Agile and the buildup and I was kind of baptized in the waters of Agile and Alister Coburn and XP and all this stuff. To me, basically, can't talk about any of the Agile engineering practices until you can reliably build from source, right? So if you can't reliably build from source, you can't start talking about TDD. You can't start talking about all these other things. So in a real kind of engineering sense, what I was interested in doing, plus I'd run into this problem over and over in my own personal kind of journey through startups and doing technology stuff as a practitioner that the deltas or small differences between environments were causing all sorts of problems and incidents, right?

08:54 Andrew Clay Shafer: So it's like you assume a certain configuration, but then there's like something slightly different in the configuration for the test environment, from the production environment. You put this stuff live and then all of a sudden it's a fire drill. So by taking and extending this notion of building from source down to the infrastructure, I could have higher confidence in that cycle. I changed the timescale a little bit because you can't do the same kind of sub-second test suite stuff. But being able to think that I have very high confidence that this infrastructure is in the proper configuration, as it goes through the life cycle is sort of the main motive for a bunch of this practice to kind of come out.

09:31 Wesley Reisz: Yeah, totally makes sense. So today it may be a little bit easier, but particularly back few years ago, when you first started talking about these ideas and these concepts. There's something that you've said before, where a lot of people want change, but not a lot of people are willing to change. It's always someone else that needs to do the change. So when you first started talking about DevOps and started talking about getting this change to happen, how did you get people on board? How did you get people to understand that we have to change?

10:01 Andrew Clay Shafer: I mean, in some ways I think that's still ongoing, right? In a lot of organizations there's still a lot of work to do, a lot of changes to be made. Actually kind of rewind a little bit, because this is related to the question you asked before about getting people to agree and work together and these new workflows. To me, I feel like I was very lucky, right? I was very fortunate, I have some skills and some insights, but at the same time, I was given a front row seat to watch a bunch of these projects and see a bunch of these themes and patterns. And there's a consequence of that from whatever my abilities are to articulate these ideas. In some ways I became the town crier and the one who was sort of projecting what I was seeing more than I, "Oh, I feel like I came up with all these ideas."

10:43 Andrew Clay Shafer: So it's like connecting ideas and seeing these patterns and then seeing them. But another thing, I always try to make clear is I feel like my superpower is recognizing other people have superpowers. There's always someone I know that no one even knows exists. So it's like, I got this prominent position in this narrative around DevOps. But every single one of these skills, there's someone somewhere that no one's heard of that I know is better than me at those things. They're doing that work in a way where it's like, I'm projecting their insight and their ability, their skills or whatever into the conversation. But when I have a chance to pick their brain, I go right to that. Right?

11:24 How does the Three Economies from Jabe Bloom apply to DevOps?

11:24 Wesley Reisz: Right. Totally. I hear you. I hear you. Okay. So there was a talk that you did that as I was prepping for this, it was called the continuous DevOps dilemmas for the digital gov DevOps meetup. And in that talk you talked about this idea of Three Economies, some work that Jabe Bloom has done. And those Three Economies kind of relate to this mindset of ops and mindset of development. Can you talk about, I guess the economies around the two and then where this third economy comes in?

11:53 Andrew Clay Shafer: So I've kind of retconned all of DevOps back through this lens that Jabe gave me. So Jabe's on my team, I brought them into Red Hat. He's working through a PhD is basically going to have to just write the thing and give the dissertation signed. And his PhD work is specifically focused on researching how organizations and cultures change their behavior to accommodate new technology. Right? So the Three Economies starts with two economies and it's this core conflict between innovation and efficiency. And it's not even necessarily about technology. It's literally about R&D throughout time and space that if you are focused on innovating, that requires some kind of experimentation and like this algae bloom of kind of information, to explore a space, to get to something new and interesting. And then efficiency is really trying to reduce variation, right? So when you come to the Dev and Ops conversation, you basically have one group who is incentivized to create novelty in some real tense instability.

12:55 Andrew Clay Shafer: And then on the other side of that equation, you have a group that is incentivized to create stability and reduce variation, right? So that kind of puts you at odds with each other, from the get-go because of the way you're incentivized. And in some cases, I feel like the way certain personalities are wired. And so that's like Dev and Ops conflict. One side is trying to make things new and unstable, the other side's trying to make things stable and reduce the variation. And that plays out in pretty much organizations of all shapes and sizes. And the more confusion conversation can kind of be retcon through this lens. And you basically have one group on both sides of that wall. So then this third economy, I don't always love every language choice that Jabe makes but he calls it the scope economy. So he calls one side the differentiation economy and then the other side’s the scale economy, and then the middle he calls the scope of economy. And it's economy in this kind of game theory sense, right?

13:48 Andrew Clay Shafer: It's like each one sort of playing a game where there's pay offs in the game. And so what he calls the scope economy is about bringing things into this middle round platform that can keep promises to both sides, right? So the scope economy in the construct that we could try to look for examples from other things, and in some ways like analyzing the actual container infrastructure, or like some of these other things from other industries is interesting in its own, right to talk through this as well. But if you think about things like Kubernetes or these platforms, they basically form this boundary object where it's allowing, and I also argue in some other talks, and I think the one that you looked through that this scope economy emerged in all these cloud natives, right? So if you look at what happened at Netflix, you basically get this thing where you can do self-service deployment as a developer, but it's also able to keep promises about the architecture and the operations.

14:45 Andrew Clay Shafer: And they build fairly reliable services based on how much I can watch Netflix. Right? And then Google is the same sort of thing. And I usually quote the SRE book because I feel like there's some pithy statements. And one of them, I usually quote, is about the SREs building these services and these libraries that are guaranteed to take and they make proper use of the infrastructure, right? So that central platform is enabling the developer self-service access, but also guaranteed to keep promises to the infrastructure. And then you could extend that same idea to some of these other qualities, right? So reliability, build the platform so that the architecture can keep promises to both sides. Observability, build these platform services so that you're already going to have instrumented code. Security, whatever, we could go through a bunch of ilities, but bringing those things into this middle place.

15:35 Andrew Clay Shafer: And then there's another language that I started adopting from another Carnegie Mellon PhD that's friends with Jabe that I got into his research and that's Dimeji, I might butcher his last name Onafuwa. And he talks about what he calls re commenting and commenting as this ongoing negotiation between two sides to manage a shared resource by recognizing they all have selfish interests, but that they're willing to negotiate in favor of the collective. And to me these platforms that emerge in all these places are manifestation of this scope economy through this lens of recommitting. And I would say, this is true, whether you're going back through the Puppet days till now. The more you can bring those two sides together, developers and operations and get them to negotiate their selfish interests in favor of the collective interests, then the better off you're going to be.

16:27 Wesley Reisz: Yeah. That's the thing I want to kind of get to is how do you bring people together to negotiate the collective interest and kind of give up some of the selfless interests? How do you get people to do that? How do you get to the meat of that problem?

16:39 Andrew Clay Shafer: This is the dilemma, right?

16:40 Wesley Reisz: That's why you're Vice-President of Transformation.

16:43 Andrew Clay Shafer: All of these hard problems are the social engineering, right? I would say that the technology pieces are relatively straightforward compared to the politics and the feelings and the ego that's attached to the artwork and the rest of the stuff you kind of have to sort through. But at the same time, I think there's some advice, right? So in a lot of organizations when you got the kind of more traditional wall of confusion style, and these two sides don't even see each other as human beings.

17:07 What is the Wall of Confusion?

17:07 Wesley Reisz: So let's stop for one second. Before we go there, you've talked about Wall of Confusion a couple of times. That's something that you've talked about quite a bit. Talk about that, then let's come back to this question.

17:16 Andrew Clay Shafer: Well, it's a metaphor and there's some threads on Twitter, even in the last week about kind of like the original of a bunch of these ideas. And so who even knows who used this first, but we'll say Lee Thompson or something, or whatever names lost history. But it's essentially this idea that both sides don't understand each other. So I have slides from 2008 or earlier, but definitely 2008 that have this wall of confusion metaphor with Dev and Ops on both sides. And so if you use a little build up is you have Dev and Ops and they're happy. And then you put a wall if confusion between them. And then I usually use this metaphor that they only through ticket systems. And so they don't even really see each other as human and then that makes people unhappy, and then the last building is they get devil heads, right?

17:58 Andrew Clay Shafer: So it was like in a lot of organizations because of the forces that are already intrinsically against each other. So one's trying to stabilize, one's trying to destabilize. And then because they don't see each other as the same or as humans or whatever, then you kind of get this thing where it's like, they don't understand each other's game. And this goes back to the Three Economies, right? So to win the differentiation economy, you have to create innovation. You have to create novelty. To win the scale economy, you have to create efficiency, you have to drive down costs. Right? And so those things, if there's not some kind of clutch in the middle of the scope economy to connect them together, that's really, really hard. But I would say going back to the other thread that the first step is if I can get them to see each other as human, that's huge, right? Let's get each other in the same room.

18:41 Wesley Reisz: Identify some people.

18:42 Andrew Clay Shafer: Yes, share some food because the other thing is a lot of times these people they're in the same company, but they're not in the same physical space.

18:48 Wesley Reisz: Not just physical space, the whole mindset.

18:51 Andrew Clay Shafer: Yeah. They're playing different games. That's the whole point of the Three Economies discussion is one is trying to play one game. One's trying to play another game. And those games are kind of odd each other. So if they can't see themselves as part of this larger mission that connects to the game that their whole org is trying to win, then it's really, really difficult. So I'd say getting them to humanize each other is a great first step. And then getting them to recognize they have collective interests. You can't resolve selfish interests in favor of collective interest until you get people to realize that there is a collective interest. And there's a bunch of kind of things to try to help people see that through visualizing the value stream or through some other discussion depends on the mission that particular organization has. But definitely getting them to realize there is a collective interest is the first hope of being able to negotiate in favor of their collective interest.

19:37 Wesley Reisz: There was something you said in that talk, and then you kind of referred to it just a little bit while ago when you were talking about the scalability, reliability, observability, security. All these different capabilities, but what you said in that talk, which I thought was really cool. I think I literally said out loud, absolutely or something like that. But it was that these are capabilities, not roles, but we've developed them in our IT organizations as roles. And then we've set boundaries around them like "I'm security. That's not what you do. That's my job."

20:06 What are you thoughts on DevSecOps?

20:06 Andrew Clay Shafer: I just gave a talk at OWASP actually this week about kind of DevSecOps or the evolution of security. And security is an interesting one to pick on. And then we can kind of talk about some of the other ilities as well. But when you look at what's happening, people say software is eating the world, but software is also eating software, right? To some degree what DevOps represents, I'd put this as kind of a funny tongue-in-cheek definition is to me software eating world is that everything that can be optimized by software will be. Everything about human performance and experience that can be optimized will be. And so to some degree, what we're talking about with DevOps is we're optimizing the human performance and experience operating software with software and with humans, right? So it's like this sort of circular meta circular definition.

20:49 Andrew Clay Shafer: And then you could extend that to all these other "-ilities" as well. So the optimization of security on both the blue team, red team, whatever, how you want to think about it, is going to be optimized by software. It's an arms race on both sides right now that this is just care something or to own something. Right? And so the more that security gets into the software life cycle, both in terms of getting upstream, building these things into the platform from first principles, then the better chance you're going to have. Also, and this is similar to DevOps in this kind of sysadmin evolution is that when we first come into this conversation and Puppet's there, there's like a whole bunch of sys admins that are like, "Oh, that's code. We don't code. That's for developers. We don't code. We'll write Bash all day long, but we don't code."

21:37 Andrew Clay Shafer: And I think there's some of that in the mindset of the security professionals right now that are like, "Oh, well, we're security and security doesn't code." But coding is like eating everything. Right? So before I came into startups and working in developer or whatever, I kind of aspire to be a scientist. And so it's like, you can't be a serious scientist right now. I mean, there's probably a few exceptions with wet labs, but even more and more of this is driven by computation and simulation and being able to code. At a bare minimum, like some kind of mathematical models for what you're trying to do is a huge advantage in every field right now. And so that just continuing to change all of every dynamic. So security, observability. So it's like, what you're trying to do is bring that re comment platform, that scope of economy platform so that you can take advantage of everyone's expertise.

22:29 Andrew Clay Shafer: So like the developer that's focused on the business logic. There's no possible way that person can be an expert in distributed systems and telemetry and security and every other ility, usability, whatever. They can't be. There's no way. Right? So you need to be able to kind of bridge all these things into these platform services that can keep as many promises as possible, making the right thing, the easy thing for that developer to focus on their business domain. And that to me is what unlocks the magic. And I think you see this to various degrees inside of these "cloud native" innovation engines.

23:10 Can you preview your QConPlus talk?

23:10 Wesley Reisz: So in just a few days you're going to be giving a talk foreign language QCon Plus that I brought up at the very beginning. It's the history of languages of infra. What's this talk all about?

23:19 Andrew Clay Shafer: I've been thinking about this a lot. And I made this comment earlier about the people that I know that I think are better at almost everything than me. Right? They're specialized in something and they spent more thought and kind of effort being great at it. And when you think about cloud kind of infrastructure automation, it's interesting because in some ways, what we see in the open is only the top of the iceberg, it's above water. And inside of Amazon and Google and Netflix or whatever, there's people that did a bunch of things, that solved a bunch of problems at scales that most of us don't really even comprehend that are experts in this thing that we haven't even got the chance to try to solve, or to try to even have the problems.

24:04 Andrew Clay Shafer: So I think that's an interesting thing to talk about is like, there's this unsung unspoken, there's some people that we kind of know that do get stage time or whatever it re invent or next day, you kind of know. But underneath those orgs, underneath the covers, there's all sorts of kind of automation problems and stability, reliability, architectural problems that have been solved in this unbelievable way that we don't necessarily have access to. So, that's kind of interesting thought. And then you look to the landscape of what's going on, but it's kind of goes back to the software's eating the world. And I'm not positive if you're familiar with Wardley Mapping, but for the audience, the idea here is that you basically have this value stream, which is somewhat familiar to people who do kind of lean and that kind of stuff.

24:48 Andrew Clay Shafer: And then those value stream or value chains depending on which lists you talk to or talk through? There's this notion in Wardley Mapping that each stage is sort of somewhere on this S-curve of commodification, right? These that begun as an innovation, there's like this Genesis and the innovation, those innovations then become products. Those products then become more standardized. They become commodities and eventually utilities. And so the whole argument or Wardley mapping, or the way you apply it strategically is looking at your value chain and helping yourself understand where you can create differentiated value versus where you should consume what's already commoditized. And then it gets more advanced about if this evolves this way, was it the things above and below it and the rest of it? So when you look at what cloud represents in a real sense, and also kind of going back to 2006 till now, you see this incredible commodification of these abstractions that are getting ever more powerful.

25:45 Andrew Clay Shafer: And the cloud computing narrative is really the kind of commodification of these innovations that we had 40 years ago, 30 years ago. Right? So now we take advantage of it as a service and then even if you look at cloud in the model from 2006, we'll call that BC. So now we're in the AD range, and then you have all of these higher level of abstractions, right? So that's one of the fundamental concepts of Wardley mappings is as things commoditize, it enables new innovations on top of them.

26:18 Andrew Clay Shafer: And just think about the proliferation of these different services and the abstractions and the power that you're getting, where in day one, EC2, you can bring up, I think there's only three different types of machines, something like that. It's very constrained a set of configurations, you don't have all the fancy, high memory, high compute or whatever sort of things to choose from. You can choose anything you want, as long as it's black. And really there's nothing else that you put compute, networking, and storage. You have S3, you have SQS, that's it. Now you go look, I don't think anyone actually could list every single AWS Surface.

26:51 Wesley Reisz: 180 of them or something?

26:52 Andrew Clay Shafer: No. Way more. There's like thousands, something.

26:55 Wesley Reisz: This is languages of infrastructure. So how has our languages that we use to deploy? You talked about Puppet and Chef earlier, how has our languages evolved from kind of the BC/world AD world to where we are today, and where do we still need to go?

27:11 Andrew Clay Shafer: We're getting more and more abstractions and more and more kind of power as a consequence of that. You could break down sort of the evolution of these generations where,, you start even before Puppet. And the things that people did to automate with scripts, and then you have the CF engine, and there's probably a bunch of tools that I don't even know about. Then you move into the kind of Puppet, Chef, SaltStack, Ansible paradigm. And there's a few other things there that are probably worth mentioning. And then in the same time you start getting things like Vagrant and Packer. And some of the stuff HashiCorp out obviously has to be kind of a central part of this narrative.

27:49 Andrew Clay Shafer: And then now you get to this proliferation of all that is kind of adding up to where any one person with essentially an internet connection and a credit card can manifest whatever infrastructure they imagine. So to me that's the power of what these abstractions allow you to do is that you don't have to kind of reinvent everything from first principles every time. You can take these patterns that have already kind of proven themselves, harden themselves and start to reuse them in more composable and more abstract ways. And then that enables the next layer and the next layer, the next layer.

28:26 Wesley Reisz: One of the talks that we're doing during QCon Plus is from Tracy Holmes of HashiCorp. She's doing a talk about Terraform and security? And I think it kind of gets to what you're talking about, that scope economy, that middle ground. Because what she's talking about is kind of encapsulating the language and then giving security as policy, the ability to kind of get into the Terraform scripts and be able to represent in that infrastructure that's being built their concerns, to make sure it's addressing the things that they need in their world.

28:55 Andrew Clay Shafer: This is a very interesting conversation for me personally, and also other people on my team, John Willis. He's the one that got me super excited about automated governance. But if you look at what is slowing down progress in a lot of enterprises, it's actually not that they don't recognize their shared interests in some cases, it's literally they have these governance risk and compliance rituals. That for lack of better reason, it's really frustrating for me. And I'm in this interesting place where you get in conversation with people and they're like, "That's what Sarbanes-Oxley says. I'm like, "That is not what Sarbanes-Oxley says. If you want to go read it with me, we can go through it line by line and it does not say that." It's like the low is not the law, and this is the thing slowing down the progress for so many organizations.

29:44 Andrew Clay Shafer: At the same time there's some exciting things that have been happening. And I think that there's ways to get out of this where the instrumentation for doing the audits, for doing the kind of like governance and the controls can be built in from first principles throughout the full software development life cycle with everything from the code review to whatever action got triggered with the level of Ansible, to something that happens in the Kubernetes platform. End-to-end-to-end, everything that happens in an organization's software, infrastructure, whatever has this policy applied to it. It's continuously compliant because it's audited and enforced. And then every bit of those actions is written to this immutable ledger. And there's some interesting work that's already been done where it's both accelerated the developer's productivity. I'm going to wager probably improve their happiness. And then also on the backside of it, huge improvements in the time spent doing audits.

30:46 Wesley Reisz: And it brings everybody together in that common area, that third economy that you were talking about. That space where everyone's kind of working towards the same goal.

30:54 Andrew Clay Shafer: Re-commenting. Let's negotiate our selfish interest in favor of the collective.

30:58 Wesley Reisz: Re-commenting, there you go.

31:00 Andrew Clay Shafer: And the best case is we all get what we want. So another shout out all back to some of my talks is I used to talk about Pareto-inefficient, Nash Equilibrium all the time. And it's like this kind of silly game theory like way economics theory, way to say that. But Pareto-inefficient system is one where it's possible to redistribute resources in such a way that no one gets less, right? So if it's Pareto-efficient, you can't redistribute it without someone getting less. But if there's Pareto-inefficiency, it means you could make it better for at least one party and no one gets worse. So we're looking for these Pareto-inefficiency so we can make it. So everyone's at least as good as they are now and hopefully someone gets better, but then the Nash equilibrium means no one will change. So from game theory, when a game is Nash equilibrium, it means no player will change their strategy. So if you have a Pareto-inefficient Nash Equilibrium, it means it's possible to make it better for someone.

32:01 Wesley Reisz: But no one changing.

32:02 Andrew Clay Shafer: Yeah. You have to change. That's the point. You have to change.

32:05 Wesley Reisz: You have to change. Okay. So if you want to hear more about it, come join us at QCon Plus here in a few days. Shafer, thank you so much for joining us. It was fun to finally get together and do this.

32:15 Andrew Clay Shafer: Absolutely. Looking forward to seeing you again in a few days.

32:18 Wesley Reisz: All right. Thank you, sir.

32:20 Andrew Clay Shafer: Cheers.