Mario Platt on DevSecOps, Platforms, and Threat Modelling

Transcript

00:04 Introductions

00:04 Daniel Bryant: Hello, and welcome to the InfoQ Podcast. I'm Daniel Bryant, News Manager here at InfoQ and Director of Dev Rel at Ambassador Labs. I recently had the pleasure of sitting down with Mario Platt, VP Head of Information Security at Cloud Margin. I met Mario on a recent SnykCon security panel that I hosted on the topic of platform security and I thoroughly enjoyed our chat. It was instantly clear that Mario has a lot of experience in this space and has done a lot of thinking about the socio-technical aspects of security.

00:30 Daniel Bryant: In this podcast, I was keen to dive into topics such as the emergence of DevSecOps and explore how this differs from the classic DevOps approach. I was also keen to understand how developers and platform teams can work together to design and build secure systems and explore how threat modeling can be used in this context.

00:45 Daniel Bryant: Hello, Mario, and welcome to the InfoQ Podcast.

00:48 Mario Platt: Thank you very much for having me.

00:49 Daniel Bryant: Could you introduce yourself for the listeners, please?

00:51 Mario Platt: My name is Mario Platt. I got interested in security when I was only 13 back in 1998 with hacking, as you do. That's how you get teenagers to like security, right? I've been working in the field since 2002. I have a lot of background experience in infrastructure, managing the servers, Linux, et cetera. I have over 10 years of security engineering and operations. My first full-time security role was a penetration test, as well. For the past seven to eight years, we have been more focusing on government risk and compliance, but I wanted to make sure that I didn't lose my technical know-how, so I still do a lot of technical stuff.

01:29 Mario Platt: I like to think of myself and consider myself a full stack security person since I'm sometimes sitting down with the technical teams actually reviewing code and implementing controls inside the city pipelines. And in the same day, I'm talking with the board about our next three-year plan. And then lately, I've also created a new course, a DevSecOp for leaders, of course, through practical DevSecOps. And that is one of the things that are applying a lot of DevOps concepts to how we manage security and how to manage security within DevOps environments.

01:58 How does DevSecOps differ from DevOps?

01:58 Daniel Bryant: Oh, super. And that's a perfect lead, actually, into my next question, Mario. So you and I were actually on a panel that sneak con a while back. So you and I have talked about this before, but you mentioned there DevSecOps. Now, when I first heard this, I wasn't super convinced. Because I was like, "DevOps surely needs security." But I know you've got some really interesting thoughts about that. Could you expand on that for us, please?

02:17 Mario Platt: Generally, there are two schools of thought with regards to the word DevSecOps. There are some people that like it, some people that don't, and I think they're both right. So what I think is we need to contextualize the word to the environment that we're trying to influence, or to the environment that we are operating in. So for organizations that have a good quality assurance approach to the way they build their products and services, then the introduction of DevSecOps makes, in my opinion, less sense, because you've got an overall approach to how you manage quality for our two products, and then extending that to include how you approach security, easy and simple. So you can leverage all of the quality assurance related processes that you may have internally to do security. And the people that are against the word DevSecOps usually that's the angle that they come from. These should all be part of what we call product quality, and it should be integrated into the way we operate. And they're right.

03:13 Mario Platt: On the other hand, you also have organizations that are not so good from an overall quality assurance perspective. They have local coverage across their code. They don't have mature practices in how they identify buds and treat them in are always chasing features with a lot of tech that they're not managing. You're always going to have it, but they're not managing it proactively. For those types of organizations, particularly when you have security teams that are used to the old model of security, securing things on prem, then that comes with a problem that if you don't call it the name, you really can't do much about it. So DevSecOps, I think, is most useful to help old world security teams. So teams that are used to old ways of working, securing on prem with a waterfall type method, et cetera. In helping them make the transition into considering what a security team DevOps should look like.

04:06 Mario Platt: Because there's such a rich body of knowledge, using the word DevSecOps, it's a way to introduce the conversation and it's a way for people to be able to have a conversation about it. From that point of view, I think it makes it useful as a transition mechanism, knowing that at some point in your maturity curve, you may want to stop calling it something different because that then may affect agency and ownership from the front lines, our digital front lines, our developers. We want them to have agency over security. We want them to own it. At some point in your journey, calling it something different becomes detrimental to that sense of ownership. If they see it as an integral part of their role identity, and then they're more likely to take ownership, they have agency and all those good things.

04:49 What's your experience of good boundaries, integration points, or maybe even common areas of friction where Dev and Ops meet on the platform?04:49 Daniel Bryant: I'm really interested in building platforms. I know you are too. And for me, there's a lot of friction between Dev and Ops in the platform space. The Ops folks own the platform and developers want to use self-service mechanisms that get their code out in front of users. What's your experience of good boundaries, integration points, or maybe even common areas of friction where Dev and Ops meet on the platform?

05:12 Mario Platt: That comes back into, particularly from a security point of view. So I like to think of integrating security in DevOps from the point of view of the practices. So we're going to have some practices where we will be identifying the things that we need to do and we're going to have some practices which are supported by tooling in the sense of integrating security controls from a CI/CD perspective. So for instance, every time I make a deploy, we use an open policy agent to check the hardening of our NGINX config, for instance, that type of stuff.

05:44 Mario Platt: So I think that's where those boundaries or points of friction may happen, where we can leverage them to better serve the whole. So for instance, a platform team's main aim should be to reduce the cognitive load required from the teams that are developing the applications. So what I mean by that is the development of patterns that the teams can use. Look, here's a docker image that you can use to deploy your software. You don't need to know how it works. You just need to know that it's in the artifact and it's here and it's really free to use in the player code on.

06:15 Mario Platt: So from that perspective, things that are more horizontal to the organization. So let's say we've got a team. Everyone uses JavaScript and GOAL, so from that perspective, it doesn't make much sense from a security point of view to be individually developing with all of the development teams, individual checks and telling them how they can integrate checks into their CI/CD pipelines. It makes more sense, both economically and from an expertise perspective, just to work with your platform team to integrate that as part of the platform itself.

06:46 Mario Platt: So things that are very specific to a particular team in the way they operate or the product that they develop, things that are very specific, should be within their realm. So the practices such as threat modeling, et cetera, is always a practice that I think should be across the board in all development and platform teams. From the point of view of not only being efficient, but people understanding what it is that security means to them, and that's, I think, a word that is thrown a lot in security, that security is everyone's job, but few people can actually tell what that means.

07:17 Daniel Bryant: That's a good point.

07:19 Mario Platt: The thing is, when I come in in the morning, a developer logs onto the laptop at 9:00 AM. What does, "Security is my job," mean to me, in the context of the things that I'm developing? And that is where having an approach whereby we stop thinking so much in terms of aesthetic, and that's a problem that team topologies is also addressed a lot, which is stop thinking about team topologies, from a security perspective, as something static. So you report to the security engineering manager or to the Caesar when you do all your job from within the constraint, the organizational hierarchy.

07:52 Mario Platt: We need to start thinking more of that team is in need of help. Let me join their team for three sprints. Let me work with them on the things that they need to do and then you move on. You disseminate the knowledge. You teach people how to perform threat modeling, how to interpret the app with some of the security tooling that you've worked with the platform team to integrate into their CI/CD platform, and then you know you've got enough critical mass of knowledge that the teams can then do it by themselves. But if we just think of you come in and every two months, we're going to do a workshop where we're going to go through this, it's not going to work. You're not going to build the organizational relationships that will make you effective in the informal network, which is my opinion from a security perspective, is usually real security happens more on the informal network than is on the formal one.

08:42 How can teams prioritize what to address when dealing with organizational and security issues?

08:42 Daniel Bryant: Yes. Good point. So you mentioned a couple of times there team topologies, and you and I are both big fans of Matthew's and Manuel's work, and we both know him quite well, which is good. Would you recommend to organizations to understand the team topologies and then the book obviously breaks down a lot of interaction models and the way teams and organizations should be structured. But would you recommend organizations get that down first and then look at security? Or is it security first? Or is it a mix of the two? If their folks are new to both these concepts, how to go about bringing it into their organization.

09:11 Mario Platt: It would be helpful if it happened at the same time, so that once people understand how these better ways of working, how these more integrated models that care for cognitive load, they care for people's actual extra work and the practices that lead to deploying X times a day, et cetera, and make everyone efficient. It would be great if the organizational design was seen as a whole. But experience tells me that probably never happened, mainly because it's not just something that I think is specific to security.

09:41 Mario Platt: It's about the areas of your organization that work in multi-year cycles mainly. So InfoSec is one of them. Finance is another one. When you have dedicated service management or sales, that's another one, it's more about the feedback loops and how people get observability from the actions that they perform. And the truth is, most CSOs, they may say they're working in an agile way, but they're not. They're still thinking about them two, three years down the line. They're not thinking about how can I make the now better. They are thinking I've got these compliance standards that I'd like to align or to achieve in the next three to four years and this is my three year roadmap on how I'm going to get there, which is not necessarily a bad thing. The problem is that you lose the opportunities to learn. You lose the opportunities to listen. And you lose the opportunities potentially to adapt to how your organization is evolving from the point of view of ownership of security, from the point of view of the types of solutions that you do.

10:39 Mario Platt: So overall, and this is something in the safety science in the safety space is much more advanced than it is in cyber security, but I think we're going to get there at some point, which is the difference between a model of a centralized control to a model of guided adaptability. Usually the problem there is the typical security teams tend to be too detached from the day-to-day development of their products and services, and they're so detached that they don't actually understand the work that people do. They don't understand the pressures. What is possible to do on the environment, the affordances provided by the tools, by the management pressures, by all of those things, they are constrained how a developer develops code. The types of pressures they feel on a daily basis.

11:21 Mario Platt: Because we tend to think we just need to deploy that control because that's what makes us secure. We need to deploy the technology because that's what we does. We lose sight of the variability into developer's daily work. And when we fail in our processes and in policies to account for that variability that is both responsible for the success of the organization, but also sometimes responsible for the filings and needs in the type we have. That variability is there. We need to learn how to deal with. It's not going to go away just because we wrote the process. It just means that we probably constrained our process so much that the developers can see how they can both meet the process into their jobs. And that's when we get the friction points between security and development.

12:05 How do you think threat modeling relates to both developers, operators, and the platform?

12:05 Daniel Bryant: Very interesting. I want to come back to something that you mentioned earlier on. I think it relates to this, as well, around threat modeling. So when I first learned about threat modeling, changed my perspective on how I think about security and so forth. I learned it somewhat later in my Dev journey, I think. How do you think threat modeling relates to both developers and Ops and the platform?

12:23 Mario Platt: The good thing about threat modeling, win then write. Let's start with that. When you refer to the threat modeling, we need to talk about Adam Shostack. He was the one coming up at Microsoft with the first threat modeling book for designing for security.

12:36 Daniel Bryant: Yeah, I've got that one. It's a great book.

12:37 Mario Platt: Yeah. I think this is also important to go through the basics. He basically distills any approach to threat modeling into four questions. What are you building? What can go wrong with it? What can you do about it? And how good of a job have you done? So those are the four basic questions of threat modeling.

12:56 Mario Platt: And there was actually, just two weeks ago, A Threat Modeling Manifesto, which Adam Shostack also did. At least I think that is making the rounds right now that I would suggest everyone have a read. But it basically still asks these four questions at the beginning. In question number one, the reason why threat modeling became so prevalent and so important to the daily practice, which is it avoids scope creep.

13:18 Mario Platt: The first question is what are you building. And there has been a problem with the risk analysis processes from a lot of security teams that I've worked with over the years, is that we go in to help a team look at their platform or have a security review or a risk analysis of a particular product or service, and we end up with the laundry list of 150 things that need to be addressed. And what typically happens is the toners and developers are going to look at it and say, "I haven't touched that part of the code in ages. I don't know when I'm next going to touch that part of the code. This does not align with anything that I'm building right now. You're basically asking me to stop everything I'm doing and do security." And you can't convince anyone of that. Unless you've had the security incident where the whole organization knows that, yes, we're going to stop and do security for a while. That's the only organizational event that can lead to you having such an affordance of stopping everything and doing that.

14:09 Mario Platt: So, because threat modeling practices start with, "What are you building," you constrain that to the feature being built, to the thing that you're developing right now. And that way, it makes it really simple and easy, and for the product owners, et cetera, to be inclined to include any security work because you're constraining it to the thing they're already touching. And I think that's a big reason why threat modeling has become so prevalent. I think that's reason number one. You will fight scope creep if you're doing threat modeling properly. And you can actually integrate into the things that people are doing exactly and it means that you no longer need to think of security as a review that you do at the end, you can think of security as, "I'm implementing this function. What threats are in this function that I'm building? What threats am I exposed to?"

14:55 Mario Platt: The companies that do this right ensure that when you integrate security into a process, it becomes part of your definition of done. So this is a security review and you need to tick this box is my definition of done ensures that things have been through threat modeling process to identify whatever we needed to identify. And also, I think a big part of what this means is that you also constraining time. So, you know you're not always going to be in the position to do a three hour workshop every sprint. That's not going to happen. So having methodologies that are going to help you constrain time, knowing that it's built into your methodology, that the highest risks threats are the first ones that you look into, or having a way to go through that, I think is key for long-term threat modeling success.15:41 How early in the design and development process for software should security be included?

15:41 Daniel Bryant: As you were talking there, I was also thinking about a chat I had a while back with Charity Majors, who is very popular in the observability space. Always learn a bunch from Charity. And she was arguing bake observability in early because a lot of folks push it off and it's, "Oh, I need to understand what's going on or add some metrics in." I was curious as you were talking there, how do you think threat modeling security relates to things like observability, being able to understand your system? We should think about that really early on in the Dev process, I'm thinking, too.

16:07 Mario Platt: I definitely think that overall in the market, something that is currently not being explored to its fullest potential. So I think their observability tools have a brilliant security potential to be used as real security tools once we start using them better and using them specifically for security. I think there's a significant open space in the market for that to be the next generation of security tools, based on observability in the data that you already have.

16:34 Mario Platt: But there is still, I think, overall not very much explored. And that's a big problem in the industry overall, is that we are still thinking about security as an add-on vendor that you're going to have, and we're not really thinking about how the technologies that they have in platform already using give you the observability we need if you can add the intelligence of understanding what are the implications to security.

16:56 Mario Platt: And I also think that's where the market is going. We are no longer going to add this tool or this product to make you secure. You know that the IT tools that you're using are going to tell you how secure you are. I think that's the direction of travel. But overall, from an observability point of view, observability is an interesting term overall, because there's the technical understanding of observability, which is more about the instrumentation of code than the making sure that you understand what's happening with your application. There's also the sociotechnicity lens, so that people such as Jay Bloom talk a lot about sociotechnicity of DevOps is a talk that I would suggest everyone to have a look.

17:35 Mario Platt: And I think that Jay's approach to observability, how he talks about it, is most useful for security because it breaks down observability into different time spans into agency of the actors. So for instance, what I mean by that is, if you are a developer or security engineer, the agency you have to perform actions in your CI/CD platform in developing anything that you're doing, are usually going to be constrained within sprints. So you know that you've done something now, you know at the end of the sprint you're going to have a definition of done which your work is going to be measured against. And that's your observability. That's your feedback loop.

18:14 Mario Platt: The challenge with security is that typically, we are trying to address risks. And risk don't work in two week sprints. That's not how things work. Inherent in the idea of risk is the element of uncertainty. So we may make a decision now, and even from the point of view of CTOs and CIOs, not working on the security feature on the next two sprints is doing nothing from the risk perspective to them. It's too short of a timeframe. It's unlikely we're going to be pushed in the next two weeks. It may happen, but the way you're looking at your system when you're at the management level is at least in six months cycles, usually two to five years, depending on how you're looking at it. And that's also part of the challenge of justifying security is that if nothing ever happens with that good money that you spent, because there's that element of uncertainty.

19:01 Mario Platt: Obviously, things are getting different because of regulatory compliance, contractual requirements, and all of that, the things that we need to evidence, but on the grounds of uncertainty and risk alone, it may or may not happen. Having that observability between not making a call today that you won't see the results of them in another two to three years time is part of the challenge of security.

19:22 Can you recommend any models or references in regards to the socio-technical aspects of security? I see you tweeting a lot about Wardley Mapping and the Cynefin framework?

19:22 Daniel Bryant: I know you mentioned a couple of times around the technologies and the socio-technical system and the processes. Have you got any good thinking tools that you could share with folks? Off mic, you and I were chatting around Wardley Mapping and Cynefin, both things I'm fascinated around. Would you recommend folks look into those kinds of models for helping them understand the value of security?

19:42 Mario Platt: Yeah, for sure. Not just understanding the value of security, but also understanding how the system, and when I say system, I'm talking about organization. I'm not talking about an IT system. How the systems that they're involved in evolve and how things that happen outside of your system affect it. So I think both Wardley Mapping and Cynefin are great ways to look at it.

20:04 Mario Platt: Another thing that I've also blogged about in my blog and I'm planning a few more posts in the next few weeks is on social practice theory. The typical way to look at security has been people processing technology. That's usually what people talk about. It's not a problematic lens, but I think the lens of social practice theory is more helpful, which looks at organizational practices from the point of view of meanings, know-how, and materials. So materials being artifacts, meanings being, "Why do I do this?"

20:32 Mario Platt: A lot of security way of thinking is kind of caught up in Taylorism. When you think about our processes, our mechanics, our machine, we just want to improve the machine, right? But people aren't machines. I think some of the lenses that we use such as people, process, technology, such as CIA, confidentiality, integrity, availability, are kind of old world lens. They're still applicable. They're still relevant. So from a practice perspective, a social practice theory of meanings, know-hows, and artifacts. From a CIA perspective, the lens that some of you write about, from the point of view is distributed immutable and ephemeral, DIE.

21:08 Mario Platt: I think those are lenses that are more aligned to our current understanding of what a socio-technical system is. This is something that wasn't really talked about much or applied to technology 10 years ago, but now it is. If we're still doing security like we were doing 10 years ago, there's a lot of learnings that we still haven't had that we need to have. And also from a CIA perspective versus DIE, distributed, ephemeral, and immutable. So again, it's not necessarily anything wrong with your models, it's just that the words we use to describe the old models, leave to the old practices that are no longer so useful or so integrated into the modern way of doing things.

21:46 Is organising and running a game day a valuable practice for developers and ops?

21:46 Daniel Bryant: Final topic. I just want to get your brief thoughts on things like red teaming. And I see a lot of folks blending chaos engineering with security testing. Any experience or thoughts on the value, your upbringing, I guess the social side of things, into the technical sphere, running game days, running fire drills, seeing what happens when a security incident pops up and how the system deals with it, both from a technical point of view and a social point of view, as well. Any thoughts on that?

22:12 Mario Platt: It is an integral part of the journey towards the focus on resilience. I think a big part of the problem that we've dealt with in security is that the language of control has the problem that it focuses on robustness. So giving you an example, when we think about floods, a lot of people think about dams, and there's nothing wrong with the dam. They're very useful. They're used throughout the world. The problem with the dam is that it's a robust control. And the inherent problem with robust controls is that once the design conditions are exceeded, they're going to fail catastrophically. Nothing wrong with it. It's just that if that's the only tool you've got in your bag, it will eventually fail once the design conditions are exceeded. It's going to fail. You can think of it from a salt marsh perspective that you get water, but that it's inundated and gets absorbed by the bottom so you only get the additional over top. So when it fails, it does not fail catastrophically. That's a resilient approach to the same problem.

23:05 Mario Platt: And I think part of the direction of traveling securities, things like, again, red teaming, et cetera, allow us the possibility in a relatively safe environment to understand how our controls act and the certain conditions that we are stress testing them under. And that allows us, if we're doing it right, to get all of the organizational learnings. And it's not just about improving the robustness of the controls, which is part of the problem in how many people talk about chaos engineering, is they're identifying a problem and either increasing capacity or putting the control to do it.

23:39 Mario Platt: What resilience is more interested in is when we were having the problem, how were the teams talking about it? What did they see that they didn't expect to see? What were their first thinking about what would be the right solution or what the problem was? Because that goes into the heart of resilience is something that happens at the individual and organizational level. It's not a thing that technology does. Technology is not resilient. It's the people that absorbed the variability that your systems can't deal with. Then when anything goes wrong, you get people on call, and it's the people that are responding to the system. That's when your technology is no longer sufficient. You need the buffer, the human knowledge, and the human capacity to deal with that.

24:19 Mario Platt: While I do think that chaos engineering, et cetera, is going in the right direction, I think as an industry, we still need to learn how to tap into the human side of it so that our people also get the benefits from the adaptive capacity and the learnings that they can take from those. So I think we should be focusing more on the people and also on the technology, but it's not technology that's going to save us. It's people understanding how their systems fail.

24:43 How can listeners get in touch with you?

24:43 Daniel Bryant: I think the answer to that final question, Mario, is a whole new podcast, right? Because I'm fascinated by this. I love John Allspaw's work, Nora Jones is doing a lot of fascinating stuff in this space. And I hundred percent agree with what you're saying, looking at what we've learned from the aerospace industry, for example, and the airplane crashes. We don't really do all that for our systems, but that's definitely a topic for another day, I reckon, because that is a huge topic and I'm sure you would have super interesting thoughts on that. At this point, I really appreciate your time today, lots of great stuff. I'll be sure to link a bunch of these things in the show notes. If folks want to get in contact with you, what's the best way? Twitter? LinkedIn?

25:13 Mario Platt: I'm more active on Twitter than anything else. So @madplatt. M-A-D Platt. On Twitter. I'm also on LinkedIn, I believe, Mario dot Platt, I believe. But I'm less active, overall.

25:24 Daniel Bryant: Super. Well, thanks for your time today, Mario.

25:26 Mario Platt: Thank you very much for having me.