Catherine: Yes. My name is Catherine Louis, I have two ongoing efforts and now three with this security. I have a company – CLL Group – that offers Agile training and consulting, designed to move organizations from this early inspecting and adapting to actually delivering and delighting.
Laurie: I am Laurie Williams, I am a professor at North Carolina State University. I have been in Agile since 1998, I got my PhD and the topic was Agile. So I have been in it for a long time. In addition to teaching, I also do industrial training and consulting in Agile and I do security research. I’ve been doing security research since 2004 and now lead a large NSA US (National Security Agency) science of security lab at NC State University. So, this is kind of bringing Agile and security together.
2. So, what brings you both to the Conference?
Catherine: We met in 2008 when I was in a company and I had a security team and she introduced some techniques to us at that time. One of the techniques was Protection Poker.
Laurie: Protection Poker is a game that teams can play. It is based upon Planning Poker, which is a game that a lot of Agile teams already play. It's used in the same kind of technique of collaboratively coming together to estimate security risk of a new feature.
Laurie: It was really from my experience. I’ve sat through lots, like hundreds of, Planning Poker sessions and I always really impressed about the amount of exchange and debate and creativity that went on in Planning Poker session to discuss. What is this feature about? And I thought – if you took that same kind of discussion and the same diversity of opinion and how Planning Poker plays people off each other because they have different numbers. And if you did that in a security context, that you would have a lot of good security discussions and because you are playing people off each other and communicating about the security risk of a new feature, people become more devious, because they say “Oh, yes. You could do that, but you could also do this” and so it just brings out the diabolical nature of security by bringing people together.
Catherine: I wanted to share a quick story: when I met her, just before meeting her, I had undergone a terribly painful security audit that was tagged on to the end of a development cycle, and so security was an afterthought. And I was really struggling to build security in – y’know fast-forward lots of Agile training and coaching later, we’re finding the two topics coming together.
Laurie: Yeah. And in most companies, I’ll say, in most industries, security is not a high priority. So, that’s kind of the base line. Many companies don’t pay attention the way they should, but Agile is particularly precarious because Agile and Lean focus on feature: “Give me the feature. What’s the value? What value can you show the customer this iteration, or this sprint?”. And security may not be a part of that. It is less likely to be a part of that if you are focusing on delivering functionality. And so what Protection Poker can do is to help you to get it so that when you do deliver that value, it’s developed securely.
5. What was it about security that really drew you personally?
Laurie: Well, me personally, I started to research security at NC State University because I have a passion for the fact that we need to develop secure software and there are nation states coming against each other, there are people who are trying to kill other people and steal money and there’s a lot of missions and if we don’t develop software securely, it makes it easy for the attackers.
6. And you were saying there is a great cost?
Catherine: Yes. In a recent survey by Dell – I think it came out in April – it’s 25.8 billion for data security breaches hitting US organizations annually. Just huge costs.
Laurie: That is right. And we all say that we were kind of disappointed yesterday when we did the workshop that there is 1800 people here and 21 people came to our workshop which kind of accentuates the fact that every day the whole world is pumping out more and more insecure software which is a concern because we’re just leaving things open for the attackers.
Katherine: Well, we are hopefully going to reach a broader platform now, with this talk, of being able to film it. So, can you describe some of the core elements that you were trying to get across with Protection Poker? And maybe a little bit about the game. So we will start with – just describe the game for me a little.
Catherine: Leading up to the game, have you ever heard the phrase “If you want to really develop a good product for a customer, you have to get into the head of the costumer.” So, in this case, we have to get into the head of the thief or the hacker. So, there is this change of thinking. We really have to think about how to abuse a product. And the beginning of the workshop is doing just that.
8. Give us an example of what you might do? What is an example?
Catherine: What came out in the…? What were some of the…?
Laurie: Well, we had participants first talk or think about it: “Just think like a thief. You are going to plan a heist. Plan a heist” And we asked them to think about how long will it take to plan and execute this heist and what is the pay off. They came up with a whole variety of things.
Catherine: One was like robbing a bingo hall where the pay-off is unmarked cash, you know. So it’s very easy and it doesn’t require much planning.
Laurie: And another that didn’t require much planning was just pilfering so you are bar-tendering and you just take out a little bit of the cash register.
9. You are using non-technical examples and then applying it onto technical examples. Is that right?
Laurie: Right. And the point of that exercise is no. 1: to think like a thief, but also to play off the fact that if the pay-off is big and some of the heists that they planned had big pay-offs, so, to get a big pay-off it generally took more planning and more work. Little pay-off, little planning. And that’s the way it is with software as well, that an attacker will work really hard for a social security number or a medical record, but they will not work so hard for a baseball score. So, when you are coming up with a new feature, we need to think about “What is it that I am protecting?” So, that exercise got them at least thinking that way.
Catherine: Yes. What is the asset? Is a database of credit cards, a database of social security IDs - and what is the ease of getting into it?
Katherine: So you start with some sort of game to look at assets, pay-off and perhaps how much effort needs to go into trying to get this heist, for example, done.
Laurie: Right.
10. So once you kind of worked that out, is there a process you take from there?
Laurie: Then we started to play the game. So, explain the game and play the game and similar to Planning Poker. Planning Poker is like a relative estimate of effort and this is a relative estimate of both – as Catherine talked about – the value of the asset and the ease of the attack. So, it’s actually two different relative estimates. So, you’re going to have a new feature you’re going to bring into your system and one of the things you think about is what data is that going to access, what’s potentially special processes I can execute and how important is that? In the eyes of an attacker, or the eyes of the owner of the data, how important is that and a relative estimate of that? Using the Planning Poker numbers – 1,2,3,5,8,13,20,40 and 100 – as the relative estimate and then the next is: how easy is it going to be to execute the attack? - and it’s possible that the new feature will make the overall system more vulnerable.
It is opening up a door for anything. But some features that you add, don’t add any more security risk, it doesn’t make the attack any easier, and some make it easier. So, the way the game is played is both of those are estimated and you multiply the two of them together to come up with the risk, but in the process of doing that and coming up with those relative estimates, they are saying, they’re kind of exploring the kinds of attacks. So they’re planning the heist: someone could do this, someone could do that. And also, covering them up. So, they may say: “Oh, well, yes someone could do that, but if we log it and alert the admin when someone does that, then we’re going to mitigate the risk”. So, just by the act of playing the game, they’ll do risk mitigation as well and close up and then the risk can go away.
Catherine: It is just done right before sprint planning so that new logging security story goes right into the sprint planning. So, it just folds right into the process.
Katherine: Because I think that’s an area where people would from a security perspective, they would be - security industry, I should say - would be concerned about Agile having these conversations and not necessarily being heavy on the documentation because we all know security industry likes documentation. So, I was wondering if you could talk a little bit about why you think it would work, it has or seen how it has worked where the conversations immediate turn into these stories that we could use within the sprints and how it covers that area of concern that security industry may have.
Catherine: My experience is that it’s representing the backlog so it is type security and it can be traced, tracked all the way to test case. So, there wasn’t any extra documentation needed.
Laurie: That is true. This may be a different subject, but a huge benefit of this Protection Poker is the learning that goes on within the team and actually, the participants felt that. We didn’t even have to tell them that. We ran a case study with Red Hat. That was an extended case study and we asked the team, before we got started with it – and it wasn’t just a little exercise; it was multiple months – “How much do you know about security?” and “How much did this help you to learn about security?” and “Does this help you focus on the right issues?”. The team did feel that they learned a lot, so just in the act of people sharing – someone could do this, someone could do that – they’re learning about the types of things that an attacker would do - so, getting into the mind of the attacker and they are also learning about techniques of what the attacker would use. The knowledge is spread around the team. Most developers and testers don’t know much about security so you can inject one or two that do, then play this game and then they’re all learning.
12. [...] My question is - How would you inject that knowledge?
Katherine's full question: Do you think that you need a similar person to a product owner but a security owner in order to distribute the knowledge about security and to help guide the conversations within a Planning Poker scenario or are people OK with discovering security within the security the Protection Poker has? My question is - How would you inject that knowledge?
Catherine: The game actually causes the knowledge to spread and I think the language now is to call it something like a guild – a security guild, or a community at practice and I don’t think it needs an owner, but I think it spreads that way.
Katherine: That’s very interesting.
Laurie: Yes, but I think that taking someone from the guild or the security team and having them being there would be good and one thing that does is that most companies that do care about security at all, do have this kind of an organization who aren’t late in the process, as Catherine mentioned. They come along at the end and tell you everything you did wrong and so if you put them into the Protection Poker and get them involved, that helps as well.
Katherine: So you are saying that as a collaborator, not necessarily a leader, so it’s to inject the information that is required.
Laurie: Right. Then, what the participants also brought up was that having just the product owner listen to the security implications of a feature, would be helpful as well because they don’t have that perspective. Like one of the features that we had them look at: we had them hack the Agile 2014 website and we gave them five different features and asked them basically to assess the risk of these particular features. One of the features was a new role that was going to able to edit all of the abstracts of all the sessions of the conference and you could see that someone might think that’s a good idea. So, let’s have this new person that has access to all the abstracts and can fix all of the punctuation problems or whatnot. But actually that feature could also easily be abused and so, the product owner may comment and say: “I want this” but not have any idea about the implications of it. So, this gets him tuned into that.
14. Do you want to explain a little bit about that?
Laurie: I guess I would say that security can be considered a non-functional requirement and the focus on features can minimize non-functional requirements as the whole including security. So, this is the way to bring it into the forefront and to look at the functional aspects of security.
Laurie: You can write any non-functional requirement as a feature. So you can have as a website user, I would like all of the response time to be one second or less so that I don’t have to wait too long. You can take really any non-functional requirement and turn it into a feature. When I work with teams I had them take those non-functional requirements, turn them into features and put them in iteration zero because they never go away. Non-functional requirements never go away – you never finish them. So you want them out there as a resource.
Catherine: So, playing the game, you look at anything new coming in, regardless of where it comes in. We need training. Is this something that could affect an asset? You see, you can look at anything in your backlog, functional or non-functional.
Katherine: OK. So it is a non-discriminating view.
Catherine: We do not discriminate.
Laurie: Equal opportunity feature.
Laurie: So, those innovation games- there is an innovation game called “product box” where you lay out what your product will look like and so we’re thinking of starting with a similar kind of thing, but write the headlines of your company in the news. What are the things, how your company could get ruined from security?
Catherine: You are completely embarrassed, everything bad happened and then we are back from there to mitigate it.
Katherine: Well, you know, I would personally predict that this will be a really big area. So, I am really happy to have spoken to you and I hope that we will be able to follow this up at some point and find out how everything went.
Catherine: Sure. Thank you.
Laurie: Thank you.