Managing Security Requirements in Agile Projects

Addressing security requirements from the early phases of software development is the most cost-effective way of preventing security defects. Most security requirements fall under the scope of Non-Functional Requirements (NFRs). As many practitioners have discovered, addressing security and other NFRs in agile projects is challenging for two reasons:

1. Mapping NFRs to feature-driven user stories is not trivial.
2. Security controls suffer from lack of visibility. Agile processes tend to bias development teams towards building features that visibly enhance the customer’s experience or fix defects.

In this article, we examine both issues.

Addressing NFRs in user stories

Several agile experts suggest methods for defining non-functional requirements in user-story driven development processes. Mike Cohn wrote on the subject with several interesting suggestions by commenters. Scott Ambler has written articles on the subject arguing that not all NFRs can be self-contained in a user story. Dean Leffingwell arguably has the most in-depth treatment of the subject where he dedicates an entire chapter to NFRs in his book Agile Software Requirements. Most of the experts agree that we can divide NFRs into two broad types:

1. Non-functional user stories: Blocks of testable functionality written in user story format. The actors in these user stories may be internal IT staff. For example: “As a security analyst I want the system to throttle unsuccessful authentication attempts so that the application is not vulnerable to brute force attacks”.
2. Constraints: These are cross-cutting concerns that may have an effect on several other user stories. They are a sort of “tax” on all relevant development efforts. For example, requiring that all developers validate data from HTTP form fields in a web application is a constraint.

Handling the first category is straight-forward. Create a relevant set of NFR stories and add them to a work queue, such as a product backlog.

Challenges with NFR Constraints

Handling the second category, however, is considerably more difficult. Agilists have proposed a few solutions to this, including:

• Adding appropriate constraints as acceptance criteria in specific user stories
• Maintaining a list of all constraints in a central repository, such as a document posted on a visible wall or on a wiki

Neither approach offers a mechanism for selecting which constraints might apply to a given story. Moreover, our experience with mapping security requirements at SD Elements is that these techniques tend to be difficult to scale. Take, for example, the following subset of security constraints for a standard web application:

• Bind variables in SQL statements to prevent SQL injection
• Verify integrity of client-supplied read-only data to prevent parameter manipulation
• Escape untrusted data in HTML, HTML attributes, Cascading Style Sheets and JavaScript to prevent Cross Site Scripting (XSS)
• Avoid DOM-based XSS in client-side JavaScript
• Use safe arithmetic to avoid integer overflow
• Disallow external redirects to prevent open redirects
• Authorize protected pages to prevent privilege escalation
• Use anti cross site request forgery (CSRF) tokens
• Validate input
• Use regular expressions that are not vulnerable to Denial of Service
• Implement transactional authentication for high-value transactions
• Do not hard code passwords

This may represent less than half of the security constraints developers need to consider when implementing a user story. Moreover, the set of constraints expands considerably when factoring in regulatory compliance such as the Payment Card Industry Data Security Standard (PCI DSS). If you start adding in other NFR constraints, such as accessibility, the list of constraints can quickly grow overwhelming to developers. Once the list grows unwieldy, our experience is that developers tend to ignore the list entirely. They instead rely on their own memories to apply NFR constraints. Since the number of NFRs continues to grow in increasingly specialized domains such as application security, the cognitive burden on developers’ memories is substantial. Thus, there is a heavy emphasis towards relying on static analysis technology to detect NFR constraint violations in code. Studies indicate that static analysis can detect up to 40% of preventable defects. This still leads to a large number of constraints that escape static analysis detection, including domain-specific security controls. This is not to mention the challenge of wading through static analysis false positives without customization.

Addressing NFR Constraint Challenges

In order to solve the problem of excessive NFR constraints, we need to address four major points:

• Prioritization: Just like user stories & defects, NFR constraints should have different priorities. For example, encoding or validating untrusted data in HTTP response headers to prevent HTTP Response Splitting is generally not as important as escaping untrusted data in HTML data to prevent persistent Cross Site Scripting. Assume that developers will rarely have enough time to address every constraint and provide a mechanism to suggest relative priorities amongst constraints. We say “suggest” because the priorities may change depending on context and developers need to make a judgment call about what the relative priorities are for their specific code. You may elect to use simple priorities like “High, Medium, Low” or a numeric scale such as 1-10.
• Filtering: Using simple criteria, you can often reduce or remove large volumes of NFR constraints for a particular user story. For example, suppose you are working on a user story that does not impact the presentation tier. You can safely ignore a large number of presentation-tier specific constraints. Using a tagging system or even simple Excel filters, you can cut down on the total number of constraints for a particular user story. Here are some example web application-relevant security filters:
  • Does this user story involve user-supplied input?
  • Does this user story involve confidential data such as passwords, credit cards, or non-public financial data?
  • Does this user story result in new or modified user output, such as web pages?
  • Does this user story involve interaction with a database?
  • Does this user story involve exposing or consuming data from a RESTful or SOAP API call?
  • Does this user story involve access to protected data (e.g. data that not all users should be able to view/modify)?
• Context: Developers are more apt to remembering & applying constraints when they are in the context of writing code or defining tickets/user stories. Ideally, if you embed your list of constraints inside an IDE or ticketing system then developers will have access to the relevant constraints at the time of coding. Keep in mind that many IDEs support embedded web browsers; so using a lightweight web or SharePoint site can achieve this goal.
• Framework: Frameworks can substantially reduce the “tax” of constraints. For example, a framework like Django that provides CSRF protection out-of-the-box will mean that developers can safely ignore a constraint unless they actively circumvent the built-in protection. In our experience, most large applications have some form of custom frameworks that can seamlessly handle the highest priority constraints. This sort of framework customization may require a high-upfront investment, but if you think of constraints as a sort of “tax” on user stories then reducing the number of constraints is analogous to permanently reducing the tax rate.

Validation

While proactively addressing NFRs is important, you still need to validate. If you have a manual code review practice, reviewers can check for NFRs explicitly listed as acceptance criteria in user stories. Many constraints, however, can be tricky or burdensome to find through manual code review alone. Static analysis tools automatically check for adherence to coding standards. Some products target security specifically. Configuring your static analysis tool to integrate with a continuous integration process can help provide consistent adherence to coding standards. Be careful, however, to not rely exclusively on code review. NFRs that protect against domain-specific vulnerabilities, such as HTTP parameter manipulation, require understanding the underlying business domain. Using a combination of manual and automated quality assurance regression testing for specific attacks can help you build a system resilient to domain-specific vulnerabilities.

Visibility

Another factor inhibiting integration of security into agile projects is a lack of visibility. Customer-facing meeting such as Sprint reviews in Scrum produce an innate bias to work on stories or fix defects that improve the user experience. Some non-functional attributes, such as usability and performance, have a tangible effect on the experience of using the system and are easy to demonstrate to customers. Others, such as security, rarely result in a net positive experience for the customer in a Sprint review. In fact, some security features such as account lockout can have a particularly negative impact on user experience. Most security features have no discernible impact for normal users. Thus, security NFRs are invisible and need to compete with more visible features for precious development cycles in the iteration. The opportunity cost of working on invisible requirements is particularly acute in the early stages of application development, precisely when developers should be building security in.

Making Security Visible

There is no silver bullet to making security a top level priority for an agile team at the onset of development. There are, however, a couple of ways to make security NFRs visible to customers and/or product owners:

• Graphing security stories: If you have sufficient application security expertise, you can produce a comprehensive set of security controls to known security weaknesses. You can then take the controls that map to NFR user stories (as opposed to constraints) and build a simple graph illustrating the number of implemented and unimplemented security user stories remain in the system. This graph can reside with your other Big Visible Charts.
  
  
  Figure 1: security user stories graph
  
  Of course, you don’t need to restrict this graph to security stories. You may elect to include other important NFRs in the same graph. You may choose to use story points rather than just number of user stories in order to reflect effort. Also, you may wish to stick to graphing only high-priority stories / controls for high risk security issues. Every story that you implement will help increase the “Implemented” count and consequently help you make the progress visible to the customer and/or product owner.
• Demonstrate exploitability: Nothing gets people to understand the ramifications of a security vulnerability like a demonstration of exploitability. This is the key difference between a security penetration test and a vulnerability assessment. Where the effort is warranted, you can demonstrate a successful exploit on a previous version of the application and then show how the new version is no longer exploitable. You can then explain the ramifications of the vulnerability in terms of impact to the business and/or customers. This can help enormously in justifying time spent on security controls.

Conclusions

Proactively addressing non-functional requirements, and security in particular, in an agile context is non-trivial. Several experts provide advice on dealing with these requirements, particularly as they relate to creating user stories. Addressing a comprehensive set of constraints is a significant challenge. Keeping in mind the suggestions discussed in this article, you can greatly improve the value of maintaining a library of constraints. Even if you can’t address all four points to improve managing constraints, any single improvement you make can yield major benefits over the long term. Adding validation provides assurance that your development team successfully adheres to NFRs. Finally, making security visible allows you to justify the time you spend on making improvements to the system that don’t normally improve an end user’s experience.

About the Author

Rohit Sethi (@rksethi on Twitter) is a specialist in building security controls into the software development life cycle (SDLC). He has helped improve software security at some of the world’s most security sensitive organizations in financial services, software, ecommerce, healthcare, telecom and other industries. Rohit has built and taught SANS courses on Secure J2EE development. He has spoken and taught at FS-ISAC, RSA, OWASP, Secure Development Conference, Shmoocon, CSI National, Sec Tor, Infosecurity, CFI-CIRT, and many others. Mr. Sethi has written articles for InfoQ, Dr. Dobb's Journal, TechTarget, Security Focus and the Web Application Security Consortium (WASC), has appeared on Fox News Live, and has been quoted as an expert in application security for ITWorldCanada and Computer World. He also created the OWASP Design Patterns Security Analysis project.