Virtual Panel: DevSecOps and Shifting Security Left

Recent attacks, that targeted SolarWinds, Colonial Pipeline, and others, have shown that development environments come ever more frequently on the radar of malicious actors. A virtual panel on the value of shifting left security, how to take responsibility for it, and the time-to-market pitfall.

Gone are the time when developers and business leaders could just delegate security to CIOs and security teams, or even worse add security to a system as an afterthought. In fact, recent PwC's Cloud Business Survey shows Cloud and organizational security are among top management concerns.

Additionally, supply chain attacks as those mentioned earlier made apparent companies should be integrating DevSecOps into their Cloud infrastructure. Supply chain security has become so critical as to be among the concerns of the American executive order on cybersecurity.

The cybersecurity landscape specifically requires a fundamental rethinking of how companies secure their building environments and ensure any code they adopt meet stringent criteria.

Also, they need a clear understanding of who is responsible for security to avoid grey areas and define appropriate processes so that building secure products becomes not only possible but also more efficient overall than building product fast, then patching their vulnerabilities.

InfoQ has taken the chance to discuss these topics with Sonatype CTO and co-founder Brian Fox, Traceable CEO and co-founder Jyoti Bansal, and Venafi VP of security strategy and threat intelligence Kevin Bocek.

InfoQ: In light of recent attacks, including SolarWinds and others, it seems that security should be a matter of concern for the whole supply chain of an organization, including internal and external contributions. Do you agree with this view and, if so, what implications can this have?

Bansal: Absolutely. The most prominent red flags are the recent software supply chain attacks heard all over the news — SolarWinds, Accellion, and Codecov to name a few. These attacks highlight how security risk can pass through a highly interconnected software ecosystem. They are telling us that the industry needs to put more focus on security in the software development life cycle, such as maintaining software bill of materials (SBOMs) and using more application signing.

However, supply chain attacks are not the only red flag organizations should be paying attention to. With the rapid growth of cloud-native, microservices based, and API-driven applications, the API attack surface of applications has drastically broadened and become the number one attack target for bad actors. Concern for these types of attacks should be rising, as well as prevention and mitigation efforts — and that goes for APIs created within an organization, as well as external APIs leveraged by an organization.

Bocek: As we saw with SolarWinds, the impact of a successful software supply chain attack can be devastating. There is very little that customers of a compromised vendor can do to protect themselves against these software supply chain attacks. Awareness of these risks is critical, and it’s critical that software development companies take action.

But it’s not only the software providers that need to pay attention here -- every business must come to the understanding that they are a software developer. They build, release, and operate software: whether they script RPA bots or run infrastructure in Amazon AWS. In doing so, every business has a responsibility to both ensure the security of the software they use and the software they build and release. Whether you are a bank, retailer, or logistics provider, you are a software developer and need to protect software developed just like all the ISVs you rely on.

Fox: Absolutely, as you said, security should be a matter of concern for all parts of an organization's software supply chain. Bad actors have become increasingly sophisticated and calculated in the ways in which they deliver attacks upstream in the software supply chain. They’re poisoning the software supply chain to get at someone even further downstream -- a vendor’s customers for instance -- by subverting the engine of trust regarding the origin and reliability of software. There are growing numbers of organized attackers whose sole focus is exploiting vulnerabilities in open source ecosystems, frequently by making their malware appear legitimate. The intensity, volume, frequency, and severity of these malicious attacks are increasing at an alarming rate. Once malicious code gets into developers’ machines and build environments, it can end up in their internal corporate networks and in the product they deliver to all their customers.

Developers must see themselves as part of the solution and become ever more vigilant in their own coding practices as they represent a clear red target with exponential cascading impacts. But, we as an industry also need to give them a no-fuss way of understanding what’s in their open source components, of choosing the best packages, and of analyzing the code they’re writing as well: which is what we’re working to do at Sonatype.

There appears to be a certain ambiguity about who is responsible for security, whether testers, security specialists, or developers. What is your take on this issue? How should the interplay among those roles look like in organizations capable of guaranteeing the security of their systems?

Fox: I don’t think there is any ambiguity - everyone should be responsible for security. It isn’t just one department or team’s job. That said, everyone plays a slightly different role. While not a perfect analogy, you can think about it as the need to play both offense and defense; you won’t win a game with just one. The work security teams do is vital to “playing defense” with advanced perimeter security — but there is also extreme importance of “playing offense” by building security into every application without slowing down innovation. By having a game plan that includes all of these elements and one that empowers developers, security teams and testers to communicate, you’re much closer to setting yourself up for success.

Bocek: There certainly is ambiguity and confusion around who exactly is responsible for securing software and the development process - in fact, we recently found in a report that just over half (58%) of security professionals believe it is their responsibility, while a similar number (53%) of developers believe software security falls under their purview. It’s this lack of consensus that is at the crux of today’s biggest cybersecurity challenge: security is not being baked into software during the development process, which has led to destructive cyber repercussions, as we’ve seen recently with the Kaseya, SolarWinds, and Microsoft attacks.

It’s just not possible for one team to keep the software build process secure - we need to incentivize developers to work with security teams from the start of development. To be clear: Developers must become responsible and accountable for the security of the software they build and operate.

Developers are often prioritizing speed and innovation, and security teams are left to pick up the pieces after software is built to keep it safe from hackers. Shifting security left, ensuring that cybersecurity is baked into software throughout the entire build process - not just once the software is shipped out - is key to guaranteeing a company’s software is secure.

Bansal: Ambiguity around security responsibility, especially in software development, is a huge problem for the industry. Not only have the two departments been siloed until recently, each team uses different types of language. With APIs, for example, software engineers are looking at a piece of software through the language of traces whereas security ops teams don’t come from the code world — they secure networks and infrastructure. While engineers need to look at software in terms of security, infosec professionals need to be more code-aware.

We need to emphasize DevSecOps as a practice, building security in from the start of application and software development, and using strategies like CI/CD to integrate security once software is deployed as well. The responsibility can not be placed on only the engineering team, or only the security team. There must be a balance — a coming together of those departments in every single organization to ensure that security is prioritized from the very beginning.

InfoQ: Too often security is seen as a factor that slows down development, which is the case, for example, when time-to-market is the top-most concern. This is possibly both a matter of culture as well as of practices. Where should organization start from in order to change this mindset?

Bocek: There absolutely needs to be a cultural mindset shift at software companies. Too much time is spent debating who is responsible for security and whether features and innovation should be prioritized. The fact of the matter is that it is not only possible, but necessary to adopt a “FastSecure” mindset across the industry, and this starts with the C-suite. Company leaders need to emphasize the importance of secure software from the beginning of development, and encourage, guide and incentivize their teams to shift security left. Leaders must hold developers responsible for the security of software they build, and hold security teams responsible for helping developers protect the software built.

Fox: It’s both a matter of culture and practices and works from the top down. People define the technique and tools that enable execution. Security does not need to be a factor that slows down development if more organizations would think of security as a piece of the development process. In fact, a key part of Sonatype’s 2020 State of the Software Supply Chain report found that organizations who focus on security as part of the development process have better productivity outcomes than their counterparts. While the majority of developers have become more aware of security, it’s difficult to implement appropriate measures when current mindsets see security issues as a reactionary problem, not a proactive problem to be solved for.

One of the best places to start an organizational transformation is by integrating security professionals into the development process from the beginning. Especially when it comes to communicating change. Then, if any security questions such as vulnerability scan results, or the validity of false positives arise, this person will be in the trenches with the information to help out immediately, not after a ton of work has already been executed. Further, Include everyone in your transformation because that’s the only way to knock down yesterday’s silos and introduce a collaborative future.

Bansal: The industry as a whole needs to shift security left — ensuring that security is implemented in the software development life cycle instead of waiting to add in security after products are deployed into production. We need industry leaders to take a stand and adopt secure development practices, making security an unambiguous priority at all levels. C-suite leaders taking a stand and incentivizing their teams to come together is absolutely critical to this effort.

InfoQ: What is the importance of the Executive Order on Cybersecurity? What changes will it bring to how organizations build software systems? Should it be replicated in other contexts, too?

Bansal:It's great to see the administration taking improvement of cybersecurity standards seriously. The gravity and widespread nature of the SolarWinds attack, and now Kaseya, clearly demonstrates that the impact of nation-state cyberattacks has reached a new level of risk. There is so much software development behind how government agencies operate and interact with citizens these days. As the SolarWinds attacks showed, software code and all the third-party suppliers in the software supply chain are the next key vector of attack and will continue to be.

But prescriptive regulation alone is insufficient. We also need industry leaders to adopt secure development practices and make security an unambiguous priority at all levels. Accountability is another part of the answer — the cost of security breaches should be sufficient to motivate vendors and IT professionals to make changes to proactively detect and prevent more vulnerabilities.

Fox: The cybersecurity executive order marks the strongest stances ever taken by the federal government to secure the United States’ software supply chain from attacks. While it is an important step, it has received some criticism from the industry as not being specific enough, leaving too much ambiguity and wiggle room without standardizations.

While on its face, the EO only applies to organizations selling software to the federal government, like the software itself, this will apply transitively to organizations providing software to those who sell to the government all the way down. This is a good way to pressure the supply chain all the way down to level up their practices and provide the transparency everyone needs.

Bocek: My worry is that this Executive Order, if adopted more widely by the entire software industry, will slow down innovation and give attackers the upper hand. There is a way for software to be built fast and secure, but prescriptive regulations for the software industry simply will not work and should not be replicated -- the federal government cannot move quickly enough to effectively regulate how software is built.

The only way the government can help protect individuals and companies from becoming victims of insecure software build processes is by incentivizing the software industry to build better. In addition, there needs to be strict financial repercussions for any company that fails to do so. As it stands, the new executive order from the Biden administration, including a software bill of materials, will only slow down software companies and give attackers the opportunity to innovate faster.

About the Panelists

Kevin Bocek is responsible for security strategy and threat intelligence at Venafi. He brings more than 16 years of experience in IT security with leading security and privacy leaders, including RSA Security, Thales, PGP Corporation, IronKey, CipherCloud, NCipher, and Xcert. Most recently, Mr. Bocek led the investigation that identified Secretary Hillary Clinton’s email server did not use digital certificates and encryption for the first 3 months of term. In 2013, Mr. Bocek led Venafi’s investigation into how Edward Snowden used cryptographic keys and digital certificates to breach the NSA. Kevin has a B.S. in chemistry from the College of William and Mary and an MBA from Wake Forest University.

Jyoti Bansal is CEO and Co-Founder of Traceable, a serial entrepreneur and a silicon valley technology visionary.Bansal believes passionately in software’s ability to change the world for the better. In 2008, he founded AppDynamics, an application intelligence company that provides enterprises with real-time insights into application performance. Bansal led the company as founder & CEO for the first eight years, and as founder & Chairman for the last year until its acquisition by Cisco for $3.7 Billion in January 2017. He is founder/CEO of BIG Labs - a startup studio aiming to co-create companies that can help define the future of software and technology. He is also cofounder/CEO of Harness - the leading continuous-delivery-as-a-service company, and cofounder of Unusual Ventures - a leading early stage venture capital firm.

Brian Fox is a software developer, innovator and entrepreneur. He is an active contributor within the open source development community, most prominently as a member of the Apache Software Foundation and former Chair of the Apache Maven project. Fox’s contributions to the open source development community include the creation of Nexus repository manager, maven-dependency-plugin, and maven-enforcer-plugin. He is most well-known known for his role as the CTO and co-founder of Sonatype. In his role at Sonatype, he is focused on building a platform for developers and DevOps professionals to build high-quality, secure applications with open source components.