One year after the official launch of GitLab public bug bounty program, it is time for the company to wrap up its results and determine how it helped improve security for GitLab and its customers.
GitLab bug bounty program brought impressive results, says GitLab, with hundreds of valid bug reports which were awarded over $500,000.
The program kept our engineers on their toes, challenged and surprised our security team, and helped us keep GitLab more secure.
Taking their program public allowed GitLab to learn a number of lessons, from how to triage reports and respond to them, to how to improve communication and transparency. In fact, the company says, they are still learning and actively looking for ways to make their program more effective.
InfoQ had the chance to speak with GitLab senior application security engineer James Ritchey to learn more about GitLab's security strategy and what a bug bounty program can contribute to an organization.
InfoQ: When it comes to security, GitLab has long adopted a policy of transparency and open disclosure. Could you please elaborate on the value this has for GitLab users and investors? Aren't you worried that disclosing the number and severity of vulnerabilities may impact GitLab's image negatively?
James Ritchey: GitLab as a company holds transparency as one of our core values, because among other things it helps to build customer trust. This extends to our vulnerability program as well. We believe sharing the details of the vulnerabilities that are found and fixed helps build and maintain trust with our customers by demonstrating the importance of building a secure product.
Trust can be lost when a security bug is mishandled and the customer is directly impacted. By disclosing full vulnerability information after 30 days, the customer has time to upgrade or apply mitigations to their environment to protect against those vulnerabilities. Being transparent about our security issues truly illustrates how invested we are in securing GitLab.
InfoQ: In the last few years, the software industry has been constantly improving the way it deals with security and how it protects its systems from vulnerabilities. For example, GitHub provides Security Alerts and code analysis to help developers detect risks as soon as possible. How does GitLab attempt to improve the way developers ensure their software is secure?
Ritchey: GitLab’s focus on enabling the developer to write secure code is reflected in our Secure product team. This team works to build tools that provide security-related information to the developer as soon as their code is pushed to GitLab. This includes dependency analysis to identify vulnerable dependencies and static source code analysis that identifies common insecure coding patterns.
The security team has also contributed tooling in this area. Our red team recently open-sourced a tool called Token Hunter that helps developers discover sensitive data in other areas of GitLab, such as issues or snippets.
InfoQ: A recent report by security firm Snyk highlighted the lack of security ownership among open source developers as one major cause of concern. How does GitLab make sure its developers are aware of their critical role in shipping secure software?
Ritchey: Security awareness communication for developers at GitLab starts at onboarding and continues through their time working on the product. When a developer is onboarded they are provided information on where to find secure coding resources. This includes videos of past secure coding training and a set of guidelines for common vulnerabilities.
Additionally, the application security team has different initiatives throughout the year. One example is bi-weekly office hours that anyone can attend and discuss application security related questions. Another, which will be a first, is that we are planning a Capture The Flag event at this year’s Contribute, our annual community event. We get together to get face-time with one another, build community, and get some work done.
InfoQ: A year ago you launched a public bug bounty program that awarded security over 150 researchers, a total of about $560,000. Could you please summarize how this program performed for GitLab and what you learned through the process?
Ritchey: We received a huge response from the community over the past year, and during this time we learned a lot. One of the most important lessons was that we needed to scale; specifically scale our communication and procedures. There are many reports and reporters and only a handful of us on the GitLab side. If we didn't scale, then we'd be smothered in the volume of reports. Our answer to this was to automate as much as possible. For example, an automated bot enabled us to reduce our average time to first response from 48+ hours to less than seven.
Another automation example was that for issues we’ve already triaged and are tracking, reporters would very often ask for updates on when the vulnerability would be fixed. So our automation team developed another bot which would comment on the report with an expected fix date once the issue was scheduled by our product team.
Besides scaling, another significant lesson was that we needed to increase hacker engagement and keep it at a high level. There are many programs which the reporters can choose from on HackerOne, so why should they participate in ours and stick with it? You’re competing for the attention of reporters from over 1000 other programs to choose from. So an important thing for us was to listen to the feedback from reporters currently engaged in the program.
One of the top suggestions was that they wanted to reduce the time of bounty payouts. Previously, we were rewarding bounties once the issue was resolved, which could take anywhere from 1 to 4 months depending on the severity of the issue. So after listening to the feedback, in September 2019 we changed how we reward bounties to pay a partial bounty of $1000 upfront at the time of triage. The remainder would be paid when the report was resolved or 90 days had past, whichever came first.
InfoQ: Establishing a bug bounty program is a key approach to improving the security of a Cloud service. Could you share some suggestions on how to create and run a successful bug bounty program?
Ritchey: I suggest learning from our lessons and other programs which have been around for a while.
- Start a public program, and start it sooner than later.
- Scale with automation.
- Keep hacker engagement high in your program.
- Pay competitively, especially for High and Critical severity vulnerabilities.
- Pay fast.
- Have a large enough scope.
- I’d also encourage you to be transparent about your security issues. Why?
- It can show how dedicated you are to securing your service, thus instilling trust in your customers:
- Exactly what were the issues.
- When the issue was reported.
- How fast you fixed the issue.
- By staying secret about all of these things:
- There’s no accountability.
- There’s no visibility into how committed they are to securing their service
GitLab is a Cloud-based platform for development and DevOps encompassing source code management, wiki, issue tracking, continuous integration/continuous development, and more.
