In this podcast Manuel Pais, InfoQ Lead Editor for DevOps, talks to Alanna Brown Director of Technical Product Marketing at Puppet and Nicole Forsgren, PhD in Management Information Systems and CEO at DORA, on the State of DevOps Report 2017.
Key Takeaways
- Three new areas of research in 2017: leadership, automation and organizational performance for non-financial organizations.
- Transformational leaders have a clear business vision and communicate in an inspiring way, and provide intellectual stimulation, care for their followers' needs, and praise accomplishments.
- High performing not-for-profit organizations (such as government) are twice as likely to achieve their goals, just like commercial organizations.
- Medium performers are in the middle of the J curve effect where performance initially improves (via quick wins) but then gets worse (as technical debt surfaces) until it definitely improves again (for those that resist reverting to old ways).
- Survey data analysis involves rigorous statistical integrity checks, followed by data correlation, prediction and inferencial tests to gather new insights.
- C-level executives have to chose between ignoring technological transformation or leveraging DevOps to keep their organization competitive via improved technical practices and a culture of continuous improvement.
Subscribe on:
Notes
Evolution of the State of DevOps Report and Team
- 0:44 - Gene Kim joined the survey team in 2012, who then invited Jez Humble to join in 2013.
- 1:05 - Over 4.000 survey responses in 2012, making it the largest of its kind at the time.
- 1:20 - Nicole Forsgren joined the team for the 2014 study and she brought IT knowledge depth and academic rigor.
- 1:50 - Gene, Jez and Nicole formed DORA in 2015, who now produce the yearly report in collaboration with Puppet.
- 2:43 - DORA stands for DevOps Research and Assessment and offers tools for organizations to benchmark themselves on their technology transformations.
What's New in the State of DevOps Report 2017
- 3:25 - Every year the survey design is improved and new questions are added.
- 3:35 - Three new areas of research in 2017: leadership, automation and organizational performance for non-financial organizations.
- 4:10 - A common trait of the most successful DevOps transformations is a highly engaged leadership team.
- 4:45 - Transformational leaders have a clear business vision and communicate in an inspiring way, and provide intellectual stimulation and care for their followers' needs, and praise accomplishments.
- 5:25 - Transformational leadership characteristics are closely linked to high performing teams.
- 5:39 - Leaders have strong influence over teams' technical practices and new process adoption, which in turn lead to improved performance.
- 6:32 - High performing, DevOps driven, commercial organizations are twice as likely to exceed profitability, productivity and market share.
- 6:45 - This year's survey measured performance of not-for-profit organizations like government or socially-focused.
- 7:10 - High performing, DevOps drive, not-for-profit organizations are also twice as likely to achieve their goals, such as delivering quality products; operating them efficiently; high customer satisfaction; and achieving mission goals.
- 7:35 - DevOps is a real differentiator and value driver for all kinds of organizations, in all industry verticals.
- 8:15 - Tranformational leadership is highly correlated with employee net promoter score.
Survey's Results on Automation
- 8:50 - High performing teams are by far more automated across technical practices such as configuration management (33% more than all the rest), test automation (27%), deployment (30%) and change approval process (27%).
- 9:30 - Medium performers do more manual work around deployments and change approval processes than low performers. Also last year's results showed medium performers were doing more unplanned work and rework than low performers.
- 9:52 - Medium performers are in the middle of the J curve effect where performance initially improves (via quick wins) but then gets worse (as technical debt surfaces) until it definitely improves again (for those that resist reverting to old ways).
- 11:45 - As we uncover additional complexity, progress becomes more difficult. But if we keep improving our architecture and technical practices our overall performance gets better.
Survey Mechanics and Data Analysis
- 12:50 - Research design process is hypothesis-based. Questions are well thought out in advance to validate or not those hypothesis.
- 13:20 - It's a challenge every year to decide which questions to keep, which to expand on and grow, and which to exclude to keep it at 15-20 minutes.
- 13:55 - Survey data goes through a process of cleaning, scrubbing and an initial set of rigorous statistical reliability tests such as discriminate validity, conversion validity and internal consistency and bias checks.
- 14:42 - After the initial checks, data correlation, prediction tests and inferencial tests are performed to gather new insights.
- 14:58 - The data analysis might then reveal new insights such as the J curve effect on medium performers, which in turn might be validated (or not) by other data, in an iterative process.
- 16:50 - Bias checks include early vs late responders bias, single source bias. Variability is introduced into the data to ensure it's not biased.
- 18:35 - Diversity in data is achieved partially thanks to a large initial target responders (aggregated from email lists from Puppet, Gene and Jez, and academic list).
- 19:40 - This is the only report in our industry that goes through all these statistical controls.
- 20:50 - Survey's target population is self-selecting DevOps practioners, the goal is not to assess who is doing DevOps, but who are the best performers from those that are already familiar with DevOps.
- 22:33 - Goal is to know what practices are predictive of improving software delivery and organization performance among those with similar-ish work.
- 24:07 - Survey is under represented in terms of female participation (5-6% while sysadmin community historical data shows 9-10%, and globally in Computer Science women represent around 20% of population).
- 25:52 - In the 2017 survey 12% of participants self identified as belonging to an under represented group.
Impact of DevOps as Demonstrated by the Report
- 27:15 - DevOps shift is happening, whether or not people buy into it.
- 27:20 - Gartner predicts by 2020 half of the CIOs who have not initiated a DevOps transformation will be asked to leave.
- 27:40 - C-level executives have to chose between ignoring technological transformation or leveraging DevOps to keep their organization competitive via improved technical practices and a new culture of work.
- 28:45 - Some people will have misguided expectations on how long this kind of transformations can take.
- 29:09 - Amazon took 4 years to re-architect and implement their digital transformation.
- 29:27 - We need to have realistic expectations on the effort required and embrace the continuous improvement paradigm.
- 29:52 - We are on an improvement journey, there is no such thing as "being DevOps" by a certain date.
DevOps Enterprise Summit London
- 30:25 - Nicole, Jez, Gene and Nigel Kersten present key findings, surprises and behind the scenes on the State of DevOps 2017 report.
- 30:53 - Gene is keen on the transformational leadership part, while Jez is keen on the lean product management part.
