In their recent survey, OverOps, a continuous reliability platform vendor, found that within DevOps investment initiatives, organizations invest the most towards enabling the constant flow of software development. The survey revealed that practices like continuous integration (54%) and continuous delivery (42%) are among the top areas of investment. The "2020 State of Software Quality" survey also showed that automated code analysis is being adopted more as engineering teams embrace cutting-edge technologies and practices.
The survey was taken by 600 IT professionals which included developers, QA professionals, DevOps engineers, and site reliability engineers (SREs). The majority of respondents mentioned that they release new code/features anywhere from once a fortnight to multiple times a day. This shows a trend to move fast, whereas the important components of software quality success like testing and observability were reported at 34% and 21% respectively.
Surprisingly, the majority of survey respondents (70%) indicated that quality is paramount over speed for them. This highlighted that they prioritized testing as much as possible in pre-production so that code is of the highest quality before they release it, even if that means slight roadmap delays.
Herb Krasner, an advisory board member of the Consortium for IT Software Quality (CISQ), stated, "The move to DevOps and the increasing velocity at which teams are delivering software is unprecedented. At the same time, they are dealing with much higher levels of risk and vulnerability."
Emphasizing this point, survey participants further revealed that improving code quality (60%) and increasing developer productivity (52%) are primary goals. Reducing Mean Time to Identify/Restore (MTTI/MTTR) found its place near the bottom of the list.
Reflecting on the next area of interest for their organizations, DevOps and the topic of microservices were reported in 58% and 45% of responses, respectively. Static and dynamic code analysis techniques are gaining popularity, striking out the need for manual processes that are prone to errors. Other techniques such as observability, SRE, "shift left", and "shift right" are also in this list.
However, 45% of respondents said they were under pressure to move fast, leading to insufficient time to ensure quality. Lack of automation (48%) is reported as the number one challenge for application quality, while lack of visibility, data, and metrics (25%) followed by inefficient monitoring processes/tools (29%) were among the top challenges.
Over half of the survey respondents revealed that they encounter critical or customer-impacting issues in production at least one or more times a week. This appears to correlate with the findings in the GitHub's DevOps Survey where 47% of developers stated testing is the number one reason for release delay, with bugs being found late in the release process.
As a result, over 40% of critical production issues are first reported by end-users or customers rather than the internal monitoring and alerting tools. This leads to two out of three software developers spending at least a day per week troubleshooting issues in their code.
When posed with questions about finding the root cause of the recurring production errors, 34% of respondents stated that they believe they don't have the right processes/culture in place. Lack of team members (24%) and relying on foresight/lack of sufficient tooling (22%) were among the other reasons that errors unintentionally make it to production.
Looking at the logging and alerting, log management (63%) and alerting (61%) tools were preferred in the production pipeline. To ensure quality during the software delivery life cycle, the majority of survey respondents relied on a mix of manual and automated testing.
For evaluating the quality of their applications, most of the participants indicated that they relied on service uptime (42%) and unit test pass/fail percentage (41%). This provided a notion that there is no universal standard for assessing software quality.
The complete OverOps 2020 State of Software Quality report can be downloaded via the OverOps website (registration required).
