The track Optimizing Teams for Fast Flow - Surviving in the Post-agile Aftermath at QCon New York 2023 comprised two talks in the morning that went into replacing an agile process with engineering and conversational software delivery using AI.
Bryan Finster gave a talk about Agile Rehab: Engineering for Improved Delivery.
The biggest problem in software delivery is applying solutions to problems we don’t understand, Finster said. If you want to use agile practices, first think about why you want to use them and what’s wrong that needs to be solved, he said. The main problem they had was uncertainty, where they needed to engineer solutions to deliver better value.
The approach taken was to descale for delivery using domain-driven design. They decomposed their system into subdomains, define interfaces, and looked at the dependencies. Next, they did a reverse Conway maneuver to shape the organization toward the desired architecture. They assigned parts of the system to product teams and created a small platform team.
To drive improvements, they defined practices, Finster explained. These practices were that parts of the system should be independently deployable, where coordination is done through API. They also mandated a 90% test coverage level, but that didn’t work out as expected, Finster said. It resulted in many meaningless tests. He stressed to avoid that.
To implement continuous delivery (CD), Finster created a dependency map of the practices described in the book. This map showed which practices should be in place to enable other practices. Finster mentioned that CD has some automation, but it is mostly about behavior, he said.
The approach taken was called "minimum viable process." They started with no process and only added what added value, Finster mentioned. Some examples were refining work as a team, pairing, and synchronous code reviews. They stopped doing retrospectives. Instead, they had daily inspections and improvements.
Don’t mandate code coverage, Finster said. Their testing "sucked" and tests were flaky while having 90% coverage. They had pointless unit tests and testing lagged development. Their main problems with testing were a lack of testing knowledge and vague requirements.
They started with BDD, defining requirements as acceptance tests, which turned out to be powerful for the problems they had with requirements. They aimed for thin slices of work, something that could be completed in less than two days.
Finster mentioned that they designed their pipeline for operations. CD is there to respond to incidents and safely solve problems in operations, he said. They always use their emergency process.
One outcome of the approach that they took was that they became dependable because all of the work was small. Another outcome was that their engineers loved development again.
Finster mentioned some of the lessons learned:
- Use agile, but only what makes sense
- If it can be done with engineering, do it that way
- The talent is there; people need to have the right problem to solve
- Take CD seriously; start with continuous integration to make it work
Lee Rehwinkel and Katharine Chajka delved into how artificial intelligence (AI) can help enhance the software development process by adopting AI as a Value Stream Agent in their talk Leveraging AI to Identify and Address Inefficiencies in Your Delivery.
Chajka demonstrated a value stream map, revealing that only a small portion of organizations’ time (8%) is spent on actual code writing. She argued that even if AI were to minimize the code-writing time to zero, it would barely affect the overall timeframe of developing new software features. This is because most time in the software delivery process is consumed by planning, designing, testing, and releasing tasks.
Going beyond code-writing, Chajka showed how AI can also play a significant role in managing the value stream. She proposed that AI could aid in identifying bottlenecks where work isn’t progressing smoothly and take measures to streamline the delivery process.
Rehwinkel expounded on how AI can eliminate technical jargon from value stream management, making it more accessible across the organization. AI enables team members to communicate in layman’s terms, thereby avoiding technical language that might be only understandable to those with a background in value stream management.
Rehwinkel demonstrated a functional example of an AI Value Stream Agent using ChatGPT. The example began with a typical executive query like "what should I be worried about". ChatGPT was able to scrutinize flow metrics, identify trends, and generate an analysis. This analysis, although detailed, was loaded with technical jargon. Rehwinkel then used a second ChatGPT step to create an executive summary that translated the information into more understandable terms. He showed how ChatGPT can also serve as a "Truth Checker", ensuring that the drawn conclusions align with the underlying data.
Rehwinkel highlighted some challenges his team faced while developing a production application using this methodology. The issues include time-consuming ChatGPT response chains, which could take up to several minutes and thus lead to a subpar user experience. He also noted that these AI models are susceptible to "jailbreaking", where users can manipulate their queries to alter the designated role of the ChatGPT model.
