Interfacing between Linear Waterfall and Agile Approaches

This article depicts the best practice approach for integrating Agile approaches and specifically Scrum development with traditional overarching linear approaches, specifically waterfall methodology. The Agile PMO, properly defined, can be positioned to secure Agile-Scrum benefits while maintaining the necessary overarching control.

The challenge

Over the last two decades, various Agile approaches have been introduced and practiced. Of these, in the last 5 to 7 years, Scrum has gained the most popularity resulting from a combination of simplicity, ease of use, and effective public relations. Scrum success in software development organizations has been a powerful driver for roll outs across products, industries and businesses. It was exacerbated by focused marketing efforts on part of Scrum evangelists. Unfortunately, most of these organizations were not structured in a way that supports Agile-Scrum. Even more so, scrum in its raw and pure form is not suitable for the majority of organizations.

The first wave of failed Agile-Scrum implementations brought about an even stricter admonition. The main assertion of Scrum zealots has been that failed implementations are caused by partially embracing the true scrum spirit. The full benefits of the approach can only be attained if the entire organization is reengineered. This fanatical attitude left many project teams across organizations big and small, struggling with their already idealistic implementations. Some have been figuring out on their own, how to combine the contemporary and traditional worlds. Other teams have completely abandoned the Agile-Scrum concepts reverting back to the traditional linear waterfall approach and method. Yet other teams, ridden with guilt, manage Scrum by name only, and hiss vehemently at any project management proponent who is unfortunate enough to advise on re-embracing Agile in a more cognizant approach.

The concepts which are presented and embodied in Agile-Scrum are too good to be ignored; however implementing it outside pure software development requires adaptation.

Complex scenarios for Agile

The main hurdle in achieving the benefits of Agile- Scrum outside software development is integrating it with existing control mechanisms. These control mechanisms are stipulated by various organizational prerequisites and are normally actualized implementing Linear Waterfall methods. Four of these typical organizational prerequisites are depicted below:

• Big global corporates – in these hierarchical matrix organizations, top down portfolio control is the rule of the day. The free spirited agile approach has a tough time adjusting to the rigorous controls. Specifically the inherent Agile plan-free concepts, make Agile-Scrum difficult for the organization to swallow.
• Highly regulated industries – some industries are required by compliance and governance bodies to have a strict binding control mechanism. These can be for example medical equipment, aircraft, and pharmaceuticals research and product development business units. While individual teams might operate Agile-Scrum, the development process must follow rigid Linear Waterfall methods for traceability and governance.
• Complex predefined products –integrated products which include hardware, software, and imbedded are developed as a contract with an end customer based on predefined requirements. In these cases the degree of requirement flexibility is small, though larger than what is anticipated initially. Agile-Scrum concept of a fully flexible backlog suffers considerably in these cases.
• Generic IT departments – much of the daily and weekly activities in maintenance driven IT departments is ad hoc. Changes to the daily schedules are numerous and immediate. Constant interferences with the team work are the norm. The concept of time boxing and no interference is difficult to maintain in these situations.

Naturally – many times the four discrete categories detailed above, actually mix; so it is common to find a complex product in a global big corporate which is required to comply with firm regulation.

Based on practical experience, the recommended approach to tackle these scenarios is by empowering the Agile PMO; it acts as an enabler, driver and translator between the emerging Agile-Scrum teams and the Linear Waterfall elements.

Refer to the table below for specific guidelines

The Agile PMO – leading the hybrid organization - guidelines

Our work with global organizations developing and delivering mobile technology is the basis for two of the above recommendations. We consulted the specific company regarding their product delivery challenges, or should I say problems… their products were late, over budget and delivering reduced scope. Not unlike situations with many product development, IT and software organizations.

The project organization rolled out Scrum method two years prior with limited success. They invested in training and coaching however where at a loss, on connecting Scrum to portfolio management and reporting. This resulted in scrum teams losing alignment with the strategic business objective, a nasty side effect of Scrum implementations. The software Scrum teams were also misaligned with the hardware elements of the product, which were managed as part of an overall program.

We scheduled an initial meeting with approximately 16 stakeholders: product owners and project managers. As we entered the room, it seemed there was an unseen yet tangible wall dividing the room, on one side the Scrum product owners and a few Scrum masters, on the other side project managers and PMO members. During the meeting, as we were gathering information, invisible daggers flew - each group was criticizing the other for the delivery blunders. Members of the Scrum teams accused the ‘traditionals’, as they referred to them, of interfering with their process: changing the requirements, moving people away from the project, enforcing the change control process, and showing up on morning stand ups blaming team members for miss delivering. Members of the project management organization and specifically the PMO said that the Scrum fanatics as they called them, were late with their deliveries, inflexible with changes, and couldn’t provide reasonable progress and status reports.

While we knew that the only way to solve the problems was to integrate the methods, we first had to create an environment of trust and collaboration. We identified a PMO champion that was accepted by both teams as reliable and trustworthy. Together, we developed a change plan, using Kotter’s eight steps approach as a guideline. We searched for the low hanging fruits and found two: increasing transparency of the Scrum weekly achievements and protecting Scrum teams from management interference during sprints. For the second ‘low hanging fruit’ we introduced adaptability to time box durations, suggesting a longer Sprint that would later be useful for integration of Hardware program elements with Scrum. This caused push back from several Scrum zealots, and as I explain elsewhere, you really can’t please all, thus we had to operate with voiced objections throughout. Objections also arose from the ‘traditional’ project stakeholders who didn’t appreciate being told to stop meddling with the projects, not allowing the teams self-manage. By then we had been able to secure support from a kernel group of advocates and exhibit tangible progress. We were diligently working with the PMO champion on translating burn down charts to phase gate view for control purposes. We defined requirement traceability, the backlog of user stories and incorporated the dictionary, translating between sprint planning, execution and the top down Waterfall life cycle. We situated the PMO as a transformation layer and renamed it: the Agile PMO. The defined activities of the Agile PMO alleviated the requisite for strict controls mandated by the global corporate, ensuring the benefits of a Scrum process for project delivery. Our next steps revolved around the hardware elements of the product; we formed a multidisciplinary team of subject matter experts with the goal of enabling Scrum concepts in hardware projects. Most of the work was analyzing the components and process of the hardware project, figuring out process points which could be accelerated. The received benefits were numerous; we defined a Kanban process for hardware scope management and reduced projects duration and costs accordingly. In a year subsequent to launch, we were exhibiting improved results. The Agile PMO was a key differentiator and had a pivotal role in contributing to the success of project and product delivery.

Important best practices to remember that support the concept of an Agile PMO:

• Implementing Agile-Scrum as a restricting admonition is exploiting the adaptive nature of Agile;
• There is no one right way – no one size fits them all;
• There is no silver bullet – you can create what works for you. Kanban is a great tool to use;
• Being agile and adaptive also allows being flexible with how one uses the methods, process and Methodology;
• Time boxing is great as long as you are flexible to changing the durations of the time box if necessary;
• Sometimes the client isn’t directly available, in these cases marketing and product management are a proper alternate;
• Arbitrary rules don’t complete projects, people do! Empower your team and yourself to choose the appropriate mix of approaches that enable product delivery.

Summarizing

In my book about the Agile PMO I describe how PMOs can succeed only if they create and enhance value through smart portfolio selection (more about that in a future white paper). With the emerging of Agile approaches and specifically Scrum methods new opportunities have become apparent. Integrating them into an existing control structure – typically presented by a waterfall lifecycle – can be frustrating. We have defined a new key player – the Agile PMO which can be positioned to create a transformation / translation layer between the approached and methods, contributing to higher success levels of these integrations. Michael Nir, President, Sapir Consulting, can be reached at here.

The above whitepaper is excerpted from: The Agile PMO, in print. Join Michael for a 2 day workshop in location globally. The odd couple – Marrying Agile and Waterfall. Learn Breakthrough tools and techniques for integrating Agile in a Waterfall environment; immediately increase value from the AGILE PMO in hybrid setting and Increase Agile / Scrum hybrid implementation success.

About the Author

Michael Nir - President of Sapir Consulting - (M.Sc. Engineering) has been providing operational, organizational and management consulting and training for over 15 years. He is passionate about Gestalt theory and practice, which complements his engineering background and contributes to his understanding of individual and team dynamics in business. Michael authored 8 Bestsellers in the fields of Influencing, Agile, Teams, Leadership and others.
Michael's professional background includes significant expertise in the telecoms, hi-tech, software development, R&D environments and petrochemical & infrastructure industries. He develops creative and innovative solutions in project and product management, process improvement, leadership, and team building programs.