Key Takeaways
- When faced with a problem, people in technology companies often feel pressured to jump to a solution quickly
- These solutions may work but are often not optimal and may have unintended consequences
- Applying a rigorous approach to formulating problems, diagnosing the root causes, performing relevant corrective actions, evaluating the effects, helps create a better understanding of the work and improves day-to-day job practices of people
- Lean thinking brings a structured approach to exposing and removing waste in the development process
- Different categories of waste require different approaches to resolving them
Today tech companies are facing the challenges of the current digital age: increasing user expectations, technology disruptions and fierce competition. Managers and IT workers tend to proactively push technology solutions in order to solve problems without being sure of the effects and they often fall short…or sometimes options are never evaluated. They lack a proper and rigorous approach to formulating problems, diagnosing the root causes, performing relevant corrective actions, evaluating the effects, and at the end create a better understanding of the work improves day-to-day job practices of people.
In this article, we will meet Marek, the CTO of a French mobile development company and Kevin a developer. Through patient observation of a developer’s work and how the software pieces were flowing or being stopped, Marek revealed a waste in the validation & deployment process, which seriously impacts developers’ productivity.
The story continues with the attempts, fails and successes of Kevin’s actions following the structure of the method tool at the core of lean: the scientific method for problem solving, Plan-Do-Check-Act method (aka PDCA).
Through this detailed real-world example, you will see how lean management posture and problem solving concretely help agile teams: this mobile application agile team succeeded to improve its productivity by 15%.
Mobile solutions partner
BAM French company based in Paris delivers mobile solutions and expertise to corporate and startup enterprises to helping them to tackle their business challenges. BAMers work to provide value to their clients as efficiently and quickly as possible, using Lean Startup approach and Scrum method with 1-week sprints.
Deploying an application is really time consuming, and costs money.
As lean engineers, BAMers follow a nice chain of steps to create this value:
- The product owner (PO), considered as the Client, formalizes a feature in a Trello Card (Trello is the software tool used as digital taskboard).
- The PO prioritizes the new features.
- Each week, the technical team commit to develop a set of features.
- Each feature follows its own steps before being considered "Done":
- A developer takes the ticket and work on it. He/She is now responsible of this feature and will have to get this ticket validated by the PO.
- The developer codes the feature, the tests, refactors the code, etc.
- Once the code is written, it is reviewed by at least one other team member.
- Once their feedback has been taken into account, the ticket is immediately deployed on the staging platform.
- The developer in charge of the ticket now checks on all the devices specified in the "definition of done" that the developed feature is working accordingly to the feature description written by the PO in the ticket.
- The developer informs the PO that this feature is developed and can be tested in a new version of the application.
The crucial part here is that there is no stock, no batching. Each and every feature goes directly on the validation environment, and ideally in production once validated: BAMers use the One Piece Flow technique. BAM’s practices discourage starting other tickets before checking the previous feature on the devices.
If the feature does not match the ticket, or if a bug is spotted, the dev has to fix it right away: he/she has to act quickly in order not to jeopardize the validation process.
The deployment of features is a critical process for BAM.
Let us take a real-world project: we worked for an energy company which wants to digitalize its whole business, from the subscription to the billing and the customer care. The BAM team is composed of three developers and one developer-architect, all full time.
Here is a problem solving approach using the scientific method (PDCA, Plan Do Check Act) which demonstrates the benefits of this approach.
P for Plan
What is the problem?
Marek, CTO of BAM who is coached by Marc (Operae Partners), during a Go & See on the project, noticed the deployment and validation time of a feature was very long, around 20 minutes. Every member of the team spends around an hour a day deploying features to validation devices: this time needs to be reduced at deployment time.
Historically, BAMers tried externalizing the whole building process: merging a feature would automatically start a new deployment.
So they used Bitrise which is a specialized mobile deployment service. It worked well, but had several major drawbacks. First, the whole build environment has to be recreated at each deployment. It is automatic, but really slow: the whole deployment and validation on Bitrise was lasting around 35 minutes!
Building on Bitrise or CI was slower than on development machines.
An automatic build process seemed like a good idea at first, because it enabled the developer to start another task while waiting for the deployment to finish. The downside of this is that they lost One Piece Flow and increased the WIP (work in progress).
Starting a new task while the deployment was still running was not efficient: the developers had to stop working on the new feature in order to test the previous one, maybe fix a bug or two that were not spotted in the development environment, etc.
Eventually, BAM completely stopped deploying using Bitrise (35 minutes to deploy) and went back to the first, manual approach (around 20 minutes, deployment and testing). The developer had to wait during the deployment, but he/she could do other pending tasks (reading and answering his/her emails, update the problem-solving sheet).
At BAM, there is no standard maximum time for deploying and testing tickets: on Marek CTO advice, the target to reach was 10 minutes (see figure below).
What are the impacts?
Sprint |
Points |
Total features |
Time lost per week (17 vs 10 min) |
Sprint 9 |
147 |
51 |
5h 57min |
Sprint 10 |
98 |
34 |
3h 58min |
Sprint 11 |
158 |
57 |
6h 40min |
For BAM’s client which contracts with BAM in time & material, the equivalent of one developer-day per week is wasted in waiting for deployments to finish.
And the client must absorb these wastes: each time a feature is deployed, he/she has to update the application through many manual steps (motion waste): the installation process takes around 2 minutes on his devices (one Android, one iOS), and repeated once or twice a day.
For the developers, this deployment time is a waste and an unpleasant process.
For BAM as an enterprise, this means a waste of productivity: 1 man day a week is spent deploying, which means 2 or 3 user features not developed during the sprint for the client. At the end of a project, this means 5% less features: for a 10-sprints project, it means between 20 and 30 features not implemented!
What is the standard process?
Now let’s look at the sequence of steps required to build a mobile application:
Ideal Step |
Ideal Time |
Build the updated application |
1’’ |
Install the update on the devices |
1’’ |
Manually test the feature on production devices |
Dependent on the feature |
Notify the client that the new feature is available |
1’’ |
Kevin, a mobile application developer, took a time watch and measured the time spent by each action required to deploy one feature:
Ideal step |
Step |
Waste family |
Time needed |
Build |
Android build |
Waiting |
2’08’’ |
iOS build |
Waiting |
4’44’’ |
|
Install |
Walk to the device lab and unplug the devices |
Transport & Over-processing |
30’’ |
Install the update on all devices |
Motion |
3’16’’ |
|
Test |
Test the feature |
4’16’’ |
|
Install (back to initial state) |
Tidy the device lab, re-plug the devices |
Transport & Over-processing |
1’25’’ |
Notification |
Back to computer, login, Trello, move the card |
Transport |
45’’ |
The steps Kevin wanted to improve were the build and installation steps.
But what are the causes?
Kevin identified two main causes for those slow builds.
The first one is the build time, almost 7 minutes. Currently the whole application (binary packages and JS source code) was rebuilt twice, for Android and iOS. The binary part is almost always the same during the development of a feature, but rebuild is done anyway: it is an over-processing waste.
The second cause identified was the installation, as a motion waste. 3 minutes are needed to update the application on four devices. During this step, developer has to:
- launch the HockeyApp application (private app store for developers);
- refresh the application list;
- find the application;
- download the whole application update;
- install the whole application update.
Ok, now what actions to take?
Regarding the application build, a new process is required which would allow rebuilding only the updated code without recompiling the whole application. This may require a new build tool.
Regarding the installation time, developers need to install only the updated code on the devices, without re-downloading a full update for each feature.
Kevin saw several interesting things in CodePush tool:
- The build time: instead of rebuilding native modules, it just bundles the assets and JavaScript code.
- The installation time: it downloads only the updated JavaScript, and not the whole 10-20Mb application.
For what expected results?
The expected result of this new build process was to respect this new standard/target, that is to build, install and test new features on devices in less than 10 minutes, including the 4 minutes of feature validation time.
In order to achieve that result, the build and installation time needed to be cut by half (going from 13 to 6 minutes).
Do
Kevin installed the Codepush native module with the default auto-update feature, rebuilt the native application, installed it on the devices, and pushed an update.
First impressions: the build time was faster; the installation time feels faster too. Kevin kept it, and continued to do soft deploys for a week.
> It was quite horrible!
Why? The developer would never know when the application was being updated, what version of the code it was running. The developer just started the app, waited for the update which would never come, killed the app, get an update...
So Kevin improved the CodePush integration in the app: he added an update button with the installation status, and he displayed both the CodePush version and the id of the deployed commit.
Manually installing updates allowed developers to really feel that they were in charge of the update. Displaying the version deployed also allowed them to improve their communication with the Product Owners: when a bug appears, they know the exact commit used to detect it.
Check
Here are the results:
- The build time went from 10 min to around 5 min.
- The installation time went from 3 min to under a minute.
- The installation being faster, developers often keep the devices plugged to the device lab: they gained another 30 seconds or so at this step.
Overall duration went from 35 min with Bitrise, to 17 minutes manually, and eventually around 9 min with CodePush
Act
By solving this problem, Kevin created several new “work standards”. A standard for maximum build, install and testing time, and a new deployment process to respect this standard.
The first action he took was to talk about this problem on a horizontal scale: Kevin did a Yokoten session where he shared this problem and the proposed solutions.
The second action was to write the new process for implementing CodePush in several apps. Every new employee at BAM now is aware of this and trained in the initial training sessions.
The final action Kevin took was to run a “CodePush Dojo” and encouraged everyone to come and install CodePush on their app with his help.
If we focus on the lean approach we can describe the following sequence:
- Go & Sees from the CTO are crucial to make developers see the wastes they have to undergo, and help them identify those wastes by themselves.
- Be careful with “silver bullet” automatic solutions which tries to solve many problems at once (Bitrise).
- Focus on the one-piece flow by avoiding having multiple pieces of work in progress.
- By reducing lead time, you identify and quantify wastes to be removed or reduced.
Conclusion
Facing the challenges of the current digital age, managers and IT workers tend to proactively push technology solutions in order to solve problems without understanding what is going on in detail and without really checking if the situation has improved.
As a manager or worker, you should definitely step back and make your people think about how to improve their work. The Lean system has been proven to be a structured approach to help you and teams to succeed. And in the IT world, it’s called Lean IT and it will definitely help you identify meaningful problems and help you tackle them.
About the Authors
Kevin Raynel is an architect, software developer, and coach at Theodo and previously BAM. He has developed mobile and web applications for the past five years using a variety of technologies. In addition to the technical part of his job, he is focused on helping his company and his clients develop applications in the most efficient way possible with lean thinking and problem solving.
Marc Legru is Agile/Lean IT coach for CEO/CTO, managers, team leaders and teams. He helps CEO/CTO startups to grow in a sustainable way. He is partner at Operae Partners, a consulting firm specialized in Lean IT and Lean Services for startups and traditional organizations. Marc has authored the first French Lean Management initiation MOOC with the French startup UNOW. Follow him on twitter: http://twitter.com/MarcLegru
Stéphane Wojewoda is helping people and corps to grow and develop themselves in a sustainable pace. He likes to keep up with the last trends, in tech & culture. He's happy when the teams he works with reach the point where they fell they understand why they do what they do, and they achieve to do the right thing right, at the right time, with just enough effort.