This article is part of the Agile Manifesto 10th Anniversary series that is being published on InfoQ.
It is human nature to look for patterns in solving problems and we have a tendency to want to reuse what we already know. Sometimes this is a useful tactic but often the problem we face is not the same as the previous problems so our previous way of doing things is not best suited for tackling the next problem. In the world of software development we often see the same desire to find the “one true way”, “the BEST method”.
Interestingly, most often, our better approaches are based on our knowledge and experience from the past. In other words, we rarely discard what we've already learned; we simply build on top of it. Its time to move on and create new approaches, if not new belief systems.
In the last few years, when I think about Agile, I've come to accept that I've moved on (building on top of it).
With my time-machine™, if I went back a decade, my journey to “nowhere” (a place beyond a prescribed set of rules), started with some core XP practices, then grew to broader Agile cultural foo, then Lean come along, and I also picked up stuff like AgileUX, DevOps, Product Discovery, etc.
In other words, over the years my understanding or perspective about Agile methods evolved as follows:
Back in the good old days, I believed clean code was everything and that software shops had to worship their code, because it was the only true asset they had. Then I worked with some really hep, successful software tycoons and realized how little attention they paid to clean code. In fact, they saw code as a liability and tried to get rid of it as quickly as possible.
This does not apply to every company out there. Some companies desperately need Refactoring and TDD mojo. For some, the technology and implementation solutions change so rapidly, that if they are not innovating and creating the latest, greatest stuff, no matter how clean their code is, they won't survive. Some companies struggle with lack of information to make decisions, while others are choked due to information overload. Some struggle due to poor management, while others lack execution.
I truly enjoy the irony of watching some folks trying the "one-size-fits-all" approach with Agile. There is a huge spectrum of companies all with different sets of pain points, each needing different solutions at different points in time. One size doesn’t fit all!
There is no doubt that over the last decade, Agile has helped me resolve many of the typical challenges we've faced during the delivery of software products. However for me over the last 4 years, the bottleneck has shifted from software delivery to discovering the right product fit, the set of challenges I'm facing cannot be fully solved by applying Agile and Lean thinking.
Over the last decade a lot has changed around us. From Nature of Business, Demand for Distributed Development, Tools and Technologies used, availability of powerful Web and Mobile Platforms to just about everything. Consider the amount of change we’ve seen, I think its very important for our principles and approach to evolve to accommodate this new context.
Couple of years ago, I wrote a blog titled "I’m not Attacking the Agile Manifesto Principles, I’m begging for Improvement." This post captured the essence of some challenges I was facing applying Agile principles at work.
I do think the Lean-Startup community has addressed many of those concerns. I haven’t seen much new come out of the Agile community in the last few years, but when I look at the Lean-Startup community, I feel a breeze of fresh perspective.
At Industrial Logic, we’ve gone ultra-lean and pretty much elimintaed much of the standard Agile practices like backlogs, user stories, iterations, planning meetings, retrospectives, daily standups and in some cases even TDD and Pair Programming. Instead we've been practicing Customer Development, Validated Learning (A/B Testing, Fake Features, …), Continuous Delivery and other Lean-Startup ideas for a couple of years now. So far we are loving it and we do see it working well in our current context. However, like always, there is a need to figure out things as we go along. And that's where it becomes fun.
Few years ago, Startup companies esp. in the Social web and Mobile development space had some interesting ideas around software develpoment built on top of Agile and Lean thinking. Over the last few years the Lean-Start-up community has packaged this knowledge really well. They’ve encapasses all the meaningful Agile and Lean thinking, but at the same time, they have given a whole new dimension to agility and lean.
I believe Lean-Startup will heavily influence the Software community over the next few years.
Following is how I view the Agile Ecosystem today:
The Lean-Startup community is great. However I’m also excited about things happening outside the software development space, that helps me understand the software ecosystem better.
For example, for many years, I’ve know the following about software development:
- Uncertainty is inherent and inevitable in software development processes and products. See this link.
- For a new software products, the software ecosystem will not be completely understood or known until after the system is in use (may be not even after that.)
- Predictability Paradox – The more time we spend planning and estimating software projects upfront does not mean our forecasts will be more accurate. See this link.
- It is impossible to completely specify nor test an interactive system. See this link.
- A problem is not fully understood until after you have developed a solution. See this link.
- If something hurts, doing it more frequently will alleviate the problem
- Throwing away code more frequently, keeps the overall cost down
- Slowing down more often and being aware of things around you (tighter feedback loops), makes you go faster
- Software evolves more rapidly as it approaches chaotic regions
- There is no such thing called Best Practices, its all contextual.
- Usually most successful product bloats up and dies under its own weight
- Kaizen (small, gradual, consistent improvement) can only take you so far. Kaikaku (disruptive change) can help you achieve quantum leaps of improvement.
I was able to kind-of explain why I noticed these behavior in software development but could never really explain the underlying principles. Agile and Lean methods did help me address a lot of these challenges. However I was still unable to explain the underlying principles. Until I stumbled across the following two concepts:
- Cognitive Complexity: I was lucky to attend a keynote from Dave Snowden at Limerick, Ireland in 2008. He got me hooked onto Complex Adaptive Systems, especially his Cynefin framework. He has some very neat ideas that the software community can benefit from. If you see many organizations know that they need to change, but are afraid to make the move. They are looking for fail-safe approaches to deal with change. Dave explains the core problems with this approach and how Safe Fail Experimentation can really help organizations. Many organizations struggle with measuring the wrong thing and forcing people to game the system. Dave’s ideas on Retrospective Coherence or Fundamental Attribution Error and Weak Signals Detection can be extremely helpful to address some of these concerns. Also I particularly like (his take on) Narrative.
- Affective Forecasting Psychology: I'm blown away by the anecdotes and studies presented in Daniel Gilbert's book Stumbling on Happiness. It has helped me understand the "filling-in" trick the mind plays while projecting ourselves into the future (or past) and its limitations. I've just started my journey into this subject and there is a long road ahead.
I’m going to leave you with an old Sanskrit sloka as some food for thought.
No Matter how fine you chop the Sandalwood, it does not give up its fragrance,
However old a businessman gets, he never gives up his profit,
In spite of crushing the sugarcane in a machine, it does not turn bitter,
However knowledgeable you become, don’t cease to learn.
About the Author
Naresh Jain is Director, Asia Operations & Agile Coach @ Industrial Logic. From Organizational Transformation to enhanced Developer productivity, Naresh helps organizations embrace, scale and sustain essential Agile and Lean thinking. Currently, Naresh spends a large portion of his time evangelizing and building Industrial Logic's eLearning platform. His current leadership role demands that he performs various critical roles including understanding market needs, prioritizing & designing solutions, collaborating on marketing & sales strategy, hands-on development (coding & testing) and finally deploying & managing the production servers.
