Agile 2015 – Industry Analyst Panel: Agile Trends and Future Direction

At the recent Agile 2015 conference the Agile Alliance hosted the fifth annual industry analyst panel briefing in which a group of commentators answered questions on the theme of agile trends and future directions.  The panel consisted of:

• Melinda Ballou, IT Industry Analyst, IDC
• Steve Denning, Author & Management Consultant
• André Grard, Senior Analyst, VDC Research Group
• Tom Grant, Senior Consultant, Cutter Consortium
• Ron Jeffries, Extreme Programming (XP) Innovator, Author, Coach, and Trainer
• Jim Newkirk, VP of Service Engineering, CenturyLink Cloud (Moderator)

The session was recorded by the Agile Alliance and is available for viewing here.

The session started with the panellists briefly introducing themselves.  

Then Jim asked the panellists to describe what they would say to a business executive who is struggling with their (non-agile) software development process.    The panellists all stated that the starting point is to understand what the challenges are and what problems the organisations face – not talking about “agile” but talking about the realities of the software development and helping them understand the nature of software development, that it is a learning and exploratory activity rather than a predictive process, and then discussing better ways of building software products.  An important point that was raised was the need to explore metrics of success and how the organisation identifies that progress is being made (their definition of done), how the work is being done and how work is prioritised.

Jim asked them to address extend their responses based on the scenario of the organisation struggling with their agile adoption.  The responses to this question focused on alignment – are the goals of the teams aligned with the goals of the executive (do the teams measure their success by the speed they deliver products and the executives focused on innovation).  Another point was to explore the level of executive support that exists for the agile implementation, what approach has been taken, what is the nature of the problems they face, and exploring whether there is real alignment between the teams and the management approach. 

The next question related to an environment where executive leadership feel they do not have to be agile, rather they feel that agile is something that the teams need without impacting the management philosophy or behaviours.  How to get the leaders to adopt an agile mindset?

The responses to this dealt with the long history of traditional management practices which were successful for many years – the only way to convince leaders to adopt a radically different approach is to take them and show them how other organisations (both outside their industry and their competitors) have adapted and contrast the modern successful management approaches with the approaches which are no longer successful.  Steve Denning stated that for large organisations today “the choice is very simple – change or die”.   The use of Innovation Games was suggested as a way of helping the executives understand the new ideas, as well as using pilot programs and providing internal proof by starting small and showing the benefits. 

An issue which was specifically identified was the fear of middle managers in many organisations as they see the significant changes that agile adoption results in for their roles.  A common pattern is the reduction in the number of middle manager roles and the migration into more hands-on roles such as Scrum Master or Product Owner.  There needs to be a clear transition path for these managers and the importance of strong executive support to overcome this resistance was strongly emphasised.

Jim then asked the panel to think back five years and identify one thing that the agile movement was wrong about and one thing they should have done differently:

• Wrong: thinking this was a methodology for software development, it is in fact a revolution in the way every part of the organisation can and should operate.  Agile thinking and techniques are moving into every area of the business today and making a huge difference.
• Wrong: the rigid ideas about scalability – there is no single way and context is everything.
• Wrong: not the content but the way things were communicated.  There was not enough clarity about how agile is not a set of practices but a way of thinking that is radically different.
• Wrong: allowing the rigid “there is only one way” zealots too much free reign and not actively challenging that mindset publically.  This has reduced significantly today but it was an impediment to acceptance of good agile thinking.  Beware – there is a new generation of “one-true-way” idealists coming to the fore in areas such as DevOps, UX, Lean and other areas and this thinking needs to be countered.
• Should have done differently: more proactively reach out and incorporate other ideas sooner rather than allowing the fracturing that happened in the community.

The next question was what are the things that the Agile Alliance could do to help with the adoption of agile approaches?

• Looking at particular practice areas and problem areas and funding research and exploration and the production of supporting materials which talk about ways to overcome the problems, for example technical debt.
• Agile at the team level has largely been successful and is now well understood, agile at the development organisation level is working now it is time to look at agile beyond software as that is where the significant improvements to productivity and organisational success will come from.
• Exploring the impact of complexity in organisations, dealing with the Internet of Things, Systems of Systems and other such advances and providing information to help organisations take advantage of the opportunities inherent in these new frontiers.
• Supporting the areas of the world where the human potential is not being fully realised such as parts of the third world – using agile thinking to drive frugal innovation and making a difference in the poorer parts of the globe.  Partner with education institutions to disseminate the knowledge.  
• Become a trusted centre of information where people can come to regarding all aspects of agile thinking, not just software development but all aspects of work in modern organisations.
• In addition to working from the top of organisations, provide the information that the people who do the work of product development and operational activities need for them to understand how to be agile in their workplaces.
• Have a broader focus – from teams to whole departments, from software development to systems development, from software development to whole organisations.

The next question was about the “certification wars” and how some trainers are now offering multiple certifications, how does that change the dynamic around certification?

• The Agile Alliance could take on showing the world what it means to actually become agile which is beyond attending a two or three day class and gaining a certificate.  Telling the stories of why becoming agile is not about simple knowledge but requires deep learning and expertise and showing the world why this is a way of thinking and a fundamental mindset shift.
• Standards and competence is importance and certification is a step in this direction.  The validity of these standards is something that needs to be protected.  The profession needs to become more professional and more competency based. 
• Competency needs practical assessment – it’s more than just knowledge.  Make the practical component visible and transparent.   Find ways to really measure if people can really do what they claim – not tests but competency.
• Engage with the education system (schools, universities) and help ensure that the education programs at that level teach people the right skills so they come to the workforce actually able to do the work.

The final question was: What is something that has not been talked about that should be?

• Agile is broader than software development, move beyond software development to areas such as imbedded systems, systems and product development. There needs to be a rigor and discipline so these practices can be applied in heavily regulated environments.
• Challenge and resist the pressure to bring into agile the practices from the non-agile era that are trying to constrain and restrict the collaborative, self-organising ethos of agile. An example is the notion of estimation – we need to estimate and budget but we do not need to identify the “requirements” in order to do so, we have better approaches today.  There are too many old ideas which are creeping back and need to go away because they don’t work.
•  The tension between the way the organisation as a whole is being run and the way agile teams are being run.  The change needed is at the organisation level and it is important for the economy as a whole as it is stifling the ability to innovate. Agile has the solution – we need to communicate these ideas to the organisation as a whole.
• Going further than even the organisation, using agile thinking to address global problems – these ideas can be used to address the large and complex problems the world faces.  Looking at the global environment as a complex system of systems and using agile ideas to empower people to tackle the big problems.
• We need to get a better handle on what software value is – software runs the world today and we don’t examine the underlying why of our software products.  There is a lot of literature on diffusion of innovation and we need to find ways to incorporate these ideas into the agile lexicon.