The journey has already begun
In his recent article 'Tradeoffs: Giving up Certainty', Paul Dolman-Darrall explored the trade-offs companies typically face when contemplating a shift from conventional waterfall-based methods to more agile approaches. As the author rightly concludes, effective progress on this journey often emerges from trial and error, plus fine tuning of methods found to be effective. This is of course the case with most forms of innovation, as Darwin observed.
In most large, typically corporate, enterprises, such innovation is in fact a continuous process. If you dig deep into the IT department of any large organisation you will find many examples of successful adoption of agile delivery practices even if the organisation itself remains heavily waterfall-centric. These pockets of innovation typically emerge through “bottom up” endeavours usually instigated by focussed individuals or small team who are quick to see the local benefit offered by such practices.
Despite their (usual) success however, such practices rarely morph to the corporate level. Worse, they rarely make it to the status of recognised best practice, so the numerous small gains seldom turn out to be incremental or even sustainable. This is not only regrettable given the huge benefit on offer, but is the source of great frustration among those who have seen the light and despair at the prospect of working on never-ending, and often doomed, enterprise programmes.
So how do you progress on the journey to truly sustainable IT Agility at the Corporate level?
Setting out the Goals
So if the journey has already started, are you sure you’re clear about the destination? Or to put it another way, what are your particular reasons for aspiring to become more agile? Is it speed, quality, responsiveness, better return on investment, or other reasons, or any combination of the above?
Speed
This would be the expectation of most senior executives. Products need to be launched now – the market can’t wait, and the competition will have stolen a march. Agile methods tend not to focus on speed as such, but on delivering value early and frequently. Strong discipline is applied through appropriate methods and techniques to ensure that this focus is established and sustained through a regular “hearbeat” of delivery iterations. In fact, sustainability becomes a key word – we can all work much harder and faster for a while, but not indefinitely. Sooner or later, something (usually quality) suffers, giving rise to the “more haste, less speed” experience.
Predictability
When you commit to multi-million dollar / pound / euro projects, typically lasting 2 years or more, you naturally want to be sure that you are going to see successful delivery at the end of it. In fact your career (and those of many others) may depend on it. You are naturally going to want to see a credible project plan which shows how progress will be made over time, ticking off recognisable and meaningful milestones along the way.
There is a paradox here in that executives tend to stick to tried and tested methods even though those very same methods are routinely failing to deliver. The failure level among large-scale waterfall-based transformation programmes is notoriously high. As an Agile Consultant colleague often remarked, “If you always do what you always did, you’ll always get what you always got….”.
Even so, it is not surprising that the Waterfall model, with its logical sequencing of activities over time offers that “warm feeling” to senior executives who have so much at stake. But as many have found to their cost, the predictability of the waterfall model often turns out to be an illusion. Unfortunately this only becomes apparent late in the project when integration testing is found to be running late, and warning signs are emerging from early User Acceptance testing that what is being delivered is not what the users had originally intended.
Return on Investment (RoI)
Sometimes, the cost doesn’t matter too much as long as they can be justified by the benefits. For many, therefore, the goal of Agility is to establish an unrelenting focus on benefits, the earlier the better. That way, a project’s payback will look healthy, confidence will be sustained, bottom line improved and bonus cheques cashed. This is certainly in keeping with the principles of Agile Delivery, and is a natural goal for any dynamic executive.
As a word of caution though, senior executives sometimes fail to grasp the engineering necessities of large, complex transformation programmes. For example, you might say that the most valuable part of a house is its roof, but there are compelling engineering reasons why you would want to start with the foundations, then the walls, etc. The same is true for IT-centred business transformation programmes.
Incremental benefit
Agile-minded enterprises generally seek wider “spin-off” benefits that are above and beyond the original project scope. Such benefits may include streamlining of the systems estate for example, or re-engineering of a particularly costly or error-prone system into something more flexible and dependable.
When the goal is more to do with Enterprise Agility (that is, general operational responsiveness to change in the marketplace) as opposed to IT Agility (usually taken to relate to development practices), then there can be substantial gain from improvements to individual system components, or even the creation of new teams offering particular expertise, emerging as “side effects” from the core programme.
Survival
The most primitive of instincts, organisations large and small are in a constant battle to sustain their very existence. In the present economic climate, the prevailing mood is that survival is the new growth. Agility in this respect then refers to the ability to anticipate and respond effectively to danger and threats, as well as opportunity.
What’s holding us back?
If we are all more or less in agreement as to the main goals for pursuing greater agility, and there is a strong desire to make the journey, then why are so many organisations failing to make the breakthrough? Sometimes it’s down to ill-advised or ill-conceived efforts. Most of the time however, corporate inertia or “drag” eventually causes well-intentioned initiatives to stall and ultimately fail. The drag is usually down to cultural and other factors that are commonplace in large organisations. Here are the main ones from my experience -
“Safety first”
Nobody wants to fail. In fact, large organisations base their governance structures around personal accountability and the expectation of success. Objectives are assigned, and people are judged according to their achievement against those objectives, often using measures and targets designed to demonstrate how effectively those objectives are being achieved. Remuneration is then aligned with these measures and targets. A strong aversion to risk then develops.
In the “agile” world, failure is not exactly welcomed, but nor is it frowned upon. Instead, a “fail fast” mindset leads us to explore different ways of making something work until we find the right one. By its nature, this approach is not very intuitive or appealing to traditional executives who expect to see a firm plan in place at the outset. So once again we opt for the waterfall, with its well-structured delivery phases but sub-optimal delivery effectiveness.
Focussing on wrong goals
Larger organisations tend to be dominated by Financial and / or Marketing professionals driven by the need to reduce costs, increasing revenue and / or margin, or similar goals. Of course these are worthy goals if you are preoccupied with balance sheets and next quarter’s Profit & Loss statement, but unfortunately they often filter down into inappropriate systems delivery challenges.
The most common examples we see of this are systems closure targets (on the basis that there is a direct correlation between number of systems and support costs), and process automation (evidently to enable headcount reductions).
Taking a truly Agile perspective however, the focus becomes much more on delivering products that will be well received in the marketplace (subtly different from increasing revenue per se), or optimising the user or customer experience. This may seem like splitting hairs, but by failing to appreciate the link between cause & effect, organisations often find themselves with sub-optimal processes where costs taken out by one efficiency initiative soon make their way back into the process in response to poor customer experience or lack of scalability.
Cultural & Geographic barriers
Geographic co-location is strongly encouraged for agile projects due to the obvious communication and collaboration advantages. This isn’t always practical for large, geographically dispersed organisations, or comes at a high cost and inconvenience to individual team members whilst still failing to achieve the expected results.
Furthermore, organisational boundaries and prevailing working practices can act as blockers to true cross-team collaboration by creating functional silos with ill-conceived “hand-offs” usually resulting in delays and higher overall costs. This in turn leads to defensiveness and “back covering” when targets are missed or expected to be missed. Such practices are often reinforced through well-intentioned but highly detrimental reward structures based on individual attainment of pre-defined (and in many cases, arbitrary) targets.
This is in stark contrast to the ethos of self-organising teams comprising the right people with the right skills and attitude to do the job, working at a sustainable pace, rewarded based on their collective efforts, and given space and the right support to grow and develop professionally. High levels of personal motivation plus strong team-working (even when not co-located) are in fact widely recognised traits of Agile teams, and is undoubtedly one of the main contributing factors to the higher productivity levels usually experienced compared with traditional Waterfall-centric teams.
Making it happen
There are several changes that any organisation, or groups or even individuals within an organisation, can make in order to become more agile. All that is required in most cases is the right mindset, a real commitment to wanting to do things better or more effectively, space to make the changes (or to use the acknowledged management speak – empowerment), plus some coaching where appropriate.
Value-driven delivery
This is as good a place to start as any. Work through your overall project goals, collectively (with delivery and operational people) and in depth, to ensure there is consensus as to the value being sought, and the delivery scope that will realise that value. Then explore what specific items of scope will realise particular value or benefits. Finally, work through a natural and logical delivery schedule or roadmap that offers an optimum path that will give early benefit whilst also following a logical sequencing of build (i.e. remember to build the walls before trying to build the roof…).
Surely this is common sense planning, I hear you say. True, but it’s astonishing how often project teams fail to take the time to perform this task, or when they do, fail to do it effectively. The result can be excessive investment of time and resources before any tangible value is realised, or real feedback obtained. Opportunities are then missed to establish and build confidence and belief in the delivery programme, leading to twitchy stakeholders and sponsors, and a dilution (and eventual loss) of active support.
Smaller & Shorter deliveries
Once you have established a value-based delivery roadmap, and probably an agreed Release Plan, attention should then be given to identifying distinct ‘building blocks’ that can be built fairly quickly, preferably independently, and integrated into the whole. Agile projects tend to specify these ‘building blocks’ in the form of user stories (see Cohn), with recognisable capability which serves a particular purpose and carries pre-defined acceptance criteria. Decomposing capabilities in this way applies equally to large programmes and small.
Engender true collaboration
The challenge is to get people talking to each other, routinely, even if they are not co-located. In fact, it can be even more beneficial for dispersed teams who may struggle to establish the level of rapport needed for true team-working, and are likely to resort to emailing. Emails of course serve an important purpose, but they are no substitute at all for genuine discussion intended to drive up understanding or to collaboratively find resolutions to pressing problems.
Daily “stand-up” meetings work well when properly managed, and people needn’t be present at the same location for them to work effectively.
Periodically getting together to work through problems, designs, plans, solutions, and so on is also an essential part of engendering team-working and collaboration. Yet it is surprising how frequently such opportunities are rejected in favour of teleconference-based approaches in the interest in saving travel costs. Sure, teleconferencing is a powerful technique when used properly, but it cannot fully replace the need for face-to-face meetings and discussions.
Early testing & deployment
With focussed releases, and smaller building blocks, the next big challenge is to perform testing at the earliest opportunity. In large organisations, testing resources are generally grouped into distinct teams and their capacity structured into timeslots, on the assumption that this achieves the optimal use of those resources. This may indeed be the case from the testing team’s perspective, but not usually so for the delivery team.
Support local methods adoption
As already mentioned, there will undoubtedly be pockets of good agile practice across your organisation. Active steps can be taken to support and nurture this so that their successes can be shared and built upon. Networking and cross-fertilisation between teams can also allow existing skills and experience to be shared more widely, and for best practices to be nurtured.
In any event, take care not to become overly prescriptive in imposing excessive centrally-defined standards on local teams. This undermines their independence and their ability to experiment with new practices.
Picking up speed
Ok, so there’s much more to becoming agile than shortening delivery cycles and working more collaboratively, but these are the basic building blocks. Only once they are gaining traction can you seriously consider the next significant steps towards achieving your Agile goals.
Leadership
Success breeds success. People who have worked on successful projects become a powerful source of leadership potential for future projects, and strong and effective leadership lies at the very heart of successful projects, agile or not. Nurturing talent in this way requires careful management, but done properly will significantly enhance your organisation’s ability to deliver successfully in the future.
Test automation
Automation of test cases is one of the most difficult challenges for large organisations, yet can be the most fruitful, especially when combined with continuous integration. In fact, effective continuous integration becomes impractical without extensive test automation. Many of the leading agile trailblazers are already seeing substantial improvements from this.
Some degree of automation will already be commonplace especially for regression testing of batch processes. Taking it to the next level requires a leap of faith, as well as a fair degree of investment, and should not be tackled without strong expertise and experience.
Changing Financial processes
Possibly the most challenging task of all (and probably best left until last…) will be to radically re-engineer your financial management processes in order to move away from the conventional annual budgeting lifecycle. This way of working seriously dilutes flexibility, undermines true accountability, and ultimately offers only limited financial control.
Instead, more dynamic funding models (such as the Beyond Budgeting approaches mentioned by Michael Azoff in his 10th April posting) offers a more continuous process which achieves greater empowerment and, in fact, clearer accountability and better overall control.
Seeing it through
Each organisation will of course be different, as will its journey to becoming more agile. Regardless of the journey though, perseverance in driving improvements, and strong change management to ensure that the improvements stick, will ensure that you are moving in the right direction. Good momentum from the early efforts should help build buy-in and support, and in turn let you take the journey to the next stage.
A word of caution though – there is no end to this particular journey. The further you progress, the more opportunity you will see to improve. And remember, your competitors may already be on their own journey, so there is no time to lose.
Bon voyage!
About the Author
Ian Evans is an IT Programme Manager with over 25 years’ experience of systems development, project & programme management, mostly in large Telecommunications settings. With a solid grounding in Software & System engineering, he now specialises in the deployment of enterprise transformation solutions including effective business change and benefit realisation. His background and interests also include Methodology leadership, particularly relating to the nurturing of Agile practices within corporate environments. He has previously presented on this topic at industry conferences.