A Hard Look at the Organizational Implications of BPM

Introduction

There is a natural symbiotic relationship between SOA and Business process management (BPM). SOA is all about changing how both business and technology groups see the way an organization functions, focusing on the capabilities provided - the service - rather than the tools and mechanisms in daily use, and BPM is all about providing executable models of the business processes that can orchestrate those services and provide a unified view across the manual and automated parts of a process.

Not surprisingly, BPM is one of the current hot topics in the SOA world because it has the promise of delivering the ultimate business "silver bullet": making it easy to change the business to meet an ever-changing market. However, most of the existing literature and certainly the majority of product whitepapers focus only on the cost reduction and "business agility" to be gained from adopting a BPM strategy. To gain real benefit from BPM requires more than modeling the existing business processes so that they may be automated-it requires organizational change so that the processes are optimized not to make them easier to implement, but so that they give more value to the people who depend on them.

This article examines the conceptual BPM project from the following perspectives: what is involved to deliver the project, what are the enablers of the project and what are the total costs of ownership (TCO) of the project. Organizations considering BPM as a "silver bullet" need to bear in mind that BPM is still software development, it critically depends on the proper infrastructure and services being in place, and the ultimate success of reducing the total cost of ownership of a BPM project over its entire lifecycle still depends on a disciplined approach to software implementation. Therefore, before significant investments are made in reengineering processes and deploying BPM solutions, businesses need to commit to making the organizational changes necessary to allow realization of any lasting value from BPM.

The Promise of BPM

The aim of BPM is to enable an organization to realize better business performance, but technology alone cannot be expected to achieve a business objective because the very nature of introducing technology into a business process requires inherent change within the organization. The attraction of BPM is the promise of better aligning IT with the business by enabling business owners to take a more active role in the development and delivery of their core business systems.

BPM is seen as a way to achieve some important business objectives:

• Lower costs
• Increase efficiency
• Increase adaptability and flexibility
• Reduce risk in system implementation
• Provide better governance and compliance reporting
• Deliver better customer service

In achieving these business objectives, a large part of the value proposition of BPM products deals with reducing the cost of systems development and support. This particular aspect of BPM implementations is the focus for the rest of this article.

Prototypical Business Users

One of the oft-cited reasons for looking to empower business owners with more control over the systems they use is "the IT department is just too slow" in providing new functionality business users feel they need to be more effective. As a result, many users "roll their own" tools to support their day-to-day operations using business productivity tools such as Microsoft Word or Excel. In most cases, these tools cause problems for the business from an accountability and compliance perspective because there is no good way to ensure the decisions made are based on "official" business information or processes.

Another issue is that because these tools are document-centric, they are often passed around to colleagues who customize them for their own use, resulting in many similar, but different, special-purpose tools floating around the organization. In many cases, the question of which version of a particular spreadsheet was used to make a calculation or business decision becomes critically important because over time errors in scripting macros are corrected or thresholds and costs have been updated.

The business user will likely argue that they are forced into this type of scenario because they can't add functionality to their back-office and decision-support systems themselves, but they certainly can produce spreadsheets, checklists and other process documents using standard productivity tools that are the basis of their day-to-day work.

The business problem with this pragmatic approach is that the details of how business gets done is often in people's heads or homegrown tools and not easy to change because it is encoded in so many different places. In such an environment, any standardized or legal requirements for compliance reporting are virtually impossible to provide.

Bespoke & Customized Systems

It is also important to understand the reason that IT can quite often be seen to be slow, and therefore costly, in making changes to support changing business requirements. One of the main reasons is highlighted by Joel Spolsky in the discussion of the Five Worlds of Software Systems in his book Joel on Software : most of the business-critical systems for either back-office tasks or to support knowledge workers either fall into the "consultingware" or "internal" worlds.

The types he describes also include a "shrinkwrap" category. It is distinguished by a larger user base that have alternatives of similar products and the product's ability to be installed into different environments across a variety of operating system and hardware platforms of compatible configurations.

Spolsky describes consultingware as a variant of shrinkwrapped software that "requires so much customization and installation that you need an army of consultants to install it, at outrageous cost." In this way, it is similar to his definition of internal software which is intended to work only to solve a particular business problem and be deployed into a known hardware and software environment. It also means that the users are pretty much forced to "like it or lump it" regarding the usability and functionality of the business systems they use on a daily basis. The main reason for building systems this way is that the speed with which they can be developed and deployed is directly tied to the short-term return on investment for the organization.

This situation is unfortunate for many reasons, but an important one is that the business pressures for delivering custom software as quickly as possible fail to consider research by Barry Boehm of the Software Engineering Institute (SEI). This research indicates only 30% of the total lifecycle costs of a software system comes from its delivery. The rest of the cost of the software comes from the operations and maintenance/enhancements performed once it is in production until it is eventually decommissioned.

Another issue which is often a result of these same business pressures is that in the rush to deliver working systems, little process is followed regarding documenting or otherwise capturing the rationale for important decisions made during delivery of the software. In the best case, a disciplined methodology is used that requires the creation of project artifacts such as the Software Architecture Description described by IEEE 1471-2000 or at least the use of agile methods with an appropriately skilled team. The use of Agile methods means that the organization is betting on the experience and ability of the lead developer to ensure the project meets the user's requirements, but the rationale for those decisions is still not normally captured explicitly as part of the process. In the worst case, haphazard or lackadaisical processes mean that not only is very little written down, but the validity and quality of what may be there is easily questioned.

These issues directly contribute to the difficulty and corresponding cost involved in changing systems falling into the "consultingware" or "internal" categories. Fundamentally, changing these systems is hard because:

1. The rationale for critical design decisions is lost or otherwise not readily available;
2. These systems are not generally built or customized with the anticipation of change to the process they are implementing or supporting;
3. The bespoke aspects to these systems are generally not developed with operations and management in mind.

Item #1 is a problem because each time the system needs to be changed, the design of the system needs to be reverse-engineered so that the people making changes at least know where they're starting. Even the use of tools to assist this process does not fill in the rationale, making developers and architects guess as to why things are the way they seem to be. Rarely will the time be taken to understand why the decision may have been made in the first place. The time and skills required to perform this part of the process before any changes can be considered can result in significant cost before the "real work" of delivering the new requirements is started.

Item #2 is a problem because often systems supporting the business are based on operational or tactical thinking rather than a more strategic understanding by the architects and developers of the business and how it may vary and change under different circumstances. Without an awareness of how the system will evolve over time, architects and developers can unintentionally make it extremely difficult to change these systems in response to changing business requirements as they attempt to respond to the time pressures of trying to implement the immediate business requirements.

Item #3 is a problem because the operations team will need to reactively discover how to diagnose problems within the system and determine if it is operating normally, thus adding further cost and complexity to the system just to allow them to "keep the lights on" for critical business systems. In extreme cases, it could take days to isolate the root cause of something as trivial as a configuration problem, or it is impossible for developers to reproduce a system issue that regularly occurs in the production deployment, resulting in a series of trial-and-error type changes that can reduce the overall integrity and stability of the system they're trying to fix.

The Business Case for Change

Adequately addressing the above problems requires the organization to make an informed "pay now or pay later" decision in a strategic context about the quality of the tools they provide to their employees. An appropriate analogy from automotive maintenance illustrates part of this choice.

If you ignore the signs that say "no customers in the service area" and take a peek at the tools used by professional mechanics to work on your car, you'll notice that they're a high quality, industrial brand like Snap-on or Teng. They are most certainly not what you'd see walking the aisles of a nation-wide discount store.

On the surface, the industrial #2 Phillips screwdriver looks nearly identical to the Brand-X, el-cheapo one from the discount store. However, if you use them both to attempt to try and remove the same set of salt-corroded, stubborn screws living under your car, you'll notice that the tip of Brand-X one is probably slightly twisted and bent and doesn't quite fit the last screw as well as it did the first. In comparison, the industrial one likely doesn't show any difference before and after the job is finished (other than new deposits of dirt and rust).

Once the tip of a Phillips screwdriver is twisted and bent, it's virtually useless and more likely to damage the screws you use it on because it doesn't correctly fit them. If the Brand-X screwdriver costs $2 from the discount store and you can only use it on 10 highly-corroded screws before it needs replacement, and the industrial screwdriver costs $12 and you can use it on 5000 highly-corroded screws, then the only way to determine which one gives you the most value is based on how you're going to use it.

If you're never going to work on your own car, and you just need a screwdriver 10 times every year, then the $12, industrial-strength screwdriver probably isn't a sound investment. On the other hand, if you were a mechanic and might need the screwdriver 100 times a week, or 5000 times a year, then the el-cheapo one starts to look less attractive. We can then use our hypothetical numbers to create some quantitative metrics to facilitate our decision:

Table 1 - Brand Comparison Metrics

Case #1 - The Household

Calculating the total cost of ownership for one year of each option gives the following results:

• El-cheapo: $2
• Industrial: $12

Stated another way, it would take six years (and six screwdrivers) before the overall cost of ownership of screwdrivers would be equal, but, based on the expected use, the industrial one would last for 500 years. At least you'll have a family heirloom.

Case #2 - The Mechanic

In this case, the total cost of ownership for each option in one year gives the following:

• El-cheapo: $100
• Industrial: $12

Of course, the difference here comes from the amount of use expected, but the el-cheapo option is over eight times more expensive over the first year of operation. Maybe those industrial-grade tools provide more value than just the calendars after all.

I'm sure everyone agrees that the above example is very straightforward. For the mechanic, the screwdriver is a key tool used to enable their business. Therefore, they're going to invest in higher quality tools that will cost them less over time. It just makes sense.

So, why should the considerations used when deciding the investment in the critical software tools to enable your business be any different? Are these tools less important to your organization than the screwdriver is to the mechanic?

If the answer to the last question is no, then decisions regarding the cost of these systems must be made in the strategic context of your business, not just based on the costs to deliver the tools themselves. Taking decisions only based on costs of delivery in the "here and now" can ultimately only cost more over time.

The only people who can make these decisions are the business leaders of the organization, because they know the business best. They're the ones that understand how often the type of fasteners on cars change, requiring the purchase of new tools, and how many older cars they plan on servicing each year where they can use the older tools. However, these decisions can only be made successfully if the information provided to them about the costs and utility of these tools is accurate.

What to Change

Making a commitment to BPM tools will require change both in how business users view the tools and processes they control and in how IT delivers systems so that they can be effectively leveraged in the context of BPM. One thing that will not change based on applying BPM or any other technology tool is the core business of the organization, so it is essential that the who, what, hows and whys of the core business are visible and understood by everyone.

Change for the Business User

Contrary to the common vendor message that BPM products will allow business users to change the core business processes without any involvement from IT, in actuality, the line between business users who have the tools to change business processes and programmers in the IT department begins to blur. This fact needs to be understood explicitly by both the business and IT, and it is one of the major organizational changes necessary for successfully applying BPM tools.

Exposing the business processes explicitly to business users and allowing the user to change them organizationally, even in the form of being able to change business rules rather than the overall process, means that those changes become as much of a software release as if newly compiled code from the IT department was deployed into production. Therefore, business users accustomed to being able to create individual tools and processes using Word and Excel, need to understand that they've just become part of the IT department's software release cycle. This means that the accountability, version control, testing, deployment and rollback aspects of software configuration management and deployment systems must be applied to each process change, no matter how small. The vendor painted picture of business users "effortlessly" changing business process is, at best, the equivalent of abstract art. Failure to adequately set the expectations of newly empowered business users regarding their new roles and responsibilities will have disastrous effects on BPM projects.

Another aspect of applying BPM tools is that, even if it is not part of the initial roll-out, the goal of using BPM tools is to optimize the IT systems to support the business and not the other way around. It is also intended to surface the business processes themselves so that the business can be optimized, resulting in organizational changes to the way people do their day-to-day work. Part of this change will require business users to move their reliance on Word and Excel to define and support business processes to the tools provided by the BPM environment. If they cannot, or if this change is resisted, there is no way that the governance model and compliance reporting for the organization can be improved.

Change for IT

The role of IT within the organization also needs to change significantly, including the details of how systems are built, changed and managed. While it does not explicitly require it, adopting a service-oriented approach to system delivery will greatly facilitate the success of the BPM effort. Additionally, IT will need to manage the enabling BPM software components as well as the artifacts created by the business users, so they too need to understand that changes to processes by business users must be treated as software releases.

Since BPM is a strategic investment, IT must also develop a corresponding strategic plan so they can identify what capabilities will be required to support the BPM processes and when those capabilities will be required. These capabilities are illustrated as layers within Figure 1.

Figure 1 - BPM Layers

Many BPM tools provide connectors to existing CRM, ERP and database systems to allow data to be directly extracted for use within business processes. However, introducing a business-specific layer of services and information using an SOA adds an extra layer of insulation between the back-end systems and the business data they contain.

Establishing the IT strategy, information and services roadmap is one area where the investment decisions illustrated by the screwdriver example become extremely important. To move to a service-oriented approach to delivery, IT needs to focus on the functionality and information that is core to the business rather than focusing on creating operational and tactically focused solutions.

Core business definitions, information and communications paths must be identified and exposed in a way that is meaningful to the business in order to provide the raw building blocks that BPM solutions require. Each of these supporting services will be shared between many processes and so operationally must be much more robust and thus be developed more like shrink-wrapped software than before. The shift from "internal" to "shrink-wrapped " will likely require significant changes in the processes, tools and skills used to deliver IT systems within the organization, because it is these core services that represent the fundamental business abstractions and operations to enable the business. These core processes will not be implemented using the BPM tools.

The governance challenges for IT are also much greater, as well as being more visible, when supporting BPM and SOA environments. Systems are more inter-dependent, so change control and configuration management are absolutely critical. Strict planning and adherence to the principles of loose coupling must be adopted across IT to ensure that changes made to one system to meet new requirements don't cause a "ripple effect" across the organization. Effective governance can pro-actively reduce the operational and development costs of IT systems in these areas. In addition, the business users now able to make changes to processes and business rules need to be included and involved in the IT governance efforts so they have an appropriate appreciation of how business and IT must work together.

Finally, IT must revisit the way that it provides basic metrics such as total cost of ownership to the business leadership. Adopting BPM may lower costs and increase the adaptability and flexibility of the organization, but the changes to BPM business processes are still governed by the existing rules of software development-including the relationship between when a defect is identified in the delivery process and how much it costs to fix. If not supported by the proper governance and processes, excessive changes to BPM processes because "it's so easy to do it" and the resulting costs involved in releasing the proposed change into the organization's production environment may actually increase the total costs of a particular process rather than reducing it.

Some people see BPM as an opportunity to blur the lines between development and production and bypass the perceived overhead of releases previously imposed by IT. While this decision is ultimately a business one, eliminating rigorous release processes may equate to operating without a safety net. The implications and potential legal consequences of an undetected defect released into production may not be worth the risk.

Calculating the TCO of BPM

By now, it should be clear that calculating the TCO of BPM is not a straightforward task. One of the first questions you need to answer is: "where do I start?" followed closely by: "the total cost of what, exactly?"

BPM can't really work in isolation, at least not and deliver real value to the business. It needs suitable services and information available as building blocks for business processes. These building blocks should be far more than the auto-generation of WSDL interfaces for existing applications. As previously mentioned, BPM is a strategic initiative, even when incrementally adopted, so it is difficult to draw a box around something identifiable and convenient to use when attempting to calculate TCO.

The following is a non-exhaustive list of potential cost factors associated with adopting BPM:

• Licenses for the BPM software
• Licenses for any required software adapters for legacy or 3rd-party systems
• Additional hardware and software infrastructure to run the BPM software
• BPM training for users and IT operations
• Identifying/extracting the existing business processes and tools from business users
• Representing those processes using the BPM tools
• Training IT developers and staff on building "shrinkwrapped" software
• Definition and training on new/revised IT governance policies
• Education of business users about software release cycles and the importance of configuration management
• Eliminating business user dependencies on Word and Excel as process automation tools
• Releasing new and updated processes into production
• Many, many, many more...

Even limiting the scope to the TCO of a single process can be difficult:

• If the process leverages shared services in the infrastructure, does the cost of building the "shrink-wrapped" service get split across all services using it? If not, whose budget bears the cost?
• Do changes to the process to reduce redundant steps that hadn't been visible before fall into the operations and maintenance phase of the original process or should it be considered a new process?
• If the capabilities required by a new process necessitate software or hardware upgrades to the underlying BPM or supporting environments, is this cost only included in the new process, or is it part of the costs associated with the base platform?

As BPM becomes more and more a part of the organization, it becomes harder and harder to calculate these costs beyond adding up the invoices relating to "BPM projects."

Both BPM and SOA are intended to eventually begin exhibiting characteristics of a reinforcing system, meaning as the more capabilities exist within the system, the more potential new capabilities there are as a result of the interaction possibilities between the participants. This "snowball effect" is a very real benefit of the layering illustrated in Figure 1, but it is only possible if the appropriate governance processes and controls are in place. Without proper management, the cost vs. value scales tip sharply in the other direction.

Because BPM and SOA are both strategic initiatives and have the potential to permeate the enterprise, both should be seen in terms of investment and potential value rather than in terms of costs alone. This isn't to say that attempting to determine TCO is meaningless, but both BPM and SOA provide pieces of business infrastructure, so it is similar to trying to determine the TCO of your network. If you want meaningful answers for the TCO of BPM, you will need to be very specific about the context and questions being considered.

Conclusion

While it may resemble and be sold like any other software package, BPM solutions really represent investments in new business infrastructure. Like any other change in the enabling infrastructure of an organization, deploying BPM has the potential to effect change nearly everywhere. Without this change, it is difficult to realize the full benefits from an investment in BPM.

BPM is also much more than automating business processes, because it is unlikely that all of the appropriate enabling components and systems already exist. While the business processes may exist today as manual procedures performed to knit separate interactions with bespoke internal business applications into a coherent unit, automating these processes will eventually require that these applications be exposed at a conceptual level, e.g. "process order" rather than a sequence of specific keystrokes and mouse clicks in an existing user interface. This level of change is the essence of moving to a service-oriented IT infrastructure by separating the "how" from the "what" of the foundational supporting processes. It is also this change that will deliver more value from the processes because they are expressed in ways that are more meaningful to the business.

The value from BPM does not come without costs, however. These costs span the obvious software and hardware costs of deploying BPM software, but they also come from implementing the process and procedural changes necessary to make BPM work. Business users need to become aware that their custom, home-grown tools, utilities and processes created using productivity applications such as Word and Excel have always been critical business assets, but have not been treated as such. Providing these users with the capabilities of BPM tools effectively seconds the business user to the IT department, because they are now in the position of creating and modifying the critical business systems of the enterprise with potentially immediate impact. While this change may come as a surprise and meet with resistance, business users need to understand the implications and potential ramifications of one or two ill-considered mouse clicks within a BPM tool. Doing so will allow them to appreciate that the overhead of the configuration management, testing and release cycles required for BPM process changes protects both themselves and the business from potential mistakes and financial losses.

To be done effectively, both BPM and SOA require IT to increase its understanding of the core business and take a more cooperative and active role in ensuring IT activities support the business objectives. The way the core systems are delivered must also be more "production grade" as the focus must shift from building solutions for operational and tactical problems to building services as building blocks supporting strategic business objectives. As strategic assets, these systems will have many more requirements for operational robustness and maintainability than systems previously delivered, and they will require that IT develop or enhance its skills in these areas. Much more effort will need to be directed towards governance, operations and monitoring in order to both more easily meet legislative and regulatory reporting requirements as well as increase efficiencies and guarantee service levels across the business.

Finally, recognizing that investing in BPM is providing new business infrastructure means that it is more difficult to isolate and calculate traditional metrics such as the total cost of ownership. If carefully considered and applied as part of an overall strategic initiative, BPM has the potential to provide tremendous value to the business, but only if the business is willing to invest in the organizational change required to make it a success.

About the Author

Andrew S. Townley is the founder and Managing Director of Archistry Limited, a professional services firm focused on helping our clients apply new and emerging technologies to deliver measurable business value.

References

Boehm, B.W., R. Turner, Balancing Agility and Discipline: A Guide for the Perplexed. Addison-Wesley, Boston. 2003.

Miers, D., P. Harmon, C. Hall, The 2006 BPM Suites Report, Version 2.0. Business Process Trends, September 2006.

Glushko, R.J., T. McGrath, Document Engineering: Analyzing and Designing Documents for Business Informatics & Web Services. MIT Press, Cambridge, MA. 2005.

IEEE (2000). IEEE Recommended Practice for Architectural Description of Software-Intensive Systems, October 2000.

Kaye, D., Loosely Coupled: The Missing Pieces of Web Services. RDS Associates, Marin County. 2003.

Linthicum, D.S., Next Generation Application Integration: From Simple Information to Web Services. Addison-Wesley, Boston. 2004.

Newcomer, E., G. Lomow, Understanding SOA with Web Services. Addison-Wesley, New Jersey. 2004.

Senge, P.M., The Fifth Discipline. Currency Doubleday, New York. 1990.

Spewak, S.H., S.C. Hill, Enterprise Architecture Planning: Developing a Blueprint for Data, Applications and Technology. Wiley, New York. 1992.

Spolsky, J. Joel on Software. Apress, Berkeley. 2004.

Weill, P., J.W. Ross (2004), Ten Principles of IT Governance, Harvard Business School Working Knowledge website, July 2004