Service-Oriented Architecture Maturity

This article first appeared in Computer magazine and is brought to you by InfoQ & IEEE Computer Society.

While some have touted service-oriented architecture as “the next big thing,” many SOA efforts have failed to achieve organizational objectives. A proposed new SOA maturity model addresses this gap by accounting for the different motivations for SOA adoption by IT administrators, business managers, and enterprise leaders.

The service-oriented architecture design concept evolved from prior efforts to wrap or encapsulate data processing functionality in such a way that the consumer need not know how or where the data processing was done. SOA differs from previous approaches in several ways.

First, a well–evolved - and still evolving - set of XML-based open standards defines everything from service description (for example, WSDL - the Web Services Description Language), to communication with the service (for example, SOAP and Representational State Transfer - REST), to discovering and connecting to the service (for example, universal description, discovery, and integra­tion - UDDI), to details on combining services to create composite services, transaction completion, and security. These layers of functionality are, in turn, organized into an architectural stack; an enterprise service bus (ESB) is a realization of this collective capability in a software platform.

Second, a service’s functional scope can range from very low-level or fine-grained up to functionality that maps directly in scope, terminology, and interest to business users.

Third, the means for communicating with services, re­gardless of scope, is an Internet protocol. This means that where and by whom a service is performed is in principle limited only by the reach, availability, and reliability of the Internet itself. The actual location of servicing is transparent to the user, and, because an intervening service directory lookup occurs during service invocation, this location can change without affecting the user. The actual processing can also change without impacting the user because the inter­face to the service remains the same; to the user, processing is a black box whose contents are defined by the rough equivalent of a service-level agreement. In fact, there’s no requirement that a computer provide the service; only that the exchange of information to the service be conducted using standard, XML-based messages over an Internet pro­tocol such as HTML or SMTP. While these protocols can be used over public or private networks, SOA standards are designed to enable interoperability over public networks, even though many firms choose to operate them over pri­vate networks using Internet-based transport protocols.

SOA capability can be viewed through multiple lenses. From the IT perspective, SOA offers another, newer way to deal with long-standing issues related to reuse, main­tenance, and enterprise application integration (EAI). For transactional business managers, SOA provides a means to locate IT-enabled services that better serve their inter­nal or external clients through, for example, improved organization and visibility across organizational and extra­-organizational data, enhanced client interactions, and more efficient business-process execution. And for enterprise leaders, SOA makes it possible to realign the organization’s structure to achieve a better flow of, and increased visibility into and control over, end-to-end value streams, adding value on behalf of clients and suppliers.

While consultants, vendors, and some early IT adopt­ers have touted SOA as “the next big thing,” professional publications have noted the failure of many SOA efforts to achieve organizational objectives and provide a return on investment, going so far as to proclaim SOA dead.[1] How­ever, we believe these failures can be explained by current SOA frameworks’ inability to account for the different motivations for SOA use by IT administrators, business managers, and enterprise owners. To address this gap, we propose a SOA maturity model that adapts the basic Capability Maturity Model Integration terminology but reflects the continuous transition from IT-to enterprise-based motivations.

Understanding SOA

The primary inducement for SOA was the IT commu­nity’s desire to move from large-scale, tightly integrated applications designed to meet specific functional needs to more modular, interoperable, and reusable functions.[2]^{,[3] ,[4] ,[5]} The need to access functionality encapsulated in, for example, routines, objects, and components beyond the scope of the application in which it was originally embedded led to the creation of cross-application interfaces such as APIs and the common object request broker architecture (Corba). The need to share this functionality across new and existing ap­plications in turn motivated EAI. This was initially achieved by combining “wrappers” to encapsulate functionality with “connectors” to export functionality to another application, but the number of such connectors grew exponentially, and changing these as new application versions appeared became untenable. SOA emerged as a way to expose ge­neric functionality for use by any other application, thus reducing the N(N – 1) number of inter-application connec­tors to N connectors.

However, existing applications were not, in general, de­signed as a collection of well-encapsulated services to be externally shared and used by other applications, let alone to meet higher-level and generally more abstract business requirements. Where encapsulated functionality did exist, it was at too low a level. A solution to this was to create new, higher-level services by wrapping existing functionality exposed in other ways such as APIs, screen interactions (via “screen scrapers”), or database calls and then to “wire” these together into a composite service. Invariably, standardized languages emerged to meet this need such as XLANG and, more recently, the Business Process Execution language (BPEL).

The use of SOA as an IT design principle presents sev­eral opportunities for operational efficiency - specifically, SOA can lead to a reuse of IT assets (the encapsulated ser­vices), in turn lowering IT development costs, decreasing development project time, reducing development risk, and leveraging existing IT investments. Loosely coupled services reduce vendor lock-in and create a flexible infra­structure to distribute processing, resulting in server cost efficiencies and greater reliability.

Although the path to SOA has been driven largely by IT’s need to more readily respond to business needs while at the same time reducing its own costs, the concept of an IT-provided “business” service, as enabled by SOA, makes it possible to directly define business requirements as services, rather than translating business requirements into lower-level IT-developed or acquired artifacts, such as application programs and other COTS products. The functionality that these composed services offer can be, in principle, directly aligned with and defined by business needs. The resulting shift from defining functionality at the application level to business-defined service functionality could eventually alter the way IT relates to, and integrates with, the business units within organizations. Such an evolution would result in IT administrators working more closely with the business units, not only on service develop­ment but also on the associated business-process redesign necessary to achieve the desired functionality.

SOA can also be thought of as a means to organize and utilize distributed capabilities currently under the control of different ownership domains, breaking down barriers and allowing enterprise-wide access to functional capabilities previously hidden in stove-piped applications. SOA provides a way to discover these capabilities and combine them to meet a business user’s needs. The user then accesses these combined services via a service interface, in much the same way as a custom-made application.

As organizations define, develop, and deploy services, the resulting services repository provides the opportunity for reuse, both for developers in creating new applications (transactional services) and for individual users in combin­ing internal and external services (mashups). Many early SOA adopters have leveraged this form of creative, user-driven service use. For example, websites such as Google, eBay, and Amazon expose services for developers to uti­lize and combine with other services to create situational applications that help solve problems arising in specific organizational contexts. In short, SOA places customiz­able power in the hands of users and enables their active participation in the IT development process.

Developers also can use services as building blocks for responsive business processes. By building services, firms can increase organizational agility and rapidly introduce new business models through the flexible delivery of functionality and improved value-chain collaboration. The creation of a service-to-service information ex­change can ensure a common data view across disparate application areas (composite services), while mashups can generate situational information delivery. In the long run, SOA injects event-based transparency into organi­zational operations.

SOA Drivers

The complementary but often overlapping rationales for SOA adoption at the IT, business management, and enterprise levels are intuitively appealing and, on the sur­face, perfectly sensible.[6]^,[7] Based on an extensive set of semi-structured interviews we conducted in 2008 with SOA practitioners at 20 leading US organizations[8], we have identified six distinct SOA drivers, arranged in a continuum with IT motivations at one end and enterprise motivations at the other. These are not mutually exclusive but instead blend objectives that all parties care about to various degrees.

IT - related drivers

IT - based drivers for SOA adoption include infrastructure efficiency, reuse, and application/data composition and integration.

Infrastructure efficiency

SOA improves infrastructure efficiency by breaking up a large, monolithic application into separate components (services). The rationale for doing this is that the large application no longer requires an equally large, monolithic mainframe computer to run on. If this were a major application in terms of size and number of transactions, only the largest of existing com­puter systems would be able to accommodate it, leading to considerable costs and limitations on improving its performance (commensurate with the availability of even larger, more powerful servers to run it). Componentizing the application-breaking it down into SOA “chunks” - makes it possible to move from very expensive, high-end processors to COTS-like commodity servers. Also, load balancing could be done at the sub-application level - if, for example, the front-end transaction that initiated the processing required more “horsepower,” this could be accomplished simply by adding additional servers to that aspect of the application.

With the advent of cloud computing infrastructure, ef­ficiency could become an even greater SOA driver. While it is certainly possible to move entire applications to the cloud, it may be more efficient to do this on a per-SOA service basis.

Reuse

By far the most common reason for adopting SOA is the possibility of reusing services[9]. While this represents a savings in time as well as cost to the organi­zation, any gains from reuse are largely accrued to IT and, as such, are an IT rationale for implementing SOA. However, while SOA has the potential to reduce both costs and future application delivery times, firms rarely achieve this; in fact, in our study, few could even measure whether they are doing so[10]. There are many reasons for this, not the least of which is that services are often encapsulated in a SOA offering in such a way that they anticipate need: a case of build it (as a service) and they will come (and discover and reuse what we have already built).

Application/data composition and integration

EAI was a primary driver for SOA and its predecessors. Business unit managers sought data and application functionality that transcended siloed applications such as customer relations management (CRM), supply-chain management (SCM), enterprise resource planning (ERP), and human resource management (HRM). For example, they wanted a single view of the customer/client even though such data was kept in disparate applications.

To serve this need, IT developed connectors so that one application could “talk to” (exchange data or func­tions with) another application. Initial efforts to reduce the resulting N(N – 1) network connection problem, such as Corba, suffered from a lack of standardization. SOA and its open standards such as WSDL and SOAP made it possible to standardize connections to each functional application. Wiring languages such as XLANG and BPEL in turn let de­velopers create composite applications and data views that transcended specific applications and responded more effectively to business needs.

Further driving this use of SOA is the availability of third - party SOA - style connectors for the major application software providers, which transfers responsibility for keep­ing up with version changes to those specializing in doing so; before, each organization would have to potentially rewrite and at least test the N – 1 connectors each time they implemented a new version.

Enterprise-related drivers

Enterprise-based drivers for SOA adoption include busi­ness analytics and processes, organizational flexibility and agility, and enterprise transformation. 

Business analytics and processes

Business analytics relates to monitoring and, when necessary, intervening in the default conduct of day-to-day operations. Tradition­ally, this task has been relegated to data warehousing and data mining experts, who keep historical transaction data with time stamps on them and then post-process the data to evaluate trends and identify correlations that might suggest different patterns of organizational or customer interventions.

SOA, in conjunction with complex event processing (CEP), gives managers real - time business intelligence so that they can potentially intervene in situations as they develop. Moreover, this data need not come from within the organi­zation and its transaction processing; rather, it can originate in wikis, social networking websites, online reviews, and other streams of near-current information. One form this can take is business activity monitoring (BAM) alerts, which rely on services that capture and monitor complex events. Another is mashups that convert preexisting services as­sociated with data of interest into user-specific applications or actionable data presentations.

SOA can also help organizations define and improve their business processes. Business process management (BPM), Six Sigma/lean manufacturing, compliance - driven development, and other business management strategies can lead to efforts to redesign and redeploy the business process, which invariably requires using a business pro­cess management suite (BPMS) to regulate, monitor, and incrementally change the process execution pattern while providing operational visibility to the process owner. With few exceptions, BPMSs necessitate access to underlying computer-based application and data functionality in the form of SOA-defined services. BPMSs also spawn events of interest to BAM alerts and key performance indicator (KPI) dashboards. Furthermore, business processes, properly de­fined, are services in their own right and hence are building blocks upon which developers can construct higher-level services representing partial or complete value chains.

Organizational flexibility and agility

Many business analytic and process management efforts focus on specific unit needs and initiatives such as improved customer expe­rience, increased supplier/client visibility into the process, regulatory compliance (for example, with the Basel II accord, the Sarbanes-Oxley Act, or the Health Insurance Portability and Accountability Act - HIPAA), and recur­ring events that lead to client dissatisfaction and reneging or poor operational performance. The solution to these problems is to ferret out the offending business process, gain metric or operational control over it, and attempt to operationally improve it or at least intervene in its by-transaction operation.

Beyond such tactical problems are strategic operational issues. For example, a client base’s changing needs requires adding new or different operational capabilities, and these in turn necessitate modifying how the organization functions. Responding requires flexibility; sensing the need for change and responding require agility. Both flexibility and agility demand the ability to rapidly reconfigure how the organi­zation interprets data and changes its execution patterns.

Here SOA can play a major role. However, planning for SOA service delivery means anticipating needs derived from business process execution flexibility and focusing on CRM and customer servicing data as well as external evaluation data.

Enterprise transformation

Some enterprises seek to reorganize themselves from being a functional or geographic silo to become an end-to-end value chain beginning with the client/customer. Others determine that they no longer deliver products but services (via products). And yet others pursue a type of “creative destruction” to radically innovate what they offer and do.

Whatever their motivation, many organizations trans­form themselves and their underlying units to match market realities before competitors, with perhaps little or no organizational baggage, can do so. This requires a means to transition from the as-is to the desired to-be con­figuration while continuing to operate and use existing systems. In short, such organizations demand extreme flexibility in their underlying operational systems, includ­ing their IT application systems.

SOA meets this need by enabling IT - based systems to expose basic functionality without dictating how it is to be used and by whom. In this case, however, it is the overall business transformation plan that drives the definition of the underlying services.

SOA Maturity Model

The primary difference among these SOA drivers is the degree to which they relate to IT concerns or higher - level business needs. To help organizations define current practices as well as guide executives on how to transform simple IT enablers to enterprise transformation enablers, we propose a new SOA maturity model.

Motivation

There have been many efforts to describe, rationalize, and project the adoption of emerging information technologies, including Richard Nolan’s stage hypothesis model and the Software Engineering Institute’s Capability Maturity Model. While SEI initially developed the CMM to distinguish soft­ware engineering levels and encourage migration to better practices, it subsequently generalized the model for use in other domains as CMM Integrated (CMMI). More recently, several organizations have developed their own CMMI-type models for SOA, notably IBM[11] and Wipro Technologies[12].

In constructing our SOA maturity model, we first consid­ered the question: along what dimension should maturity be defined? In normal CMMI development, maturity levels reflect the transitions from an ad hoc application of a particular practice to one that is repeatable, measured, managed, and optimized. This essentially represents a difference in orientation rather than maturity of use within a fixed orientation. As such, what we propose could be more accurately termed a capability orientation model. As organizations become more mature in their use of SOA, they move toward more fully realizing SOA’s ability to contribute to business operations and the organization’s service orientation as a whole.

SOA Maturity Levels

As Table 1 shows, our proposed SOA maturity model has five levels: initial, managed, defined, quantitatively managed, and optimized. These levels use the same basic CMMI terminology but reflect the changing locus of mo­tivation for SOA adoption. Each maturity level indicates the principal capability needed to achieve a higher-level capability - that is, one that moves away from IT-dominated reasons for SOA use toward enterprise-level transforma­tional objectives. SOA maturity has six dimensions. To distinguish the standard CMMI maturity categories from our model’s orientation view, we include “SOA view” as the primary attribute distinguishing the maturity progression.

Initial maturity

At the initial maturity level, most firms adopt SOA for IT - related reasons and seek to develop finer-grained software components, largely from exist­ing APIs, to achieve greater interoperability and platform efficiency. They have not yet adopted any form of SOA governance and tend to loosely define services to more fully embrace past and current efforts with minimum change. They also attempt to shield the business from SOA costs by burying them in existing project overhead or “rainy day” funds set aside in the budget for new initia­tives. There are no set metrics for success. The general objective is service reuse within current and future IT (level 2) projects.

Managed maturity

As they define and create more and finer-grained services, organizations need to employ common service definitions and some form of registry to manage service policies. Services are still defined rela­tive to internal IT needs but have shifted from APIs and wrappers around existing functionality to a more stand­ardized software architecture and standardized data and resources, leading in turn to reuse capability and the integration of disparate data across application platforms. As services are exposed through project costing, the busi­ness begins to see the benefits of leveraging SOA.

Defined maturity

A fundamental shift occurs as firms move beyond the simple, IT-focused management of services toward actual definitions of new services driven by business requirements and defined in business func­tionality terms. Service definition is now directly tied to business requirements capture, and SOA becomes an effective means to support data analytics and business process redesign. Business service definition methods become more important, and there is a need to develop a full set of service-related policies and metrics. Using the emerging software architecture created at the managed maturity level, IT can begin external sourcing of services in addition to internal servitization.

Quantitatively managed maturity

The redesign of business processes lets organizations be more agile and flexible, but to quantitatively manage all of the services, they must create an ESB and define services in terms of business needs and in advance of their use in processes, mashups, and so on. Metrics evaluate the efficacy of new or modified services and relate these to organizational return on investment. The alignment of IT and business governance metrics leads to “service thinking.”

Optimized maturity

At the final level of optimized maturity, firms define, develop, and implement busi­ness processes that are themselves services. This leads to the creation, in conjunction with senior managers, of a truly adaptive enterprise service architecture. The process of optimization moves outside the firm along value-chain lines that extend to upstream suppliers and downstream clients. This level of maturity lets organiza­tions realize the full benefits of SOA at both the IT and enterprise level.

SOA Maturity Cube

Organizations can use our proposed SOA maturity model in at least two ways. The first is to identify their current level of SOA maturity according to the six defined attributes; the second is to determine what they have to do to reach the next maturity level. The model therefore offers a multidimensional view of SOA maturity: an orga­nization can evaluate its maturity in terms of how well it executes a perspective-constrained practice (the normal CMMI view) or based on how far it progresses from the narrower IT-driven viewpoint toward the broader enter­prise-transformation viewpoint. While a fully developed “SOA maturity cube” is beyond the scope of this article, Figure 1 shows what its general layout might look like.

Figure 1- SOA maturity cube. An organization can evaluate its SOA maturity in terms of how well it executes a perspective-constrained practice (the normal CMMI view) or based on how far it progresses from the narrower IT-driven viewpoint toward the broader enterprise-transformation viewpoint.

Service-oriented architecture, like many other technology-led innovations, has an adoption and dissemination cycle that can span multiple years. This cycle depends not only on the mode of execution but also on organizational perspective. CMMI-style maturity models offer one way to explain differences among organ­izations in the adoption and use of SOA practices, but in our experience, the standard CMM maturity levels do not accurately capture the observed dichotomy between theory and practice.

Our research suggests that CMMI-style maturity models in general lack an important perspective - the perceived effect on the organization - that could account for the adoption, or lack thereof, of new technologies.

In addition, the sometimes competing views about SOA maturity uptake and delivery by IT, business managers, and organization leaders could be the result of undeclared assumptions regarding where, and to what end, SOA should be deployed. Finally, additional work is needed in SOA met­rics. We encourage other researchers to further develop our model with both qualitative and quantitative measures.

An SOA maturity model must incorporate both perspec­tive and execution maturity. Progress must be made across a 3D space, with movement from an IT-driven perspective toward an enterprise-transformation outlook - embracing governance, metrics, drivers, and even terminology - likely trumping execution refinements within a particular perspective.

About the Authors

Richard Welke is director of the Center for Process Innovation and a professor in the Department of Computer Information Systems in the J. Mack Robinson College of Business at Geor­gia State University. His research focuses on business process management and innovation in service-oriented enterprises. Welke received a PhD in management systems from the State University of New York at Buffalo. He is a member of the Association for Information Systems (AIS) and the Association of Business Process Management Professionals. Contact him at rwelke@ceprin.org.

Rudy Hirschheim is the Ourso Family Distinguished Professor of Information Systems in the E.J. Ourso College of Business at Louisiana State University. His research interests include IT outsourcing, IT management, and IT governance. Hirschheim received a PhD in information systems from the University of London. He is a fellow of the AIS. Contact him at rudy@lsu.edu.

Andrew Schwarz is an associate professor of information sys­tems in the E.J. Ourso College of Business at Louisiana State University. His research interests include acceptance of new technology, IT - business alignment, IT governance, IT outsourc­ing, and emerging technologies. Schwartz received a PhD in management information systems from the University of Hous­ton. He is a member of the AIS. Contact him at aschwarz@lsu.edu.

 Computer, the flagship publication of the IEEE Computer Society, publishes highly acclaimed peer-reviewed articles written for and by professionals representing the full spectrum of computing technology from hardware to software and from current research to new applications. Providing more technical substance than trade magazines and more practical ideas than research journals. Computer delivers useful information that is applicable to everyday work environments.

[1] R. Smith, “InformationWeek Analytics: State of SOA,” In­formationWeek, 21 Feb. 2009;

[2] M. Ren and K.J. Lyytinen, “Building Enterprise Architecture Agility and Sustenance through SOA,” Comm. AIS, vol. 22, article 4, 2008;

[3] R. Varadan et al., “Increasing Business Flexibility and SOA Adoption through Effective SOA Governance,” IBM Systems J., vol. 47, no. 3, 2008, pp. 473-490.

[4] A. Arsanjani et al., “SOMA: A Method for Developing Ser­vice-Oriented Solutions,” IBM Systems J., vol. 47, no. 3, 2008, pp. 377-396.

[5] S. Khoshafian, Service Oriented Enterprises, Auerbach, 2006.

[6] H.A. Smith and J.D. McKeen, “Creating a Process-Centric Organization at FCC: SOA from the Top Down,” MIS Quar­terly Executive, vol. 7, no. 2, 2008, pp. 71-84.

[7] J.P. Glaser, “Too Far Ahead of the IT Curve?” Harvard Business Rev., July-Aug. 2007; 

[8] R. Hirschheim, R. Welke, and A. Schwartz, “Service-Oriented Architecture: Myths, Realities, and a Maturity Model,” MIS Quarterly Executive, vol. 9, no. 1, 2010, pp. 37-48.

[9] H. Luthria and F. Rabhi, “Service Oriented Computing in Practice—An Agenda for Research into the Factors Influ­encing the Organizational Adoption of Service Oriented Architectures,” J. Theoretical and Applied Electronic Com­merce Research, vol. 4, no. 1, 2009, pp. 39-56.

[10] R. Smith, “InformationWeek Analytics: State of SOA,” In­formationWeek, 21 Feb. 2009;

[11] A. Arsanjani and K. Holley, “Increase Flexibility with the Service Integration Maturity Model (SIMM): Maturity, Adop­tion, and Transformation to SOA,” IBM developerWorks, 30 Sept. 2005;

[12] S. Inaganti and S. Aravamudan, “SOA Maturity Model,” BPTrends, 1 Apr. 2007.