SOA anti-patterns

A lot of emphasis has been placed on implementing Service Oriented Software according to best practices and principles. But how about the worst practices? In this article, Steve Jones from CapGemini goes over some of the most egregious and thorny antipatterns based on his experiences in the industry and discussions with other SOA thought leaders.

The purpose of patterns is to define how systems should be built in repeatable ways; the purpose of anti-patterns is to help you see when that hasn't been done.

Format

Each anti-pattern presented in this article will follow the following format:

• Description - What it is?
• Effect - What you see?
• Cause - What behaviours led to that effect?
• Resolution - What can be done to solve the problem?

Antipattern: The Shiny Nickel

Also know as the "Magpie"

Description

Used to incorporate the latest technology buzz within your SOA for the sake of telling people about it. For example, introducing an ESB for 2 services that have no translation or routing requirements.

Effect

SOA projects are characterised by their use of new technologies and their product procurement cycle more than the actual solution being delivered. Projects are talked about as being the "BizTalk", "WebSphere Process Server", "SAP Netweaver", "Oracle Fusion" etc project rather than the functionality the project aims to deliver. They are also characterised by the sheer volume of technology and a massive turnover in technologies being used. Technologies are rarely re-used from one project to the next as IT delivery moves to the next shiny nickel. Each project lays down another technology stratum that increases the cost and complexity of support and creates an integration challenge for future projects. This pattern sometimes evolves into the Technology Altar when allowed to.

Sometimes Shiny Nickel creates project paralysis as the project is continually delayed waiting for the "next" technology which will solve everything.

Cause

The Shiny Nickel anti-pattern is often caused by lax procurement rules and by a lack of common strategy in IT. Often driven by "golf-course" product selection from business and IT, these products go hunting for a solution. Often product procurement takes place independently from projects, which leads to technology decisions on projects being driven by a desire to minimise shelfware. Pet projects and pet technologies go hand in hand with both feeding off the other, projects are created because the technology can solve them, and the technology is used to deliver a pet project.

Shiny Nickel is also caused by technologists reading websites and going to vendor presentations and not investigating the detail of the solution before the actual project.

Resolution

Fixing the shiny nickel is about knowing where you want to get to. It means defining what you want your architecture and enterprise to look like in a few years, and understanding the steps, standards and therefore technologies that will take you there. It also means getting a grip on IT procurement so decisions are ratified against the roadmap.

Antipattern: The Technology Altar

Description

IT sacrifices business focus on the technology altar; all focus is on technical purity and a single IT "truth" for the enterprise.

Effect

This anti-pattern takes the form of a series of strategic IT projects that are not well understood by business users and deliver limited noticeable benefit. Business objectives are relegated to second place and business projects are de-prioritised while IT creates its strategic backbone, bus or SOA infrastructure. IT program is eventually cancelled before completion and never delivers the benefits promised. Single Canonical form, Enterprise Wide data models and infrastructure and a "single" solution to all business problems are all proposed as sensible solutions.

Cause

Sometimes linked to the Shiny Nickel anti-pattern this anti-pattern occurs when IT has become clearly disjoint from its business and is able to dictate both budget and focus for IT investment. The IT organisation focuses on "strategic" IT projects which tend to be focused on delivering a step-change in the organisation after several years of investment and the creation of a solid base. By trying to create an infrastructure in a single big-bang the IT organisation becomes separated from the actual requirements and tends to sacrifice sensible solutions is favour of technical perfection, it is this belief in the primacy of IT that leads to things like single canonical form and single EAI or ESB solutions being proposed without recognising that the world may actually change.

Resolution

IT has demonstrated an inability to manage its own budget effectively; the IT budget should be separated between Operations and Investment, with Operations covering only "business as usual" costs. IT Investment should be given to the business to spend on IT projects that it wants to see; IT must then look across these areas and "sell" common strategy elements across the business.

Antipattern: Percolating Process

Description

Organisations start with a detailed Process map and then attempt to "fit" this into a Service Oriented Architecture; this refactoring leads to process becoming the dominant feature and leads to a Process Oriented Architecture (POA) rather than SOA.

Effect

An organisation's "Services" come in two basic types, firstly end-to-end processes that co-ordinate lots of individual steps, and secondly a large set of fine grained services that represent individual steps. Any hierarchy or structure is solely from the basis of process. The fine grained services proliferate and become difficult to manage while the large business process elements become difficult or impossible to change. The systems slowly, or quickly, stagnate and lead to solutions being built on top of the existing solution and the general treating of the process oriented system as a legacy application.

Cause

The organisation has decided to map the processes and end to end and in detail; the work has created a series of grand "end to end" process models that are categorised by their large number of steps and the lack of sub-processes that they use. Once this exercise has been done to a great level of detail it is decided to make it SOA. The problem is that the valid business services have not been identified and thus the process maps have been created without a proper service structure. This makes identifying valid services a tricky process particularly when looking for cross-functional or horizontal services.

The challenge here is that the organisation has not started with clarity on its services and hence has created a Process Oriented Architecture. Retrofitting services into such an environment creates a sub-optimal system that is liable to have all the issues of other process based systems, such as COBOL.

Resolution

The first resolution is to independently of the process map create your services architecture. This will provide the structure for breaking down the processes and creating a clear hierarchy of use. Next, this service architecture should be over-laid onto the process map to understand where the cuts should be made. The current solutions can then be refactored to create a more service oriented solution by attacking the major inflexibilities in the system and then looking towards incremental change of the current systems.

Antipattern: Point to Point Web Services

Description

Web Service point to point is STILL point to point; doing a bad practice in XML doesn't make it better.

Effect

An organisation believes that they are creating a next generation loosely coupled architecture just because they are using Web Services. Web Service calls are invoked directly, using a URI which is hard-coded into the WSDL. After a period of time the multiple web service calls and dependencies lead to the Spaghetti infrastructure that EAI aimed to solve. Having a highly disorganised and interdependent systems model leads to increased cost of change and a high degree of fragility of the enterprise. When one service is changed a number of other services either fail or behave in unpredictable ways. There is a lack of clarity as to how one service depends on another and what the impact of change is across these services. Consumers then start demanding multiple versions of services, and these add further to the spaghetti in the enterprise.

Resolution

The first step is to indirect all of the calls and this can be done in a number of ways; the least invasive is to "proxy" all web-service requests via some form of mediation. So while the host application is still calling webservice.mycompany.com, this is proxied so mediation and routing can be applied if required. The next stage is to understand the different dependencies and identify those which are valid and those which should have been done in a more managed way. A clear governance plan then needs to be created to identify and manage dependencies and versions and to help manage the Web Service infrastructure.

Once this level of management has been created it is time to start considering what the enterprise Service model should be, that requires the creation of the business service architecture and then overtime evolving the current infrastructure to better represent the business that pays for it.

Antipattern: Splitting Hairs

Description

Splitting into two separate tiers of Service and Process, with separate rules and governance results in divergent solutions.

Effect

Applications and the enterprise are considered in distinct layers with those layers having different governance models, different technologies, and often at the enterprise level distinct project teams.This layering results in competition between "Service" and "Process" teams as to who should build new functionality; if the hairs are split further it results in Data, Security and other elements all being considered independently of each other and having different approaches to interaction and management. The IT estate begins to solidify around technology layers, rather than along business lines. Work is created to "keep teams busy" or misallocated to different parts of the architecture because one side or the other shouts the loudest. It becomes simple to call Process from Process, or Data from Data etc, but becomes difficult to call Process from Service and Data from Service as teams create their own special ways of working. The rate of change in the IT estate slows first in one layer and then in another. There is a lack of standardisation across the enterprise and the layers start being considered as a number of discreet, and now legacy, systems.

Resolution

The first step is to break the teams away from being centred around technologies and layers and becoming centred around the business services that you are trying to deliver. While the technologies for services that contain higher level orchestrations and those which deliver core logic will be different this should only be apparent inside the services, externally there must be a common mechanism and standard for how services are discovered and consumed.

This anti-pattern tends to occur when organisations become fixated on the technologies they are using to deliver rather than on the capabilities they are attempting to deliver with it. Keeping people focused on the function over the technology helps prevent this layering solidifying. It is important to clearly identify the right technology in which to implement a given set of capabilities, and then to make that set of capabilities exposed and accessed in a common way.

Antipattern: IT2B

Also known as "The people's republic of IT".

Description

Creating "business" services based on the belief that IT understands the business results in services that meet neither IT nor business goals.

Effects

IT decides to do SOA and then presents the results to the business. They create a services structure without consulting the business based on a best-guess using previous project learnings. IT then creates specific orchestrations through these services which represent their view on the "optimal" business processes. This is often done using a packaged solution and IT fails to take into account the cost and impact of change, which dooms the exercise. The service architecture created is then used as the basis of future IT project planning decisions, but as this does not match the business view it fails to deliver the expected benefits and becomes extremely strained as IT struggles to manage change within an invalid architecture. The IT estate becomes more complex and unmanageable with a large amount of fragility as the dependencies between services increase.

Resolution

Similar to the technology altar, the first stage is to clearly align the business with IT. It's possible that the reasons IT undertook this approach was because they did not receive clear long term direction from the business. The first stage must clearly identify what the services should be and then work on identifying how current services can be broken down and re-assembled along the correct lines. There should be clear business aligned governance within IT to make sure that future decisions are based around business drivers rather than ad-hoc best efforts decisions.

Antipattern: DIY Transport

Description

An organisation decides to build its own proprietary XML-RPC stack or other technology in order to have more "control".

Effect

To allow your service implementation team to start to develop their own concept of a protocol, transport or any service delivery mechanism. For instance, SOA over < ahref="http://www.ietf.org/rfc/rfc1149.txt?number=1149">RFC 1149 for interconnential service providers is probably not needed. This is often done in the mistaken belief that a companies' requirements are "unique", or because the technical team is planning for future possible requirements. The solution becomes more and more complex as the in-house team has to turn into a product development group, integration with other solutions becomes harder due to the lack of standards support and upgrades to new COTS and non-COTS solutions are delayed while the in house product is created.

Resolution

Rip out the proprietary solution and replace with a standard based Open Source or COTS product. The earlier this is done, the more money it will save in the long run.

Antipattern: Nobody Home

(credit goes to Dan Marchant)

Description

Used to allow a service implementation team to define services without a glimmer of hope for a client to actually use the service.

Effect

Services are created based on perceived rather than actual needs; these result in a proliferation of services without any consumers. However these services still require management and deployment. Services are exposed by developers from IDEs because it is "possible" rather than because it needs to be done.

Resolution

Create a company "register" of services and have a regular policy of removing services that have no consumers. Also educate people on your enterprise SOA so they know the sorts of services that are required.

Antipattern: Too many Cooks in the SOA

Also known as "A maze of twisty services, all alike".

Description

Multiple services with the same functional behavior are added to the registry without each group checking for service functionality beforehand. It is the proliferation of services that are identical to each other but are supported by different teams or within the same team in the enterprise.

Effects

Projects are more interested in building services than in re-using them; the numbers of services proliferates while the amount of functionality increases slowly. A culture of build and expose becomes endemic and the enterprise becomes a mess of services all attempting to do the same thing. Projects do not benefit from re-use, so costs are not reduced in development and the proliferation of services makes management of services more difficult. Change is in particular much harder as it must be applied in several different places.

Resolution

The reason that projects create lots of services is that they are allowed to. Governance and tracking needs to be put in place to enforce re-use. A simple recommendation is to fund service development differently from project specific functionality, thus re-using service functionality from elsewhere frees up additional budget for the project functionality, and the creation of genuinely re-usable services results in additional investment in the project to help create those services.

This needs to be combined with a simple search facility that enables people to quickly find the services that already exist.

Antipattern: UBER service

Description

The anti-pattern of creating a very limited set of services, or even just one, to try and provide all functionality within the enterprise SOA, related to Singleton SOA Anti-Pattern. This causes the functionality and operations being added to a service, to be mushed into a group of functionality that may or may not be related, creating tight coupling between them.

Effect

The desire to not create more services becomes a consuming passion for the organisation and services gain larger and larger interfaces representing more and more diverse functionality. Updating services begins to take longer as the conflicting requirements of these different pieces of functionality make change harder to implement. Behind these large service boundaries a separate layer of services often develop which are then aggregated behind the "uber" service interface. These lower level services become directly used via covert means within the enterprise, meaning that visions of centralised management become lost.

Often this approach occurs in connection with a DIY approach to security and management, the reduced number of services making the custom development simpler. Unfortunately this re-inventing of the wheel leads to a lack of standardisation making future package and service purchase decisions more complicated and it is very brittle to future change.

Resolution

Firstly determine what the true enterprise services, then go out and buy products that make this easier to manage. Then start breaking down, or even just removing, the uber-services. Implement the centralised governance using a proper governance solution and manage interactions between the services using a proper mediation engine.

Antipattern: A Million Services all in a row

Description

Standard anti-pattern where organisations go "Service Crazy" with a massive number of services created to result in a quantity rather than a quality of service.

Effect

Organisations approach SOA as a race to expose as many services as possible, to enable as much re-use as possible in a period of time as short as possible. This tends to be driven by technologists using IDE tools to "right-click" and expose elements as a web-service.

The organisation developers taking pride in the "number" rather than the quality of services is a prime symptom. This is all done with a total lack of a "big picture" as to what the organisational services should be. These efforts tend to be driven out of IT organisations that are often disconnected with the business and are focused purely on Web Services. These approaches are driven by a desire to directly expose technical code. No mediation or grouping occurs and even non-functional elements are exposed in the desire to create as many services as possible.

This results in a large number of normally fine grained services with a very close binding of Web Services to implementation. There is a total lack of structure with no business alignment. Added to this there becomes a large amount of point to point calls being made that help to create the start of the "rats nest" problem. The basic root cause of this is that the technologists mistake SOA for Web Services and as a result try to create as many as possible to look cool.

Resolution

The first step is the cleansing and categorisation of services for deletion or refactoring, mostly deletion. This should include the complete removal of created artifacts, if they haven't been used yet. The IT organisation then needs to start to understand how the business operates and focus on creating services for the business rather than engaging in a web services arms race. This means the creation of the over-arching "big picture" and migration plan for the existing services to the "To-be", generally involves simply asking the business what they want!

Antipattern: Architectural Stovepipe

Description

An enterprise architecture that levels too many expectations on present and future applications. One example occurred during the MOM era, where in order to conform to requirements from the enterprise architecture contained applications suffered unnecessary complexity.

Effects

An Enterprise Architecture group seeks to establish a "grand plan" with a big and bold vision for the future. This architecture has been created separately from implementation teams and seeks to address large strategic challenges of the business. Unfortunately it becomes a tenant of faith that all of the architecture should be implemented as it is, which leads to excessive complexities on even the simplest project. The Enterprise Architecture group holds a compliance position over projects and isn't burdened by technical or project realities; this begins to create a clear separation between the enterprise vision and the projects, as a culture of avoidance is created. The Enterprise Architecture often fails to be modified as business objectives change, or is modified in a yet more complicated and "visionary" way.

The Enterprise Architects become disconnected from the business and are seen like White Robed philosophers coming down from Mount Olympus preaching wisdom and visions of the future; this is beginning to be seen more an entertainment than an enlightenment.

Finally the Enterprise Architecture is abandoned and the resultant projects and infrastructure become yet another stratum of the enterprise infrastructure.

Resolution

Firstly the Enterprise Architecture must be "kept real"; this means both in terms of its vision for the business and the practical realities of delivery. Having a visionary architecture that is created first and only then mapped to technology doesn't result in the best architectural vision; it must be tightly bound by what can be done, as well as what should be done in future.

The Enterprise Architecture must have a clear practicality and implementation roadmap; it must say what the steps towards the final vision are and must present the costs that justify each of these steps. The vision should be that of the business rather than of an architectural "purity" that exists only for its own aims.

Generally the architecture team should be led by someone who understands implementation problems and challenges.

Antipattern: Defensive SOA

(credit to Miko Matsumura)

Also known as "Optimistic SOA".

Description

The default answer produced by your Information Technology group whenever asked is "we already ARE doing SOA". Certainly there is nothing wrong with this response if it were true. But beware "definitions" of SOA that do not meet the organizational requirements. This is a SOA Governance problem, in the sense that SOA for your organization needs to be defined to the extent that the benefits are clearly articulated, measured and audited. Traditional software development is filled with extremely bright people. The significant challenges faced by organizations grappling with the large list of external changes and the difficult processes all seem to invite developer focused solutions. But in many cases you can't solve the problem in the mindset you used to create the problem.

Effects

A number of change management mechanisms have emerged as a way to deal with changes. All of the technologies listed here are virtuous and powerful. Seeing the presence of these techniques in and of themselves does NOT denote an anti-pattern.

The concern is that the use of band-aid solutions to solve short-term pain forestalls the advancement of solutions to the chronic problem of fundamentally inflexible IT.

The use of the following solutions UNDERNEATH the service interface can be cause for alarm-because agility in SOA is defined in policies and metadata constructs (like BPEL) which sits ABOVE the service interfaces. Once again, there is absolutely nothing inherently wrong with any of the following technologies, save for when their use precludes and obstructs the advance of network centric SOA.

1) Aspect Oriented Programming

A powerful system which enables transformation of software systems through "join points", "code weaving", "point cuts" and "cross cuts". AOP is primarily a design time programming model, thus subject to all of the complex lifecycle processes involved in testing and deploying new software projects in the Enterprise. Using AOP to do things which could better be done by a policy enforcement point is a warning sign.

2) JRS175 Metadata within Java Code

This capability allows metadata tagging to be subsumed within Java code. This has many uses but can also be abused as a way of injecting declarative policy sets within Java code. This has the disadvantage of being configurable only by Java programmers.

3) Spring and other frameworks

Another powerful system for managing change involves the use of powerful programming frameworks. Spring, for example, has a variety of features, but most noteworthy are the Inversion of Control features which enable dependency injection. These features can be used to manage highly changeable software environments.

4) Test first development

By building a robust use case based test harness before implementing code, a virtual "contract" is built. The semantics of that contract are all of the test conditions for the service. This seems to be a strong approach to change management, as it provides a solid way of assessing if changes would impact use cases. Unfortunately, the test cases are hard to verify in a way that captures all possible changes and externalities.

5) Source Code Reuse "Repositories"

Leveraging source code repositories for reuse is not SOA. Reusing a deployed service is transacting against a live Enterprise asset. Reusing source code does not save the organization from the complete application development, testing, provisioning, deployment, approval and other application development lifecycle requirements.

None of these mechanisms intrinsically enables the alignment of business and IT constituencies because the complexity and control of policy and of the model gets stuck squarely in the development organization.

As implementation techniques for services, these are well and good, but if your entire architecture can sit inside of a single virtual machine, you may have a problem.

Cause

In 90% of cases, when business people go to conferences and come back with a buzzword, they are wrong and need to be appeased. Therefore, when developers see business people coming back from SOA conferences, they react in this defensive way. Unfortunately, SOA is within the 10% of the time where the business user actually has it right. Flexible IT, Business Agility, Business IT alignment are all sorely lacking in tightly coupled land, and SOA can actually help.

Resolution

This requires business people to actually try to understand what IT are doing. It also requires IT people to understand what business people are trying to do. Mechanisms like service interfaces, BPEL diagrams and other abstractions can help smooth the conversation.

Warren Buffet has a saying "If you can't understand it, don't invest in it." This applies strongly to SOA. If a business person is sold a very complex UML diagram or a mind boggling looking development process, it's incumbent upon them to ask for a clearer articulation-and it's incumbent on the IT people to provide that. If the SOA implementation plans coming out of IT are filled with extremely Byzantine gobbledygook, you need to send IT back to the drawing board.

What aligns business users and techies in SOA is that it should be operating at a level of abstraction that is easily understandable to both sides. If that's not the case, it's time to get worried.

Reference: