Why SOA Should Be Viewed As “Dependency-Oriented Thinking”

This is the link for the second part of this article: How SOA Governance (and SOA Management) Should Actually Be Done.

I’m going to start with a rather confronting assertion. I claim that no matter how mature your organization’s SOA practice is considered to be, SOA is probably a disappointment to you. As a SOA practitioner who has had a ringside view of industry developments over the last decade and a half, I believe I know why this practically has to be so, and I’m about to share some startling insights with you.

SOA emerged as a buzzword around the turn of the millennium and rapidly became an industry of its own, making careers and determining winners and losers in the technology marketplace. From a business perspective, the promise of SOA was simple and three-fold. It promised greater business agility, sustainably lower operating costs, and reduced operational risk. A decade and a half later, however, the return on the massive investment made by an enthusiastic business community has been anemic at best. Not all SOA projects have failed spectacularly, but most have failed to deliver results commensurate to the hype. SOA today elicits derision in greater measure than it does respect.

The good news is that there isn’t anything wrong with SOA itself. It’s that we as an industry have misunderstood what SOA was about, and have gone about adopting it the wrong way. If we unlearn some of those bad ideas and teach ourselves better ones, we can solve the same business problems with far more dramatic success, often using many of the same technology tools. It’s our approach that’s faulty.

Let me illustrate.

If you had to reduce the principles of SOA to one practical rule of thumb for designers, what would it be? I’m willing to bet it will be “Avoid point-to-point connections”. That’s the rule of thumb I’ve heard dozens of SOA experts provide as advice at more than a few organizations I’ve worked for. On the face of it, that sounds like perfectly reasonable advice, because everyone knows that point-to-point connections cause rigidity and brittleness, and SOA is all about flexibility and agility, right?

But I’m about to tell you now that this simple rule of thumb epitomizes all that is wrong with the practice of SOA today. Not only does it misstate the problem, its blind application can lead practitioners to believe they are building flexible systems when they are doing the very opposite. It can also cause them to turn their backs on some very simple and flexible solutions that they wrongly believe to be rigid.

Here’s what conventional wisdom would claim to be an example of bad design.

If asked to suggest a better alternative, designers may respond with this.

But let me show you a well-known example of a “point-to-point connection” that is not rigid at all.

We know this system is not rigid because this is how the web works. A browser that bookmarks the URL of a page need not be left with a stale link just because the actual web server hosting that page has changed its physical location. Yet, if we only look at the connection between System A and System B in the above diagram, it would appear to run afoul of our rule of thumb, “Avoid point-to-point connections”.

Let me now show you a converse example of a mediated connection that is not flexible.

If System Y can see an internal data element within System X, then no matter how many intermediary nodes there may be between them, a change to that element will affect System Y. This is a textbook case of rigidity, or “tight coupling”. Following the rule of thumb of avoiding point-to-point connections has not helped us at all!

These two simple examples should illustrate that the approach to SOA commonly followed in the industry is incorrect and is quite possibly causing us to ignore some good design patterns and to implement some poor ones.

But is there an alternative?

There is, and it lies in the fundamental concept of dependencies.

The term “dependency” is not a fancy word with an abstruse technical meaning. It means exactly what you think it means. If Entity A is dependent on Entity B in some way, then it has a dependency on Entity B. That’s all there is to it.

UML even gives us a formal notation to specify a dependency, and this is just a simple dashed arrow.

Looking at systems through the lens of dependencies (rather than in terms of point-to-point versus mediated connections) can help us achieve better designs. Let me show how.

For starters, let’s recast the two previous examples in terms of dependencies, using the UML notation.

Once we move away from the (arguably unhelpful) notion of “connections”, we can see that in the second case (Fig. 7), System A has no dependency on System B, because the directory or DNS takes responsibility for abstracting the location of System B. The directory/DNS has a dependency on System B, because it needs to be updated whenever the location of B changes, and System A in turn has a dependency on the directory/DNS to tell it where to find B, but there is no direct dependency between A and B as in the first case (Fig. 6). It has been effectively broken by the prior lookup.

The underlying nature of the second example is also obvious when drawn using dependency notation.

It’s clear that the existence of mediators does nothing to remove the fundamental dependency that System Y has on System X. In the day-to-day organizational SOA context, designers routinely employ expensive mediation technology (ESBs) and are often blissfully unaware that they have failed to decouple systems because they have neglected to analyze the real dependencies between them. ESB-oriented architectures are therefore not a substitute for Service-Oriented Architecture.

I hope I have now established the superiority of the dependency-oriented approach over the simplistic rule of thumb (“Avoid point-to-point connections”) as the practical way to do SOA.

This was just the five-minute overview. There is a great deal that goes into a formal dependency-oriented method, and the word limit on this article prevents me from describing it fully here. I will merely touch upon a few important points, then refer you to my book “Dependency-Oriented Thinking: Volume 1 – Analysis and Design” for a more in-depth treatment of the topic.

How technology can introduce extraneous dependencies

It’s bad enough when designers fail to eliminate dependencies between systems in the course of implementing “SOA” solutions. It’s much worse when their tools and technologies inadvertently introduce extraneous dependencies into the environment.

If you use SOAP-based Web Services, then I have some bad news for you. You have probably, and entirely unknowingly, introduced extraneous dependencies into your environment through the use of WSDL. Why do I consider WSDL a dependency hotspot? This diagram should explain better than words.

You may be able to corroborate this from your own experience. How often do you have to issue new versions of a WSDL file because of a relatively minor change? How many versions of a WSDL do you need to maintain concurrently to avoid impacting older clients?

The lack of proper science around interface design can make SOA technology a source of dependencies rather than a means of alleviating them.

Most organizations have no formal model of version control

Ask yourself what your biggest bugbear with SOA has been. I’m betting that the answer is “service version proliferation”. I’ve seen this in virtually every organization I’ve worked for, including those with a highly mature SOA practice.

New service versions are created in response to changes in the signature of operations, but as almost always happens, there is an ad hoc feel to the way changes are handled.

I have rarely seen a formal specification of what constitutes a “breaking change” to an interface and what is a “non-breaking change”. Here are my lists.

Even rarer is the use of these definitions as a basis for version numbering. The formal dependency-oriented method I have developed has these simple rules.

• Service versions are expressed in a three-level format (“x.y.z”, where x, y and z are non-zero numbers). E.g., 2.1.7
• Changes that are not visible at the interface only increment the lowest version number (“z”).
• Changes that are visible at the interface but still support older clients (i.e., “non-breaking changes”) increment the middle version number (“y”).
• Changes that are visible at the interface and will not support older clients (i.e., “breaking changes”) increment the highest version number (“x”).
• Interface definition files such as WSDL will be versioned in terms of “x.y” but will internally only refer to “x”.
• Clients will only be referenced in terms of “x”. A version “x” client will always see the latest “x.y” version of an interface, which is by definition a non-breaking change.
• The implementation of an “x.y” interface will be the latest “x.y.z” version of the logic, which is by definition an invisible change.

When applied in conjunction with a couple of data design principles, versioning becomes simple, intuitive and manageable.The following overview diagram is a teaser.

(Click on the image to enlarge it)

REST and SOAP have different dependency characteristics

It may be wrongly surmised from my comments on WSDL that I consider REST to be far superior to SOAP. On the contrary, I consider both SOAP and REST to be valid approaches to SOA, but they each need to be approached with an understanding of the peculiar dependency characteristics they exhibit.

SOAP-based systems usually follow a process coordination model called orchestration (the WS-BPEL model), while REST-based systems follow a model that is closer to choreography (strictly speaking, a hybrid between orchestration and choreography). The following diagrams should provide a quick overview.

SOAP and REST may look superficially similar from an invocation perspective, and the only differences between them may seem to stem from encoding (JSON versus XML) and wire protocol (SOAP/WS-* versus HTTP). However, the dependency graphs that emerge from an analysis of a complete business process can throw up surprising differences, as the above diagrams illustrate.

There is a great deal of sophistication and conceptual subtlety in the proper design of both SOAP- and REST-based systems. Designers can learn to harness the power and the elegance of both approaches using a single formal method. That method is Dependency-Oriented Thinking.

Summary and Conclusions

I’m aware this article with its various “teasers” has stopped tantalizingly short of describing the dependency-oriented method in full, but if it has whetted your appetite to read further, then it has achieved its purpose.

Let me leave you with two final diagrams. The first provides an overview of the entities of relevance to SOA at various levels in an enterprise. The second provides an overview of dependency principles to be applied at each of those levels to arrive at a suitable design.

For a more in-depth look at Dependency-Oriented Thinking, read the eBook: Dependency-Oriented Thinking: Volume 1 – Analysis and Design.

This is the link for the second part of this article: How SOA Governance (and SOA Management) Should Actually Be Done

About the Author

Ganesh Prasad has 27 years of IT experience and has been a developer and application designer through four generations of technology - mainframes, minicomputers, client-server and the web. His experience as an architect over the last 10 years, especially in the Shared Services area, has led him to pioneer radically simpler and less expensive alternatives to traditional corporate IT strategy.