Service Design Patterns, the latest addition to the Martin Fowler Signature Series authored by Robert Daigneau with a contribution on consumer driven contracts from Ian Robinson, addresses web service design patterns that cover the entire service lifecycle from creation through to continual improvement. The patterns are categorized by service lifecycle phase, associated with patterns from PoEAA and EIP and illustrated with code examples in Java or C#.The book concludes with a useful evaluation of patterns for their impact on service evolution.
InfoQ caught up with Robert Daigneau to discuss some of the ideas behind the book and some relevant applications of them. The books publishers, Pearson Education, has made available an excerpt on the Resource API pattern for the InfoQ community.
InfoQ: The book does a great job at classifying the patterns but in your opinion, do there exist groupings of patterns across these classifications that are ideally implemented together and some that are mutually exclusive?
Robert Daigneau: The book narratives explore many of the possible relationships that can exist between patterns in different categories. I cannot provide an exhaustive listing of these combinations, but I can point out several "cross-category" groupings that are relatively common, natural, and complementary:
- (RPC API or Message API or Resource API), i.e. any "Web Service API Style" and ...
- Message API and ...
Again, this is not an exhaustive list. There are innumerable ways in which the patterns may be combined. Also, I cannot say that there are specific combinations of patterns that are ideal. All design patterns have benefits and trade-offs. Developers and architects should carefully consider the context (i.e. specific constraints like your budget, team skills, schedule, organizational politics, rate of change, etc.) for the problem at hand in order to determine what pattern combinations will likely offer the best balance between their inherent advantages and disadvantages in the short and long-term.
The book only calls out one cross-category combination as being mutually exclusive. This is the combination of the RPC API and Single Message Argument. The latter pattern is applied when developers working with an RPC API use a "Code-First" strategy wherein the service API and Service Descriptor are derived from the signatures of a class method. Other than that, there are certainly cross-category combinations that are awkward or unnatural. For example, I have yet to see a good example of an RPC API that uses Media Type Negotiation. Another example is the pairing of Resource APIs and Service Descriptors. REST proponents generally discourage this combination for reasons that are explored in great detail in the book. Nevertheless, frameworks like WCF support this combination to facilitate automated generation of Service Connectors.
I should point out that the back cover inset shows how patterns within a specific category may be selected. It shows, for example, when developers may "Pick One" or "Pick Zero or More" patterns within a given category. Therefore, when the option is to "Pick One", then the pattern choice is mutually exclusive in that category.
InfoQ: Which patterns are more applicable to enterprise environments versus the Web and vice-versa?
Robert Daigneau: I'll assume that when you say "enterprise environment" you're referring to situations which call for a relatively higher degree of intra-company or B2B collaboration, review, consensus, rigor, and governance. Perhaps the parties must ensure compliance with certain regulatory or contractual obligations. Given these assumptions, a "web environment" might have fewer constraints. For example, a start-up providing a Software-as-a-Service (SaaS) product for social media might offer web services whose design can evolve with fewer constraints and oversight. In any event, the patterns described in the book are not specifically aligned to either of these scenarios. That is, they may all be used in enterprise and web environments.
I should point out that even Resource APIs that don't use Service Descriptors can be applied in environments which require a high degree of governance. Some believe that descriptors must be used in these situations, however, their use does not ensure proper governance. In fact, the prime reason to use descriptors really isn't for governance, it's to provide developer tools with the information necessary to use services and to help in the generation of client side code (i.e. Service Connectors). In any event, since descriptors don't guarantee governance, and Resource APIs typically don't use them, then it is possible to use Resource APIs in enterprise architectures too. The parties must, however, devise a different set of processes to manage API changes. One of the best ways to do this, for all API styles, is with the Consumer-Driven Contract.
InfoQ: Chapter 6 includes a section on SOA infrastructure patterns, are these applicable to a ROA?
Robert Daigneau: Yes, the SOA Infrastructure patterns mentioned in the book (i.e. Service Registry, Enterprise Service Bus, and Orchestration Engine) can certainly be used with ROA. Hopefully your readers won't mind if I provide a quick overview of ROA before I discuss each of these patterns and examine how they might be used with it. Resource-Oriented Architecture (ROA) is a set of concepts and design guidelines proposed by Leonard Richardson and Sam Ruby in their book, "RESTful Web Services". It entails, amongst other things, a process whereby logical resources, URI patterns, representations, and linking protocols (for hypermedia) are identified. ROA also emphasizes HTTP and adheres to the REST architectural style.
The Orchestration Engine (pattern) is totally compatible with the ROA concept. Keep in mind that the Web Service API Style that results when following ROA is a Resource API. ROA is more concerned with the external interface that clients communicate with rather than the internal web service implementation. Therefore, one could use ROA to create a Resource API and implement some services such that they will route certain requests to an Orchestration Engine. In many cases, a Workflow Connector will be used to achieve this end.
The book attributes three core functions to the Enterprise Service Bus (ESB) ... message routing, message translation, and protocol-bridging (e.g. the ability to receive a message over HTTP and forward it to a message handler over JMS). Many vendors and SOA aficionados assign additional responsibilities to ESBs (e.g. authentication, authorization, workflow/orchestration, etc.). In any event, the ESB is compatible with ROA as well. One could, for example, design a Resource API using ROA techniques and publish this API as a set of "Virtual Services" that receive requests into the bus. The bus would also transform requests into the format required by the actual target web service, which may use any of the Web Service API Styles, and route them using the required transport protocol (e.g. HTTP or other).
Service Registries are typically used within "enterprise environments" that adopt RPC APIs and Message APIs. This pattern hasn't seen significant use with Resource APIs that are created through the ROA methodology. Keep in mind that I define registries as being central repositories that store such things as message definitions, service usage policies (e.g. authentication, encryption requirements, etc.), and Service Descriptors (e.g. WSDL). Their goal is to promote reuse and standardization of these artifacts, assist developers with service design and composition, and facilitate governance in corporate IT environments. Most registries to date have relied, in part, on WSDL and XSD artifacts to facilitate governance. While the originators of ROA aren't fond of WSDL, they were open to WADL (another type of Service Descriptor) when they wrote their book. Unfortunately, WADL hasn't seen wide-spread adoption, and REST proponents dismiss the very concept of Service Descriptors. In reality, most "service metadata" for Resource APIs created through the ROA approach is documented through traditional means (e.g. web pages, word documents, etc.).
Another function of the Service Registry is to support "Run-time Discovery". [Aside: Many SOA proponents attribute "Service Metadata Management" (the topic of the prior paragraph) specifically to "Service Repositories" and "Run-Time Discovery" to Service Registries. I purposely merged these concepts in the book]. The traditional approach to discovery typically involves a client (application) which uses a Service Connector that communicates with a Registry at run-time to find a particular service (address) and then "bind" to it. This objective (i.e. finding and binding) is achieved through a starkly different technique with ROA, and is described in the Linked Service pattern. In this pattern, the client contacts a "root web service" at the beginning of a conversation in order to acquire information on services it's interested in. This information may, for example, be provided in an Atom-Pub Service Document. After that, the client parses each web service response to find URIs for services that it's interested in and the media types consumed or produced by those services. I'm not sure that the approach to service discovery that we've seen offered to date by Service Registries will ever be compatible with ROA or REST since Linked Services are essential to both concepts, and the links in responses tend to be quite dynamic and specific to individual client-service interchanges. There's a lot more to say about these topics, but we'd probably be going far beyond the scope of this interview.
InfoQ: The book sticks to Java and C# based frameworks based on popularity, do you consider them the most mature in the market? Are there some patterns for which you would recommend using specific frameworks or are the capabilities comparable across the board?
Robert Daigneau: I wouldn't say that the frameworks featured in the book are more or less mature than others. I only chose these languages and frameworks for pragmatic reasons. I started with the premise that if I limited the number of languages and frameworks that were used to explain the patterns, then the reader's understanding would be facilitated. I selected Java and C# because most developers are familiar with their syntax. Once these languages were selected, the choice of frameworks was easy. In the case of C#, it was obvious that WCF should be used. As for Java, JAX-WS and JAX-RS were selected because they are both in Java EE. Given this context, I believe it's important to say that the patterns discussed in the book are not supposed to be technology specific. Rather, they can be implemented with a variety of platforms and technologies like Ruby (on Rails), PHP, JavaScript, JQuery, Perl, Google Protocol Buffers, Apache Thrift, etc. My goal was to codify common design solutions and principles that will be relevant and valuable despite the inevitable changes in technology or industry specifications.
Regarding the frameworks mentioned in the book, JAX-WS is appropriate for creating RPC APIs, while JAX-RS is suitable for Resource APIs. Either framework can be used to develop Message APIs. WCF is unique in that it can be used to create services for all of web service API styles.
InfoQ: If you were given the chance to include a chapter on anti-patterns in the book, what would be some widely present anti-patterns in existing codebases that you have come across.
Robert Daigneau: Many of the most common service design anti-patterns are hinted at in the forces section of each pattern narrative. Amongst other things, these sections discuss solutions that have been used to solve specific problems, but haven't always worked out so well. And this is the essence of what anti-patterns are ... common solutions that often turn out to hurt more than they help. Here are a few that are called out in the book:
- Projecting the Domain Model or Database (Tables) out through the Service API: Developers often annotate Domain Model objects with serialization attributes and use these objects in service requests and responses. While this is a quick, easy, and common approach, it creates a very tight coupling between clients and these back-end entities. This means that whenever a change is made to a domain object (or a table corresponding to the annotated object), that change will ripple out to clients forcing them to change as well. Likewise, if a client asks for a change in a request or response structure, then these back-end entities will probably have to change as well.
- Putting too much business logic in Web Request Handlers: Web services often contain all of the logic required to implement a business function. This logic may entail input data validation, calculations, database access logic, and transaction management. The problem is that this code can become quite lengthy and very complex over time. Such logic is often duplicated across web services too.
- Flat APIs: Developers who create RPC APIs though a "Code-First" approach often create flat APIs. This situation is recognized when a web service method has long parameters lists. The problem is that these signatures are inherently inflexible and fragile. Developers often incur breaking changes whenever the need arises to add, change, or move parameters.
- Fine-Grained Services: Fine-grained services force clients to get or set relatively small data-sets and therefore encourage "chatty conversations". This, in turn, leads to problems with latency, and increases the potential for partial failures (i.e. situations in which the client, server, or network independently fail). For these reasons, service APIs should be approached in a fundamentally different way than Domain Models or similar layers that wrap access to database tables.
- Using Web Services Everywhere: Web services can certainly make it easier to reuse business logic and data, but they should not be used everywhere. The problem is that web services are extremely expensive to call due to the costs of serialization/deserialization processes, data transmission over the network, and the server intercept and routing processes. Developers must also be prepared to handle partial failures.
Fortunately, the book addresses these problems.
InfoQ: With the explosion of mobile apps on devices varying hardware resource constraints, what is the effect on the client infrastructure patterns described in the book? Do they need modifications or do these same principles apply?
Robert Daigneau: You bring up a very important consideration. Mobile clients are constrained by limited power, memory, and disk. All client infrastructures should therefore be designed with this in mind. Consider, for example, garbage collection. Many platforms have moved away from this feature since, when the collector kicks in, it can really place a significant burden on the device's battery. The take-away is that all client infrastructures used on mobile devices should be light-weight and conserve the device's resources.
Let's take a look at some of the infrastructure patterns mentioned in the book. The Asynchronous Response Handler pattern addresses the problem of blocking while the client waits for a request. Implementations of this pattern probably don't need to be designed specifically for mobile devices. In fact, many mobile frameworks prefer an asynchronous interaction pattern. Service Connectors probably won't need to be designed especially for mobile devices either. However, the trend is towards more connectors being implemented as Service Gateways (a type of connector whose interface is not derived from a remote procedure or Service Descriptor) that communicate with Resource APIs. This API style tends not to force mobile developers to have to redeploy their applications in "lock-step" whenever a service changes, as is often the case for RPC APIs.
I would suspect that there will be less use of Service Interceptors in mobile applications to do things like up-front message validation, logging, and the like; their use implies that the device would have to do more work. I also think we'll see less use of the Idempotent Retry pattern on mobile devices for similar reasons. Instead, we'll probably see most mobile applications simply catch service interaction failures and give a warning to the user. Finally, I don't think we'll see mobile clients using Orchestration Engines either since, due to the nature of their responsibilities, they tend to be rather heavy-weight infrastructures that consume significant resources. Furthermore, it doesn't really make sense to have a mobile application orchestrating long-running processes.
About the Book Author
Robert Daigneau has more than twenty years of experience designing and implementing applications and products for a broad array of industries, from financial services, to manufacturing, to retail and travel. Rob has served in such prominent positions as Director of Architecture for Monster.com and Manager of Applications Development at Fidelity Investments. Rob has been known to speak at a conference or two.
“This excerpt is from the book, ‘Service Design Patterns: Fundamental Design Solutions for SOAP/WSDL and RESTFul Web Services’, authored by Robert Daigneau, published by Pearson/Addison-Wesley Professional as part of the Addison-Wesley Martin Fowler Signature Series. ISBN 032154420X, Oct. 2011, Copyright 2012 Pearson Education, Inc. For more info please visit the publisher website.