A new "Open Source SOA" book by Jeff Davis provides a wealth of invaluable information on selection and usage of the open source products for SOA implementation.
In his new book "Open Source SOA" Jeff Davis discusses advantages of the open source product usage for building SOA solutions. He starts his book starts by discussing main SOA principles and overall implementation architecture and then provides criteria for choosing a product for every layer of proposed architecture. The book then goes into details of each product, describing how particular product works, how it can be used in SOA implementation and how it can be integrated with other selected products.
Manning provided Infoq readers with the excerpt from the chapter two of the book, focusing on selection of the open source products for SOA implementations The book is packed with an extremely useful - and hard to find - information about existing open source SOA products, their capabilities and the best ways to leverage them. A lot of software examples, contained both in the book itself and on the Manning book's site, make the content of the book even more practical for developers.
InfoQ had a chance to interview Jeff.
Infoq: In your book you define SOA as a collection of "discrete, reusable business services that can be used to construct new and novel business processes or applications", but then you compare it to different distributed computing approaches. How much do you consider SOA as a technical - distributed system - vs. business problem? How do you distinguish between services, business processes and applications?
Jeff: That's a great question! First off, I would consider SOA an architectural paradigm that provides a blueprint for how you develop software solutions. In that sense, it is a technical challenge driven by business needs that now demand greater flexibility, better integration and sharing between applications, and improved agility to respond quickly to market changes. If you embrace the SOA approach, applications become less monolithic in nature, and begin to more resemble mashups. Why? Because applications can be composed from a variety of services, and these services can leveraged by many different applications/clients.
I would consider business processes as an orchestration of many services, which may or may not include a human element (such as for tasks that require a human-in-the-loop). With services as building blocks in SOA, applications and processes can be written to leverage them.
Infoq: A book touches on a Web Services (SOAP) vs. REST controversy. Do you consider that moving forward Web Services and REST will peacefully coexist? Do you see any specific areas where usage of one is advantageous?
Jeff: The main advantage to SOAP is the WSDL. It provides you the specification necessary for how to invoke a given service. Tooling, such as provided by the .NET and Java platform, can thus easily consume a WSDL and basically code-generate the backend classes needed to interface with the service. Personally, I think that's of great value. Of course, this adds to the complexity of SOAP, and lots of people are horrified when they have to try and understand a WSDL. Now, there are efforts underway in the REST world to provide something analogous to a WSDL (but without its admittedly arcane nomenclature). For example, the Web Application Description Language, or WADL. It serves a similar function to a WSDL. Also, applying common security patterns in the REST world doesn't seem to be as established as in SOAP, where WS-Security is becoming more widely embraced. I think REST will continue to thrive for consumer type web services, but in the enterprise, I think SOAP will gain increasing acceptance.
Infoq: In your book you are equating service interface to service contract and suggest to use WSDL for both. Do you consider WSDL to be sufficient for the service definition? How do you foresee defining such things as service QoS? Service/information semantics?
Jeff: You're right, a WSDL really defines the parameters, operations, content requirements and endpoints of SOAP-based web services. As a consumer of a service, this is obviously the most important thing when it comes to interfacing with a service. Obviously, what a WSDL doesn't address is what the service really is for, nor is it a true registry of what services are available (unless you stuff everything in one-big WSDL, which is done a lot). I do have an overdue bonus chapter I am working on that covers implementation of a registry using WSO2's registry product (my blog page will post the free chapter when it is available at http://jdavis.open-soa.info/wordpress/). As for QoS related semantics, it appears as though WS-Policy is often being extended for such purposes. For example, using Apache Synapse, you can define policies that drive how many times a given client can access a service, and manage service priorities.
Infoq: When talking about core characteristics of SOA you are service's statelessness. Later in the book you are also talking about SCA (stateful) conversations. How can these two paradigms coexist?
Jeff: I suppose in a perfect world all services would be perfectly stateless and the Packers would win the Super Bowl every year. Unfortunately, that isn't always feasible or practical. Some complex services require a more interactive exchange with a client. I think that should be avoided when possible, but sometimes you can't. I think SCA's conversational support is pretty clever, but it does take some time to figure out and understand from an implementation perspective (hopefully my book will help in that regard).
Infoq: In your book you have an SOA diagram, which depicts presentation as one of the SOA layers, but you do not discuss either products or approaches for implementing presentation. Does SOA, in your opinion, requires specialized presentation and what can you recommend for its implementation?
Jeff: I purposely didn't want to get into the presentation side of things, because there are a bewildering number of choices that one can use. However, many of the modern GUI frameworks such as Flex, Tibco GI, GWT (SmartGWT in particular), Ruby on Rails, all provide pretty strong support for interfacing with web services, be they REST or SOAP in nature. In the perfect world, I suppose, the client layer could simply access and consume the various services exposed through your SOA environment, but in practice, a web application is likely required to manage page flows, security and the like. However, the web application can obviously be client in its own right of the available services.
Infoq: You have a great criteria table for evaluating Open Source products, but it does not contain "Adherence to standards” as evaluation criteria. In your opinion, how important is standard support for the open source product selection. Also many open source products, for example, JBoss and Mule, allow you to purchase support for their products. How important, in your opinion, is an ability to buy support for a product?
Jeff: That's a good point, and an omission on my part, insofar as the evaluation criteria. Obviously adherence to standards is important, particularly when it involves how your services will be exposed. In the SOAP world, I think it's very important that your services framework create services that are WS-I compliant, for instance, to smoothly work with .NET and Java clients. Also, if you are developing BPEL scripts, it may be very important to you that you can move those scripts from one framework or application to another and not have it overly sprinkled with proprietary extensions. However, such as in the case with Mule, is the fact that it isn't a JBI compliant container relevant? Perhaps if you want to move your adapters from one ESB provider to another it is, but usually you are selecting a product with the intention of sticking with it for some time (and Mule obviously adheres to many of the standards as it relates to JSON, RSS, SOAP, REST etc), and those considerations may not be foremost on your mind (as compared with ease of implementation, scalability etc).
As for being able to purchase support, I think that provides a nice comfort level, and can help ensure the long-term viability of the project. If you look at it, most of the real successful open source projects have some sponsoring company that derives revenue in some fashion from it, be it through selling support or consulting implementation. What I do tend to eschew is the model where the open source product is limited in functionality in some way, and you have to buy a commercial cousin to get what you really want. I much prefer the model used by JBoss and WSO2 over that of Mule or Active Endpoints, for example. I do think it's very important to give back to the community, even if a sponsoring company is behind it, as that helps advance the product and contribute towards its long-term success.
Infoq: When discussing service registry you are using the words registry and repository more or less interchangeably. Do you consider them to be one of the same? Also registry discussion in the book revolves mostly around supporting design-time artifacts, while registry product selection includes many LDAP implementations, which are optimized for supporting run-time (identity) information. Do you consider run-time usage of registry as well as design-time support?
Jeff: I think the old notion that dynamic discovery of services, as originally espoused by UDDI, is terribly naïve, and that's why nobody really does that. I think a service registry's main focus is how WSO2 and Mule are now advancing it -- that being more as a collaboration and support tool. I think services themselves can be adequately described, insofar as their technical specification, through a WSDL or WADL.
Infoq: In your evaluation you are separating ESB from service component framework, suggesting usage of Apache Tuscany and Apache Synapse. Do you consider both required for successful SOA implementation?
Jeff: That's a good observation. I think an ESB is best suited for acting as middleware (or mediation) between various protocols, and is also a great choice for managing security and policy enforcement. I don't think an ESB is an appropriate choice for building reusable components and services. I think a framework such as Tuscany or OSGi are best suited for that. An ESB is obviously very good at wrapping existing legacy services that you want to expose through various other protocols, and you don't want to gut what you have and start over. In that respect, an ESB is very good at service-enabling some of your older applications. Part of the confusion originally surrounding ESB is that they were touted as the swiss-knife for SOA, but that neglected how you actually build your services and components.
One of the real focus areas in the book is how to integrate technologies such as Synapse, Tuscany, jBPM, Drools, and Esper to best leverage their respective capabilities all-the-while doing so in a fashion consistent with the precepts of SOA.
Infoq: Although you are mentioning SCA tooling - Eclipse plugin - the book does not really discuss its usage. Do you consider this tooling to be ready for wide usage, or in your opinion it is still experimental?
Jeff: I think it's coming along, and I do use the Eclipse SCA plugins. That said, some of the features such as the SCA diagrams, while nice to view occasionally, are not that useful when it comes to actually building your configurations, at least in my opinion. I may be a bit old school, but I prefer to know what's being done "under-the-covers”. In contrast, I think the diagramming features available through plugins such as for jBPM are very useful, from a development standpoint. I am sure the visual tooling for SCA will continue to evolve and improve.
Infoq: From the SDO discussion, presented in the book, the advantages of its usage are not immediately obvious. If WsDL is used for service design anyways, what advantage does SDO provides versus "normal” XML marshalling?
Jeff: You really have read the book closely! In my experience, I tend to use SDO primarily as another XML/Java binding technology, not unlike JAXB, Caster or Axiom. Since it is integrated tightly with SCA, this is a natural. In particular, if you develop your services in a top-down fashion where you design your WSDL first, you can then use SDO's WSDL-to-Java utility to create your Java binding classes (I think a top-down approach is really essential for building consistent and well-design services). I've used this approach with fairly complex WSDLs, and have been very satisfied with the results. Of course, SDO also offers support for what is call disconnected change sets, where you can compare changes made off-line to the original - sort of like a build in change-log. In practice, I haven't seen this used much yet, but it has some interesting possibilities.
Infoq: BPM discussion in the book does not seem to specifically address process modeling. Do you consider process modeling to be a requirement for a successful BPM implementation? Can you suggest any Open Source BPM modeling tooling, which is compatible with jBPM.
Jeff: I think there is probably some merit in using a process modeling tool, especially when you are building highly complex models. However, I think that is often the exception and not the rule. Obviously, there are products in the BPEL world that use BPMN notation models that strive to encourage business analysts and subjects matter experts to develop processes. Then, some code-generation occurs that produces BPEL as an output (Intalio uses this sort of approach). However, this can be very tricky in practice, because the BPMN notation is really a lot richer than what BPEL supports, so a complex translation layer occurs. The resulting BPEL code, because its code generated, should really not be modified or the round-trip solution can't be preserved. This type of solution is often problematic, in my experience.
That said, I think the jBPM diagrams that are used to construct models through the Eclipse plugin are purposely designed to shield the modeler from the underlying implementation. In other words, even non-developers could you use it in a fashion similar to Visio to construct workflows, but the implementation details can then be plugged-in by the developer. In contrast, products like Active Endpoints BPEL designer assume that the modeler is conversant with the nomenclature of BPEL, which can be a challenge for those not trained in it. In the book, I demonstrate some fairly complex models using jBPM and describe how SCA can be used in tandem with it.
Infoq: A book goes into a lot of interesting details on usage of the events stream processing for implementing BAM solutions. Do you envision usage of events stream processing for starting business processes and/or services.
Jeff: Absolutely! I think CEP should be woven into the very fabric of the services and processes that you design. If you build your services with CEP in place up-front, it's really pretty straightforward. Even if you don't intend initially to do much with the events that are generated, you have the flexibility down the road to do so by just adding some filtering rules and patterns. In my opinion, it is analogous to a cross-cutting concern such as logging - build it in up-front, and you'll be very glad you did later. At a minimum, it's an easy way to track how often your services are being consumed and or monitoring your errors. In the book, there are some case studies on how jBPM can emit events that are then consumed by Esper, which is the open source CEP product documented in the book.
Infoq: A book does a great job of describing usage of Drools, but the relationships between rules, services and business processes is still not completely clear. Can you elaborate on these relationships?
Jeff: I think rules are really the glue that ties together how services can be woven together to form composite applications or processes. In this respect, I think a rules engine is really critical to SOA, which is why it's a topic discussed in detail in the book. For example, the workflow within a business process is often driven by decisions. For example, if you have an order processing workflow, the decisions which relate to which approvals are necessary are, at their foundation, business rules. So, business rules are critical to business processes. Drools RuleFlow, which we discuss in the book, describes how you can even create business processes within the context of Drool itself.
The problem historically that has hindered the widespread adoption of business rules in the enterprise is how do you integrate a business rules engine within your applications. If you embed the rule engine in the application, you haven't necessarily accomplished all that much, because your rules may still be buried within the confines of the application. Instead, if you pull those rules out into more stand-alone decision services, you bring much greater visibility to them. Further, they can then be more easily centrally maintained through a business rules management repository, such as what Drools Guvnor provides (described as well in the book). When you pull the rules out of the application code and into a central repository that can be accessed by business users, you really begin treating business rules as true assets that can be managed, monitored, and more readily changed.
Infoq: One of the selling points of ESBs is transparent implementation of security. You are discussing some of the aspects of security in the Synapse description, including WS-Security implementation. One of the things that you are not talking about is identity management for credential conversion, when routing requests between services that can belong to different security domains - scenarios covered, for example, by WS-Trust or OpenID. Any suggestions on products in this area?
Jeff: You're right, I didn't really get into identify management, which itself is a fairly broad topic. Obviously, we've seen some standards in this area emerge over the years, such as SAML. However, I think because of its complexity, it hasn't been widely adopted outside of large enterprises. OpenID, on the other hand, is enjoying a lot of success, but it is more focused on web applications, not web services. Until recently, there wasn't a lot of good open source products in this area, but WSO2's identify product, which supports OpenID, appears to be doing pretty well, and is worthy of a good hard look by anyone interested in this area.
Infoq: Any word of advice for the people that start using Open Source products for their SOA implementation?
Jeff: Buy my book. In all honesty, we are very blessed by the quality and selection of open source products now available that can assist in a SOA implementation. Obviously, in my book, I necessarily only cover a selected product for each product category such as ESB, CEP, BPM, services/component framework and business rules, but many of those I didn't cover are also outstanding choices. I think it's really important to assess the vibrancy of the community when selecting an open source product, and having a sponsoring company behind it does lend some important muscle. As I often tell people, I think it is far riskier to go with a commercial offering, because, especially in this environment, companies go out of business frequently or are acquired, and you are left with a proprietary product that likely has a small user base. On the other hand, the popular open source products like I've covered in the book have many thousands of users, the source code is freely available, and have a multitude of developers who are likely intimately familiar with it. Proprietary, commercial products are risky!
Jeff: Thank you for the excellent set of questions - they were very thought provoking!
InfoQ readers - use 'infoq35' discount code when purchasing this book at Manning.com