SOA at one point was discussed as a game changer, something that was going to revolutionize software development and delivery processes, unite business and IT, and significantly lower software maintenance costs.
SOA was (and still often is) a gold mine for software vendors, who were very active in defining numerous (often contradicting) supporting standards and delivering software products which were supposed to simplify design and implementation of SOA systems, but in reality often added confusion by defining those products as SOA.
Many companies embarked on servicizing their software implementations and some achieved great success, while others failed. SOA was on the top of analysts’ reports, then was proclaimed dead and then reborn under slightly different names – REST, cloud computing, etc. There are literary thousands of SOA books and articles covering many aspects of SOA from architectural design to nitty-gritty implementation details
And still, after all this, there are a lot of disagreements between SOA practitioners not only how to successfully create SOA solutions, but even what exactly SOA means and what the necessary SOA components are.
In this virtual panel, InfoQ talks to 5 SOA experts about the significance of SOA, its relationships with other architectural styles, best technologies and approaches used for successful implementations of SOA solutions and SOA future.
The panelists who have answered our questions are:
- Jim Webber – Chief Scientist, author of “Developing Enterprise Web Services - An Architect's Guide”.
- Claus T Jensen - IBM Chief Architect for SOA-BPM-EA technical strategy
- Stefan Tilkov - RESTafarian & Former InfoQ SOA Community Lead Editor
- Steve Jones – Global Head of Master Data Management at Capgemini
- Mike Rosen - Director of Enterprise Architecture at the Cutter Consortium
The following questions were put to them:
- Q1: SOA means a lot of things these days: What is your sound bite definition? Can you compare and contrast SOA with “Integration”, ROA, and EDA?
- Q2: There has been a lot of discussions about relationships between SOA and BPM, ranging from statements that they are the same to the statements that the need to be explicitly separated. What is your opinion?
- Q3: A new analyst's darling is cloud computing. What is the relationship to SOA? Related to this is Big Data, how is it related to SOA?
- Q4: One of the reasons of falling popularity of SOA is projects failure. What’s the #1 thing you think SOA projects should do to succeed?
- Q5: Do you foresee a SOA 2.0? How would it look?
Q1:SOA means a lot of things these days: What is your sound bite definition? Can you compare and contrast SOA with “Integration”, ROA, and EDA?
Mike Rosen: SOA is an architectural approach to developing solutions where a 'service' is the main concept of analysis, design, implementation, deployment, and operations". To get the most benefits from SOA, it is best done at a scope that spans organizational boundaries. Service designs must incorporate both business and information requirements.
While SOA is one approach to implementing integration, and when done well, is often a more flexible approach, integration it is only one aspect of SOA. In the past, integration has most often been done in a limited scope where enterprise wide consistency was a secondary concern. A SOA approach typically uses integration techniques to transform and expose resources as services that can be available on an enterprise scope.
ROA and EDA are other architectural styles that have 'resources' or 'events' respectively as their primary design driver. These architectural styles have different overall goals for system flexibility and optimization. They are not subsets or supersets of SOA, nor are they incompatible with SOA. Effective enterprise solutions can be constructed by combining then together to take advantage of each approach. But that's a bit too much to go into here.
Steve Jones: SOA still means the same thing that it previously meant, namely thinking about architecture in terms of services, capabilities and their interactions. SOA is to architecture as OO was to development, a new way of thinking about the whole of IT from a business perspective, it’s that which makes SOA powerful not the specific delivery approach. So thinking architecturally in a Service Oriented Way you can then choose whether to implement parts using Resource Orientation or EDA, those two approaches are about the "how" of delivery not about providing a consistent mental framework into which various different technical approaches can be combined.
The only thing that really changed in SOA is that the vendors have put different buzzwords on the same products so aren't confusing what SOA is so much these days.
Stefan Tilkov: SOA means whatever suits the one using the term best. Seriously, I've totally given up on trying to define it, since whatever definition you come up with, there's simply no agreement to be found, and ultimately it ends up being a waste of time. Having said that, I'd like to focus on the high-level aspects that at least seem to find approval by a significant minority: A holistic approach to enterprise IT; some concept of service consumption over a network without a need to know about implementation details; trying to apply sound architectural principles to the overall system landscape; with integration not added as an afterthought and a necessary evil, but something deemed so essential that it becomes the default. I'm a big proponent of using REST as the most applicable architectural style to achieve those goals, but I concede that not everyone agrees - they just don't know any better :-)
I don't like the term "ROA"; "REST" is perfectly fine and abstract enough for my taste, no need to add something (I use "RESTful HTTP" to refer to the Web's architecture as an instantiation of that style). Events are a part of almost every architecture I've ever dealt with, but I've never seen one where _only_ events were used, or even played the central part, so I'm not sure Event-driven architecture as a concept of its own is something worth exploring. To make the connection to REST again, I do see event notifications via feeds as an interesting alternative for many scenarios - quite often polling is perfectly fine and the most scalable solution available.
Jim Webber: SOA is the design and delivery of systems-of-systems which resemble the businesses they support. Integration is joining things together - SOA gives it context. Integration tasks alone have very little value - integration is not an end in itself - though often we work with systems-of-systems that take integration for granted. I'd add here as a side note that "integration projects" are a smell.
ROA is nonsense since nobody in their right mind architects around resources alone (unless they're really doing RPC and want to make it sound less horrid). However SOA can give broad guidance to those responsible for designing systems-of-systems based on Web architecture. The key here is that SOA provides the mould - services which encapsulate a business need - while the Web provides a pervasive and robust infrastructure for supporting business protocols between those services.
EDA has been painted as a specialized activity involving low-latency busses and the like to support complex near real-time events. Actually the same design principles that work there also work at more humane latencies - all systems simply respond to events, EDA takes that with a low latency constraint.
Claus T Jensen: My sound bite, holistic, definition of SOA is this: SOA is an architectural style, a thought model, that enables Business & IT collaboration and alignment and is centered around the interaction between services and processes. Integration is certainly part of what a service oriented architecture should be concerned about, but is by far not the only thing. A lot of SOA initiatives have historically been driven by IT and focused on application integration; yet while immediate value can be achieved that way, the alignment with the business and the effectiveness of the enterprise portfolio is not enabled by integration alone. Service oriented architectures can be event driven or not - I do not personally see EDA as a different architectural style, rather perceive it as simply one aspect of service orientation.
Q2: There has been a lot of discussions about relationships between SOA and BPM, ranging from statements that they are the same to the statements that the need to be explicitly separated. What is your opinion?
Mike Rosen: BPM is an approach to orchestrating modular business functionality and information to support business processes. The 'M' of BPM stands for management. So orchestration is just one of many aspects of BPM. SOA is the technology that provides the modular functionality in the form of business services. Businesses services might be implemented by orchestrating smaller domain and utility services, but in SOA the emphasis is typically on encapsulating the orchestration, whereas in BPM, the emphasis is on exposing, managing, monitoring, and optimizing the process. BPM without SOA tends to create siloed solutions. SOA without BPM creates services without a business consumer. Together, they provide a total solution.
Steve Jones: I've said before that BPM screws up SOA (see this link) and I still think that is true. SOA actually makes great BPM and BPM is a great approach to implement the capabilities of a business service. What I mean by this is that if you think first in terms of services then you can choose to use BPM where it’s appropriate and thus combine it with non-BPM approaches (e.g. ROA, EDA, package, etc) in a consistent manner. Starting with BPM tends to drive fine-grained single operator services which isn't a good thing in terms of creating a robust architecture. The important thing therefore is to understand your services and capabilities first, then choose to implement capabilities using BPM, this gives both good SOA and good BPM.
Stefan Tilkov: I believe that BPM is a concept of its own and you can do just SOA, just BPM, or both together. I don't see processes at the core of SOA, even though I understand that BPM vendors might like to see things differently. Sometimes it's perfectly fine to hard-code processes, sometimes it's not; even then, typical BPM solutions are never the only option.
Jim Webber: Services implement business protocols, otherwise they're not decent services. Choreographing the interactions between those services could be classified as BPM, but that term is so overloaded with product offerings I don't find it useful anymore. As a Web fan, I like to see that hypermedia is used to convey business protocols over the Web (whether that's the big Web, or the little Web inside your organization). So I think choreography is alive and well. But I wouldn't conflate that with BPM.
Claus T Jensen: BPM is a discipline focused on operational optimization, and SOA is an architectural style focused on business and IT alignment. Included in SOA is the thought that one should care about interacting services and processes, in other words SOA is a very natural architectural style to adopt for an organization that cares about BPM. Having said that it is possible to derive value from BPM without SOA, though you do expose yourself to "service sprawl". It is also possible to derive value from SOA without BPM, but in all honesty BPM is a natural component in an SOA strategy as it provides a built-in business context for the architecture itself as well as immediate business operational benefits.
Q3: A new analyst's darling is cloud computing. What is the relationship to SOA? Related to this is Big Data, how is it related to SOA?
Mike Rosen: The cloud is about providing services, whether it's infrastructure, platform or software as a service. So, SOA and the cloud share a common fundamental. The implication for the cloud is that the same service design principles that make a good SOA service need to be applied to a cloud service: well defined interfaces, loose coupling, proper decomposition, common semantics, etc. Unfortunately, just like many SOA designers don't really understand this, many cloud service designers don't either. So buyer beware. Make sure the services you're getting are well designed, implemented, and operated.
A SOA solution should be able to integrate cloud based services as well as on-site services, and it should easily be possible to provision your own service on a cloud infrastructure. Like anything else, there are good ways to do this, and not so good ways, so some implementations will introduce problems with security, performance, reliability, and accountability. This will be a failure of the implementation, not of the cloud or SOA. Expect to see cloud based services increase the need for custom integration and exacerbate the lack of common semantics among applications. As an architect, you should be developing guidelines for cloud usage now to minimize these issues before they become disasters.
Steve Jones: In the book I talk about understanding the business value of a service, cloud really plays in to this as it helps to better align the delivery costs of services to their business value. That business value might be about reducing costs or shifting to utility pricing or it might be about investing in innovations and massive scale.
So SOA provides the way of understanding what the business value is and therefore where elements like cloud and Big Data can deliver value. If you have an area where it’s all about cost of service and driving costs down it might be worth shifting the application into the cloud to reduce the cost but it probably won't be worth investing in a Big Data solution as there just isn't the value to be realized.
Stefan Tilkov: There obviously is a relation in that whatever services (in the most abstract sense of the term) you consume, you're likely to do so over a network and you won't care about the implementation details. This works nicely for most IaaS solutions such as Amazon's, but I still find it hard to map something like Google's App Engine onto some SOA concept, and often feel the connection is made by those want to attach their SOA thoughts from the past to the success of the Cloud Computing model today.
Jim Webber: Not in the slightest, other than you can run your own services on other people's infrastructure, or buy (and bind) further up the stack by renting their business processes encapsulated as services. This notion of SOA in the cloud or cloud being the next generation of SOA is rather dull at this level. I'm convinced it's super exciting and tricky to run systems at such scale, but the cloud is not the savior of SOA, rather it's the next shiny object that will attract certain thought leaders, analysts and so on. If your service deals with big data, that's how they're related. Otherwise they're not: data is data, services are about behavior atop line of business data.
Claus T Jensen: Cloud is the result of combining SOA and dynamic infrastructure. A Cloud Infrastructure is:
- Completely Virtualized
- Continuously Optimized
- Dynamically Responsive
- Heterogeneous to Support Differing Workloads
But even more importantly the Cloud business model is a utility model based on pre-defined utility services that can be consumed (and paid for) on-demand. I often get the question "how do I figure out what others should pay to use my Cloud services" - in reality someone asking that question is not really doing Cloud, rather is applying dynamic infrastructure within a classical IT business model.
Q4: One of the reasons of falling popularity of SOA is projects failure. What’s the #1 thing you think SOA projects should do to succeed?
Mike Rosen: I wouldn't agree that SOA is falling in popularity. I would say that it is no longer of interest to the hype and marketing machine. Every application infrastructure and major application provider is SOA based, or moving to be so. It would be foolish to create a new solution today that wasn't service oriented. I'm not convinced that SOA projects fail any more than other software projects of their size, but I haven't looked at the numbers. SOA projects fail when organizations try to bite off too much to start, and when they don't have an architectural vision. So, the #1 thing is to have a SOA architectural vision that incorporates a SOA catalog and enterprise semantics. It should not take more than a few weeks for a good architect to formulate the vision. Then, the #2 thing is to do small, incremental projects that implement a few services at a time that provide immediate value to some project. Incorporate lessons learned into the next project and grow the architecture and service base over time.
Steve Jones: I think another key reason is that most projects were not actually doing SOA, they were doing Web Services based integration. I'm regularly told that an organization is "doing SOA" but when I ask for their solution services architecture, let alone their business service architecture, it is nowhere to be seen, all that actually exists is an old style architecture with Web Services at integration points. These people have then moved to the next shiny technical approach and that will fail for them as well, failure rates in IT remain stubbornly high. Why is that? Mainly because most of IT seems to believe, despite all evidence to the contrary, that a new technology will become the Silver Bullet that doesn't and will never exist.
A good example of these failures is the lack of MDM within most SOA programs. If you have diverse areas of the business trying to communicate then you need clarity on the key entities within the business that you are communicating around. Almost every single SOA initiative however fails to grasp this information challenge which means implementation is more complex and the business value is less clear.
So the first thing to make SOA succeed is actually DO SOA, that means thinking about your services and capabilities first, THEN working out how to deliver the services. You should be able to see services through-out the project, in the development teams, in the code repository, in the deployment guides and in the management reporting. That will help project success rates increase as it breaks projects down into more independent areas which helps approaches like agile and also compartmentalizes risk within programs. This requires a step up in discipline and a recognition that the same people, process and mentality but with a new technology will not deliver a new result, you have to change how people think, how they work and potentially add new technology in order to deliver new results.
Stefan Tilkov: Adopt REST? Just kidding. To me, the major factor is to actually start small and deliver something of value, an actual service that people want to use and actually start using. Unless you manage to do that, you shouldn't be allowed to spend any company money on things such as ESBs, registries, repositories, or any other magical piece of infrastructure that is only a means to an end, but creates no value in itself.
Jim Webber: Deliver incrementally. Don't do the 3 year "transformation program" because you'll fail. Think about SOA not as a protracted effort, but as a series of short, winnable skirmishes that happen as change is driven through your market or organization. I've talked about this a lot in the past: the practice is called Guerrilla SOA. Do that, and you have a fighting chance.
Claus T Jensen: They should understand that SOA is not about technology, and that adopting SOA cannot be done in a project it has to be done organizationally (or at least at a program level). Steve Mills, Senior VP and Group Executive IBM Software Group, stated back in 2007 that “ Service orientation does not begin with technology; it begins with the mind-set of thinking about your business and the world around you in terms of functional components.” I completely agree with that, service orientation begins with the thought model.
Q5: Do you foresee a SOA 2.0? How would it look?
Mike Rosen: Most SOA today is implemented in the enterprise tier of an application to provide 'business' services and support business process. Enterprise 2.0 solutions require an additional but different type of service in the application tier. SOA is still the right kind of architecture for building rich, collaborative, user experiences in the application tier that access flexible business processes in the enterprise. So, I think SOA 2.0 is an expansion of SOA across the end-to-end solution stack. This means that solution architects understand how to use it, and that infrastructure provides its capabilities (such as location, presence, collaboration) via well designed service interfaces.
Steve Jones: No, I don't see an SOA 2.0 in the same way as I never saw an OO 2.0. Its a way of thinking about IT and its relationship with the business, maybe 2.0 is just people actually doing SOA rather than concentrating on the next shiny technology. Key things that people should be looking at are the IT and business building blocks they need to make SOA successful. So looking at the business value and looking at how to drive IT simplicity by using elements such as MDM as a key IT/Business enabler.
There is no SOA 2.0, there is just doing SOA properly.
Stefan Tilkov: I don't think anything called "SOA 2.0" will have a chance of being successful - the term has been abused way too thoroughly in the past. I strongly believe the core values behind SOA have merit, but there needs to be a new label attached to it. Coming to think about it, maybe it would be even better if the industry stopped using labels altogether, and tried to do whatever makes sense. One can dream, right?
Jim Webber: I think the shift to the Web is inevitable. I imagine that Web APIs will become more RESTful over time, and that we will habitually open up our data in a responsible manner. The building blocks are already there - we are getting much better at understanding how to share business protocols over the Web; authentication and authorization are usable (if not exactly perfect). So SOA 2.0 will leverage this ubiquitous, secure, robust platform for widespread data sharing - it really will be the Web of everything.
Claus T Jensen: At some point in the future there is going to be yet another paradigm shift in terms of architectural thinking. It is hard, if not impossible, to imagine exactly what it would be - such is the nature of paradigm shifts, they cannot be predicted. I don't see any such change in the foreseeable future though, there is still plenty of potential in applying SOA as the key architectural approach to agility and effectiveness.
Conclusion
It seems like even experts cannot agree on exact meaning of SOA, which way it is moving and what is its future. This is what makes our jobs – software architects – so challenging and interesting. It seems like there is no “silver bullet” and we will need to continue experimenting with building new and better SOA implementations.
About the Panelists
Mike Rosen is Chief Scientist at Wilton Consulting Group which specializes in architectural consulting. He is also Director of Enterprise Architecture at the Cutter Consortium which provides research and analysis on IT and Editorial Director of SOA Institute. He is an accomplished architect and technical leader with extensive experience in enterprise architecture (EA), service-oriented architecture (SOA), software architecture, distributed technologies, and industry standards. Mr. Rosen’s 30 year career has spanned startups, CTO roles, chief architect, and product architect for industry leading middleware products. Throughout his career, Mr. Rosen has been a frequent technical speaker and author for conferences in more than a dozen countries. He is coauthor of several books including: Applied SOA: Service-Oriented Architecture and Design Strategies; Developing E-Business Systems and Architectures: A Manager’s Guide; and Integrating CORBA and COM Applications.
Stefan Tilkov is a co-founder and principal consultant at innoQ, a technology consulting company with offices in Germany and Switzerland. He has been involved in the design of large-scale, distributed systems for more than a decade, using a variety of technologies and tools ranging from C++ and CORBA over J2EE/Java EE and Web Services to REST and Ruby on Rails. He led InfoQ's SOA queue for a couple of years, has authored numerous articles and a book ("REST und HTTP", German), and is a frequent speaker at conferences around the world.
Dr. Jim Webber is Chief Scientist with Neo Technology the company behind the popular open source graph database Neo4j, where he works on graph database server technology and writes open source software. Jim is interested in using big graphs like the Web for building distributed systems, which led him to being a co-author on the book REST in Practice, having previously written Developing Enterprise Web Services - An Architect's Guide. Jim is an active speaker, presenting regularly around the world. His blog is located here and he tweets often @jimwebber.
Steve Jones is Global Head of Master Data Management at Capgemini where he is responsible for the coordination and establishment of Capgemini's global MDM offers including the scaling of the offshore Centre of Excellence and the publication and industrialization of a business centric method for MDM consulting and delivery. Steve helps clients to deliver IT estates which look like the business, evolve like the business and are costed in line with the value they deliver. Working with business and IT leaders, he helps organizations take control of their master data and align its governance to its business value.
Claus Torp Jensen is a Senior Technical Staff Member on the Business Process Optimization (BPO) Foundation team and Chief Architect for SOA-BPM-EA technical strategy, driving their alignment and integration both conceptually and practically. The BPO Foundation team has architectural responsibility for all of IBM's software products to ensure that they support the key principles of SOA, Business Process Management (BPM) and Business Agility from a client perspective. Prior to joining IBM in March 2008, Claus was Group Chief Architect, VP of Architecture and Business Development, in Danske Bank, a regional European bank.
He was responsible for driving Danske Bank's SOA initiative and SOA center of excellence since its inception in 1999, and is known as an SOA expert and evangelist. Claus holds a PhD in Computer Science from Aarhus University, Denmark.