DIY SOA: How to build your own Simple Service Repository

As a professional service-oriented architect, over the years I've had a recurring irritation in the form of the service registries & repositories. As a regular creator custodian of canonical models, technical service interfaces and service governance documentation I'd often be intrigued by the features of these technologies but I could never find one that adequately complemented my contract-fist engineering requirements and my relatively low-end budget.

If you're not familiar with these two technologies, here is a quick précis. Service repositories are used to centrally store information about the services that are present in your organisation's service inventory whereas service registries are often used at runtime to help with problems like endpoint resolution. The traditional user group for service repositories would be the architects, analysts and developers responsible for service design, implementation, governance and maintenance. Registries on the other hand are often used to support the runtime find & bind process so they tend to be of interest to infrastructure users or enterprises with large scale SOA implementations.

Both of these tools are supposed to help teams to create and maintain their SOA, but when you examine the features, functionality and scope of the tools that are currently available, in my opinion they don't address the unique collaborative requirements of small, agile, multi-discipline teams or the more basic needs of an organisation that is just getting started with SOA.

Service registries & repositories basically fall into two camps. There are the heavy weight and feature rich commercial applications from the usual big name vendors and the smaller less well appointed open-source offerings.

The issues with commercial service registries

For me, commercial service registries and repositories are basically too complicated, too demanding, too expensive or too tightly-coupled to other pieces of the vendor's platform to be of much interest to smaller organisations, agile teams or SOA newbies - even though there is little doubt that most teams could benefit from at least some of their features.

Licensing a commercial service registry & repository from a big name vendor can also be a seriously expensive proposition, especially when you factor in the cost for infrastructure, training, upgrades and maintenance over the lifetime of your SOA. If yours is a small or medium sized enterprise or your SOA is naturally small, your budget for tools to support back office architecture and analysis processes may simply not be large enough to afford one of these commercial products.

Similarly, if your organisation is just getting started with SOA, the process of buying, installing and creating the governance processes and working practices required to satisfy a sophisticated commercial tool could possibly be a distraction that you could do without.

The open source alternatives

That leaves the open-source community's offerings which sadly are rather thin on the ground and often have rather limited functionality. In many cases they're not terribly well thought through in terms of the functionality that a multi-disciplined service construction team would need. They tend to focus on either the infrastructure requirements, architectural requirements or the development requirements, but rarely bridge the collaborative needs of all three.

My “sod-it, do it yourself” tipping point...

I knew that I'd have to bite the bullet and build my own service registry when on one project a Value Added Reseller's developers told me that they didn't know how to meet my contract-first service development requirements on their proprietary vendor platform unless we adopted their service registry and repository technology.

I found this quite frustrating because SOA is based on open standards – and yet here I had a VAR who's inexperience could have cost my team a small fortune in wasted time and software licensing money just because they didn't know how to engineer SOA to my specifications without expensive proprietary tooling.

When implementing a service-oriented solution it's important to remember that there is no technical reason why a proprietary service registry and repository should be a mandatory requirement. Service-orientation is a vendor neutral discipline and as such doesn't require any one specific technology in order to be successful, including service registries & repositories (which are generally considered to be a 'nice to have' anyway)

I know this to be true because I've already built some very successful service-oriented architectures without requiring any formal service registry or repository tooling, so I know that all you really need in order to create the conditions for great SOA are a few simple bits and bobs to help you and your team to collaborate during the service realisation process.

So on this particular project I found myself in a position where I had to prove that acquiring and integrating a complex and expensive commercial repository tool was an unnecessary distraction. I decided to prove my point by offering the team an alternative repository solution that would still satisfy my engineering requirements but also foster closer collaboration and allow contract-first development. I also wanted to show that we could do this at significantly lower cost than the commercial solution being suggested by the VAR.

What I wanted from my Simple Service Repository

My overall goal for my 'Simple Service Repository' was to enable service designers and service developers to collaborate more closely and more effectively by facilitating easy access to the artefacts they'd need in order to implement SOA effectively.

Good service-oriented architecture's are designed in such a way as to support the goals of service orientation. Therefore, a service repository should help to create an environment where a high degree of service contract standardisation can occur as this benefits the core federation and interoperability goals of service-orientation. A service registry should also facilitate contract-first service development and allow service descriptions and data models to become highly discoverable (preferably at both design time and compile time).

Let me explain a little bit more about what I mean by these requirements...

Please Note: My choice of SOA implementation technology for the purposes of this discussion is SOAP based Web Services built with Java. However, I'd still be able to use some parts of the same solution with REST or with distributed components (for example, by using the registry to centralise your XML Schema based data models).

Contract centralisation and standardisation

Contract standardisation and centralisation are two SOA practices that support and improve the federation, interoperability and reuse of services and service contracts. By centralising and standardising artefacts like canonical data models (XML Schemas), service descriptions (WSDLs) and service policies (WS-Policy documents) we make them easier to find and simpler to reuse.

The standardisation and reuse of XML Schemas is particularly important because it's a key enabler for 'intrinsic interoperability' whereby services are designed to communicate with each other using the same data model (and thereby avoiding complex data model transformation during runtime).

By having all your service definitions, data models and policies in one central location you can reuse them very easily and as often as you like. Having them all together also makes them easier to maintain and will add a layer of transparency to your SOA design processes and practices that can benefit all kinds of stakeholders.

Contract-first service development

Contract-first development is vital to SOA because it ensures that the service contracts remain totally independent from their underlying implementation technology. Contract-first often entails a service designer producing the required service contracts followed by developers then 'importing' them into their build as the starting point for a service's implementation code.

This practice of allowing the contract to shape the code can often be a source of some consternation for developers because traditionally they had the responsibility for specifying distributed component interfaces as they saw fit. However, because service-orientation requires that these software assets become more generally reusable and more closely aligned with the business requirements, this previous flexibility often has to be relinquished for the greater good.

By making the process of generating services from contracts simpler and easier, we can help to overcome any potential technical objections that may arise. In my experience, this is the part that many service repositories fail to do particularly well. They often provide the necessary the contract centralisation functionality, but then make it quite hard to then get the contracts back out again for further analysis, modelling or service implementation (unless you use more of their proprietary tooling of course).

Design-time discover-ability

Finally, easy service discover-ability benefits service-oriented architectures because by making services and data models more generally visible & accessible we can reuse them more readily and help to prevent 'accidental' duplication from occurring. Duplicate, competing, unused or redundant services can confuse and dilute the effectiveness of your SOA and therefore damage its ability to serve its community. These issues can be avoided (and greater reuse promoted) if service contracts can be easily discovered by anyone at any time.

By creating one location where we can centrally store and discover service contracts (models, descriptions & policies) as well as other design-time information like WS-I BP compliance certificates, human readable interface documentation on QoS features, and other service usage information, we can achieve all of these goals and add some real value to our SOA analysis, design and development practices.

Time to get started: How to build your own 'Simple Service Repository'

So here is my rather straight forward and very low-cost alternative to commercial and open-source service repositories based on a combination of Maven's built in artefact repository and a simple service repository style dynamic web application.

Just follow my 3 easy steps and you'll have your own simple service repository in no time...

Remember: you're doing this so that the team can collaborate more effectively, so get the team involved early as this will help to build consensus and generate ideas. Tell them what you're doing and get them to help.

1. Create a Java web application (the Simple Service Repository application) using Maven.

You need something simple in which to store and publish all your SOA related content, so what could be simpler than a standard dynamic web application?

By far the best thing about this Simple Service Repository is that it's really just a Java web application with a bit of Maven thrown in to help with the contract standardisation, contract centralisation and contract-first development. Anyone who can edit a HTML page can add value to this kind of service repository simply by authoring and publishing information about your service-oriented architecture.

To get started simply create a normal web application using a Maven 'WAR' archetype. This application is the core of your service repository. It's Maven artefactID and groupID will be used later by service implementation projects whenever they have a WSDL dependency at compile time.

Because it's a standard web application, it's not limited to just storing XSD's, WSDL's and WS-Policy. It can also store BPMN/BPEL process descriptions, service descriptions, governance documentation, inventory blueprints or any other information that will bind your service construction team together. It's essentially a WIKI for your SOA but with technical service contract storage thrown in.

In my version of this application, I've stored each group of service contract 'releases' in a separate folder and made the folders within the web application browsable by default (you may have to search your application server's documentation to find out how to do this). A 'release' folder contains all the WSDL's, XSD's and WS-Policies I've created for the release with each document containing a namespace that links them all together in a predictable way.

With browsable folders switched on, if I go to "http://myrepository/r201107" I would see a folder view of all the technical contracts related to the July 2011 release. The WSDL and XSD namespaces in those documents would also contain the 'r201107' identifier at the end of the namespace after the slash '/' so that everything ties together nicely.

If I didn't want to browse folders and I preferred a prettier more human readable view, I'd simply point my browser at "http://myrepository/index.html" where my HTML pages would take over and show me whatever content I wanted to see and would allow me to navigate via hyperlinks in my web browser.

Finally, I'd advise that you use a version control repository (like SVN) to store your Simple Service Repository application and all of its content. That way you'll get basic version control features thrown in for free. This will help you to maintain the integrity of the information being stored in the repository and provide a further mechanism for collaborating and managing changes as a team.

If you're struggling to visualise what the registry application could look like at this point, have a look at the following screenshot.

(Click on the image to enlarge it)

Illustration 1: The home page of my version of the Simple Service Repository web application.

2. Build the Simple Service Repository application and deploy it to an application server

You need published changes to your repository's content to be instantly available to everyone wherever they are so step two is to use Jenkins to extract, build and deploy your Simple Service Repository whenever it notices a change in its version-controlled content. Jenkins is an open-source continuous integration server that can simply be deployed as another application on your application server.

Glassfish 3.1 is my personal application server of choice for Java web applications (but other servers would work just as well). Once deployed, the Simple Service Repository's content will be instantly available via a browser. This helps to satisfy my basic design-time discover-ability, high availability and high visibility requirements.

When Jenkins builds the application it should use Maven's 'install' phase (or the deploy phase if your Maven repository is remote) so that your Simple Service Repository application (and all the service metadata that it contains) is placed into your local Maven repository as a versioned Maven artefact. This is very handy for the next bit...

3. Build a Web Service implementation from a contract hosted in the Simple Service Repository

You want it to be easy to create contract-first implementation of your web service contracts and once more Maven comes to the rescue. Using Evgeny Goldin's maven-copy-plugin to extract the service contract(s) from the Simple Service Repository's Maven WAR artefact during the Maven 'init' phase of your service implementation project's build makes all the necessary service contract documents available to your project's code at compile-time. It's then a simple matter of using the jaxws-maven-plugin to create a service implementation framework for you in Java based on the extracted WSDL/XSD/WS-Policy documents.

The rest is plain sailing - normal boilerplate JAX-WS service implementation code. In my version I use the Cargo plugin for Maven to deploy the service implementations onto my Glassfish server and I use SoapUI's Maven plugin to run a service integration test suite during every build.

On the Jenkins build server you can configure your environment so that any change to the content in the Simple Service Repository will trigger a fresh build of all your service implementations. That way, if the service contracts for your services change, your service implementations would be immediately re-built and re-tested so that you'll know instantly if you've introduced a defect into your build environment.

In some very simple cases, the service implementation projects can be so lightweight that the project needs only a few Java class files initially. Also, because Glassfish 3.1 can do annotation based deployment, there can often be very little need for metadata like deployment descriptors etc.

(Click on the image to enlarge it)

Illustration 2: An architectural view of the Simple Service Repository in action.

A job well done?

So there you have it, version 0.1 of your Simple Service Repository is complete.

By centrally storing all your SOA artefacts and other rich content centrally on a web server and in your Maven repository you've made all your services highly discoverable at design-time and supported the SOA goals of reuse, federation, interoperability and vendor neutrality.

Your repository also simplifies contract-first development and will alert you if you introduce any changes to your service contracts that de-stabilise your service implementations. You've also gained simple folder and namespace based versioning of your service definitions and canonical data models.

To my mind the Simple Service Repository fits the brief perfectly. It's highly available and accessible, repeatable, manageable and provides a valuable platform that supports closer and richer collaboration between service designers and service developers.

More importantly it was very quick and easy to build. It required zero code (if you don't count HTML as code). It cost you about a day in development time and it draws nicely on the strengths of existing low-cost tools and techniques. It also requires very little infrastructure and will run nicely on a normal laptop. You could even host it into the cloud without too much additional development effort (try this link for cloud based Tomcat or Glassfish 3.1 application hosting).

I admit that there is lots of scope for improvement and a long list of additional features that you could undoubtedly add. But I also think that most issues will have pretty simple workarounds that you could use for now until they become a high enough priority for you to fix. I'd also argue that thanks to the continuous integration aspect whereby contract changes trigger fresh service test cycles, it can add a valuable 'alert me if I break my services' feature that I don't think I recall seeing in any other service registry or repository.

In Conclusion

Repositories are all about sharing information. However, the way that one team chooses to share information will be different from the way that other teams do it. Therefore, creating your own Simple Service Repository is a valid and legitimate solution to the challenge of sharing SOA artefacts. By doing it yourself you can create something quickly that the whole team can contribute too and which will suit your processes perfectly. Try it - because I think you'll like the control that it gives you - and it could just turn out to be the secret of your SOA success.

About the Author

Ben Wilcock is CEO of SOA Growers Ltd, a unique and independent SOA consultancy practice based in the UK. He's also a SOA Certified Professional (SOACP) and his recent service-oriented architecture work with Screwfix.com's agile development team won  Oracle Retail Week's Retail Technology Initiative of the Year 2011. As you probably noticed, he's passionate about SOA. You can contact Ben via his company website and read his blog.

Find out more...

Would you like a copy of my Simple Service Repository? Would you like me to open-source it?

Comment, subscribe or contact me for more information.

Screenshots

(Click on the images to enlarge them)

Illustration 3: The repository page from my Simple Service Repository showing one released set of documentation.

Illustration 4: The view page in the Simple Service Repository for one of my document set releases. Links for WSDLs, XSDs, Policies, WS-I BP and testable endpoints.