How REST replaced SOAP on the Web: What it means to you

Its been slowly creeping up on us, creating exciting new possibilities for our applications; APIs are changing the face of the Web. Since 2005 Programmable Web have been tracking SOAP and REST APIs available to the public.  In 2005 they tracked 105 APIs, the prominent ones being from Amazon, Google, SalesForce and eBay.  By 2008 this number had grown 6-fold to 601 APIs with social and traditional media seeing value in opening up their data to third parties. By the end of 2010 developers had over 2,500 APIs at our disposal.  While forward thinking companies like Zappos published a REST API, we also saw government and brick and mortar retail join the fray. Tesco allows developers to order groceries, Instagram created a Twitter for pictures, Face offers facial recognition as a service and you can create telephony applications with a few calls to Twilio. In 2011 we’re seeing the number of public APIs climb towards 5,000, there has never been a better time to build killer applications.

(Click on the image to enlarge it)

source: Programmable Web

The SOA Holy Grail

Anyone working on enterprise systems in the last 10 years will remember the initial tenets of Service Orientated Architecture were to decouple applications and to provide a well defined service interface, which can be reused by applications and composed into business processes. The idea of reuse and composition made SOA an attractive proposition that sent thousands of organizations on a very challenging treasure hunt. We have since read SOA's obituary and its resurrection with many stories of woe peppered with some success, but with very few achieving the holy grail of SOA. Meanwhile, the web has essentially become a service oriented platform, where information and functionality is a available through an API; the Web succeeded where the enterprise largely failed.
This success can be attributed to the fact that the web has been decentralized in its approach and has adopted less stringent technologies to become service oriented. Many early APIs were written using SOAP but now REST is the dominant force (though some are more REST than others).  The publication of REST APIs has been rapidly increasing.

source: Programmable Web

Some offer both SOAP and REST APIs, but this practice has been on the decline and REST is now preferred for most new APIs.

source: Programmable Web

XML or JSON

One of the reasons REST has been favored on the Web is client accessibility. SOAP was defined with the enterprise in mind and while the protocol is platform agnostic, SOAP XML is verbose and often painful to use in web technologies like JavaScript and Ruby. JSON is favoured due to it more compact representation, its easier to read and is the native data format in JavaScript. Interestingly, newer APIs which only support JSON are on the rise and 45% of APIs now support JSON with many new APIs offering JSON as the only data format.

source: Programmable Web

REST Implementation Woes

The biggest challenge with the shift to APIs is the nature of REST. Representational State Transfer is an architectural style for defining an interaction model over HTTP where HTTP verbs map to service operations for doing things like listing all users, updating an account or deleting an order entry.  These principles came from Roy Fielding in a paper he published in 2000. Since then REST in all its forms has taken a firm grip on software development. The biggest challenge is that the original paper only provided a set of constraints, it did nothing to prescribe a URL scheme, versioning, authentication and authorization, error handling (HTTP codes are not sufficient) or even the correct way to pass parameters to a restful resource. RESTafarians may shut me down at this point since there are strong opinions here, but observing the wildly different opinions out there, there is no agreed way to do REST.

Your REST isn’t my REST

The right way to build a RESTful architecture is not well defined and anyone that has worked with 3^rd party REST APIs will have seen that there are many interpretations of REST that makes working with different APIs difficult.  These include:

• Inconsistent authentication, even from APIs from the same provider
• Inconsistent use of HTTP Verbs. REST falls foul to human error and misunderstanding like everything else
• Inconsistent use of HTTP return codes. No well-defined scheme for handling errors. Some APIs have great error handling, other have almost none; HTTP return codes are not enough
• Varied URI schemes that don’t fit the REST model
• No agreed way to do versioning, yet it is so important given a REST API is the gateway to an application
• Hashing and signing parameters for a REST API call is often annoying and fiddly and never consistent between different APIs
• Request and Result messages are often not consistent between services, makes queries and object binding more challenging. There are no clear guidelines for creating DTOs (data transfer objects)
• Unlike SOAP web services there is usually no contract which means rest services are not self describing and with no contract things change on you without warning

The Developer API challenge

Any developer that has worked with public APIs knows that you can code to these APIs directly, but it can be awkward and frustrating. This is fine if you are just integrating with Facebook, but when you start composing your applications from multiple APIs you need a new approach.  Many API vendors have made developer's lives easier by providing clients in PHP, Ruby, and Java. However, your mileage varies with these clients since many are often not maintained.

Embracing APIs

With public APIs doubling every year, developers cannot ignore the value of these new capabilities.  Developers need help to iron out some of the REST wrinkles that make one API very different from another. Often the API clients don’t meet expectations, which has lead to frameworks as API Smith emerging as well as API services such as Gnip that provide a consistent API interface to a wealth of Social Media APIs. Additionally companies such as apigee and Mashery offer tools for exploring APIs.

Another Way

Our approach to embracing APIs is to provide a consistent interaction model.  Mule Cloud Connect provides such a model.  APIs are represented as simple objects with meta-data configured using annotations that handle the URI scheme, authentication, signing, hashing, session management and even streaming.
These Cloud Connectors can then be orchestrated in flows to allow you to do things like filter social media data to send updates to your phone, provide automated voice access to your billing account or back up your CRM data to a database.
Of course you need a place to run these flows. Mule iON is an integration Platform as a Service (iPaaS) which allows developers to compose API functionality and publish the results to their own applications. Mule iON acts as the integration layer to your application, removing the integration logic from the application code.

The value of iPaaS is that developers can decouple integration logic from their application. If you integrate with multiple APIs you’ll need custom code in your application for each API and if any one of those APIs change, you’ll need to update the application code too. With an iPaaS, all that logic can be delegated to an integration layer that is much better suited to working with different APIs handling things like security, retries, session management, throttling and errors. The iPaaS can then publish the results (a mashup) of data or functionality to mobile applications or web applications, whether traditionally hosted or running on a PaaS such as Heroku, force.com, CloudBees or Azure.

The proliferation of APIs on the Web means developers have a treasure trove of new functionality and data to incorporate into their applications.  New applications increasingly make use of these APIs to provide more context and a more interesting user experience.  However, due to the lack of standardization of APIs working with different providers is a real challenge. Developers should be looking for ways to keep their application code clean by removing integration logic and delegating it to a platform tailored to make integration much easier. 

About the Author

Ross Mason is the CTO and Founder of MuleSoft. He founded the open source Mule® project in 2003. Frustrated by integration "donkey work," he set out to create a new platform that emphasized ease of development and re-use of components. He started the Mule project to bring a modern approach, one of assembly, rather than repetitive coding, to developers worldwide. Now, with the MuleSoft team, Ross is taking these founding principles of dead-simple integration to the cloud with Mule iON, the world's first integration platform as a service (iPaaS). Ross holds a BS (Hons) in Computer Science from Bristol, UK and has been named in InformationWeek's Top 10 Innovators & Influencers and InfoWorld's Top 25 CTOs. Twitter: @rossmason, @mulejockey