Is AMQP on the way to providing real business interoperability?

Advanced Message Queuing Protocol (AMQP) came from inside of JPMorgan, thanks to John O'Hara. But his vision was bigger than just a new way to do things internally. The standard and open source technologies around it have been gaining momentum. Jeff Gould and others shed some light on where AMQP came from, who is driving it, and where it might be going.

IONA Technologies, a contributer to the AMQP specification, published a short description of what AMQP is and why to pursue it. AMQP is

• A defined set of messaging capabilities called the "Advanced Message Queuing Protocol Model" (AMQP Model). The AMQP Model consists of a set of well-specified components that route and store messages within a broker service, plus a simple set of rules for wiring these components together
• A network wire-level protocol, AMQP, that lets client applications talk to a broker and interact with the AMQP Model it implements.

In a three part series, Jeff Gould talked about the history of AMQP and John O'Hara's decision to go open source with it. After hiring iMatix to implement an AMQP project for JPMorganChase in 2003, the beta went live in 2006 and processed around 300 million messages per day.

O’Hara’s plan for AMQP was far more ambitious. From the start he wanted the new protocol to match the functionality of the high-end proprietary MOMs. It had to handle all the major use cases, including queue-style store-and-forward messaging, Tibco-style publish and subscribe, and reliable file transfer. The protocol had to be capable of carrying any kind of message, but the focus was on more efficient binary formats rather than text, because in app-to-app messaging human readability is not a paramount concern.

Gould went on to point out that it was O'Hara's quest to have an open standard that really pushed AMQP out of the bank and into the wild. The open standard was what brought players like Red Hat, Apache, WSO2, IONA, and Cisco to the table. Red Hat's MRG Messaging inside of its MRG initiative is an implementation of, and the core contributor to, the Apache Qpid project. LShift and CohesiveFT are jointly developing RabbitMQ, a "complete and highly reliable Enterprise Messaging system" that can be used to build an AMQP network or to enhance an established network. OpenAMQ from iMatix is another AMQP implementation product available. OpenAMQ was described as a product that "provides you with a framework on which to build distributed business applications which communicate using messages."

Even with this momentum, there are still detractors of AMQP. Lustratus Research, headed by somw long time IBM foks, wasn't ready to climb on the train with AMQP in 2007 when they said

guaranteed messaging is almost by definition the backbone of a company's mission-critical operations. If they weren't critical, then guaranteed delivery wouldn't be so important. So, where is the AMQP 'throat to choke'? Who do you run to when it fails, knocking out your key online systems? And going back to the testing question, who will invest in the huge expenses involved in stress testing and performance tuning to ensure it can meet the demands that will be placed on it?

Jean-Louis Seguineau talked about how AMQP is a late arrival to the game XMPP pubsub has already been playing:

XMPP pubsub extensions already address many of these features. Persistent nodes play the role of AMQP store-and-forward queues, instant nodes can be used for on demand AMQP queues. But AMQP also makes way for content-based routing, explicitly using message attributes. XMPP does not provide a similar explicit way to define content-based routing, as the pubsub extension leaves this decision to the implementation.

Alexis Richardson, of CohesiveFT, and Carl Trieloff, of RedHat, think differently about AMQP. In interviews with Gould both men talked about where they see the future of AMQP. Richardson noted it could become an internet protocol that does handles "everything that HTTP and SMTP do badly at present". In his view, AMQP takes care of what business needs:

In a business network you need trust, you need transactional integrity. What goes in has to be the same thing that comes out. You need smart routing and multiple topologies, you need to be able to manage traffic flows and do server-to-server federation. And you need complete interoperability between vendors and implementations. All this is what it takes to do transactional business messaging, and this is what AMQP delivers.

Trieloff talked about the progress of AMQP interoperability and said, "our experience has been that interop between the implementations is continually getting better, because the spec is getting better and less ambiguous, and because the developers are working very hard to track it. I think you need to have a very pragmatic approach to interop."

AMQP may not be quite ready to replace your current message oriented middleware, but it is getting closer and deserves a look if you are building or enhancing yours.