Ilya Grigorik, founder and CTO of PostRank, wrote an introduction to ZeroMQ last week:
Berkeley Sockets (BSD) are the de facto API for all network communication. With roots from the early 1980's, it is the original implementation of the TCP/IP suite, and arguably one of the most widely supported and critical components of any operating system today. BSD sockets that most of us are familiar with are peer-to-peer connections, which require explicit setup, teardown, choice of transport (TCP, UDP), error handling, and so on. And once you solve all of the above, then you are into the world of application protocols (ex: HTTP), which require additional framing, buffering and processing logic. In other words, it is no wonder that a high-performance network application is anything but trivial to write.
he adds:
Wouldn't it be nice if we could abstract some of the low-level details of different socket types, connection handling, framing, or even routing? This is exactly where the ZeroMQ (ØMQ/ZMQ) networking library comes in: "it gives you sockets that carry whole messages across various transports like inproc, IPC, TCP, and multicast; you can connect sockets N-to-N with patterns like fanout, pubsub, task distribution, and request-reply"
ØMQ is a new layer in the networking stack, a scalability layer that spreads the load of a distributed system so that it can support arbitrarily large applications. ØMQ defines the overall topology of the distributed system rather than simple point-to-point communication. 0MQ applications are concurrent without locks, and elastic over any number of threads, cores, and boxes.
ZeroMQ is community driven:
Dozens of [people] worked together for three years to build ØMQ (also known as ZeroMQ or 0MQ) as free software.
Ilya explains:
- ZeroMQ communication is message-oriented [and] hides much of the everyday boilerplate complexity we are forced to repeat in our applications. This means that if a client socket sends a 150kb message, then the server socket will receive a complete, identical message on the other end without having to implement any explicit buffering or framing.
- ZeroMQ sockets are also transport agnostic: there is a single, unified API for sending and receiving messages across all protocols. By default, there is support for in-process, IPC, multicast, and TCP, and switching between all of them is as simple as changing the prefix on your connection string.
- ZeroMQ sockets are routing and network topology aware. Since we don't have to explicitly manage the peer-to-peer connection state - all of that is abstracted by the library, as we saw above - nothing stops a single ZeroMQ socket from binding to two distinct ports to listen to for inbound requests, or in reverse, send data to two distinct sockets via a single API call.
Ilya explains:
By default, all communication in ZeroMQ is done in asynchronous fashion. This asynchronous processing model allows ZeroMQ to abstract all connection setup, teardown, reconnect logic, and also to minimize message delivery latency: no blocking means the messages can be dispatched, delivered and queued (sender or receiver side) in parallel to the regular processing done by your application. Of course, you can also control the queuing behavior of ZeroMQ sockets by setting an allowed memory bound and even a swap size for each socket. Hence, you can simulate the blocking API if desired, but asynchronous I/O is the default.
Mongrel2 is a Web server which is using ZeroMQ. Ilya explains:
Mongrel2 offers an interesting case-study of applying ZeroMQ to the world of web-servers: all inbound requests are routed by Mongrel2 via a "Push" socket which automatically load-balances the requests to connected handlers. The handlers, in turn, process the incoming requests (via Pull socket) and publish them to a "Pub" socket, to which the Mongrel2 server itself is subscribed to and is listening for its process ID (via a topic filter).
ZeroMQ's web site provides a short comparison with other communication mechanisms:
- TCP: message based, messaging patterns rather than stream of bytes.
- XMPP: simpler, faster, and more low-level. Jabber could be built on ØMQ.
- AMQP: 100x faster to do the same work and with no brokers (and 278 pages less spec).
- IPC: we abstract across boxes not only a single machine.
- CORBA: we do not enforce horrible complex message formats on you.
- RPC: ØMQ is totally asynchronous, and lets you add/remove participants at any time.
- RFC 1149: a lot faster!
- 29west LBM: we're free software!
- IBM Low-latency: we're free software!
- Tibco: we're still free software!
Ilya concludes:
Needless to say, ZeroMQ is an ambitious project, and this short introduction only scratches the surface of the full feature set. The stated goal of ZeroMQ is to "become part of the standard networking stack, and then the Linux kernel". Whether they will succeed, remains to be seen, but it is definitely a very promising and arguably a much needed layer of abstraction on top of the "traditional" BSD sockets. ZeroMQ makes writing high performance networking applications incredibly easy and fun.
Antonio Garrote agrees with Ilya:
ZeroMQ is incredible powerful, is not only fast and efficient but it also simplifies enormously the design of the communication layer for any application. It also makes possible to re-use the same communication pattern in different situations. Nevertheless, ZeroMQ can be seen, in my opinion, as complementary to queues systems like RabbitMQ rather than a drop in replacement. As with any other software system, a careful look at your application requirements will make clear which communication mechanism suits better to the problem you are trying to solve.
Are you using ZeroMQ? what's your take on it?
