Allegro has launched version 1.0 of Hermes, a REST API based message broker built on top of Apache Kafka. Whilst not containing any new features, this first major version release is based on the current stable codebase.
Hermes has a pub-sub architecture like a traditional message broker, with its current implementation being an abstraction layer over Kafka that makes it HTTP driven. It also offers additional features over Kafka such multi-DC support, a tracking mechanism, JSON to Avro conversion, and a GUI called Hermes console.
Łukasz Drumiński, a tech lead at Allegro, explains the core use case of Hermes:
Hermes fits very well into microservice architecture with HTTP as the main communication protocol.
Drumiński elaborates that this is particularly true from an integration perspective, as the production and consumption of messages works by sending and receiving HTTP requests. This means that no additional protocol needs to be introduced into the stack, keeping things familiar to developers, and also that shared dependencies can be avoided:
When you have an environment with 20+ services, code sharing, maintenance, and following updates become problematic. At Allegro we had the chance to find it out. It’s better to take out dependencies from business services as much as possible. This also applies to the message broker. It pays to use Hermes when you have an environment with 20+ services developed by many teams.
In addition to the low barrier of entry and non-invasive dependencies within the business services, Drumiński also highlights many other advantages which include:
- Reliability: No loss of messages in the event of a disaster.
- Performance: Only a few milliseconds of overhead is expected over using Kafka natively.
- Metrics: Easy to monitor the rate, latency, throughput, success, and failure of messages.
- Tracking: Using message IDs, they can easily be tracked through the system.
- Highly-available: Hermes supports multiple data centers, so if one fails the system continues to function.
Hermes also comes with Hermes console, a web application which provides a user interface for monitoring and managing it. This might include configuring topics and subscriptions, replaying messages in sequence from an earlier point in time, defining subscriptions, and looking at various metrics.
Drumiński also elaborates as to why Kafka was chosen as the underlying message store for Hermes:
Nowadays you can choose from a variety of messages brokers available on the market. We are aware of them. They are more or less mature. We stick to Kafka as it proved itself in our BigData ecosystem, being mature software which we trust and which provides us with the required functionality.
The future roadmap for Hermes includes:
- Support for dynamic environments such as Kubernetes, making it possible to auto-scale Hermes.
- TLS between Hermes and Kafka.
- Support for one Zookeeper cluster per DC as opposed to one multi-DC Zookeeper cluster.
Hermes is open source and free to use, with various contributors working on the project. The Hermes source code is available on GitHub, and a getting started (with Vagrant) guide is also available.