For global businesses to meet today’s architectural challenges with constant change and extreme scale, we need to go back to the basic principles of system design. The common element in the problems we face is the notion of events driving both actions and reactions, Neil Avery writes in a series of blog posts describing why events are important and the advantages of an event-first approach that also supports an evolutionary architecture.
Avery, author at Confluent, a provider of an event streaming platform based on Kafka, notes that it has taken us 25 years to realize that events should be a core principle in system design. Back in 2002 he built a commercial messaging system based on Staged Event-Driven Architecture (SEDA) and that’s when he understood the importance of events. In an increasingly digital world, organizations must be more real-time oriented and to succeed with this he believes they must be event-driven:
There has been a revolution; organizations must become real time; to become real time, they must become event driven.
When transitioning to an event-driven architecture, Avery’s experience is that this often is a move from simple use cases to a deeper understanding of processes and operations, and how to reason about the architecture. After a while, teams discover that although the events are useful, it’s the stream that enables them to really improve. He emphasizes that the value of an event lies in the fact that a sequence of events (a stream) represents behaviour, for instance describing how a user clicks her way through a site, or adds and removes items from a shopping basket.
To become event-driven, Avery describes two alternatives. The first uses commands as events to describe something that should be done — an event-command approach. One example of this is a traditional REST application using synchronous calls. Problems with this approach include that both the endpoint and the method being called must be known, and that changes must be made in existing parts when a new event processor is added.
The second approach uses the messaging paradigm with pure events to describe something that has happened — an event-first approach. This is what Avery recommends and notes that this moves the responsibility for dealing with an event over to the consumer. Here an event is published without any coupling to an API. The publisher of the event is not aware of any consumers of the event, which means that these can change over time with no effect on the publisher. He emphasizes by writing:
The event-first approach forces an inversion of responsibility; it is a fundamental paradigm shift of how applications are developed.
This may look more complex, but Avery points out that when trying to enhance a system built this way, the benefits will be clear. He also notes that an evolutionary architecture is a benefit from this way of thinking.
In a blog post early 2018, Randy Shoup discussed how events should be a first-class tool for developers, and noted that developers often underestimate the value of events as a tool, and consequently often miss the advantage they could give. He also pointed out that the events should be included when an interface is defined. Shoup has a long experience working for eBay, Google and Stitch Fix, and is currently VP of engineering at WeWork.
Bernd Rücker, co-founder of Camunda, the company behind the Camunda Workflow Engine, claimed in a presentation at DDD eXchange 2017 that although publishing events keeps domains decoupled from each other, if there really is a logical flow of events, this flow may become implicit and hard to follow. In such cases he prefers using a process manager to keep track of the overall process.
In a second blog post, Avery looks into the details of different styles of event-driven, streaming architectures and compares a reactive architecture, possibly using the actor model, with a streaming architecture. He also compares the different persistence models used. Today, many organizations use a microservices framework and often believe they can be improved by adding streaming, but Avery emphasizes that this is not true — an event streaming platform will replace the microservice framework, because he thinks of stream processors, producers and consumers as microservices.
Avery is also in the progress of writing more posts: part 3 with a focus on serverless, and part 4 where he will discuss the art of the event streaming application.
