The CloudEvents specification can help solve the challenges associated with the cloud event management lifecycle, like the discovery of event producers, setting up subscriptions, and event verification. Doug Davis from Microsoft spoke on Thursday at KubeCon + CloudNativeCon North America 2022 Conference about how the CloudEvents project has been focused on eventing-related pain points that might benefit from some standardization.
Currently, there is no consistency in how the events are managed in the cloud, which is causing application developers to have to write new event-handling logic for each event source.
A typical messaging-based solution in the cloud consists of different event producers going to a middleware message broker like Kafka which in turn is subscribed by different event consumers. The challenge is that the events and messages can be very diverse in terms of formats, schemas, and business logic. These architectures also need to support different messaging protocols like MQTT and HTTP.
CloudEvents, organized via the CNCF's Serverless Working Group, is a specification for describing event data in a common way and seeks to simplify event declaration and delivery across services and platforms. It's a simple specification that defines common event-related metadata independent of business logic. The project goals are to improve interoperability, enable automation, runtime generation & validation, and development time tooling.
Davis discussed what cloud event specification looks like with the help of a JSON example. CloudEvents defines extra metadata with four additional properties: specversion, event type, event source, and event ID. He also showed the difference in message formats like HTTP binary vs HTTP structured format.
CloudEvents context attributes include: required parameters like the ones mentioned above, and optional parameters like subject, time, datacontenttype, and dataschema. There are also extensions like dataref, distributed-tracing, traceparent, tracestate, partitionkey, severity etc.
He talked about how CloudEvents can help with the rest of the lifecycle, including the following use cases:
- discovery of event producer endpoints,
- consumption/subscription mechanism,
- discovery of event consumer endpoints,
- how to push to an event consumer,
- discovery of event types, formats, and attributes,
- discover supported endpoints, and
- event validation
Davis described how Discovery Service Endpoint works in sending or receiving messages. The event producer registers itself with Discovery Service. The event consumer talk to the producer after retrieving the details from the Discovery Service. He showed examples of message definitions with a mixture of s3-events using different formats and schemas. The Subscriber side of the communication needs more than the basic specification attributes. The team is looking to create a brand new spec just for subscribers with additional options like filtering, pull or push model, etc. The message delivery portion takes care of tasks like the transfer of messages, validation of messages, and retrieval of metadata from Discovery Service including message schema.
Schema Registry specification helps with standardization for storing/managing schema itself. Its goals are to keep the spec simple, protocol-neutral, and scenario/schema/messaging neutral. Schema registry structure includes attributes like schema groups, grouping / access control, and schema versions.
The new specs for Discovery API, Subscriptions API, and Schema Registry don't need to be used together, they can be used independently.
CloudEvents integrations include several popular cloud event, data, and security technologies like Adobe I/O Events, Alibaba Cloud EventBridge, Argo Events, Azure Event Grid, Debezium, Falco, Google Cloud Eventarc, IBM Cloud Code Engine, and Knative Eventing.
Davis concluded the presentation by saying that CloudEvents is a new effort and is still under active development. Its working group has received a lot of industry interest, ranging from major cloud providers to popular SaaS companies.
The plan is to have a release candidate version by EOY 2022 with support for different languages and different protocols. The working group is looking for end-user feedback on any new use cases on discovery that are missing in the current specification.
If you are interested in more information on Cloud Events specs, check out the following resources: CloudEvents website, specifications repo, and weekly calls on Thursdays @ 12pm ET (see repo for dial-in info & slack info). If you are interested in contributing, check out the CloudEvents Github project, join their weekly call via Zoom, check the meeting notes, and review the Governance model to familiarize yourself with the contribution process.