Today, Microsoft released a novel service for ingesting and processing cloud events. The Azure Event Grid takes events generated from Azure services, or custom apps, and routes them to chosen handlers. This service offers developers and operators a single event stream to tap into for serverless apps, app integration, or ops automation.
Microsoft's Corey Sanders describes this service as a "fully-managed event routing service" that "makes an event a first-class object in Azure." Azure services emit events that get routed to one of many types of event handlers. These handlers—ranging from Azure Functions to webhooks—support event filtering and reliable delivery. Microsoft claims that the Azure Event Grid scales to millions of events per second. The first 100,000 operations per month are free, and users pay $0.60 per million operations after that.
While all major cloud providers, including Microsoft, offer a bevy of messaging services for application-to-application routing or event processing, this capability is a bit unique. Microsoft is offering a single eventing fabric with managed or custom topics. They shared a commitment to deliver built-in publisher support for all their major cloud services. Their out-of-the-box handlers also give users the choice of triggering actions within Azure, or outside through webhooks. For reference, services like Google Cloud Storage emit events to Google Cloud Pub/Sub, but this is only for a subset of services, and messaging topics are self-managed. In the AWS cloud, SQS takes events from a handful of services, and CloudWatch publishes events to a few types of handlers.
To learn more about the Azure Event Grid, InfoQ reached out to Dan Rosanova, principal program manager Lead who acted as product owner for the new service.
InfoQ: Where did Azure Event Grid come from? Was this an internal product that became a public offering, such as Service Fabric? Or was this created from the start to be customer-facing?
Rosanova: Azure Event Grid was created from the start to be customer focused, but also to meet the internal needs we have in Azure, and across Microsoft, for eventing. This means we purposely designed it for scenarios common both to Azure and to our customers. Scale, reliability, interoperability, and cost were all considered very important for us.
InfoQ: Which Azure services publish events to the Event Grid today? Which regions? What's planned for later on?
Rosanova: Today Azure Event Grid is available in West US 2 and West Central US. Azure Resource Manager, Event Hubs Capture, and Storage blob service are launch publishers. We will continue to deploy this service to every region in Azure and over time onboard all Azure services.
InfoQ: Can developers publish events to Azure Event Grid from their own applications? Even if those apps are outside Azure? How would they do that?
Rosanova: In Azure Event Grid, all events are emitted from topics. There are some topics that are Azure specific topics, like your storage account. When you use that storage account, it will raise events. You can also make your own custom topics. This enables you to publish your own events to a topic and you, or others, can subscribe to that topic to receive events they are interested in. Publishing is done via HTTPS POST using a key or token in a header for authorization. My favorite way to send to Event Grid is with curl using bash variables: curl -X POST -H "aeg-sas-key: $key" -d "$body" $topicEndpoint
InfoQ: How do developers interact with the Azure Event Grid? What's the UX?
Rosanova: Azure Event Grid is an Extension Resource in side of Azure, this means that it is anchored to other resources like storage. When you want to create event subscriptions on a storage account, you actually go to the UX for that storage account. In this way events really are a natural extension to resources you're already working with in Azure. You can also interact with Azure Event Grid through its own UX, but we expect many people to use the UX of the event publisher. For custom topics, the anchor is your custom topic itself, so you just navigate to your topic and subscribe to events from there. This really is the same model and same technology used both for our internal usage and offered to customers directly.
InfoQ: What does an "event" look like? Are there required and optional parts of the header/payload?
Rosanova:aeg-sas-key: <Key From My Azure Event Grid custom topic>[{"id": "'"$RANDOM"'","eventType": "recordInserted","subject": "myapp/vehicles/motorcycles","eventTime": "'`date --iso-8601=seconds`'","data":{"make": "Ducati","model": "Monster"}}]In the above sample, the header aeg-sas-key is required, or a SAS token in a header called aeg-sas-token. Within the body the id, event type, subject, event time (in ISO 8601) and any valid JSON in the data object are all required. You don't need to preregister the event type or subject, but you can use them to enable filtering for your event subscribers, so you should think about using this in your event modeling.
InfoQ: Is this for reliable messaging scenarios?
Rosanova: Event Grid offers at least once delivery. If your endpoint doesn't 200/202 our request, it will retry with an exponential back off. Eventually this retry interval becomes one hour and after 24 hours we drop undelivered events. At least once means that while it is reliable, you may receive events more than once, particularly if you don't acknowledge them.
InfoQ: What are valid consumers of events? Custom apps? Webhooks? Azure services?
Rosanova: All of the above; we don't discriminate about where events go and we don't limit them to region or cloud. Consumers can be other Azure services like Logic Apps and Functions which have direct UX integration to Azure Event Grid, but they can also be your own HTTP endpoints run in Azure or anywhere else.
InfoQ: Microsoft offers a handful of messaging-related services including Azure Service Bus, Azure Event Hubs, Logic Apps, and more. What are the clear use cases for Azure Event Grid?
Rosanova: Azure Event Grid fills a gap in the current cloud messaging space, not just in Azure, but across all cloud providers. We have services for messaging, queuing, and telemetry, but nothing for comprehensive eventing, particularly for cross service or cross cloud scenarios. If you look at a typical eCommerce application you would probably use something like Service Bus for the sales transactions and other material or financial exchanges, something like Event Grid to let all the parts report their status, like "inventory pulled", and something like Event Hub to track all the telemetry across all the different components. The three areas that we think will be significant for Azure Event Grid are serverless, operations, and integration. Each for its own reasons and focused on slightly different audiences.