Cloudflare recently announced the private beta of Cloudflare Queues, a message queuing service that allows applications to send and receive messages using Cloudflare Workers. The new service provides at-least once message delivery, supports batching of messages, and does not charge bandwidth egress fees.
Cloudflare Queues can be configured with producer and consumer Workers that can send data between each other, with each message stored on disk. Ashcon Partovi, product manager at Cloudflare, explains:
You can use Cloudflare Queues to defer tasks and guarantee they get processed, decouple load among different services, batch events and process them together, and send messages from Worker to Worker (...) We will also support integrations with Cloudflare products, like R2. For example, you might configure an R2 bucket to send lifecycle events to a Queue or archive messages to a R2 bucket for long-term storage.
Developers can use the new service to off-load tasks from a Workers request, guaranteeing messages get delivered to an HTTP endpoint or filter, and fan-out messages to multiple Queues. As for the documentation, Queues can be used to handle bursty loads, processing them in batches. A demo which can be run on a local machine is available on GitHub.
Richard Seroter, director of developer relations and outbound product management at Google Cloud, tweets:
Cool to see Cloudflare get into the "queuing" game. That's the next step for any legit (cloud) platform. Looks simple to use with a basic feature set.
In the "what we served up for the last Birthday Week before we're a teenager" article, Cloudflare summarises the most recent announcements, following the GA of their object storage service. Jeremy Daly, author of the weekly serverless newsletter Off-by-none, comments:
I was particularly interested in R2 now being generally available as well as the announcement of Cloudflare Queues. Globally distributed queues with no egress fees? A game changer?
Discussing the pricing structure for the new service, Partovi writes:
Cloudflare Queues has a simple, transparent pricing model that’s easy to predict. It costs $0.40 per million operations, which is defined for every 64 KB chunk of data that is written, read, or deleted. There are also no egregious bandwidth fees for data in or out - unlike Amazon's SQS or Google’s Pub/Sub.
The price per million operations and the 64 KB chunk of a payload are similar to Amazon SQS pricing, with the main difference being the lack of bandwidth fees, a feature that has now been promoted on different Cloudflare services.
It is now possible to join the waiting list for the private beta where the service is available for free. During the beta, Queues are limited to 100 messages per second, a maximum batch size of 100 and 10 queues per account.