Apache Kafka is a distributed, fault-tolerant publish subscribe messaging platform. It was originally developed by LinkedIn, open sourced, incubated in Apache Foundation and is now a top level project. Confluent was formed by former LinkedIn engineers in the Kafka development group and recently announced Confluent Cloud, a fully hosted and managed Apache Kafka as a Service in AWS.
Confluent Cloud is a Software as a Service solution allowing developers to deploy Kafka based solutions in Amazon Web Services, with Google and Microsoft Cloud integrations on the roadmap. As a SaaS it offers the latest stable Kafka version using the same open source APIs developers are familiar with, performance, multiple Availability Zones in AWS and client support in Java, Python, C/C++, .NET and Go. Available throughput ranges from 1-10MBps for read loads and half of that for writes with different purchase plans as described in the datasheet.
Together with Confluent Cloud, Confluent announced the results of its second annual Streaming Data report. The report’s findings outline that adoption of Kafka is increasing, with more than half of the responders using Kafka in more than six systems in production, 43% using it on public cloud and 60% using it on AWS. More than 80% of responders use Kafka for Data Pipelines and half of them use it for microservices setup. One in four responders were working in companies with more than a billion dollar of sales per year, showing adoption of Kafka in large enterprises. Interestingly, 75% of responders had trouble finding talent in the job market with Kafka skills.
Confluent will host a webinar on June 1st for the Cloud SaaS moderated by Rob Benson, senior director of engineering. Pricing for Confluent Cloud is based on usage by measuring throughput, retention and availability. The platform is still in Early access, with interested parties having to apply through the website.