InfoQ Homepage Apache Kafka Content on InfoQ
-
Spring News Roundup: Third Milestone Releases of Boot, Security, Integration, AI and AMQP
There was a flurry of activity in the Spring ecosystem during the week of March 16th, 2026, highlighting the third milestone releases of: Spring Boot, Spring Security, Spring Integration, Spring AI and Spring AMQP; along with the second milestone releases of Spring Data and Spring for Apache Kafka.
-
QCon London 2026: Introducing Tansu.io — Rethinking Kafka for Lean Operations
Peter Morgan introduced Tansu at QCon London, an open-source, Kafka-compatible, stateless, leaderless broker that scales to zero, with pluggable storage (S3, SQLite, Postgres), broker-side schema validation, and direct writes to Iceberg and Delta Lake. Written in Rust, it uses 20MB of RAM and starts in 10 milliseconds.
-
Pinterest’s CDC-Powered Ingestion Slashes Database Latency from 24 Hours to 15 Minutes
Pinterest launched a next-generation CDC-based database ingestion framework using Kafka, Flink, Spark, and Iceberg. The system reduces data availability latency from 24+ hours to 15 minutes, processes only changed records, supports incremental updates and deletions, and scales to petabyte-level data across thousands of pipelines, optimizing cost and efficiency.
-
Uforwarder: Uber’s Scalable Kafka Consumer Proxy for Efficient Event-Driven Microservices
Uber has open-sourced uForwarder, a push-based Kafka consumer proxy built to handle trillions of messages and multiple petabytes of data daily. The system introduces context-aware routing, head-of-line blocking mitigation, adaptive auto-rebalancing, and partition-level delay processing to improve scalability, workload isolation, and hardware efficiency in large-scale event-driven microservices.
-
Spring News Roundup: Second Milestone Releases of Boot, Security, Integration, Modulith, AMQP
There was a flurry of activity in the Spring ecosystem during the week of February 16th, 2026, highlighting the second milestone releases of: Spring Boot; Spring Security; Spring Integration; Spring Modulith; and Spring AMQP; along with the first milestone releases of Spring Session, Spring for Apache Kafka and Spring LDAP.
-
Uber Moves In-House Search Indexing to Pull-Based Ingestion in OpenSearch
Uber transitions its in-house search indexing to OpenSearch with a pull-based ingestion framework, improving reliability, backpressure handling, and multi-region consistency for large-scale streaming data while simplifying recovery and supporting global, real-time search experiences.
-
LinkedIn Re-Architects Service Discovery: Replacing Zookeeper with Kafka and xDS at Scale
LinkedIn's engineering team successfully upgraded its legacy ZooKeeper service discovery platform to enhance scalability and performance. By leveraging Apache Kafka and the xDS protocol, the new architecture enables eventual consistency, supports multiple languages, and allows migration without downtime. Post-upgrade, latency vastly improved, facilitating hundreds of thousands of app instances.
-
From On-Demand to Live : Netflix Streaming to 100 Million Devices in under 1 Minute
Netflix’s global live streaming platform powers millions of viewers with cloud-based ingest, custom live origin, Open Connect delivery, and real-time recommendations. This article explores the architecture, low-latency pipelines, adaptive bitrate streaming, and operational monitoring that ensure reliable, scalable, and synchronized live event experiences worldwide.
-
Grab Adds Real-Time Data Quality Monitoring to Its Platform
Grab updated its internal platform to monitor Apache Kafka data quality in real time. The system uses FlinkSQL and an LLM to detect syntactic and semantic errors. It currently tracks 100+ topics, preventing invalid data from reaching downstream users. This proactive strategy aligns with industry trends to treat data streams as reliable products.
-
Karrot Improves Conversion Rates by 70% with New Scalable Feature Platform on AWS
Karrot replaced its legacy recommendation system with a scalable architecture that leverages various AWS services. The company sought to address challenges related to tight coupling, limited scalability, and poor reliability in its previous solution, opting instead for a distributed, event-driven architecture built on top of scalable cloud services.
-
Spring News Roundup: GA Releases of Boot, Security, GraphQL, Integration, Modulith, Batch
Following the much anticipated release of Spring Framework 7.0, there was a flurry of activity in the Spring ecosystem during the week of November 17th, 2025, highlighting additional GA releases of Spring Boot, Spring Security, Spring for GraphQL, Spring Integration, Spring Modulith, Spring REST Docs and Spring Batch.
-
From Outages to Order: Netflix’s Approach to Database Resilience with WAL
Netflix uses a Write-Ahead Log (WAL) system to improve data platform resilience, addressing data loss, replication entropy, multi-partition failures, and corruption. WAL decouples producers and consumers, leverages SQS/Kafka with dead-letter queues, and supports delay queues, cross-region replication, and multi-table mutations for high-throughput, consistent, and recoverable database operations.
-
Airbnb’s Mussel V2: Next-Gen Key Value Storage to Unify Streaming and Bulk Ingestion
Airbnb’s engineering team re-architected its internal key-value storage system, Mussel, to unify streaming and bulk ingestion while simplifying operations, achieving over 100,000 writes per second and sub-25ms read latencies on 100-terabyte tables, while leveraging Kubernetes, Kafka, and a NewSQL backend to improve scalability, reliability, and operational efficiency across its internal services.
-
Spring News Roundup: Third Milestone Releases of Boot, Security, GraphQL, Integration, Modulith
There was a flurry of activity in the Spring ecosystem during the week of September 15th, 2025, highlighting third milestone releases of Spring Boot, Spring Security, Spring for GraphQL, Spring Integration, Spring Modulith, Spring REST Docs, Spring Batch and Spring for Apache Pulsar. There were also resolutions to CVEs in Spring Framework and Spring Security.
-
PagerDuty's Kafka Outage Silences Alerts for Thousands of Companies
PagerDuty, the incident management platform used by thousands of organisations to alert them to problems on their systems, suffered a major outage itself on 28th August, 2025. In a comprehensive outage report, the company detailed the scope of the problem, the customer impact, and how it is working to prevent a recurrence.