InfoQ Homepage Apache Kafka Content on InfoQ
-
Confluent Moves Schema IDs to Kafka Headers to Simplify Schema Governance
Confluent introduces a new approach in Apache Kafka that moves schema IDs from message payloads to record headers, aiming to simplify schema governance and evolution. The update integrates with Schema Registry, improves compatibility across serialization formats, and reduces coupling between data and metadata in event-driven architectures.
-
Spring News Roundup: First Release Candidates of Boot, Security, Integration, Modulith, AMQP
There was a flurry of activity in the Spring ecosystem during the week of April 20th, 2026, highlighting the first release candidates of: Spring Boot, Spring Security, Spring Integration, Spring Modulith, Spring AMQP, Spring for Apache Kafka and Spring Vault.
-
Grafana Rearchitects Loki with Kafka and Ships a CLI to Bring Observability into Coding Agent
At GrafanaCON 2026 in Barcelona, Grafana Labs announced Grafana 13 with the new Loki Kafka-backed architecture at the ingestion layer and the AI Observability in Grafana Cloud to monitor and evaluate AI systems in real time. In particular, the new CLI called GCX was announced, designed to surface Grafana Cloud data inside agentic development environments.
-
Uber Launches IngestionNext: Streaming-First Data Lake Cuts Latency and Compute by 25%
Uber launches IngestionNext, a streaming-first data lake ingestion platform that reduces data latency from hours to minutes and cuts compute usage by 25%. Built on Kafka, Flink, and Apache Hudi, it supports thousands of datasets, enabling faster analytics, experimentation, and machine learning workloads globally.
-
Spring News Roundup: Third Milestone Releases of Boot, Security, Integration, AI and AMQP
There was a flurry of activity in the Spring ecosystem during the week of March 16th, 2026, highlighting the third milestone releases of: Spring Boot, Spring Security, Spring Integration, Spring AI and Spring AMQP; along with the second milestone releases of Spring Data and Spring for Apache Kafka.
-
QCon London 2026: Introducing Tansu.io — Rethinking Kafka for Lean Operations
Peter Morgan introduced Tansu at QCon London, an open-source, Kafka-compatible, stateless, leaderless broker that scales to zero, with pluggable storage (S3, SQLite, Postgres), broker-side schema validation, and direct writes to Iceberg and Delta Lake. Written in Rust, it uses 20MB of RAM and starts in 10 milliseconds.
-
Pinterest’s CDC-Powered Ingestion Slashes Database Latency from 24 Hours to 15 Minutes
Pinterest launched a next-generation CDC-based database ingestion framework using Kafka, Flink, Spark, and Iceberg. The system reduces data availability latency from 24+ hours to 15 minutes, processes only changed records, supports incremental updates and deletions, and scales to petabyte-level data across thousands of pipelines, optimizing cost and efficiency.
-
Uforwarder: Uber’s Scalable Kafka Consumer Proxy for Efficient Event-Driven Microservices
Uber has open-sourced uForwarder, a push-based Kafka consumer proxy built to handle trillions of messages and multiple petabytes of data daily. The system introduces context-aware routing, head-of-line blocking mitigation, adaptive auto-rebalancing, and partition-level delay processing to improve scalability, workload isolation, and hardware efficiency in large-scale event-driven microservices.
-
Spring News Roundup: Second Milestone Releases of Boot, Security, Integration, Modulith, AMQP
There was a flurry of activity in the Spring ecosystem during the week of February 16th, 2026, highlighting the second milestone releases of: Spring Boot; Spring Security; Spring Integration; Spring Modulith; and Spring AMQP; along with the first milestone releases of Spring Session, Spring for Apache Kafka and Spring LDAP.
-
Uber Moves In-House Search Indexing to Pull-Based Ingestion in OpenSearch
Uber transitions its in-house search indexing to OpenSearch with a pull-based ingestion framework, improving reliability, backpressure handling, and multi-region consistency for large-scale streaming data while simplifying recovery and supporting global, real-time search experiences.
-
LinkedIn Re-Architects Service Discovery: Replacing Zookeeper with Kafka and xDS at Scale
LinkedIn's engineering team successfully upgraded its legacy ZooKeeper service discovery platform to enhance scalability and performance. By leveraging Apache Kafka and the xDS protocol, the new architecture enables eventual consistency, supports multiple languages, and allows migration without downtime. Post-upgrade, latency vastly improved, facilitating hundreds of thousands of app instances.
-
From On-Demand to Live : Netflix Streaming to 100 Million Devices in under 1 Minute
Netflix’s global live streaming platform powers millions of viewers with cloud-based ingest, custom live origin, Open Connect delivery, and real-time recommendations. This article explores the architecture, low-latency pipelines, adaptive bitrate streaming, and operational monitoring that ensure reliable, scalable, and synchronized live event experiences worldwide.
-
Grab Adds Real-Time Data Quality Monitoring to Its Platform
Grab updated its internal platform to monitor Apache Kafka data quality in real time. The system uses FlinkSQL and an LLM to detect syntactic and semantic errors. It currently tracks 100+ topics, preventing invalid data from reaching downstream users. This proactive strategy aligns with industry trends to treat data streams as reliable products.
-
Karrot Improves Conversion Rates by 70% with New Scalable Feature Platform on AWS
Karrot replaced its legacy recommendation system with a scalable architecture that leverages various AWS services. The company sought to address challenges related to tight coupling, limited scalability, and poor reliability in its previous solution, opting instead for a distributed, event-driven architecture built on top of scalable cloud services.
-
Spring News Roundup: GA Releases of Boot, Security, GraphQL, Integration, Modulith, Batch
Following the much anticipated release of Spring Framework 7.0, there was a flurry of activity in the Spring ecosystem during the week of November 17th, 2025, highlighting additional GA releases of Spring Boot, Spring Security, Spring for GraphQL, Spring Integration, Spring Modulith, Spring REST Docs and Spring Batch.