InfoQ Homepage Streaming Content on InfoQ
-
Dataflow-Based Query Caching with Readyset
Alana Marzoev discusses the fundamentals of streaming dataflow and the architecture of ReadySet, a streaming dataflow system designed for operational workloads.
/presentations/readyset/en/mediumimage/AlanaMarzoev-medium-1729235131570.jpeg)
-
1BRC–Nerd Sniping the Java Community
Gunnar Morling discusses some of the tricks employed by the fastest solutions for processing a 13 GB input file within less than two seconds through parallelization and efficient memory access.
/presentations/1brc/en/mediumimage/GunnarMorling-medium-1729580504344.jpeg)
-
Rethinking Connectivity at the Edge: Scaling Fleets of Low-Powered Devices Using NATS.io
Jeremy Saenz discusses NATS, an open-source project for services communication, and how to leverage NATS to streamline communication and fleet management for devices at the edge.
/presentations/nats/en/mediumimage/JeremySaenz-medium-1722860985397.jpg)
-
Incremental Data Processing with Apache Hudi
The presenters discuss an introduction to incremental data processing, contrasting it with the two prevalent processing models of today - batch and stream data processing.
/presentations/incremental-data-processing/en/mediumimage/SakethBhavani-medium-1722860452431.jpg)
-
Streaming Databases: Embracing the Convergence of Stream Processing and Databases
Yingjun Wu discusses the evolution of streaming databases, and the features and design principles that set streaming databases apart from conventional database systems and stream processing engines.
/presentations/streaming-database/en/mediumimage/YingjunWu-medium-1715174692877.jpg)
-
Building a Large Scale Real-Time Ad Events Processing System
Chao Chu provides insights and practical knowledge for building streaming pipelines for an ad platform.
/presentations/streaming-pipeline-ad-platform/en/mediumimage/ChaoChu-medium-1710236223857.jpg)
-
Multi-Region Data Streaming with Redpanda
Michał Maślanka introduces the design of Redpanda’s Multi-Region feature, and describes how they leveraged Raft’s properties, a constraint solver, automatic data balancing, and tiered storage.
/presentations/redpanda-raft/en/mediumimage/MichałMaślanka-medium-1709028430357.jpg)
-
Building High-Fidelity Data Streams
Sid Anand discusses how they built a lossless streaming data system that guarantees sub-second (p95) event delivery at scale with better than three nines availability.
/presentations/high-fidelity-data-streams/en/mediumimage/SidAnand-medium-1697095755723.jpg)
-
Streaming from Apache Iceberg - Building Low-Latency and Cost-Effective Data Pipelines
Steven Wu discusses the design of the Flink Iceberg, comparing the Kafka and Iceberg sources for streaming and how the Iceberg streaming source can power many common stream processing use cases.
/presentations/apache-iceberg-streaming/en/mediumimage/Steven-Wu-medium-1689934544144.jpg)
-
From Zero to a Hundred Billion: Building Scalable Real-Time Event Processing at DoorDash
Allen Wang discusses the design of the event system including major components of event producing, event processing with Flink and streaming SQL, event format and schema validation.
/presentations/doordash-event-system/en/mediumimage/allen-wang-medium-1685003811186.jpg)
-
Resilient Real-Time Data Streaming across the Edge and Hybrid Cloud
Kai Waehner explores different architectures and their trade-offs for transactional and analytical workloads. Real-world examples include financial services, retail, and the automotive industry.
/presentations/data-streams-architecture/en/mediumimage/Kai_Waehner-m-1675025271514.jpg)
-
Building & Operating High-Fidelity Data Streams
Sid Anand discusses building high-fidelity nearline data streams as a service within a lean team.
/presentations/building-high-fidelity-data-streams/en/mediumimage/Sid-Anand-m-1648803206604.jpg)