InfoQ Homepage Streaming Content on InfoQ
-
Confluent Announces Kafka for the Enterprise with Multi-Datacenter Replication
Confluent Enterprise latest version supports multi-datacenter replication, automatic data balancing, and cloud migration capability. Confluent, provider of the Apache Kafka based streaming platform, announced last week the new features for Confluent Enterprise, to help build streaming data pipelines and develop stream processing applications.
-
Twitter Open Sources Stream Processing Engine Heron
InfoQ's Rags Srinivas caught up with Karthik Ramasamy, co-creator and engineering manager at Twitter, regarding the Open Sourcing of the Stream-Processing engine Heron, a successor for Apache Storm.
-
Neha Narkhede: Large-Scale Stream Processing with Apache Kafka
In her presentation "Large-Scale Stream Processing with Apache Kafka" at QCon New York 2016, Neha Narkhede introduces Kafka Streams, a new feature of Kafka for processing streaming data. According to Narkhede stream processing has become popular because unbounded datasets can be found in many places. It is no longer a niche problem like, for example, machine learning.
-
Data Streaming Architecture with Apache Flink
Jamie Grier recently spoke at OSCON 2016 Conference about data streaming architecture using Apache Flink. He talked about the building blocks of data streaming applications and stateful stream processing with code examples of Flink applications and monitoring.
-
Apache Spark 2.0 Technical Preview
Two years after the first release of Apache Spark, Databricks announced the technical preview of Apache Spark 2.0 , based on upstream branch 2.0.0-preview. The preview is not ready for production, neither in terms of stability nor API, but is a release intended to gather feedback from the community ahead of the general availability of the release.
-
Comparison of Event Sourcing with Stream Processing
Event sourcing and CQRS are two patterns that has emerged in the Domain-Driven Design (DDD) community. Stream processing builds on similar ideas but has emerged in a different community, Martin Kleppmann noted in his presentation at the Domain-Driven Design Europe conference earlier this year comparing event sourcing with stream processing.
-
Amazon Releases Kinesis Service Update
Amazon has recently announced an update to their Amazon Kinesis Service. In this update, three new features have been added to Amazon Kinesis Streams and Amazon Kinesis Firehose including support for Elasticsearch Service Integration, Shard-Level Metrics and Time-Based Iterators.
-
Azure Stream Analytics Publishing to Power BI Reaches General Availability
On Thursday, April 21 Microsoft announced the integration between Azure Stream Analytics and Power BI has reached General Availability (GA). Using this capability, customers can gain real-time insight into their business performance by analyzing in-flight data streams.
-
Operational Data Stream and Batch Processing at Netflix with Mantis
Operational Data Stream and Batch Processing at Netflix with Mantis
-
Apache Storm Reaches 1.0, Brings Improved Performance, Many New Features
Version 1.0 is "a major milestone in the evolution of Apache Storm", writes Apache Software Foundation VP for Apache Storm P. Taylor Goetz, and it includes many new features and improvements. In particular, Goetz claims a 3x–16x boost in performance.
-
Microservices for a Streaming World
Embrace decentralization, build service-based systems and attack the problems that come with distributed state using stream processing tools, Ben Stopford urged in his presentation at the recent QCon London conference.
-
Moving from Transactions to Streams to Gain Consistency
With many databases in a system they are rarely independent from each other, instead pieces of the same data are stored in many of them. Using transactions to keep everything in sync is a fragile solution. Working with a stream of changes in the order they are created is a much simpler and more resilient solution, Martin Kleppmann stated in his presentation at the recent QCon London conference.
-
Netflix Details Evolution of Keystone Data Pipeline
Netflix has shed light on how the company uses the latest version of their Keystone Data Pipeline, a petabyte-scale real-time event stream processing system for business and product analytics. This news summarizes the three major versions of the pipeline, now used by almost every application at Netflix.
-
Architecting Scalable, Dynamic Systems when Eventual Consistency Won’t Work
Architecting a scalable and dynamic system without caching is explained by Peter Morgan, head of engineering for the sports betting company William Hill. The values of the bets on sporting events change constantly. No data can be cached; all system values must be current. Distributed Erlang processes model domain objects which instantly recalculate system values based on data streams from Kafka.
-
The Basics of Being Reactive
A key problem with the whole Reactive space and why it’s so hard to understand is the vocabulary with all the terms and lots of different interpretations of what it means, Peter Ledbrook claims and also a reason for why he decided to work out what it’s all about and sharing his knowledge in a presentation.