InfoQ Homepage Streaming Content on InfoQ
-
Yahoo! Benchmarks Apache Flink, Spark and Storm
Yahoo! has benchmarked three of the main stream processing frameworks: Apache Flink, Spark and Storm.
-
CQRS, Read Models and Persistence
Storing events in a relational database and creating the event identity as a globally unique and sequentially increasing number is an important and maybe uncommon decision when working with an event-sourced Command Query Responsibility Segregation (CQRS) system Konrad Garus writes in three blog posts describing his experiences from a recent project building a system of relatively low scale.
-
Introducing Reactive Streams
Modern software increasingly operates on data in near real-time. There is business value in sub-second responses to changing information and stream processing is one way to help turn data into knowledge as fast as possible, Kevin Webber explains in an introduction to Reactive Streams.
-
DDD, Events and Microservices
To make microservices awesome Domain-Driven Design (DDD) is needed, the same mistakes made 5-10 years ago and solved by DDD are made again in the context of microservices, David Dawson claimed in his presentation at this year’s DDD Exchange conference in London.
-
Real-time Data Analytics at Pinterest using MemSQL and Spark Streaming
Pinterest, the company behind the visual bookmarking tool that helps you discover and save creative ideas, is using real-time data analytics for data-driven decision making purposes. It’s experimenting with MemSQL and Spark technologies for real-time user engagement across the globe.
-
Making Sense of Event Stream Processing
Structuring data as a stream of events is an idea appearing in many areas and is the ideal way of storing data. Aggregating a read model from these events is an ideal way to present data to a user, Martin Kleppmann claims explains when describing the fundamental ideas behind Stream Processing, Event Sourcing and Complex Event Processing (CEP).
-
Lessons Learned Building Distributed Systems at Bitly
At the Bacon Conference last May, bitly Lead Application Developer Sean O'Connor explained the most relevant lessons bitly developers learned while building a distributed system that handles 6 billions clicks per month.
-
Microsoft Tackles Internet-of-Things With New Data Stream Processing Service
Last week at the Microsoft Worldwide Partner Conference, Microsoft took the wraps off of Azure Event Hubs. This service – in preview release until General Availability next month – is for high throughput ingress of data streams generated by devices and services. Event Hubs resembles Amazon Kinesis and uses an identical pricing scheme based on data processing units and transaction volume.
-
DataTorrent 1.0 Handles >1B Real-time Events/sec
DataTorrent is a real-time streaming and analyzing platform that can process over 1B real-time events/sec.
-
Reactive Streams with Akka Streams
Typesafe has announced the early preview of Akka Streams, an open source implementation of the Reactive Streams draft specification using an Actor-based implementation. Reactive Streams is an initiative to provide a standard for asynchronous stream processing with non-blocking back pressure on the JVM. Back pressure in needed to make sure the data producer doesn't overwhelm the data consumer.
-
New York Times Lab Introduces Visual Stream Processing Tool
The New York Times R&D Lab has released streamtools, a general purpose, graphical tool for dealing with streams of data, under Apache 2 license.
-
Greg Young on Using Complex Event Processing
Complex Event Processing, CEP, can be very useful for problems that have to do with time e.g. querying over historical data when you want to correlate things that have happened at different times, Greg Young explained in a recent presentation.
-
Twitter Open-Sources its MapReduce Streaming Framework Summingbird
Twitter has open sourced their MapReduce streaming framework, called Summingbird. Available under the Apache 2 license, Summingbird is a large-scale data processing system enabling developers to uniformly execute code in either batch-mode (Hadoop/MapReduce-based) or stream-mode (Storm-based) or a combination thereof, called hybrid mode.
-
Yahoo! Open Sources Storm on Hadoop
Last week Yahoo! announced the open source release of Storm on Hadoop cluster. This implementation enables Storm applications to utilize the computational resources of a Hadoop cluster along with accessing Hadoop’ storage resources such as HBase and HDFS.
-
Dempsy – a New Real-time Framework for Processing BigData
A new open source project – Dempsy adds one more option for people trying to do real time processing of big data. Comparable to Storm and S4 Dempsy is most applicable to near real time stream processing where latency is more important than guaranteed delivery.