BT

Apache Storm Reaches 1.0, Brings Improved Performance, Many New Features

by Sergio De Simone on  Apr 14, 2016

Version 1.0 is "a major milestone in the evolution of Apache Storm", writes Apache Software Foundation VP for Apache Storm P. Taylor Goetz, and it includes many new features and improvements. In particular, Goetz claims a 3x–16x boost in performance.

Microservices for a Streaming World

by Jan Stenberg on  Mar 14, 2016

Embrace decentralization, build service-based systems and attack the problems that come with distributed state using stream processing tools, Ben Stopford urged in his presentation at the recent QCon London conference.

Moving from Transactions to Streams to Gain Consistency

by Jan Stenberg on  Mar 13, 2016

With many databases in a system they are rarely independent from each other, instead pieces of the same data are stored in many of them. Using transactions to keep everything in sync is a fragile solution. Working with a stream of changes in the order they are created is a much simpler and more resilient solution, Martin Kleppmann stated in his presentation at the recent QCon London conference.

Netflix Details Evolution of Keystone Data Pipeline

by Dylan Raithel on  Mar 04, 2016

Netflix has shed light on how the company uses the latest version of their Keystone Data Pipeline, a petabyte-scale real-time event stream processing system for business and product analytics. This news summarizes the three major versions of the pipeline, now used by almost every application at Netflix.

Architecting Scalable, Dynamic Systems when Eventual Consistency Won’t Work

by Michael Stiefel on  Jan 20, 2016

Architecting a scalable and dynamic system without caching is explained by Peter Morgan, head of engineering for the sports betting company William Hill. The values of the bets on sporting events change constantly. No data can be cached; all system values must be current. Distributed Erlang processes model domain objects which instantly recalculate system values based on data streams from Kafka.

The Basics of Being Reactive

by Jan Stenberg on  Jan 20, 2016

A key problem with the whole Reactive space and why it’s so hard to understand is the vocabulary with all the terms and lots of different interpretations of what it means, Peter Ledbrook claims and also a reason for why he decided to work out what it’s all about and sharing his knowledge in a presentation.

Yahoo! Benchmarks Apache Flink, Spark and Storm

by Abel Avram on  Dec 23, 2015

Yahoo! has benchmarked three of the main stream processing frameworks: Apache Flink, Spark and Storm.

CQRS, Read Models and Persistence

by Jan Stenberg on  Oct 20, 2015 4

Storing events in a relational database and creating the event identity as a globally unique and sequentially increasing number is an important and maybe uncommon decision when working with an event-sourced Command Query Responsibility Segregation (CQRS) system Konrad Garus writes in three blog posts describing his experiences from a recent project building a system of relatively low scale.

Introducing Reactive Streams

by Jan Stenberg on  Sep 30, 2015

Modern software increasingly operates on data in near real-time. There is business value in sub-second responses to changing information and stream processing is one way to help turn data into knowledge as fast as possible, Kevin Webber explains in an introduction to Reactive Streams.

DDD, Events and Microservices

by Jan Stenberg on  Jun 29, 2015 1

To make microservices awesome Domain-Driven Design (DDD) is needed, the same mistakes made 5-10 years ago and solved by DDD are made again in the context of microservices, David Dawson claimed in his presentation at this year’s DDD Exchange conference in London.

Making Sense of Event Stream Processing

by Jan Stenberg on  Mar 22, 2015 1

Structuring data as a stream of events is an idea appearing in many areas and is the ideal way of storing data. Aggregating a read model from these events is an ideal way to present data to a user, Martin Kleppmann claims explains when describing the fundamental ideas behind Stream Processing, Event Sourcing and Complex Event Processing (CEP).

Lessons Learned Building Distributed Systems at Bitly

by Sergio De Simone on  Jul 23, 2014

At the Bacon Conference last May, bitly Lead Application Developer Sean O'Connor explained the most relevant lessons bitly developers learned while building a distributed system that handles 6 billions clicks per month.

Microsoft Tackles Internet-of-Things With New Data Stream Processing Service

by Richard Seroter on  Jul 22, 2014 1

Last week at the Microsoft Worldwide Partner Conference, Microsoft took the wraps off of Azure Event Hubs. This service – in preview release until General Availability next month – is for high throughput ingress of data streams generated by devices and services. Event Hubs resembles Amazon Kinesis and uses an identical pricing scheme based on data processing units and transaction volume.

DataTorrent 1.0 Handles >1B Real-time Events/sec

by Abel Avram on  Jun 03, 2014 7

DataTorrent is a real-time streaming and analyzing platform that can process over 1B real-time events/sec.

Reactive Streams with Akka Streams

by Bienvenido David on  Apr 21, 2014

Typesafe has announced the early preview of Akka Streams, an open source implementation of the Reactive Streams draft specification using an Actor-based implementation. Reactive Streams is an initiative to provide a standard for asynchronous stream processing with non-blocking back pressure on the JVM. Back pressure in needed to make sure the data producer doesn't overwhelm the data consumer.

General Feedback
Bugs
Advertising
Editorial
Marketing
InfoQ.com and all content copyright © 2006-2016 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we've ever worked with.
Privacy policy
BT