InfoQ Homepage Distributed Programming Content on InfoQ
-
Yahoo! Benchmarks Apache Flink, Spark and Storm
Yahoo! has benchmarked three of the main stream processing frameworks: Apache Flink, Spark and Storm.
-
Release of MBrace 1.0, a Distributed Programming Framework for .NET
After several years of development, MBrace 1.0 was released last week. MBrace is a programming model for scalable cloud data scripting and programming with F# and C#. The project consists mainly of code libraries and cloud providers runtime.
-
GameAnalytics Open-Source Erlang Scheduler for Distributed Tasks
GameAnalytics, maker of a free analytics platform, has recently open sourced gascheduler an Erlang library that provides a generic scheduler for parallel execution of distributed tasks. InfoQ has spoken to Chris de Vries, one of gascheduler’s creators.
-
ELIoT: Distributed Programming for the Internet of Things
ELIoT (Extensible Language for the Internet of Things) is a simple and small programming language aiming to make distributed programming easier. A program in ELIoT may appear as a sigle program, but it actually runs on different computers, so, e.g., a variable or function declared on one computer is transparently used on another.
-
Twitter Has Replaced Storm with Heron
Twitter has replaced Storm with Heron which provides up to 14 times more throughput and up to 10 times less latency on a word count topology, and helped them reduce the needed hardware to a third.
-
Protocol Design and Implementation with Martin Thompson
Martin Thompson answers a few questions about the opportunity for developers and architects to introduce custom protocols to their system's interaction points.
-
Spark, Storm and Real Time Analytics
Hadoop is definitely the platform of choice for Big Data analysis and computation. While data Volume, Variety and Velocity increases, Hadoop as a batch processing framework cannot cope with the requirement for real time analytics. Spark, Storm and the Lambda Architecture can help bridge the gap between batch and event based processing.
-
AnyPresence Soups up Enterprise MBaaS Platform- Part 1 of 2
Mobile Backend as a Service provider AnyPresence continues to hone their chops. Launching the fifth update to their self-titled platform geared for the enterprise. Co-founder Rich Mendis provides some insights for InfoQ readers…
-
Twitter Storm: Open Source Real-time Hadoop
Twitter has open-sourced Storm, its distributed, fault-tolerant, real-time computation system, at GitHub under the Eclipse Public License 1.0. Storm is the real-time processing system developed by BackType, which is now under the Twitter umbrella.
-
Clojure Roundup: Distribution with Crane, Mathematics with Incanter, Builds with Leiningen 1.0
FlightCaster recently open sourced Crane, a tool for distributing and remotely controlling Clojure instances, currently specialized for EC2. Incanter is a Clojure library and tool that makes R-like statistical computations easy with Clojure. Also: the build and dependency management tool Leiningen 1.0 is now available.
-
CRISPY, a New Remoting Framework
With the multiplicity of existing remoting mechanisms it is often necessary to build clients in a way that allows to swap/introduce new protocols with no/minimal impact to the client’s implementation. A new framework – CRISPY - provides support for such implementations.
-
How Relevant Are The Fallacies Of Distributed Computing Today?
Tim Bray of Sun Microsystems writes of the Fallacies of Distributed Computing; He observes that despite its profound implications when designing distributed systems, “you don’t often find them coming up in conversations about building big networked systems”.
-
Article: Why Do We Need Distributed OSGi?
Recently, an early release draft of a Distributed OSGi requirements and design document has been published, along with a reference implementation as part of Apache CXF. In a new article, Eric Newcomer writes about the current status of distributed OSGi and explains the reasons for standardizing it in the first place, and its significance to the OSGi specification and community.
-
Google Introduces Binary Encoding Format: Protocol Buffers
Google caused a stir by releasing Protocol Buffers, a binary serialization format. We take a look at what exactly Protocol Buffers are and what alternatives are available in ASN.1 or Facebook's Thrift.
-
Interview: Randy Shoup Discusses the eBay Architecture
In this interview from QCon San Francisco 2007, Randy Shoup discusses the architecture of eBay. Topics discussed include eBay's architectural principles, horizontal and vertical partitioning, ACID vs. BASE, handling data inconsistency, distributed caching, updating eBay on the fly, architectural and coding standards, eBay's search infrastructure, grid computing, and SOA.