During the HTML5 Track at QCon New York 2013 Matt DeBergalis gave a talk on Meteor, the open-source real-time web application framework that Debergalis co-founded. According to DeBergalis, the client versus server-side pendulum has swung many times: from the mainframe (server-side) to the desktop (client-side) to the web (server-side) and now to the modern web where clients get increasingly capable and more and more work happens in the client again. However, the tools to build these modern web applications, DeBergalis argues, have not caught up. The goal of Meteor is to provide the developer with the tools to build these types of real-time web applications more easily and consistently.
Today's modern web applications consist of two disparate parts:
- The server-side may be built using e.g. PHP, Java or Ruby, using specific server-side APIs, and tools for building software and managing dependencies.
- The client-side uses one or many JavaScript libraries, HTML, CSS and is built using its own build too, for instance Google Closure Compiler.
The two communicate using essentially a custom wire protocol, usually encoded using JSON over HTTP or Websockets sending commands: send me this piece of data, persist this new piece of data, render this piece of data etc.
Meteor is rethinking this approach and aims to make everything real-time. For this purpose it does not deem HTTP a good fit, instead, communication (beyond delivering the web resources) happens via websockets using its DDP (Distributed Data) protocol. The DDP protocol is a generic protocol (encoded using JSON) that supports three core operations:
- Subscribe: using the protocol a client logs on and says: "I'm interested in data collection X, send me an initial snapshot of all its data, then keep sending me deltas so that I can keep my cache up to date."
- Publish: "property X of entity Y has changed its value to Z."
- Remote-procedure call: these are used to execute a remote procedure and to return the result in a fault-tolerant way.
Meteor comes with a DDP implementation for server and client written in JavaScript (Node.js with Fibers on the server). However, the protocol is implementation-agnostic, so alternative implementations can be written. For instance, there is a .NET client library already.
The second component of Meteor is its view engine, automatically keeping the DOM up-to-date with underlying data, similar to what AngularJS and Web Components support.
The third component of Meteor is packaging. Since Meteor components are not just server-side or just client-side, current package systems are not suitable. Meteor's packaging system distributes both the server and client-side components.
Meteor itself decides what parts of your code base need to run in the browser and what parts will run on the server. DeBergalis says that there is currently work underway to use static source code analysis techniques to better partition code, so that only the code that the client absolutely needs is pushed.
Real-time pushing of data is not the only real-time aspect of Meteor. One of the most striking elements of the demo was hot code pushes: whenever the developer saves a file, Meteor not only automatically reloads the code on the server, it also automatically packages up the code and pushes it all the clients, ensuring that all clients are running the latest and greatest version of the client-side code compatible with the server-side code.
Meteor is an interesting full-stack approach to real-time web application development. To learn more about it, consider reading the documentation and look at examples at its website, there is also a book written on Meteor, which we reviewed here on InfoQ.