Asynchronous, Event-Driven Web Servers for the JVM: Deft and Loft

Asynchronous, event-driven architectures have been gaining a lot of attention lately, mostly with respect to JavaScript and Node.js. Deft and Loft are two solutions that bring "asynchronous purity" to the JVM.

InfoQ had an interview with Roger Schildmeijer, one of the two creators, about these two non blocking web server frameworks:

InfoQ: What is Deft and Loft?

Roger: So before I start to describe what Deft and Loft is I would like to start from the beginning. September 2009 Facebook open sourced a piece of software called Tornado which was a relatively simple, non-blocking Web server framework written in Python, designed to handle thousands of simultaneous connection. Tornado gained a lot of attraction pretty quickly and became quite popular because of its strength and simplistic design. At this time a lot of developers out there became aware of the "c10k problem" (from Wikipedia: The C10k problem is the numeronym given to a limitation that most web servers currently have which limits the web server's capabilities to only handle about ten thousand simultaneous connections.)

In the late summer of 2010 Jim Petersson and I started started to discuss and design an asynchronous non-blocking web server/framework running on the JVM using pure Java NIO. I would say that the main reason for this initiative was our curiosity about the potential speed improvements that could be achieved with a system similar to Tornado but written in Java. We knew that we could never create an API as clean and simple as Tornado's.

(Clean APIs have never been the advantage of Java, if you ask me)

We got something up and running within the first 48h and saw some extraordinary (very non-scientific) benchmark results. This was the feedback we aimed for and Deft was born.

Just to clarify, I would be the last person to suggest that someone should throw away their current system using Tornado (or some other asynchronous non-blocking web server) and replace it with Deft. Deft is still pretty young and has a lot to learn (there are a lot of issues to be implemented). By the time this interview is published I hope that the next stable Deft release, 0.2.0, will be ready and released.

After a couple of weeks of Deft hacking we started to discuss how Deft would look like if we had used another language, like Scala. It would be very gratifying to create something with nice performance but also a system that had a clean and elegant syntax. This was the seed for another project, and Loft was born. To make another important clarification: Loft is still very much in its infancy and there is yet no stable release available.

The main features that set Loft apart from Deft are: 

1. it's written in Scala
2. it uses a feature in Scala called continuations. The reason for this is to make asynchronous programming easier. (Will explain that in detail below)
3. the initial version will be a pure Tornado clone

InfoQ: Would you like to explain to us the motivation behind those architectural decisions?

Roger: We wanted to create something similar to Tornado that runs on the Java Virtual Machine. There are a lot of good (multi-threaded) open source web servers and web frameworks (apache tomcat and jetty are two popular examples) out there already, and we didn't want to compete with those beasts. Instead we wanted to build something that was good at handling thousands of simultaneous connections. The single threaded approach was already tested (and proved successful) by frameworks like Tornado.

InfoQ: What is a typical development workflow in Deft - from coding and debugging, up to deploying and monitoring?

Roger: This is a very good question and something that is very important to address. However, because Deft is so new and its user base is currently small, I'm afraid I can't give an example of a typical development workflow. That is something we hope a growing community will help flesh out.. 

The biggest difference from the coding that most Java developers do is that everything executes inside a single threaded environment. The benefits of coding that way are that you don't have to use explicit locks and you don't have to think about deadlocks because of inadequate synchronized code. The downside is that you are not allowed to do blocking operations. A blocking operation will stall the entire server and make it unresponsive

Deft 0.2.0 will contain an experimental JMX API used for monitoring. Some examples of things that could be monitored are the number of pending timeouts/keep-alive timeouts, number of registered IO handlers in the event loop and the select timeout of the selector.

InfoQ: Are there any benchmarks that evaluate Deft's performance?

Roger: We have made some (non-scientific) benchmarks against a simple web server using a single request handler that responds with a 200 OK HTTP status code and writes "hello world" to the client (The code for the benchmarks is also available on github.com/rschildmeijer). Against a simple "hello-world server" we have seen speed improvements by a factor of 8-10x (compared to Tornado). The entire results from the benchmarks are available on http://deftserver.org/.

InfoQ: How does Deft compare with other solutions like Tornado, Node.js or Netty?

Roger: Tornado and Node.js are the two other asynchronous web servers that we used in the benchmark. We didn't include Netty because it felt a little bit like comparing apples with oranges. But I wouldn't doubt if Netty showed numbers equal to (or greater?) the results we have seen for Deft. Netty, the successor to apache mina, is a really cool socket framework written by a really smart guy (Trustin Lee). 

InfoQ: What was the motivation for Loft and how does it use continuations?

Roger: So finally time to show some code! (I would like to (once again) remind you that Loft is pretty much in the starting blocks and the code snippets for Loft are the proposed design)

A simple request handler for Loft could look something like this:

  @Asynchronous
  def get() {
    val http = AsyncHTTPClient()
    reset {
      val id = database get("roger_schildmeijer");   //async call
      val result = http fetch("http://127.0.0.1:8080/" + id); //async call
      write(result)
      finish 
    }
  }

  val application = Application(Map("/".r -> this))

  def main(args: Array[String]) {
    val httpServer = HTTPServer(application)
    httpServer listen(8888)
    IOLoop start
  }
}

The main method contains the canonical way to start a Loft instance. The ExampleHandler.get is where things start to get interesting. Inside the method two asynchronous calls are made. Asynchronous programming is often conducted by supplying a callback as an additional parameter, and that callback will be called when the result is ready. And if you have two or more consecutive asynchronous calls, you (usually) will have to chain these calls together. 

E.g.:

database get("roger_schildmeijer", (id) => http fetch("http://127.0.0.1:8080/" + id, (result) => write(result)));

So what is actually going in the ExampleHandler.get method above?

You might have noticed the "reset" word in method, this indicates that some Scala continuations are about to happen. Continuations, as a concept, are hard to grasp. But when the magic disappears, continuations are just functions representing another point in the program (It contains information such as the process's current stack). If you call the continuation, it will cause execution to automatically switch to the point that function represents. (Actually you use restricted versions of them every time you do exception handling).

Just so I don't confuse anyone. The two asynchronous methods "get" and "fetch" must be implemented in a certain way in order for this example to work.

InfoQ:  The asynchronous, event-driven paradigm has gained lots of attention lately. Why do you think is that and how do you see this trend evolving in the future?

One reason that systems like Tornado, node.js and Netty have received a lot of attention in recent years is because of big social networks that need a huge number of idle connections.

As long as you and I use services like Facebook and twitter I think the need for systems like Deft, Loft and Tornado will exist.

As a final note I would like to add that we are looking for contributors that are interested in supporting Deft (current status: two committers, two contributors) and/or Loft (two committers).