Client side load balancing of Ajax applications

When it comes to load balancing options today, most clustered web applications opt for a server side solution, either through software or hardware. Digital Web Magazine recently published an article that discusses how one company has implemented the load balancing logic for their EC2 backed application in the client tier.

The article begins by looking at the requirements of a loading balancing solution:

1. Distribute loads among a cluster of application servers.
2. Handle failover of an application server gracefully.
3. Ensure the cluster of servers appears as a single server to the end user.

Lei Zhu, author of the article, then surveys conventional load balancing approaches used on the server side, starting with “Round Robin DNS” and notes:

Unfortunately, the key drawback to round-robin DNS is that it fails the second requirement mentioned above—if one of the two servers fail, the DNS server will continue to send requests to it, and half of your end users will experience downtime.

He continues by pointing out the drawbacks with dedicated software and hardware solutions that reside ahead of the cluster:

1. There is a limit to the number of requests the load balancer itself can handle. However, this problem can be resolved with the combination of round-robin DNS and dedicated load balancers.
2. There is an extra cost related to operating a dedicated load balancer, which can run into tens of thousands of dollars. The backup load balancer generally does nothing other than wait for the primary to fail.

In introducing the concept of client side load balancing, Lei asks the reader to consider how load balancing for a desktop application could work:

The desktop client randomly selects a server and attempts to retrieve data. If the server is not available, or does not respond in a preset time period, the client can select another server until the data is retrieved. Unlike web applications—which store the client code (JavaScript code or Flash SWF) on the same server that provides data and resource—the desktop client is independent of the server and able to load balance servers from the client side to achieve scalability for the application.

To extend the concept to the web, Lei breaks down the major components in a typical Ajax application:

1. Client-side code: JavaScript and/or SWF (for flash clients)
2. Resources: images, CSS (Cascading Style Sheets), audio, video, and HTML documents
3. Server-side code: backend logic that generates data requested by the client

Items 1 and 2 are relatively static content, and typically do not have the load balancing requirements of item 3. Lei suggests utilizing reliable servers, or services such as Amazon S3 service, for these. Turning his attention to item 3, Lei outlines his strategy:.

Just like the desktop client above, we can embed our list of application servers into the client code. The web client contains a file called “servers.xml”, which has a list of available servers. The client tries to communicate (whether via AJAX or Flash) with every server in the list until it finds one that responds.

Browsers prevent client code making server calls to domains other than where the code originated from, however Lei proposes solutions for both Flash and JavaScript that solve the problem.

1. No separate server of device is required - “a special load-balancing device is unnecessary—there is no need to configure load-balancing hardware, or to make sure that the backup functions the same as the primary load balancer.“
   
2. Servers can be physically separate - “since the client is selecting servers instead of having a fixed load balancer redirecting traffic, the locations of the servers are unrestricted “

Finally, the article shows how the technique was used to make VoxLite, a video messaging application, highly available and scalable. The application was built to utilize Amazon's EC2 and S3 services, however Lei was not able to architect a load balancing solution that didn't have a single point of failure:

Many web applications hosted on EC2 use a single EC2 instance with dynamic DNS to load-balance requests to a particular domain. If the instance that provides the load balancing fails, the whole system can become unavailable until the dynamic DNS maps the domain to another EC2 instance.

To overcome this issue, VoxLite finds the list of available servers by issuing a HTTP GET request against S3, which returns the list of servers. The list is maintained by a cron job in the EC2 instance with the following logic:

1. Load and parse http://s3.amazonaws.com/voxlite/?prefix=servers.
2. If the current running instance is not listed, write an empty file to the bucket with the key servers/{IP address of EC2 instances}.
3. Verify if other servers listed in the bucket are running properly by testing the connection using the internal AWS IP address. If a connection cannot be established, remove the server key from the bucket.

Depending on your requirements, client side load balancing offers an interesting and innovative option in the continuum of load balancing solution architectures. Lei concludes:

By using client-side load balancing with S3 and EC2, it is easy to build an elastic, scalable and robust web application.