The first four NoSQL datastores are available as Add-ons for the Heroku PaaS (platform-as-a-service) platform. Using the Add-on system that was introduced in October 2009, CouchDB from Cloudant, Membase from NorthScale, MongoDB from MongoHQ and Redis were made available for Heroku users.
Heroku is a Ruby platform provider that currently hosts about 70.000 applications. It's main benefits are the simple setup (via git), dependency handling and seamless scaling for ruby applications. It is based on Amazon's EC2 on top of which Heroku added custom infrastructure management for serving and scaling ruby applications.
Heroku's VP Tech Oren Teich as well as the contributors of the NoSQL Add-ons provided us with additional information which is integrated in the article.
Heroku sees NoSQL databases as very beneficial for its customers, especially for those that have to address problems with large datasets, unstructured, user-provided content and graph structures. Other attributes of NoSQL datastores like high availability, fault tolerance, scalability and partition tolerance are similarly important.
Due to the high demand of customers Heroku decided to go for NoSQL datastores. At the same time the vendors approached the company, so it was "the perfect storm" in Oren's words.
The open Add-on system and marketplace enabled the third party vendors to easily integrate their infrastructure and services with the heroku system.
Oren notes on the status of the NoSQL add-ons:
"The current state of the various Add-ons differs. Northscale memcached and Mongo by MongoHQ are in full production release. Cloudant is currently in public beta, and Redis To Go is in private beta. This state only describes the Heroku addon not the underlying datastore. Some of the Add-ons have been available for months, such as Memcached by Northscale. Others have had a rich beta program where we've had the opportunity to get feedback from hundreds of users."
For other NoSQL Heroku Add-ons, the company would like to encourage users to state their interest and vendors to step up to provide the necessary connecting infrastructure.
We asked the four providers about their experiences of integrating with Heroku, their interfacing protocols and provided capabilities. Other questions covered migration support from relational databases as well as documentation references and future development.
All providers host the instances themselves and just provide access to the Heroku users either by REST interfaces or programmatic APIs (i.e. ruby gems). The Add-ons integrate with the Heroku provisioning system to scale transparently with the needs of the applications. Most of the solutions offer the existing capabilities of the underlying datastores, with enhanced support for fault tolerance and monitoring. For migration from relational datastores existing open source solutions should be used.
MongoDB by MongoHQ
With the MongoHQ add-on it is possible to visually manage databases. It also provides backup and replication.
The connection to the hosted MongoDB datastores is achieved via MongoMapper, the ruby object mapper or Mongoid, a object document framework or the Mongo ruby driver.
NorthScale Membase and Memcached
Integrating with Heroku was straightforward as NorthScale manages the service. Updates of the installations is transparent to users (and to Heroku). Northscale worked with Heroku to develop an API that both fit into their requirements for Add-Ons and also fits with the Membase architecture. Provisioning of the services is done by Heroku and made available directly to the user. NorthScale is also integrated into the Heroku ticketing system.
As membase is a drop-in replacement for memcached, all existing tools can be (re-)used for membase and on Heroku as well.
On the client side, NorthScale worked with Evan Weaver, the creator and maintainer of the Fauna memcached client for Ruby, to enable proper client behavior within a Heroku multi-tenant environment. Membase is compatible with memcached, so existing Ruby on Rails can be switched transparently to Membase including ActiveRecord support with membase as a backing store.
Currently about 2000 users use Memcached Server on Heroku. NorthScale Membase Server, will be available as a Heroku Add-on in mid August.
Northscale recommends incremental migration from RDBMS to caching solutions and NoSQL stores.
Redis To Go - James Bracy
With the system built on EC2 scaling is no issue as well as adding new servers through the Heroku provisioning system. Monitoring was more difficult. Redis To Go uses a set of internal tools and Server density to monitor the status of Redis instances and servers.
Redis To Go is currently being used in several production applications. Currently the biggest user is FlightCaster. The response from people on Heroku has been positive
The Redis add-on has an API for the provisioning service. To communicate with Redis there are several open source solutions. Documentation can be found at the redistogo website.
Backups are done daily. If the web interface is used the backups can be downloaded directly. The system will also notify users when they are nearing the provisioned capacity of their instance.
Cloudant - Alan Hoffman
As Cloudant has already experiences with hosting CouchDB instances (also on EC2) the integration with Heroku was straightforward. The instance deployments are controlled by cloudant and access for the Heroku users is provided via the CouchDB REST interface.
There were Heroku users that were using Cloudant even before the release of the add-on, who provided positive feedback.
The add-on is API compliant with CouchDB 1.0. It provides all of the capabilities of CouchDB including incremental map/reduce and peer-based replication. Cloudant additionally has built in redundancy; storing multiple copies of each record on different servers. The service also automates typical data management tasks such as database compaction and view-builds.
The default Ruby gem for interfacing with CouchDB is CouchRest. There is an overview article about getting started with Heroku and CouchDB. Rails specifics are described at the rails wiki.
