Couchbase recently announced a new storage engine for its server called ForestDB.
At its heart, ForestDB uses a Hierarchical B+-Tree based Trie (HB+-Trie) as opposed to a B-tree. It is append only and supports Multi Version Concurrency Control (MVCC) which allows concurrent access to multiple readers and a single writer thereby mitigating locking overhead.
ForestDB also utilizes Write-Ahead Logging (WAL) that maintains in-memory data structures for documents that are not yet indexed into persistent storage. This approach helps in building write intensive highly scaleable applications.
As a result, Couchbase expects some serious performance gains when this beta product is incorporated into a shipping product. Additionally, ForestDB lays the foundation for offering Transaction/ACID support in future versions of Couchbase. Code for ForestDB is available on github.
InfoQ caught up with Cihan Biyikoglu, Director of Product Management at Couchbase.
InfoQ: How do these recent announcements help developers specifically?
ForestDB will eventually sit at the heart of Couchbase Server and Lite and impact many core tenets of our platform. For example, ForestDB storage engine builds a path to future durability improvements, all the way to bringing in ACID-ity for better data protection and enriched guarantees to developers. We expect to also see significant performance and scale gains, leading to more responsive applications that can take advantage of new innovations in hardware. There are too many dimensions to mention here but those are the few at the top.
InfoQ: If I were a developer using the different language SDKs for Couchbase would I be impacted in any way? Can you comment on the roadmap for the SDKs to exploit this server-side feature?
Developers will gain a number of guarantees that will make development much simpler with Couchbase with the addition of ForestDB. Imagine having complete freedom to choose eventual consistency to strict consistency per operation or having transactions with all the added benefits of NoSQL performance, scale and flexibility with your modern Big Data apps. All of that is possible with ForestDB.
InfoQ: Will ForestDB be the storage engine for both Couchbase and Couchbase-lite?
Yes both Couchbase Server and Couchbase Mobile will use ForestDB extensively.
InfoQ: How is the Write Ahead Log (WAL) in ForestDB critical to gaining better performance?
WAL gives us a great deal of performance in managing our writes so we can optimize to the best strategy for batching our writes. WAL also sets up the foundation for durability guarantees and can even build the path to extensive transactional guarantees in future.
InfoQ:Do you expect other vendors (SQL and NoSQL) to adopt ForestDB?
That is certainly our expectation. They can get the same benefits Couchbase gets from the modern storage engine and hopefully they contribute back to the community as well.
InfoQ:How can developers contribute to ForestDB?
ForestDB is open source and is in beta today and we will continue to enhance this engine and innovate in this space. You are welcome to contribute to the effort through github.
NoSQL adoption is steadily increasing and the line between traditional SQL and NoSQL is blurring. ForestDB seems to be aimed at bridging the gap between these development paradigms.