Last year, Netflix Cloud Database Engineering (CDE) team introduced Dynomite. Dynomite is a proxy layer, aiming to turn any non-distributed database into a sharded, multi-region replication aware distributed database system. Dynomite is built to provide high availability and can survive from a whole region in AWS failing through an Active-Active setup.
Netflix is already using Dynomite with Redis in their production systems. In a series of blog posts, Netflix is going through use cases and features of Dynomite. In the first one of the series, we can better understand how does Dynomite help Redis scale linearly on AWS.
Dynomite can extend the eventual consistency model to tunable consistency. The client can select consistency level of DC_ONE where reads and writes are propagated synchronously within the local Availability Zone(AZ) and asynchronously outside it. Alternatively, she can select DC_QUORUM where operations are propagated synchronously within a quorum number of nodes in the local region and asynchronously outside of it.
As shown in the tests, Dynomite can scale from 3,6,12,24 all the way through 48 nodes with linear throughput increase. This happens both in DC_ONE and in DC_QUORUM setups. Dynomite adds a minimal overhead in terms of latency, which in all tests, even with DC_QUORUM is not more than a couple of milliseconds. DC_QUORUM has higher latency and lower throughput but can provide better read and write guarantees to the client.
Finally, Dynomite also supports Redis pipelining, which can batch several requests together for a potential benefit of up to 50%. Using Dynomite, a client can improve throughput against latency, so this is something to be taken into account from the use case perspective.
The next article in the blog series, will deal with handling anti-entropy in Dynomite and how to achieve higher consistency.