The Linux Foundation has announced the general availability of Valkey 8.0, the open source in-memory storage solution developed as a successor to Redis. By introducing a dictionary per slot and embedding keys directly into dictionary entries, developers can achieve up to 20% more capacity, allowing for the storage of additional keys per node.
Released under a BSD license, Valkey is a high-performance key/value datastore that supports a variety of workloads, including caching, message queues, and serving as a primary database. Valkey can operate as a standalone daemon or in a cluster, with options for replication and high availability. Kyle Davis, senior developer advocate and open source leader for Valkey, writes:
While this is a major version, Valkey takes command set compatibility seriously: Valkey 8.0.0 makes no backwards incompatible changes to the existing command syntax or their responses (...) Because this version makes changes in how the software uses threading, you may want to re-evaluate your cluster’s infrastructure to achieve the highest performance.
The first major release from the Valkey community comes just five months after Redis switched to the SSPLv1 license, which prompted a fork by former maintainers due to its restrictive nature, as previously reported by InfoQ. Madelyn Olson, principal software engineer at AWS and Valkey project maintainer, comments:
Redis moved away from open source almost 6 months ago, and I still vividly remember discussing on Slack with my former Redis contributors what we were going to do (...) We recently hit 500k container pulls, have contributions from over 100 individuals across the server and client ecosystem, and have over 40 organizations support us.
Explaining the improvements in memory handling, Harkrishn Patro, software engineer at AWS and Valkey contributor, writes:
The first optimization is a dictionary per slot (16,384 of them in total), where each dictionary stores data for a given slot. With this simplification, the cost of maintaining additional metadata for the mapping of slot to key is no longer required in Valkey 8. To iterate over all the keys in a given slot, the engine simply finds out the dictionary for a given slot and traverse all the entries in it.
Other optimizations include a new multi-threaded architecture and key embedding into dictionary entries, which eliminates an extra memory lookup and reduces memory overhead by 8 bytes per key.
Source: Valkey blog
To further improve memory efficiency, the team is discussing plans to overhaul the main dictionary with a more compact memory layout and introduce an open addressing scheme. Andi Gutmans, VP & GM of Databases at Google, comments:
Great to see the innovation velocity in Valkey! Likely to turn out to be one of the most successful forks in open source history!
Recent research by Percona suggests that Valkey has now emerged as the leading open source alternative to Redis following the relicensing controversy:
Over 70% of respondents with Redis deployments said that the shift in Redis’ licensing has motivated them to seek alternatives. And although there are several options already available in the marketplace—including DragonflyDB, KeyDB, and Skytable—it appears that the most likely usurper of Redis’s former market dominance is Valkey.
Valkey 8.0 is now available for download.