AWS has announced the general availability of Amazon Aurora PostgreSQL Limitless Database, a relational database designed to provide automated horizontal scaling. This new option can handle millions of write transactions per second and manage petabytes of data, all within a single database environment.
According to the cloud provider, the new Limitless Database provides compatibility with PostgreSQL 16.4 and employs a two-layer architecture consisting of multiple database nodes within a DB shard group—either routers or shards—to scale based on workload. Routers are cluster nodes that handle SQL connections from clients and forward SQL commands to shards, while shards are nodes that store subsets of tables and data, processing queries received from routers.
Source: AWS Blog
The new capability enables developers to scale relational database workloads beyond the limits of a single Aurora PostgreSQL writer instance, eliminating the need to create custom application logic or manage multiple databases. Channy Yun, principal developer advocate at AWS, explains:
Once you have created the DB shard group and your sharded and reference tables, you can load massive amounts of data into Aurora PostgreSQL Limitless Database and query data in those tables using standard PostgreSQL queries.
Announced in preview at last year’s re:Invent, the new database supports three types of tables: sharded, reference, and standard. Sharded tables distribute data across multiple shards, splitting the data among them. Reference tables are fully replicated on every shard to facilitate join queries, while standard tables work like regular PostgreSQL tables, stored on a single shard. Yun adds:
You can convert standard tables to sharded or reference tables to distribute or replicate existing standard tables or create new sharded and reference tables (...) To achieve a high degree of parallel processing, Aurora PostgreSQL Limitless Database utilizes two querying methods: single-shard queries and distributed queries, which determines whether your query is single-shard or distributed and processes the query accordingly.
Aurora Limitless is not the only distributed ACID-compliant database, as established competitors like Yugabyte and Cockroach also offer PostgreSQL-compatible distributed solutions. Developer feedback has been mixed, with some questioning the name and highlighting the current limitations. In a popular Reddit thread, user ragbait0 writes:
Everybody stands down. It is just sharding, with an overly ambitious marketing name. Again.
Similar to Aurora Serverless, the setup of the Limitless Database requires specifying a minimum and maximum capacity, measured in Aurora Capacity Units (ACUs), across all routers and shards. The initial number of routers and shards is determined by the maximum capacity set during the creation of a DB shard group and does not change if the value is modified later. While some users question whether the new option also requires a "limitless" budget, user FalseRegister adds:
It is limitless, the same way serverless has no servers.
Aurora PostgreSQL Limitless Database supports only the Aurora I/O-Optimized DB cluster storage configuration, with each shard offering a maximum capacity of 128 TiB and reference tables limited to a size of 32 TiB. It is currently available in select regions, including Northern Virginia, Ohio, Hong Kong, Tokyo, Ireland, and Stockholm.
So far, the cloud provider has not announced plans to support additional database engines.