Paypal's Hera framework supports MySQL and Oracle database connection multiplexing, read-write split, sharding, and automatic SQL eviction capabilities. Petrica Voicu and Kenneth Kang from PayPal's development team spoke at QCon New York's 2019 Conference on Tuesday about the data access gateway. Hera (which stands for High Efficiency Reliable Access to data stores), recently open sourced under Apache 2.0 license model, and is used to scale several PayPal applications.
Hera was originally written in C++ but recently migrated to the Go programming language. It also provides secured connections between applications and databases as well as automatic transaction application failover between replica databases.
Voicu started the presentation by talking about typical database access by applications and microservices where each database connection takes a long time to establish. Using a DB connection pool helps by pre-establishing the connection, caching and reusing the connection for each client request. But it creates a shadow process. Also, when you have lot of microservices, you need to configure the pool cache large enough for availability.
Some of the solutions to address these challenges include the following:
- Partitioned connection pool
- Shared connection pool
- Multiplexing
They developed Hera to address the constraints of these different database connection management approaches. Hera is basically a shared connection pool multiplexer that's based on the concept of "microservice first DB proxy." It helps with the abstraction of several different languages (from different client apps) connecting to the database.
Voicu and Kang discussed the following principles and constraints of using Hera framework in client applications:
- Correctness
- Minimal configuration
- Thin client
- Minimal change to wire protocol
Read write split is also supported, where each database instance exchanges lock information with other databses in the cluster. DB write queries are separated and directed to one node. This reduces traffic between the database nodes. Hera automatically converts application query using SQL rewrites for the query to be sharding compatible. Sharding helps with controlled data redistribution. Hera keeps the sharding logic in the server so each application doesn't need to manage sharding on the client side.
Its resiliency features for query oversubscription include a surge queue to put the service requests in a queue and process as soon as another service request is complete. Hera also takes care of the slow query eviction to make space for many typical queries instead of one expensive query. Transparent application failover is accomplished using techniques like read replica retry. It also frees database connections on each Hera node after the queries are executed.
The framework can be used for monitoring query usage which helps with database maintenance without customer impact.
If you want to contribute to Hera framework, check out their website for more details.
