Recently, Cloudflare open-sourced Pingora, their Rust-based framework to create HTTP proxy services. This framework has been open-sourced under the Apache License version 2.0. As a proxy solution, it seamlessly handles HTTP/1, HTTP/2, gRPC, and websocket traffic, featuring adaptable load balancing and failover mechanisms.
In addition to its foundational support for the creation of proxies, clients, and servers, Pingora extends its capabilities with libraries and APIs that facilitate building services utilizing HTTP/1, HTTP/2, TLS, or even TCP/UDS. Pingora has processed almost one quadrillion Internet requests through Cloudflare's extensive network to date.
For security and regulatory adherence, Pingora integrates with the widely recognized OpenSSL and BoringSSL libraries, ensuring FIPS compliance and incorporating advancements in post-quantum cryptography. Currently, Pingora plays an important role in developing services that manage a vast amount of Cloudflare's traffic.
Pingora presents a more secure alternative with its memory management compared to traditional C/C++ services. Cloudflare experienced a notable reduction in memory safety errors, which boosted productivity by allowing more focus on feature development rather than bug fixes. Additionally, for those operating performance-sensitive services, Pingora's multi-threaded architecture ensures fast and efficient performance, saving significant CPU and memory resources. This is especially useful for tasks where the speed and cost of the system are crucial. Furthermore, Pingora offers highly customizable APIs, making it an ideal choice for users looking to create advanced and personalized gateways or load balancers.
On average, Pingora cuts down the wait time for websites to start loading (known as "Time to First Byte" or TTFB) by 5 milliseconds and reduces longer wait times by 80 milliseconds for most users. Pingora uses internet connections more efficiently by sharing them across different tasks. This means it doesn't need to set up new connections as often, saving time usually spent confirming the security and reliability of these connections (through TCP and TLS handshakes).
Source: How we built Pingora, the proxy that connects Cloudflare to the Internet
For all the Cloudflare customers combined, Pingora needs one-third of the number of new connections every second compared to their old system. For a major customer, using Pingora, Cloudflare improved the rate at which connections could be used again from 87.1% to 99.92%. This resulted in the need for 160 times fewer new connections. Interested readers can explore Pingora's programmable API by building a simple load balancer.
The tech community on Reddit and YCombinator took note of this announcement. One of the conversations on YCombinator highlighted that, unlike nginx or caddy, Pingora is not a ready-to-use binary. It's a collection of packages designed for building a customized reverse proxy to suit specific needs. Another comment on Reddit expressed anticipation for the SSL documentation, with a keen interest in discovering features such as hot-reloading for new certificates.
The announcement post also called out the importance of this framework as there is growing recognition of memory safety's importance throughout the industry and by the US government. With this shared objective, Cloudflare is partnering with the Internet Security Research Group's (ISRG) Prossimo project, aiming to promote the integration of Pingora into the Internet’s key infrastructure.