Cloudflare recently described a prototype called Cache Transcoding that compresses eligible cache content, mainly uncompressed text such as HTML, JSON, CSS, and JavaScript, using Zstandard before storing it on disk. The hyperscaler estimates that the approach could provide petabytes of additional effective cache capacity, although broader testing is still needed.
According to the article, the additional processing required is relatively small, as the content is compressed once when it enters the cache and decompressed when served. Compression reduced the size of eligible content by about 2.8 times, allowing existing servers to store more data and reducing the amount of data transferred between data centers.
To achieve this, Cloudflare relies on Zstandard, the lossless compression algorithm developed by Facebook for real-time use, together with Pingora, its Rust-based proxy framework. Aashi Patel writes:
A small increase in CPU gives Cloudflare petabytes of effective cache capacity and reduces the data transferred between our data centers. The encoding cost is paid once when an asset enters the cache. The storage and bandwidth savings continue every single time that asset is reused.
Cloudflare's Cache Transcoding only compresses uncompressed responses that contain compressible text, return successfully, and are at least 4 KiB. It excludes range requests, precompressed or binary content, and responses with an unknown size. The threshold avoids processing many small objects while sacrificing only about 1% of eligible data, and both this threshold and the Zstandard compression level can be adjusted based on CPU and storage trade-offs. Patel adds:
Transcoding does not mean compressing everything. Images, video, and fonts are usually compressed already. In our traffic sample, this media slice represented 21.4% of requests but 63.3% of bytes. Compressing it again would burn CPU for nothing.
According to Cloudflare, compressible text such as HTML, JSON, CSS, and JavaScript accounted for 67.3% of requests and 22.3% of bytes. Of this content, about 71% arrived uncompressed and could be effectively compressed.
Source: Cloudflare blog
While some practitioners question the use of the word "transcoding" for an encode/decode approach, others on Hacker News discuss the compression options. As Cloudflare initially considered limiting transcoding to popular content, user MayeulC writes:
Weird, I would have compressed cold content instead, if the goal was to save on CPU time during decode.
User CodesInChaos asks instead:
I'm confused by how this affects range requests. Without compression, those can be easily satisfied by reading the relevant part of the cached complete file. But how are they handled now?
Tests were performed with and without Tiered Cache to measure the impact of compression on local caching and transfers between cache tiers. The prototype remains under development, with further testing planned across different compression levels, content types, object sizes, and cache scenarios.