BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Topics

Choose your language

InfoQ Homepage News Cloudflare Migrates JavaScript CDN Serving 9B Requests a Day to Its Developer Platform

Cloudflare Migrates JavaScript CDN Serving 9B Requests a Day to Its Developer Platform

Listen to this article -  0:00

Cloudflare has migrated cdnjs, its open source CDN for JavaScript and CSS libraries, entirely to its Developer Platform, replacing publishing infrastructure spread across Cloudflare and Google Cloud Platform with Workers, R2, Workflows, Queues, Durable Objects, KV, and Containers. The migration also moved R2 into the role of source of truth for published package files while preserving existing URLs, package contents, and Subresource Integrity (SRI) hashes.

Cloudflare said cdnjs now serves approximately 9 billion requests per day, averaging 108,000 requests per second across more than 330 Cloudflare data centers. The service reports a 98.6% cache hit rate and is used by approximately 12% of websites. Cloudflare described the migration as an example of dogfooding its Developer Platform at the scale of a widely used public service.

The migration builds on a 2020 architectural change in which Cloudflare moved cdnjs file serving to Workers and Workers KV, replacing dedicated origin machines for normal traffic while retaining an external origin as a fallback. Cloudflare also introduced precompressed Brotli and gzip assets to improve delivery efficiency.

The publishing path, however, remained distributed. Google Cloud Functions periodically checked npm for package releases, Google Cloud Storage held packages, Pub/Sub handled messaging, and a virtual machine running git-sync synchronized repository content. The system used 26 Cloud Functions divided into alphabetic shards to monitor package updates. At the same time, the GitHub repository had grown beyond 1.1 TB of packed storage, while published files were represented in both GitHub and KV.

Previous cdnjs publishing and serving architecture (Source: Cloudflare Blog Post)

The new architecture makes Cloudflare R2 the source of truth for published files. KV stores package metadata, versions, and SRI hashes, while a Worker handles requests and Workers Cache provides the caching layer. Published content is additionally mirrored to DigitalOcean Spaces as a fallback if R2 cannot serve a file.

Package ingestion is now orchestrated through Cloudflare Workflows. A scheduled workflow checks npm and GitHub for releases, downloads packages into R2, and launches processing workflows for individual files. The processing pipeline extracts package contents, minifies and compresses assets, stores the resulting files in R2, updates metadata in KV, and refreshes the Algolia search index. Workflow state allows processing to resume from the last completed step after a failure.

New cdnjs architecture using R2, Workers, KV, and Workflows. (Source: Cloudflare Blog Post)

Compression presented a constraint because the existing processing algorithms require an entire library to be buffered in memory. Cloudflare therefore uses Containers for compression rather than running the processing directly in Workers. Cloudflare said it is exploring streaming support that could eventually allow this processing to move to Workers.

Preserving package bytes was critical because changes to minification or compression could alter SRI hashes. The migration also exposed platform limits, prompting Cloudflare to raise Worker subrequests from 1,000 to 10 million and Workflow steps from 1,024 to 10,000, with up to 25,000 configurable. The resulting architecture uses R2 for artifacts, KV for metadata, Workers for delivery, and Workflows for publishing.

About the Author

Rate this Article

Adoption
Style

BT