Khan Academy's engineering team shared their story of scaling to handle 2.5x their usual traffic by leveraging a serverless architecture on Google App Engine and their static content delivery providers Fastly and YouTube.
The online learning platform's usage grew 2.5 times in March compared to last year, and has not decreased. A lot of learning has moved online due to the COVID-19 pandemic, and consequently usage reached 30 million people in April. Marta Kosarchyn, VP of engineering at Khan Academy, says that two key components of their infrastructure - Google App Engine and Fastly CDN - were instrumental in ensuring that their platform could scale in response to rising demands.
Khan Academy's architecture is based on serverless components and serving static content. According to Kevin Dangoor, principal software architect at Khan Academy, they chose serverless on Google App Engine to "scale as our usage increased, without requiring us to scale an operations team to match". The data store is based on Google Cloud Datastore, and the platform uses Memcached for in-memory storage and caching. Internally, Khan Academy runs a custom content management system (CMS) with a versioned content store.
The other key component - the content delivery network (CDN) - is Fastly. Fastly serves as the first point of contact for all requests, and videos are loaded from both YouTube as well as Fastly. Although the primary video service happens to be YouTube, they have a fallback service which serves videos from Amazon S3 via Fastly for institutions where YouTube is blocked. S3 serves as a data store in this case and Fastly caches the videos for subsequent serves, but it's a costlier alternative. Kosarchyn says that they also cache their frequently used queries, preferences and session data in Memcache.
Even though most of Khan Academy's traffic is served from Fastly and Google App Engine, there is a "lot of dynamic behaviour around that static content", says Dangoor. This includes tracking study progress of individual learners, progress reports, recommendations for the next steps in a course, assignment creation by teachers and parents, notifications and integrations with external test providers. The compute cost for these form a big part of the overall costs, says Dangoor. They are currently focused on migrating this codebase to Go, with an aim of reducing the CPU load. Offloading most of the scaling part has allowed them to focus on their key goal - which is providing educational content.
Khan Academy also has some components running on Kubernetes. Their data management (analytics pipeline) runs on Kubernetes on GKE.