The Modern Edge

In this series of InfoQ articles, we take a look at multiple dimensions of the edge story. We review considerations for virtual and physical security. Another piece explores the data streaming aspects of edge that are more about “how” and not “where.” We also explore the dynamics of constantly-changing edge systems, and how cloud-based edge services are lowering the barrier for experimentation.

This InfoQ article is part of the series "Edge Cloud". You can subscribe to receive notifications via RSS.

Too often we hear fanciful stories of the purpose of ‘edge computing’. I was raised on tales of self-driving cars (which probably should be able to drive without an internet connection), AI-driven security cameras, and oil drilling platforms which can’t process their own data. These use-cases are figurative and fanciful, and while they may exist someday, they aren’t a real part of our lives and jobs. To combat that I thought it would be fun to share some features of how interesting companies are using edge computing in ways you might not expect to solve real problems and help build a better Internet.

npm at the Edge

npm was acquired by GitHub a few weeks ago. What you might not know is every npm package you download is powered by edge computing! 

We can say pretty confidently there's about 11 or 12 million JavaScript developers in the world. And we know because that's how many people are using npm. - Isaac Schlueter, Creator of npm

As is common with many services, npm began by carefully dividing the world into ‘static’ files and ‘dynamic’ responses like npmjs.org searches. Static files could be cached and delivered by a CDN, while dynamic responses had to travel all the way to a central origin to be responded to. In the words of Isaac Schlueter, creator of npm: “Once you have a CDN, you have a really high degree of control over [how files are delivered], especially if you're talking about GETs and especially if you're talking about fetches of downloads of archives that can be long lived… that's just how NPM is architected.”

Unfortunately using a CDN is not without consequences. Static files can’t be customized based on who is viewing them or what that viewer is looking for. Similarly, dynamic responses are slow as they may have to travel halfway around the world and hard to scale as they rely on inflexible infrastructure. npm was built on JavaScript, but they found themselves with a full time engineer devoted to writing not JS, but bespoke CDN configuration languages. As their service became more powerful with features like npm for Enterprise, it became very necessary to deliver different packages to different users, something a CDN was not very good at.

npm uses Cloudflare Workers which run JavaScript in Cloudflare’s points of presence around the world. Workers takes the V8 JavaScript and WebAssembly engine which runs as a part of Google Chrome and runs it on Cloudflare’s network of thousands of servers in hundreds of locations around the world. As V8 doesn’t need to launch a dedicated process or container (or Kubernetes pod) for each customer’s code, it’s possible to perform serverless computing at grand scale without the ‘cold starts’ which plague traditional serverless systems. Code running in a Worker starts in single-digit milliseconds, and requires as little as one tenth the memory overhead of a Node.js process.

Inside their Worker code npm makes decisions about whether a package is private or not. For private packages they ensure that the user has the appropriate authentication token (using the standardized WebCrypto API) before delivering the file.

To understand the transformation, let’s consider performing a similar type of authentication on a request using VCL, the configuration language of the Varnish Cache used by several CDNs, versus being able to write code with JavaScript and Cloudflare Workers. The VCL code has been abridged for space:

sub vcl_recv {
  /* unset state tracking header to avoid client sending it */
  if (req.restarts == 0) {
    unset req.http.X-Authed;
  }

  if (!req.http.X-Authed) {
    /* stash the original URL and Host for later */
    set req.http.X-Orig-URL = req.url;
    /* set the URL to what the auth backend expects */
    set req.url = "/authenticate";
    /* Auth requests won't be cached, so pass */
    return(pass);
  }

  if (req.http.X-Authed == "true") {
    /* we're authed, so proceed with the request */
    /* reset the URL */
    set req.url = req.http.X-Orig-URL;
  } else {
    /* the auth backend refused the request, so 403 the client */
    error 403;
  }

#CDN recv

  ...etc...
}

sub vcl_deliver {
  /* if we are in the auth phase */
  if (!req.http.X-Authed) {
    /* if we got a 5XX from the auth backend, we should fail open */
    if (resp.status >= 500 && resp.status < 600) {
      set req.http.X-Authed = "true";
    }

    if (resp.status == 200) {
      /* the auth backend responded with 200, allow the request and restart */
      set req.http.X-Authed = "true";
    } else if (resp.status == 401) {
      return(deliver);
    } else {
      /* the auth backend responded with non-200, deny the request and restart */
      set req.http.X-Authed = "false";
    }

    restart;
  }

#CDN deliver

  ...etc...
}

async function handle(request) {
  // Make an authentication request that is identical to the
  // original request, but a GET with no body.
  let authUrl = new URL(request.url)
  authUrl.pathname = "/authenticate"
  let authResponse = await fetch(authUrl, {
    ...request,
    method: "GET",
    body: null
  })

  if (authResponse.status === 200) {
    // Client is authenticated.
    return fetch(request)
  } else if (authResponse.status === 401) {
    // Authentication server wants credentials from the client.
    return authResponse
  } else {
    // Every other response from the authentication server becomes 403.
    return new Response(null, {
      status: 403,
      statusText: "Forbidden",
    })
  }
}

As a programmer, I certainly find the code easier to decipher. This means edge computing isn’t just relevant for esoteric use-cases, it’s relevant for each and every CDN requirement developers have. When compared to deploying similar code in a central location using something like Node.JS, deploying to a CDN offers a massive scale advantage for npm. Any site being powered by a CDN is already being served by thousands of servers, this means any code they write is already horizontally scaled to handle their hundreds of thousands of requests per second.

Building Your Own Edge

Anycart looks like any other grocery delivery service. When you dig deeper you find one big exception: it’s very fast. Not the delivery, the website! It’s also growing very quickly:

This growth isn’t coming from paid ads or marketing, it’s coming purely from the speed of the site translating into a better experience and better SEO than its competitors.

CEO, Rafa Sanches, has built a unique setup which eschews any public cloud provider. Instead, he leases dedicated servers in dozens of locations around the world. In fact, he has over 30 dedicated points of presence for his application, and seven more for his database, entirely built and managed by him.

I will never, never, never, never, never, NEVER build on the cloud - Rafa

To Rafa, the public cloud is exceptionally expensive. He is able to get dedicated machines so affordably that his entire hosting bill is under $2000 a month, for a complex service with more than 200k users a day. The secret to getting such cheap hardware is being able to always accept the best (cheapest) deal. He can afford that because he doesn’t need to worry about the availability of any one of his instances. He uses MySQL with Galera Cluster to arrange all seven of his database points of presence into a master-master replication configuration. That configuration means he can write to any one of the instances and have it appear everywhere.

Careful programming has ensured that his system expects all processing to be done asynchronously, so replication lag which would be an issue is minimized. Batch jobs are processed on a 48-core machine which his team only pays $180 a month to run (the same machine on AWS would be over $3000). His users are also generally only touching their own records (like their cart and their recipes), meaning conflict resolution is rarely an issue between replicas. This system means his users get blazing fast reads and writes from anywhere on earth, and he gets to sleep well knowing it would take quite the act of god to take his database offline.

Similarly, his application servers are run on thirty dedicated machines in different corners of the world. Cloudflare global load balancing is used to map his incoming web traffic to the nearest host, and to perform health checks to remove hosts which might be offline.

Many of us make performance decisions based on ‘best practices’, generally accepted minimums required to build something of quality. For Rafa though, things are more simple: “How can we build a user experience that is so good that if Google doesn’t put us first, they’re doing a bad job.” As an example of what that looks like, they don’t use the industry standard tool for frontend programming: React. It’s simply too slow. 

Outsmarting Performance

What can cleansing herbs teach you about performance? They taught Shalom Volchok that e-commerce platforms are not ready for the modern web. Shalom’s family ran a major herbal product company that wanted to perform A/B tests. An A/B test is where a web developer deploys two variants of a page, shows each variant to a portion of visitors, and tests that perform better. Unfortunately for them, every method of doing A/B tests with existing systems delivered subpar performance. In fact, every e-commerce vendor they tried simply didn’t have the capability to both do things dynamically and with performance.

So Shalom founded Outsmartly, a startup e-commerce platform which is built on the edge. Like many new ideas, they had to build the tools to build their product. Namely, they constructed a Cloudflare Workers-based hosting platform which allows for variants to be deployed and served directly from the edge. Features like multiple site variants being deployed at once and integration with most of the modern web landscape (React, Angular, Vue) all had to be built from scratch. With their platform they are able to host sites which load in tens of milliseconds almost anywhere on Earth, but which can have many of the capabilities of a traditional web server.

The dream location would be to render a full application within the CDN’s cache — effectively that’s exactly what Cloudflare Workers enables for our platform. For large ecommerce companies, this page load improvement can equal millions of dollars in incremental revenue. The performance is really that astonishing. — Shalom Volchok, co-founder Outsmartly

They now can deliver static websites that support an unlimited number of A/B tests (and the statistically relevant analytics to go with them). These sites are just as fast as any static site deployed directly to a CDN, but actually support the tools modern businesses rely on. It goes to show you what technology can do when it meets a problem ready to be solved.

The Future

The future of web development and computing is going to look a lot like the present. Hard working and passionate people will build amazing things, and they will often not get the credit they deserve. I believe in that future that websites will be faster for those people who are most disenfranchised by the Internet of today. Platforms will be more powerful and have less moving pieces, less APIs, and less languages. Ultimately edge computing isn’t going to solve someone else’s problems, it will solve yours!

About the Author

Zack Bloom is the Head of Developer Marketing at Cloudflare. He is the creator of open source projects which have gathered over forty thousand stars on Github.

In this series of InfoQ articles, we take a look at multiple dimensions of the edge story. We review considerations for virtual and physical security. Another piece explores the data streaming aspects of edge that are more about “how” and not “where.” We also explore the dynamics of constantly-changing edge systems, and how cloud-based edge services are lowering the barrier for experimentation.

This InfoQ article is part of the series "Edge Cloud". You can subscribe to receive notifications via RSS.