The Discord team hit a hard-limit on the performance of BEAM -- the virtual machine at the core of the Erlang Open Telecom Platform (OTP) -- when dealing with large data structures, and so they resorted to interfacing Elixir with Rust to enable their system to scale up to 11 million concurrent users.
Discord, maker of a multi-platform voice over IP (VoIP) application platform, has been known for using Elixir on top of the Erlang virtual machine to scale their system up to five million concurrent users. Recently the user base has grown to eleven million, with millions of events per seconds. A recent optimization they tried to implement required a higher performance dealing with large data structures than what Elixir and BEAM appeared to be able to provide.
The BEAM VM is pretty speedy and getting faster everyday. It tries to take advantage of persistent data structures where it can, but at the scale we operate, these large lists could not be updated fast enough.
Specifically, Discord engineers needed a sorted data structure capable of supporting a large number of concurrent mutations with the ability to give back a list of affected indices. So, they started to analyze which Elixir data structures could meet those requirements, including MapSet, List, and Ordset. None of them were suitable, especially when considering their worst-case performance on collections of 250,000+ items. A similar research conducted among available third-party packages held no useful results either and for very much the same reasons.
At this point, Discord engineers took a completely different approach and attempted to use a Native Implemented Function (NIF) to reach their goal. NIFs are a mechanism that enables the use of native code from within the BEAM virtual machine as if it were just another Elixir or Erlang built-in function. Rust seemed the natural choice as a language to build a NIF, thanks to both its "zero-cost abstractions" approach and its strong safety guarantees that promised to minimize the chances to crash the BEAM VM or leak memory. Additionally, the Rust ecosystem already provides an open-source library to make it easier to write BEAM NIFs using Rust, Rustler. Rustler provides facilities to generate interface boilerplate, encoding and decoding, and error management.
After several iterations, Discord engineers could write a Rust implementation of a SortedSet which delivered a significant 6.5x best-case performance improvement and a huge 160x worst-case performance improvement. The new SortedSet was subsequently used as a foundation for all Discord guilds, i.e., roughly speaking, user groups.
This project showed how effectively Rust can be integrated with the BEAM VM, providing not only performance and safety, but also reliability at scale, without losing the advantages of programming in a higher-level, functional language such as Elixir.
