In a two-installment series, Stephen Crane and Khyber Sen, software engineers at Immunant, recount how they ported VideoLAN and FFmpeg AV1 decoder from C to Rust for the Internet Security Research Group (ISRG). The series includes plenty of detail about how they ensured not to break things and optimized performance.
The AV1 decoder used in VideoLan VLC and FFMpeg, dav1d, has been under development for over six years and contains about 50k lines of C code and 250k lines of assembly. As Crane notes, it is mature, fast, and widely used. It is also highly optimized to be small, portable and very fast. This strongly suggested to port it instead of rewriting it from scratch in Rust.
The first choice engineers at Immunant had to make was whether to do the porting step by step or transpiling the entire codebase using c2rust to get an unsafe but runnable implementation in Rust, then refactor and rewrite it to make it safe and idiomatic. They decided to go the c2rust route because of two major advantages: the possibility to test the ported code while refactoring it and the reduced need for expert domain knowledge it required.
We found that full CI testing from the beginning while rewriting and improving the Rust code was immensely beneficial. We could make cross-cutting changes to the codebase and run the existing dav1d tests on every commit. [...] The majority of the team on this project were experts in systems programming and Rust, but did not have previous experience with AV codecs. Our codec expert on the project, Frank Bossen, provided invaluable guidance but did not need to be involved in the bulk of the effort.
The task of refactoring the ported Rust code into safe, idiomatic Rust was encumbered by several challenges, some related to mismatches between C and Rust, such as with lifetime management (borrowing), memory ownership, buffer pointers, and unions; others arising from dav1d design, strongly relying on shared mutable access across threads.
Thread safety issues related to shared state were addressed through locks using Mutex and RwLock and validating at runtime that a thread could access data without introducing delays with Mutex::try_lock() and RwLock::try_read() / RwLock::try_write().
This approach was just fine to handle the cases where only a single thread needed to mutate a value shared across thread. However, dav1d also relies on concurrent access to a single buffer from multiple threads where each thread accesses a specific subrange of the buffer. Instead of using the more idiomatic Rust approach based on using disjoint slices exclusively assigned to different threads, Immunant engineers resorted to creating a buffer wrapper type, DisjointMut, responsible for handling mutable borrows and ensuring each of them has exclusive access.
Two other challenging areas were self-referential structures, mostly used for cursors tracking buffer positions and links between context structures, and untagged unions. Since Rust does not allow to have self-referential structures, cursor pointers were replaced by integer indices while context structures were unlinked and referenced through function parameters. Untagged unions were converted into tagged Rust unions were convenient, while in other cases the zerocopy crate helped reinterpret the same bytes as two different types at runtime to avoid changing the union representation and size.
A major goal of the porting was to preserve performance, so Immunant engineers took care to monitor performance regression throughout the refactoring stage for each commit. As they progressed in the transition to safe code, they realized performance was mostly impacted by rather subtle factors such as the cost of dynamic dispatch to assembly code, bounds checking, and structure initialization. Finally, they dealt with finer optimizations related to branching, inlining, and stack usage.
The work on performance optimization brought a significant reduction in the overhead introduced by the porting, which went down to 6% from 11%. Overall, the process of porting dav1d to rav1d took over 20 person-months with a team of 3 developers and required more manual effort than it was initially foreseen, says Crane, but it showed it is possible to rewrite existing C code into safe, performant Rust and solve all threading and borrowing challenges.
For applications where safety is paramount, rav1d offers a memory safe implementation without additional overhead from mitigations such as sandboxing. We believe that with continued optimization and improvements, the Rust implementation can compete favorably with a C implementation in all situations, while also providing memory safety.
There is much more to learn from the process that led to the creation of rav1d than can be covered here, so do not miss the original write-up for the full detail.