Rust Analyzer is an experimental IDE/latency-oriented Rust compiler. This is an emerging endeavour within the Rust ecosystem, which is aimed at improving the IDE experience with Rust.
Compiler performance has always been a major focus of Rust tooling development and compile times have been steadily improving across releases. However, as Igor Matuszewski explained in his Rust Belt Rust Conference talk, Rust IDE support is an area of active work:
Despite the landscape shifting a lot during these last three years, including a proliferation of new tools and improved integration between tools, it feels like the Rust IDE story is not yet complete.
This work is being carried through under the guidance of the RLS 2.0 working group, which includes among its main components Rust Analyzer. InfoQ has taken the chance to speak with Aleksey Kladov, main contributor to Rust Analyzer, and Rust Core Team member Steve Klabnik to learn more about it.
InfoQ: Rust has been gathering lots of interest recently, and the language has been evolving and growing its ecosystem/tooling at a great pace. What is Rust's current maturity status and what can we expect in the next few years?
Steve Klabnik: "Maturity" can mean many things. For me, the metric is companies using Rust in a real product. Today, that looks like:
- Amazon
- Microsoft
and more. Three out of five FAANGs ain't bad.
In general, Rust is slowing down, and getting less new features, and more refinement to existing stuff. Now that async/await has shipped, for example, more work is being put into things like diagnostics. Rust will get a few more big features in the next few years, but similarly to how async/await is mostly good for networked applications, they're features that are extremely valuable in a particular niche, but may not be huge to all Rust programmers. For example, "const generics" let you write code that's generic over integers, rather than just types, and this is going to be very good for numerics libraries. But in general, these features are being added more slowly than previous major ones.
InfoQ: Can you briefly explain what limitations the current Rust compiler has when it comes to IDE integration? What are the goals of the Rust Analyzer project?
Aleksey Kladov: The limitations here are not specific to the Rust language, but a general for command line vs IDE compiler.
The main thing is that the command line (or batch) compiler is primarily optimized for throughput (compiling N thousands lines of code per second), while an IDE compiler is optimized for latency (showing correct completion variants in M milliseconds after user typed new fragment of code). As usual with throughput vs latency, these two goals require pretty different optimizations (and even high-level architectures). In general, it's hard to retrofit low latency requirement on a compiler which was developed solely with big throughput in mind.
Another thing is difference in handling invalid code. A traditional compiler front-end is usually organized as a progression of phases, where each phase takes an unstructured input, checks the input for validity, and, if it is indeed valid, adds more structure on top. Specifically, an error in an early phase (like parsing) usually means that the latter phase (like type checking) is not run for this bit of code at all. In other words, "correct code" is a happy case, and everything else can be treated as an error condition. In contrast, in IDE code is always broken, because the user constantly modifies it. As soon as the code is valid, the job of IDE ends and the job of the batch compiler begins. So, an IDE-oriented compiler should accommodate an incomplete and broken code, and provide IDE features, like completion, for such code.
The overarching goal of rust-analyzer project is to provide a single Rust compiler with excellent scores along both the latency and the throughput dimensions. The road towards this goal is long though, so currently we are in the phase where there are effectively two front-ends:
- rustc, which is a very mature batch compiler.
- rust-analyzer, which is a very-much experimental IDE/latency-oriented compiler.
This front ends share a tiny bit of code at the moment, and the immediate tactical goal is to share more of the easily shareable code between them.
InfoQ: Does this project supersedes Rust LSP implementation?
Kladov: Not at the moment; rust-analyzer is an experiment, and we are not quite ready to recommend it as the official LSP implementation. However, the current tentative plan is indeed that rust-analyzer supersedes RLS in the near-ish future.
InfoQ: Can you share some details about which direction the compiler refactor will take?
Kladov: The main idea is to make the compiler more lazy. The single most important trick an IDE can use to achieve low latency is avoiding as much work as possible. For example, to provide code completion, you generally need to analyze code on the screen and its immediate dependencies; you don't care what's written in the other five millions lines of code in your project. The idea is simple, but to get a compiler to not look at the extra code is actually pretty tricky, and that's where the bulk of work will go. Some more specific things we plan to do are:
- Transitioning to a full-fidelity syntax tree representation, which includes white space and comments.
- Adding "multi-crate" mode, where a single compiler instance can process several compilation units at once.
- Making the compiler process persistent and adding ability to send diffs of the input files to the compiler.
All those things are already implemented in rust-analyzer, but in a rather proof-of-concept manner. The tricky bit is moving them all into the production compiler without breaking user's code in the process.
Rust Analyzer is still alpha quality and requires building from source:
$ git clone https://github.com/rust-analyzer/rust-analyzer
$ cd rust-analyzer
$ cargo xtask install
Interested readers can check out the Rust Analyzer manual if they wish to start experimenting with this new tool.