Compiled, Typed, Ruby-Inspired Crystal Language is Ready for Production - Q&A with Beta Ziliani

InfoQ recently covered the recent general availability of the Crystal language and provided an overview of the key features of the new 1.0 release. Crystal, which features a syntax close to Ruby’s while striving to achieve C-like performance, has caught the attention of Ruby developers and the programming community at large. Crystal 1.0 comes 12 years after the language was ideated by its creator and already has a relatively vibrant ecosystem.

Following the community excitement about the new release, InfoQ interviewed Beta Ziliani, who heads the Crystal team.

InfoQ: Tell us about you and your involvement with the Crystal development team. What motivated you to be a part of the team?

Beta Ziliani: I saw the birth of the Crystal language when I was working with Ruby at Manas, somewhere around 2009. I remember thinking at the moment, "Hey, this is crazy! I wonder if it will work for large projects?" I probably voiced my concern to Ary Borenszweig (its creator). The idea of a compiler that needed the entire program to compile was ... unusual.

By 2010, I moved to Germany to do my PhD in the topic of programming languages. Specifically, my research was in improving the programming experience of writing proofs with the computer, by creating a new typed meta-language. In 2015, I finished my degree and moved back to Argentina to become a researcher and a professor at the National University of Córdoba. Here, besides continuing to develop my project with a PhD student, we formally described the semantics of the Lua programming language.

During this time, I got in contact with Brian Cardiff, former Team Leader of Crystal and member of the core team, who wanted to understand the core ideas of the language from a more abstract point of view than just the code of the compiler. During my stay in Germany, the language grew significantly, and its design space was so unique that it was worth looking at from an academic perspective. Therefore, I started a project based on our experience with Lua.

Unfortunately, this project saw little advance since the students I assigned to work on it finally decided to do their PhDs abroad. With my background as a programming languages researcher and a strong appreciation for Crystal, the position as head of the Crystal team at Manas was a good fit. Managing a language is a new challenge for me that I’m eager to tackle.

InfoQ: Can you give us a quick primer on programming languages taxonomy and where Crystal fits in that? What are Crystal’s overarching goals?

Ziliani: In short, Crystal feels like a dynamic language, but it’s fast and secure like a very performant static language. In Crystal’s words, it’s a language for humans and computers. This concept is best described with an example: say we have classes Dog and Cat, both having a method talk. Then, we create a function make_talk that receives an object and it calls the talk method in it. In a static language like Java or Rust, we need to specify that Dog and Cat have a common interface, say, Talkable, and then in the function make_talk we need to specify that the object has the Talkable interface. From a human perspective, it means we need to write a significant amount of code that has the only purpose of making the compiler understand that your object indeed talks. Instead, in a dynamic language like Ruby or Python, as long as the object we’re passing to the function has the method talk, we’re good to go, no need for boilerplate code. But we pay a high cost for this convenience: if there is a bug in the code and we end up providing an object that doesn’t have that method, we’ll only find it when running the program. If the program wasn’t tested thoroughly before production, that means it will likely crash to the user! Here is where Crystal comes into play: its typechecker will statically check if the object has the expected method -- like a statically checked language -- but without requiring the boilerplate code that static languages require. For this particular example of cats and dogs, you can write the exact same code in Ruby and compile it with Crystal (although we like to stress, Crystal isn’t Ruby with types). In essence, humans can write programs easily, without the noise that typecheckers typically add.

When we say it is also a language for computers, we mean two things: first, our programs won’t execute a lot of trivial checks just to ensure that the program doesn’t break -- the typechecker provides the required guarantees. And second, that our programs have a small memory usage. For instance, Crystal’s objects (like in Rust) don’t have a virtual table, unlike in Java, Ruby, and most OO programming languages.

To conclude, let me add an important point: some people migrate to Go when seeking performance in concurrent settings. Crystal shines here too, having a concurrent model inspired in Go, yet staying close to the Ruby we love.

InfoQ: Crystal 1.0 generated a significant amount of interest in the software community, especially among Ruby enthusiasts. Creating a programming language that would be fun, productive, easy to read, catch errors before they happen, and all of this with first-class performance may well be like squaring a circle. What are the tradeoffs that you have to manage?

Ziliani: I can only think of three. The first one is that many Ruby developers are used to modifying their code and seeing the changes immediately in their running program. With Crystal this isn’t an option: you have to recompile your program and reload it. Some complain that compilation is somewhat slow (see the next question), but to me, the real problem is that it isn’t incremental. The second tradeoff when moving to Crystal is that it is a young language, with thousands of shards (its libraries) publicly available, yet not as many or as mature as in other widely used languages. Finally, the third one is that it requires knowing what typing is -- a concept that isn’t in every programmer’s mind (particularly newcomers). And the Crystal typechecker is rather unique, as described before, so even if you are familiar with typecheckers, it takes some time to adjust to Crystal’s.

That said, in my experience, most developers that tried Crystal and decided not to work with it mostly did so because of its former experimental status or the lack of some particular library (second point above), and not because of the other two reasons I just mentioned. I assume this will tend to change over time, in particular now with the 1.0 release. If there are other reasons, I’d love to hear them!

InfoQ: Some developers reported slow compilation times that may penalize an iterative development process. What drives up compilation time? What are current workarounds? What do you have in mind to address this in future releases?

Ziliani: I think the problem isn’t that compilation is slow, but rather, that it isn’t modular. With modular compilation, you can just compile the affected code and dynamically re-plug it into the main program at each iteration. However, the only way we know to do proper modularization of code is to have a stricter typechecker that enforces type annotations and heavily restricts dependencies in the code. And this goes strongly against the language’s design (and its beauty). As Ary mentioned in the past, we really need to do research to find a way to improve the compiler’s efficiency. This is topping my list of priorities for 2.0 (see below).

But let me stress that the compiler isn’t slow. For instance, compiling Crystal’s compiler and stdlib, summarizing more than 120KLOCs of Crystal code, takes roughly a minute to compile in my i5-8265U CPU @ 1.60GHz. If you only need to do it once every now and then, it is pretty decent. And if you need fast iterations with TDD, it is interesting to note that the compiler slices the program to only consider the fraction affected by the test. Thus, you can easily just compile and run the affected tests and relevant code instead of the whole program.

And for web development, things requiring fast iterations like HTML generation can be better solved with a different approach than in a dynamic language, for example with a dynamic template language or an API-centric design (as in Jamstack).

InfoQ: A general availability release is admittedly a public badge of stability and an important moment. What key features made it in the 1.0 version? Which were postponed to future releases and why? What is planned for Crystal 2.0?

Ziliani: Compared to the previous release (0.36.1), the major change -- besides regular bug fixes and improvements -- was admitting that the language got stable enough to deserve the promotion to 1.0. There are pending discussions of possible improvements to the language, but we decided we needed to be practical and stop seeking perfection. As for which features were postponed to future releases, there are a few that aren’t breaking, so they might land in minor revisions before 2.0: Windows and multithreading support, and general improvements to the stdlib. Then, for 2.0 we consider those that require significant research to be done. For instance, I mentioned already that we don’t have yet a good alternative to keep the spirit of the language and allow modular compilation.

Another interesting point to consider is if we want to add variance annotations to our generic types and how to avoid having to do dynamic dispatch on certain cases. And the list goes on. I would like to work on simplifying the passage of blocks of code to functions a bit, something that is inherited from Ruby and that is related to anonymous (lambda) functions in other languages.

InfoQ: Evan Czaplicki, the creator of the Elm language, presented interesting ways that a programming language may succeed that only loosely relate to commonly used metrics (e.g., GitHub stars, forks, feature list). In your opinion, what makes a programming language not only interesting but successful? How would you define success for Crystal?

Ziliani: There are several factors. What GitHub stars show is that there is interest in the language, and that’s somewhat related to its features. But on its own, it isn’t sufficient. A language might gain a lot of initial momentum, and get lots of stars, only to start sunsetting a few years later. Rarely someone will un-star a project, so it is important to see that the number of stars isn’t stalled after the first peak.

For this to happen, you need your language to grow continuously and prove that it works in production, without failing to deliver on its promises. For instance, if our programs crash, or if we start requiring type annotations everywhere, the interest in Crystal will probably drop quickly. The language’s growth requires investment, and this we were able to do thanks to our generous sponsors. Unlike stars, the fact that these sponsors haven’t gone away and that others keep joining the list is a trustworthy metric of the language still having potential in people’s eyes and delivered on its promises enough to justify a sustained contribution. Another important aspect highlighted by Evan is the community: we need to actively sustain and expand a healthy environment in which everyone feels welcome. We are always working actively on this matter, it’s never-ending work.

InfoQ: There is considerable research published in academic journals and presented at specialized conferences. What do you think are the most interesting ideas currently investigated that may have a significant impact in the professional world?

Ziliani: As someone with a recent past in software verification, I find it hard not to be biased in that direction. I truly believe that we are going to see more of that shortly. In fact, it is already happening with several companies and start-ups verifying software for different domains from smart contracts in blockchain to IoT devices. From the perspective of a programming language, you don’t need full verification to build better software. For instance, Crystal already avoids the NullPointerException by distinguishing the type of an uninstantiated object, and it also recognizes aborted paths of execution. Pushing this a bit further, one can think of empowering the typechecker even more to be able to capture other interesting cases. As an example, it’d be great to be able to statically determine that the index of an array will always be within the bounds.

But this is a dangerous path! If you give it too much power, then it becomes really hard to use. The research community is still experimenting with general-purpose languages with strong guarantees that increase their safety without compromising much of their usability.

InfoQ: Crystal is an open-source project created at Manas, a company that specializes in technical and scientific software solutions and employs most of the core team. How is Crystal used at Manas? What other companies are using Crystal?

Ziliani: Currently, Manas employs only two core team members. Since the language was created at Manas, it was a natural occurrence that members of the core team were employed by them, not an intended thing. But addressing the important part of your question, there are a growing number of companies adopting Crystal; with two major ones being our generous donors, 84codes and Nikola Motor Company. Besides these two companies, there are dozens more we are aware of, and there are likely a myriad more that only use Crystal for some small project, (sort of as a test), and we don’t have them on our radar. I hope the 1.0 milestone will drive more people to use it for larger projects too. There is also an interesting number of OSS being built on top of Crystal. Off the top of my head I can list the Lucky and Kemal web frameworks, the Mint DSL for single-page applications, the Invidious alternative video provider, and the list continues.

InfoQ: What can the community contribute to Crystal’s development and how?

Ziliani: Let me start by saying that the community already responded amazingly: we have +400 contributors just in the compiler repo, shards in the thousands, the financial support of contributors from all over the world, and we keep incorporating active members as part of the core team. If we could ask even more from the community, I guess that putting Crystal in production is at the top of the list. After the years of support we’ve received to get Crystal to 1.0, a proper celebration would be to start using it. That’s what we’ve all been doing this for. It is also important to spread the word, have other people know the language, and that can come in many forms: adding a Crystal banner in the application; writing tutorials; participating in forums to help newcomers; etc. And I won’t lie, we’re lagging behind with the issues at our repository; we could use more people to help us solve them. One way is hiring more people full-time, but for that, we need to increase the incoming contributions. The other is by increasing the number of code contributions. Either way, we welcome all the help we can get.

About the Interviewee

Beta Ziliani is Team Leader at Manas.Tech, leading the development of the Crystal programming language. He is also a professor at the National University of Córdoba (UNC) in Argentina, where he teaches programming languages and databases. He has a PhD and did significant research in the area of Programming Languages. He’s the proud father of two wonderful kids. His pronouns can be he/him or they/them.