JSIL is a .NET to JavaScrip compiler converting Common Intermediate Language (CIL) into cross-browser JavaScript that runs on all major engines - V8, SpiderMonkey, Chakra and JavaScriptCore-. JSIL compiles C#, VB.NET and F# into corresponding JavaScript, generating readable code, as the one shown in the next example:
| using System; using System.Collections.Generic; public static class Program { public static void Main (string[] args) { var array = new[] { 1, 2, 4, 8, 16 }; foreach (var i in array) Console.WriteLine(i); var list = new List<int>(array); foreach (var j in list) Console.WriteLine(j); } } | JSIL.MakeStaticClass("Program", true, [], function ($) { $.Method({Static:true , Public:true }, "Main", $sig.make(1747428, null, [$jsilcore.TypeRef("System.Array", [$.String])], []), function Program_Main (args) { var array = JSIL.Array.New($asm01.System.Int32, [1, 2, 4, 8, 16]); var array2 = array; for (var k = 0; k < array2.length; ++k) { $asm01.System.Console.WriteLine(array2[k]); } var list = (new JSIL.MethodSignature(null, [$asm01.TypeRef("System.Collections.Generic.IEnumerable`1", [$.Int32])], ])).Construct($asm01.System.Collections.Generic.List$b1.Of($asm01.System.Int32), array); for (var a$0 = list._items, i$0 = 0, l$0 = list._size; i$0 < l$0; i$0++) { var j = a$0[i$0]; $asm01.System.Console.WriteLine(j); } } ); }); |
A number of .NET applications have already been compiled into JavaScript including XNA and MonoGame games, WebGL and others. We have talked to K. Gadd, the woman behind this project, to find out more about the difficulties porting CIL code to JavaScript.
InfoQ: How hard it was to compile CIL into JavaScript? What was the hardest part?
K. Gadd: It's difficult to describe the difficulty in a general sense. The early stages of the project were essentially investigation; I built a simple prototype based on an open-source C# decompiler, and my JavaScript code generator was actually a heavily modified version of their C# code generator. This was an easy way to sort of 'prove out' the basic principles of doing something like this: "What parts of the language and runtime map cleanly to the browser, what is the performance like, what is the quality of current decompiler technology, and how complex is the decompiler in order to achieve these goals?" The early results were promising enough - and with relatively minimal effort - that I decided to treat it as a serious project.
At that point, things got more complex. The vast majority of IL features - arithmetic operations, field and property access, etc - are trivial to implement at a basic level so getting a big set of test cases working and producing decent code is not a big undertaking. The vast majority of the work lies in the corner cases - non-obvious consequences of ECMA CLI spec details, obscure features relied on by the C# compiler, and features that are simply very difficult to implement. If I were to write a compiler aimed at 'from-scratch' development, like Script#, the right approach would have been to ignore those features unless they proved their worth. Given my goal to port existing software, however, I had to buckle down and do the hard work to figure out how to support those features.There are particular cases where things were pretty difficult. In general, my lack of a background in compiler engineering (or computer science in general) meant that I had to do a lot of learning and experimenting to solve problems. For a simpler example, handling 'ref' and 'out' parameters adequately has been an ongoing effort due to the fact that JavaScript has no equivalent feature, and emulating it requires somewhat sophisticated automated transformations of code. The transformations necessary to handle those parameters had many serious bugs initially and after 2 years the implementation is quite complex. This is at least a 'solved' problem in the sense that I do not regularly encounter problems with it, but it is a part of the codebase I expect will need work in the future.
One of the major challenges involved is actually somewhat counter-intuitive - generating good JavaScript from IL not only requires decompiling the IL, but reversing some optimizations performed by the compiler and then applying new optimizations of my own. Doing this correctly without manual guidance from a developer requires a very, very robust knowledge of static analysis and other related topics, as without that you cannot implement optimizations without introducing significant bugs into user code. The static analysis and optimization portions of JSIL have probably been the source of most of the bugs I've fixed over the last year, and my approach to these challenges is still ultimately ad-hoc and based on partial knowledge. Anyone with an interest in tackling this sort of problems should go into things prepared to spend a ton of time reading up the relevant literature :) On the bright side, these optimizations do at least produce very worthwhile results, and in many cases are the difference between running 50x slower than the MS CLR and running 5x slower.
InfoQ: I have seen several examples done, a XNA game, a Mono one, a WebGL sample. Could you enlist all the various types of .NET applications that you have succeeded to compile so far?
K. Gadd: There is not really a single list. At this point dozens of XNA games have been ported, most of them without my intervention (other than to fix the occasional bug report). One or two developers experimented with doing automated ports of Silverlight applications and had some measure of success. There are a couple projects using JSIL as the foundation for their own game development or application development framework; the most notable one I'm aware of is Fusee by a team from the Furtwangen University of Applied Sciences. Some work is ongoing to enable porting a larger set of applications based on Mono and MonoGame - this requires making it possible to translate the entire Mono standard library along with more complex user-authored libraries. Smaller test cases like the WebGL example demonstrate that it is possible to construct a web application 'from scratch' using C# and JSIL, similarly to how you might use Script# or Saltarelle.
In practice this is a matter of completeness; even the successful ports I've seen tend to touch missing features in various libraries and frameworks - it's just the case that most of those features aren't necessary for a working port. For example, despite the fact that 2D XNA games 'work', there is absolutely no support for networking or threads, two incredibly essential features for many large-scale games. The same ends up being the case for any kind of application you might want to port: You have to evaluate how much you depend on the .NET standard library, and how much energy you can dedicate to filling in the gaps.
InfoQ: Is there anything missing, some CIL code you can't compile yet to JS?
K. Gadd: At this point the failing test cases that show up on the JSIL issue tracker are not usually due to unsupported parts of IL. There are certainly instructions and particular features of the .NET runtime that I don't implement. In some cases, like threading and locks, this is because they are impossible in JavaScript, but in other cases I simply don't implement them because I've yet to see a compiler produce them. All JSIL development has been driven by test cases, so without a test case for a given feature or instruction, the odds are low that I will implement it. In many cases it is possible to work around a missing or impossible feature by paying careful attention to the higher level goals a developer has - many uses of locks can safely be ignored, for example, because the browser guarantees that user code executes synchronously on a single thread.
One big missing check mark in the past was support for unsafe code - pointers, pinning, etc. This is used fairly often in game development, so lacking it was a big obstacle for more complex games, and in particular for the use of libraries like MonoGame. It's a very complex feature that relies on many aspects of .NET, so adding it wasn't trivial, but once JSIL got more sophisticated and the test suite became more mature, it was possible to begin work on it. At this point JSIL has substantial support for pointers and unsafe code, and while it does not perform as well as it would in a native environment, this lets you bring a lot of code like that over and run it in the browser. With the exception of things that are truly impossible, I think the same can be true for any other missing feature - it's just a matter of having the right test cases and the opportunity to take time to add the feature the right way.
InfoQ: Are there any problems running the compiled JS code on any of the browsers?
K. Gadd: Oh, certainly. Each of the four major JS runtimes out there (IE's Chakra, Safari's JSC, Firefox's SpiderMonkey, and Chrome's V8) is a unique and troublesome beast. I've had to spend time debugging many engine-specific issues and even more time identifying engine-specific performance issues and workarounds. I had to spend a lot of time reading source code to some of those engines and working with browser developers to try and understand those issues and document them. This sort of work can produce incredible results - modern JSIL code runs tens to hundreds of times faster than code the compiler produced a year ago - but it's very expensive, and you can't ever finish it, because the browsers move faster than you do. Sometimes a new browser release will halve your framerates and you won't know why, and you just have to deal with that - it's a cost that comes with doing development for the web.
In terms of browser limitations and bugs, however, the picture is pretty good. Google and Mozilla do a great job of delivering browsers that *work*, and work consistently, even if performance is sketchy sometimes - Over the past two years I think I've only encountered around 5-6 bugs in JS engines or browsers, in total, despite often testing using bleeding-edge releases of their software. Browser vendors do a great job of addressing these issues if you can deliver a test case.
Feature support can be a big issue, certainly. Internet Explorer has always been lagging behind in this area, with feature support that trails years behind Chrome or Firefox. As a result some of the simple JSIL examples are able to function in relatively 'modern' releases like IE9 or IE10, but a truly demanding game or application is not likely to work until IE has an opportunity to catch up further with the state of the art. IE11, at least, demonstrates Microsoft's commitment to closing the gap - it sounds like they're shipping a partial implementation of WebGL.
Supporting a browser you can't test is unpleasant. Safari's been a pain in the neck since day 1, because I don't own a Mac and very few of the visitors to JSIL and its demos use them. It's unfortunate, because that combination of factors means it's hard to justify giving it any more attention. On the other hand, their JS engine is high-quality and developed by some smart people, so the issues there are mostly with HTML5 APIs like Canvas or bugs in the browser. I don't want a browser monoculture, but I have to admit I wouldn't mind if we had one less platform-specific web browser out there. ;)
InfoQ: What is your roadmap for JSIL? What do you still plan to do with it?
K. Gadd: JSIL does not operate on a rigid roadmap; my general approach has remained the same over the past 2 years. I have a collection of test cases that I maintain support for, along with new test cases I'd like to implement. I have a small set of applications I'd like to port that either can't be ported yet, or are not completely functional given the current state of the libraries and compiler. Finally, I actively monitor the issue tracker for the JSIL GitHub project, and users do a great job of requesting missing features, identifying bugs, or simply complaining about how awful the compiler is. :) I combine all that with a healthy dose of my own whims and interests to try and keep the project focused on solving real problems for real people, without getting too narrowly focused on one area.
As a result of that approach, despite the fact that I've been almost laser-focused on my direct goal - porting XNA games - JSIL is still quite usable for a variety of purposes; F# and VB.net code compiles just fine, you can consume APIs like System.Drawing (given an adequate implementation), and there's actually no game-specific logic built into the compiler itself. This makes me confident that in the future, as long as people want to write C# and want to run code in a browser, JSIL will be a tool that can help them get things done.
In the past year some great sponsors have appeared to help spur progress on JSIL, and their needs helped set a roadmap - Mozilla, for example, sponsored the recent work on unsafe code and was instrumental in the MonoGame port getting off the ground. In the future I hope partnerships like this will continue so that the project can thrive.
The future does hold some interesting challenges for the project; one of my original goals was to try and produce 'high-quality' JavaScript, something a human could theoretically have written and that a human would be able to read easily. Sadly, this did not survive contact with reality - it's impossible to get good performance without generating the kind of JS that gets spit out of a minifier like Google's Closure. At some point it may become necessary to entirely rethink how JSIL produces JavaScript if it's really going to compete with alternatives like GWT or Emscripten.
It will continue to be challenging to maintain another one of my original goals - from the beginning, I wanted to enable people to *port* their existing work, without having to rewrite it or build a new application from scratch. This means minimal changes to existing code (if possible, ZERO changes), and only adding minimal amounts of new platform-specific code. This is a really tough balance to strike. Features like structs and pointers are simply at odds with the fundamental model behind JavaScript, so making existing code work unmodified can't be done without compromises - especially if it's already relying on platform-specific details, like how the Windows file system works or a particular feature of a library like Direct3D.
About the Interviewee
K. Gadd is a generalist programmer and jack-of-all-trades from Northern California, United States. She's been working in software and related industries for about 9 years now, being involved in a variety of roles on a mix of games and classic software products like websites. The common thread beneath everything she does is that she really loves working with creative people, so she's passionate about doing stuff that makes them happy - building better tools, working with them to design cool stuff, or simply putting their work in as many hands as possible.