In July, the Digital Energy unit within GE Energy Management (created when GE Energy was split up in July) disclosed that they were in the process of porting their Smalltalk-inspired programming language, Magik, from its own proprietary Virtual Machine, MagikSF, to the JVM.
Magik is a dynamically typed, object-oriented programming language that supports multiple inheritance and polymorphism. It predates Java, having originally been introduced in 1990. It is used in GE Energy's Smallworld technology platform, and was designed to implement complex applications for enterprise utilities such as power distribution and telecommunications. In the telecommunications domain, for example, GE Energy Management have a product called Physical Network Inventory, which allows telecoms companies to plan and manage their copper, fibre and cable telecoms networks. On the utilities side, Electric Office provides similar functionality to power distribution companies.
The language has quite a similar look and feel to Ruby. Here's a simple "hello world" example:
Magik> _for i _over 1.upto(3)
_loop
write("Hello world ", i)
_endloop
In Ruby this would be:
1.upto(3) do |i|
puts("Hello world #{i}")
end
Output in both cases would be:
Hello world 1
Hello world 2
Hello world 3
Although the port was only publicly announced in July of this year, the GE Energy team started with a proof-of-concept a year earlier, in July 2011, with a small team working with early releases of Java 7. Team lead/architect George Marrows told InfoQ,
That was very successful and we have been working on a product release since then. We've written a parser, compiler and runtime for the Magik language, integrated the large majority of our C libraries and written a completely new graphics subsystem using Java2D, all to a very high level of compatibility with the existing system.
Whilst it is not yet complete, the ported version is able to run full applications. Critically, given the millions of lines of Magik code that GE Energy's partners and customers have written, the port is aiming to be entirely compatible with the MagikSF version, and developers can simply recompile the Magik source on the JVM port. "We're confident that our language implementation is very, very close to MagikSF's, and our library support is very good and improving all the time as we run more existing Magik code and fix any incompatibilities," Marrows told us.
As you might expect given Magik's dynamic type system, the port makes heavy use of Java 7's invokedynamic bytecode instruction. Marrows told us,
We use invokedynamic liberally, for method dispatch obviously, but also for binding complex literals into their place of use and resolving global variables. Duncan MacGregor has led the charge in the team on this – at one point it seemed he'd spotted a new performance improvement using invokedynamic pretty much every week.
The experience using invokeDynamic and java.lang.invoke has been very good. One way to think of it is as an API on the JVM JIT compiler's method inlining functionality, and as such it works very well. We've been pleasantly surprised to see the performance improvements that have come from pretty much every update to Java 7 since its first release last year, and it looks like we're about to get another improvement from the Java 8 "lambda form" work.
From the technical point of view, the JVM has a number of capabilities which are not supported on MagikSF. Amongst them:
- MagikSF is interpreted - the JVM offers JIT compilation
- MagikSF uses green threads - the JVM has native threads
- MagikSF is 32 bit only - the JVM also supports 64 bit
- MagikSF has its own garbage collector which is generational but not concurrent - Java offers multiple GC choices with a mostly concurrent collector supported in Hotspot.
Of course the JVM also gains from the considerable resources and expertise large software vendors like Oracle and IBM are able to devote to the platform.
The resulting performance improvements that Marrows' team are seeing are impressive - the Java port is already faster in the vast majority of the benchmarking tests the team have run, up to 30 times in some cases. As Marrows notes, they are also gaining from the improvements made to the performance of invokeDynamic and the JVM over time - Java 7 update 9 performs better than Java 7 update 2 and so on. More significantly, Oracle has re-worked invokeDynamic as part of the lambda project in Java 8, and this version is being back ported to Java 7 (it is planned for inclusion in the forthcoming update 12). The new version is again considerably faster in many cases, though Marrows told us they have some examples of code that show a performance deterioration when run with the current test builds of update 12 when compared to update 9; they will be discussing these with the Oracle team. The team have had great support from Oracle thus far, and in particular Rémi Forax of the JSR 292 expert group, who "helped us get started with JSR 292, and is a tireless contributor to the MLVM (multi-language virtual machine) mailing list".
Thought it wasn't part of the original plan for the project, the team are also now porting the graphics system using a mixture of JNI and Java 2D - "JNI to integrate with the C libraries, which provide access to our database and perform optimised geometry and other calculations. Java 2D to render maps and UI."
The team have a fairly well finalised, homegrown interop for going from Java to Magik which is used heavily in the new Java 2D-backed graphics system. For example
def_slotted_exemplar(
:line_style,
{ { :colour, colour } },
{ :simple_style_mixin},
{:|com.gesmallworld.magik.commons.drawingsurface.LineStyle|})
$
_private _method line_style.|getColour|()
>> .colour
_endmethod
The vertical bars here |getColour| are used to preserve the case for Java since Magik is not case sensitive.
They are less happy with the current interop from Magik to Java since it requires the developer to write wrapper code:
public class Char16Vector {
@MagikInitialiser
public static void exposeThisClass() {
//Expose an instance of this class to Magik
}
@MagikMethod(name="nth()")
public Char16Vector nth( Integer index ) {
return new Char16Vector(value.substring(index-1, index));
}
}
Oracle's new JavaScript implementation, Nashorn, is able to do this much more cleanly using the Dynalink library. Writing on his blog, Jim Laskey provides a good example of how well this can work, using Nashorn to talk to Twitter with the Twitter4J library. Marrows told us that Dynalink is something he and the team intend to look at.
As well as access to the Java Virtual machine, the team sees a couple of other plus points to being on the Java platform. Speaking at JavaOne Brazil, Marrows mentioned access to both Java developers and the Java libraries. In addition he told us,
...we do appreciate having the power and flexibility of the JVM to support any improvements we might make in future.
More generally, we have taken the opportunity to simplify and replace some parts of the MagikSF environment that we had developed ourselves to date, for example the graphics support, file I/O etc. We envisage that we will do more of this in the future.
Magik is one of a growing number of languages, including Groovy, JRuby and Nashorn, that are taking advantage of invokeDynamic on the JVM. JSR 292 really has helped make the JVM a universal environment for programming languages.