As programming languages become more powerful, they also become more complicated. To an experienced developer, C# holds few surprises.
But to a novice who needs to write code for his non-IT job, it can be a bit daunting. This scenario led to the creation of Essence#.
Alan Lovejoy created Essence# as a superset of syntax found in ANSI-Standard Smalltalk.
As the # suffix implies, this is a .NET compatible language. More specifically, it runs on top of the Dynamic Language Runtime (DLR).
Recently we spoke with Alan Lovejoy, creator of Essence# about his creation.
InfoQ: Before we get into the details of the language, can we talk a little about the history of Essence#? Was this originally a research project or was it built for a specific use case?
Alan Lovejoy: Essence# was built for a specific use case: Trading financial markets. I and my business partners use it to write “trading plan scripts” that are executed whenever a trading signal (to buy or to sell) is generated by our market analyst software. A particular trading plan script can generate a trading plan that is appropriate to the situation at the moment the trading signal is issued. For example, it may vary the size of the position based on the current balance in your trading account, or use 3 profit targets instead of just one because all trends are in the same direction (all trends up or all trends down,) or it may optionally use timed exits because the trend on the daily chart, the trend on the hourly chart and the trend on the 15-minute chart conflict with each other and it’s a narrow range day.
We would also like to make it possible to write our market analyst software in Essence#, but that’s for the future.
My trading partners are not programmers, and so they needed a programming language that they could use that didn’t require them to master computer science. So they left the computer science up to me :-). Essence# meets their needs. We feel it’s easier than TradeStation’s EasyLanguage, which is used for similar purposes, and it’s far easier than C#, which is what everyone else must use in order to write trading strategies and trading plan generation logic on NinjaTrader, which is what we use. The fact that NinjaTrader is implemented in C# is the reason I implemented Essence# as a .Net language, using the Microsoft Dynamic Language Runtime.
InfoQ: At InfoQ we have a wide mix of readers. The C# and Java developers tend to think in terms of classes with a fixed set of features while others of objects that can and should be extended at runtime. And then we have some in the middle, like Python where you can modify functionality at runtime but you’ll be accused of “monkey-patching” if you do.
What is the correct mindset for working with Essence#?
Alan: That completely depends on the use case. Essence# can be used as though it were a language like Java or C#, or as though it were a language such as Python, or as though it were a language such as Smalltalk. The best use case for being fully and shamelessly dynamic is hosting software development tools in the language, so that a programmer can use the development tools to operate on and change a program while the program is running. I haven’t tried that yet, but I designed the language to support it.
My language design philosophy is that a programming language should make it possible to do whatever you want, but should do so using as little syntax as possible. I view the programmer as a professional and artist, not as a cog in a machine who shouldn’t be trusted. That’s why I designed Essence# to use an optimistic typing paradigm which assumes that the programmer is innocent until proven guilty. In other words, Essence# is dynamically typed. There is no static type checking that can be specified in the syntax of the language, and the Essence# compiler does not attempt to enforce type safety at compile time. But Essence# is nevertheless strongly typed, although type safety is enforced solely at run time.
InfoQ: Sitting between Essence# and the actual runtime is the DLR or Dynamic Language Runtime. Some of our readers are familiar with the DLR from its use in IronPython and IronRuby.
How heavily do you use the DLR? Do you see it as a collection of helpful functions or is more core to how Essence# works?
Alan: I use the DLR rather heavily, but I have also gone my own way in some respects.
I use LINQ expressions to generate all code. I use the DLR’s ParameterExpressions for all method/function parameters, and for all stack-resident variables. I rely on the DLR closure implementation for the Essence# lambda functions. And I use the DynamicMetaObject protocol:
Essence# objects implement the IDynamicMetaObjectProvider interface, and so can be sent messages by other languages that also use the DLR’s DynamicMetaObject protocol. And Essence# uses the DynamicMetaObject protocol in order to operate on any “foreign” objects it may encounter.
The Essence# compiler generates DLR dynamic call site for each and every message send, regardless of the receiver. The compiler does not and cannot know the type of the object that is receiving a message, so it just emits a DLR CallSite for all message sends, and the CallSite for a message send is always an instance of the ESMessageSendBinder class. An ESMessageSendBinder figures out, at run time, how to implement the message send. That’s done one way in the case of native Essence# objects, done another way in the case of the CLR primitive types, done yet another way for any non-essence# objects that implement the IDynamicMetaObjectProvider interface (using the protocol of DynamicMetaObjects and the DLR’s standard set of DynamicMetaObjectBinders as fallback binders,) and done yet another way for all other objects (using LINQ.Expression trees.)
The bottom line is that Essence# objects are fully interoperable with code written in other .Net programming languages. The methods of Essence# objects can be called by foreign code (however, the fields of Essence# objects are never accessible externally, not even in the case of other Essence# code.) And Essence# code can instantiate instances of foreign types, can access the fields and/or properties of those types, and can invoke any methods defined by those types.
In fact, that’s how we are able to use it with NinjaTrader.
Language Highlights
To fully review Essence# we would have to also review Smalltalk, so instead we are just going to look at some of the features and design concepts that may be novel to C# developers.
Namespaces in Essence#
Namespaces in Essence# are unlike anything you see in C# or VB programming. Rather than just being part of a name, they are actual objects that one can interact with. In addition to classes, programmers can store loose variables, constants, and functions in a namespace. You may find it beneficial to think of this as a formalization of the pseudo-namespace conventions used by JavaScript libraries such as jQuery.
There are four primary namespaces in any Essence# project. The first is the Root namespace, which is the parent of all other namespaces. Under this is the default namespace, which is where “all of the core system classes reside”. Unless you specify a different namespace, all code is executed in this context.
For working with external libraries, there is a CLR namespace and its children. From the documentation:
The "CLR" namespace is the default parent namespace for all root namespaces known to .Net and the CLR. Unless overridden by the program, all namespaces that are direct or indirect children of the CLR namespace are automatically bound to a .Net/CLR namespace with the corresponding qualified path name, where "corresponding" means "without the prefix Root.CLR." Consequently, there is a default, standard qualified name for the Essence# class that corresponds to any type (struct or class) known to the CLR. (The runtime system makes it appear as though there is always an Essence# class that shadows/represents/corresponds to every CLR type, although what it actually does is to create any such classes dynamically, when and as needed--but only once, of course.)
The last of the primary namespaces is called “Undeclared”. If you try to reference an undeclared variable, and don’t have the compiler set to abort in that scenario (Option Explicit Off in VB parlance), then the undeclared variables are stored in this namespace.
Items stored in a namespace must have one of three visible levels. “Public” is the same as public in any other language. “Local” is similar Java’s “package private” in that anything in the same namespace can access to the namespace entry. The final option is “InHierarchy”, which is like Local expect child namespaces can also access the entry.
Object State Architecture
This is a hard concept to introduce because we don’t have a commonly used term to refer to a “class of classes”. Haskell has the term “Type Classes”, but that implies more than we want to say. If we were to use the biological classification system, the next level above “class” would be “phylum”. In Essence# it is called “Object State Architecture”.
If we were to apply the concept of object state architectures to C# we would find six basic architectures: primitives, value types, reference types, finalizable reference types, arrays, and static classes. Each of these architectures has specific implications when it comes to runtime issues such as memory layout and garbage collection.
In Essence#, most of the classes will derive from one of the following object state architectures.
- Abstract: A class whose instance architecture is #Abstract cannot have any instances.
- Stateless: The instances of a class whose instance architecture is #Stateless cannot have any state at all. For example, the class Object.
- NamedSlots: The instances of a class whose instance architecture is #NamedSlots can have named instance variables ("fields" in CLR-speak.) The instance variables ("fields") are dynamically typed; they work as though they had the C# type "Dynamic." (Note that there are more specific object state architectures that can also have named instance variables. So #NamedSlots is just the most abstract or general case.)
- IndexedObjectSlots: The instances of a class whose instance architecture is #IndexedObjectSlots can have any number of indexable slots--including none at all. They can also optionally have named instance variables. In both cases, the slots work as though they had the C# type "Dynamic." Such objects are the Essence# equivalent of C# object arrays.
- Indexed[Type]Slots: These represent arrays of various types. The current list is Byte, Char, HalfWord, Word, LongWord, SinglePrecision, DoublePrecision, and QuadPrecision (128-bit floating point).
There is a variety of system architectures as well. These architectures are used for the language’s infrastructure rather than for custom types. Some of the more interesting architectures include:
- Message: A Message instance specifies a message that was or could be sent, along with the message arguments, if any. Instances are created by the run time system when and as needed, although application code may also create and use instances. Message instances cannot have programmer-accessible named instance variables.
- Namespace: Namespace instances serve as dynamic namespaces at runtime (See the documentation on namespaces for a far more detailed description]). Instances may optionally have programmer-accessible named instance variables.
- Pathname: A Pathname instance serves as a hierarchical key whose elements are Strings. It's used for identifying namespaces, file pathnames, URLs, etc. Instances may optionally have programmer-accessible named instance variables.
- Block: A block is an anonymous function with full closure semantics. The implementation uses CLR delegates. Blocks cannot have programmer-accessible named instance variables.
- Method: A method is executable code that runs in the context of a specific class, with full access to the internal state of the distinguished object that receives the message that invokes the method. Methods cannot have programmer-accessible named instance variables.
- Behavior: A Behavior is a proto-class. There can actually be instances--it's not abstract. Instances may optionally have programmer-accessible named instance variables. (See the documentation on classes for a more detailed description]).
- Class: A Class is a full Essence# class which is a subclass of Behavior, is an instance of a Metaclass, and whose instances (if it's allowed to have any) can be an object (value) of any type. The term 'class' is usually intended to refer to an (indirect) instance of the class Class, but technically can refer to any Object that can create instances of itself, such as a Behavior or a Metclass (i.e., any instance of Behavior or anything that inherits from Behavior.) Instances may optionally have programmer-accessible named instance variables. (See the documentation on classes for a more detailed description]).
- Metaclass: A Metaclass is an Essence# class which is a direct subclass of the (Essence#) class Behavior. A Metaclass is an instance of the class Metaclass, and its instances must be Classes. A Metaclass can have only one instance which is called either the canonical instance or the sole instance. Note that the superclass of the Metaclass of any root Behavior (e.g., the metaclass of class Object) is (and must be) the class Class. Instances may optionally have programmer-accessible named instance variables. (See the documentation on classes for a more detailed description]).
- BehavioralTrait: A Trait is a composable unit of behavior. Traits can be "used" by a class or by another Trait with the effect of adding the methods defined (or used) by the Trait to the method dictionary of the using class or of the using Trait. A BehavioralTrait is a Trait usable by any BehavioralTrait or by any Behavior (i.e., by any instance of the class BehavioralTrait, or by any instance of the class Behavior, or by any instance of any subclass of either the class BehavioralTrait or of the class Behavior.)
- HostSystemObject: A "host system object" is simply an instance of any CLR type which is not a formal part of the Essence# runtime system. One of the requirements for an Essence# class to represent a CLR type (which may or may not be a "class" as the CLR defines that term) is that its instance type must be #HostSystemObject.
Traits
Traits are Essence#’s answer to multiple inheritance. Traits are sets of functionality that can be imported by a class, but they are more than just cut-and-paste methods handled by the compiler. So first, let us talk about how they differ from normal methods.
With one exception, methods imported from a trait should be indistinguishable from those implemented locally by the importing class or trait. The exception is that methods defined in a trait bind to non-local variables (e.g, class variables) based on the namespace environment of the trait that defines the methods, and not based on the namespace environment of the class or trait that uses (“imports”) those methods.
Currently Essence# does not allow instance variables to be defined or referenced by traits. That may change in the future.
Traits don’t have to be used entirely as-is; they can be combined to create new traits. Combining traits isn’t like combining interfaces where you have to include every method from each source. Instead, you have to work with three operators.
The '+' operator allows you to combine the unique methods of two traits. As a way of dealing with name collisions, if a method is defined in both traits being combined with the '+' operator, the method is excluded.
Since that is probably not what you want to happen, you can use the '-' operator to remove the unwanted method from one trait so that the method will be expressed by the other trait. Or you can use the '@' operator to rename the method on one of the base traits.
Essence# is available on CodePlex under the Simplified BSD License.
About the Interviewee
Alan Lovejoy is currently a discretionary trader specializing in automated trading systems. Before becoming a trader, he worked for many years in the software industry, initially as a programmer, and then as a software engineer, software team lead, software/solution architect, enterprise architect and finally as Director Of Engineering And Architecture for a small startup company. He worked on trading/financial software for BankAmerica and JP Morgan. His first exposure to Object-Oriented programming was using Smalltalk for BankAmerica on a prototype of a foreign-exchange trading platform, staring in November 1985. In addition to Essence#, he is the author the Chronos Date/Time Library (Smalltalk,) and of “Smalltalk: Getting The Message,” a primer/tutorial for Smalltalk.