Concrete: Rich, Customizable DSL Editors for the Browser

Domain Specific Languages (DSLs) allow users to write down or at least read and modify business logic in a concise way. DSLs lack the clutter of general purpose languages and their focus on a domain can make them smaller and easier to learn. 
Great. Except now users have to learn the syntax and vocabulary of one or more textual DSLs in order to get work done. Worse yet, they have to do so with plain text editors that don't even give them basic syntax coloring or any other help.

Language Workbenches and other solutions aim to help with that problem, allowing DSL creators to build powerful editors with syntax coloring, auto completion of relevant symbols, validation and more. XText  for Eclipse and Jetbrains MPS are just two examples.

Concrete is a new tool for building powerful DSL editors that run in browsers, using only HTML, Javascript and CSS. 

The Concrete editor can be customized for a particular DSL with a JSON-based grammar, the look can be customized with HTML, Javascript and CSS.
Martin Thiede, the creator of Concrete as well as the modelling framework RGen, showed Concrete in a talk at RubyConf 2010. The Concrete website also has some demos that show how customizable the editors can be. One demo is the formula editor sample. Instead of making the user write formulas in a linear way, it can do some basic mathematical layout. 

InfoQ talked to Martin Thiede about Concrete, how it works and how to use it, with a few bonus questions thrown in for readers who might be unclear about Modelling.

InfoQ: What is Concrete?

Concrete is an editor for Domain Specific Languages (DSLs) which runs in the web browser. It needs to be set up for a specific DSL by telling it the structure of the language (abstract syntax) as well as the look of the language elements (concrete syntax). Once set up, users can write sentences in that DSL and will be supported by syntax highlighting, auto completion and instant constraint checks.

InfoQ: How does Concrete work?

The core of the Concrete editor is a Javascript widget which needs to be embedded into a web application. It loads and stores data (the DSL sentences) in JSON format, most probably via AJAX. The data consists of objects with properties holding either primitive values, other objects or references to other objects.

The setup of the language structure (abstract syntax) is done in JSON as well. Concrete needs to know which language elements exist, what properties they have, how they can be nested and how they can reference each other.

As a default, the language elements have a textual look, much like what we are used to from current text editors with syntax highlighting. However, this look can be customized using almost any kind of web design technique based on HTML and CSS. This allows for a great amount of visual variety including icons, boxes, different fonts, etc.

At present, Concrete works with Firefox and Chrome.

InfoQ: What can Concrete be used for?

Concrete is useful whenever a DSL is to be exposed on the web. In contrast to a DSL made available in a text area, Concrete DSLs provide much more user support. As Concrete DSLs are external DSLs, they also don't suffer from security problems and don't confuse users unfamiliar with an internal DSL's host language.

One of the most interesting applications are DSLs for administration interfaces. An example is an online registration platform for sport events where customers are able to edit the registration forms with input fields, constraints, etc by themselves. They actually make use of a DSL for the domain of online registrations and Concrete can greatly improve the user experience with that DSL. Once a description has been created, it can be stored in the database and the web application will interpret it whenever participants access the registration platform in order to sign up. This example can be tested live here. 

In contrast to web applications which directly make use of the edited data by interpreting it, other Concrete based applications may just act as editors and the input data may be used elsewhere in the system later on. Storing the data on a server or in the cloud would allow for distributed and collaborative editing environments without the need for any special client software apart from a browser.

Concrete based editors can also work on the local filesystem, though. An example of this kind is the EMF/ECore compatible "mmedit" metamodel editor which can be used to define the abstract syntax of a DSL. This tool makes use of a simple Ruby backend included with Concrete which handles access to the filesystem. "mmedit" is available as a Ruby gem and here.

InfoQ: What's the advantage of Concrete over existing editors?

Compared to general purpose text editors like Emacs, Vim or Textmate, Concrete is always tailored to a specific DSL and can thus offer more support. It provides context sensitive auto completion, both for DSL commands and for references and if offers DSL specific constraint checks.

Compared to DSL aware editors like EMF/xText, Concrete allows for much richer graphical representation of the DSL. Another advantage is that Concrete is very lightweight and simple. In cases where you can't or don't want to use Eclipse or MPS, you might still be able to integrate Concrete because it's just an extension not a whole environment.

Yet another aspect of Concrete is that DSL setup is very dynamic. Concrete can easily switch between different concrete syntaxes at runtime just by exchanging HTML templates. This is demonstrated by the "mmedit" example (after starting "mmedit", open the properties dialog, pick a different layout, click "ok" and reload the page). It can also switch between different versions of a DSL by loading the JSON language descriptions from a database. There is no need for a compilation step or prebuilt code.

InfoQ: What exactly does the JSON format look like?

Concrete uses plain JSON with two extensions. First, each object has a special property called "_class" which indicates the kind of language element the object is an instance of. Second, references to non-child objects are expressed using identifier strings. In many cases, the qualified name of the target object is used as identifier. Here is an example:

[{"_class": "Statemachine", "name": "AC", 
  "states": [
   {"_class": "State", "name": "Off", "transitions": 
    {"_class": "Transition", "targetState": "/AC/On"}}, 
   {"_class": "State", "name": "On", "transitions": 
    {"_class": "Transition", "targetState": "/AC/Off"}}
  ]
 }
]

 The objects are of classes "Statemachine", "State" and "Transition". The transition within State "Off" references State "On" by means of the qualified name "/AC/On". The qualified name consists of the name of the target object, prefixed with the names of its parent objects. Optionally, Concrete can use different strategies for building target identifiers.

JSON is also used to describe the structure of the language that Concrete understands (abstract syntax). The format is the same as the one for user data described above. However, the language it refers to by means of the "_class" properties is a special built-in language for the purpose of describing language structures. Here is an example:

[{"_class": "Class", "name": "Statemachine", "features": [
  {"_class": "Feature", "name": "name", "kind": "attribute", "lowerLimit": 1, "upperLimit": 1, "type": "String"}, 
  {"_class": "Feature", "name": "states", "kind": "containment", "type": "State"}]}, 
{"_class": "Class", "name": "State", "features": [
  {"_class": "Feature", "name": "name", "kind": "attribute", "lowerLimit": 1, "upperLimit": 1, "type": "String"}, 
  {"_class": "Feature", "name": "transitions", "kind": "containment", "upperLimit": -1, "type": "Transition"}]},
{"_class": "Class", "name": "Transition", "features": 
  {"_class": "Feature", "name": "targetState", "kind": "reference", "lowerLimit": 1, "upperLimit": 1, "type": "State"}}
]

The main elements of the built-in language for describing language structure are "Class" and "Feature". This example describes a simple lanuguage consisting of the language elements "Statemachine", "State" and "Transition" (each an instance of "Class"). Statemachine and State have a name, Statemachine contains States, State contains Transitions and Transition references a State (each property is expressed as an instance of "Feature").

As the language for describing language structure is just another language for Concrete, Concrete can be set up to edit these language structures. In fact, one of the examples which come with Concrete is such an editor.

People familar with modelling terms call sentences of a DSL a "model". The model for describing the language structure is called the "metamodel". This is why the example editor for editing Concrete language structures is called the "metamodel editor". "mmedit" is a special variant of a metamodel editor which is EMF/ECore compatible. Concrete comes with a converter for turning ECore metamodels into Concrete metamodels.

InfoQ: How can users customize the look of the editor or the language?

Concrete uses a default look for all language elements. If you want a type of model element to look differently, just tell Concrete the piece of HTML and CSS which defines the look. Here is an example:

 <div class="ct_element ctc_State">
    ...
    <span class="ct_attribute ctn_name">
      ...
      <span class="ct_slot"></span>
      ...
    </span>
    ...
    <div class="ct_containment ctn_transitions">
      ...
      <div class="ct_slot"></div>
      ...
    </div>
 </div>

By means of the CSS classes "ct_element" and "ctc_State" Concrete knows that this snippet of HTML defines the look of all elements that are a "State". Any HTML can go inside the outer "div" tag. The classes "ct_attribute" and "ctn_name" designate the property "name". Again, any HTML can go inside this "span" tag. Finally the class "ct_slot" marks the place where the actual primitive value will be displayed and edited. The same applies to contained elements ("ct_containment") and refereces to other elements ("ct_reference").

In most cases, extra CSS will be used to define the actual look. Since there is no limitation regarding the use of CSS with Concrete, the editor can be integrated seamlessly into existing web designs.

Also note that HTML and CSS snippets can be created dynamically by a web application using JSP, PHP, RHTML, HAML, etc.

InfoQ: How does the formula editor example work? Do you parse the input?

As Concrete is a projectional editor, there is no parsing involved. The formula editor DSL uses the same browser based rendering mechanism as any other DSL in Concrete. Here is the concrete syntax definition for the square root element as an example:

<div class="ct_element ctc_Sqrt">
    <div>
      <div class="ct_containment ctn_expr">
        <span style="display: inline-block; padding: 3px"><span class="ct_slot"></span></span>
      </div>
    </div>
  </div>

  
  div.ctc_Sqrt {
    display: inline-block;
    padding-left: 19px; 
    background: url(sqrt_vert.png) no-repeat left bottom;
  }

  div.ctc_Sqrt > div {
    background: url(sqrt_horz.png) no-repeat right top;
  }

The square root sign consists of two images, one for the horizontal line on the top and one for the vertical line on the left. These images are assigned to two "div" elements which surround the actual content "div". When the content expands, the lines automatically get longer. In general, any layout technique used in web design can also be used with Concrete.

Modelling

InfoQ: With a Concrete frontend, is there anything special required on the backend? Do I need to use a modelling framework like RGen?

There are no special requirements on the backend software, it can be implemented using any language or framework available. An application embedding Concrete just has to provide the language structure in JSON format, the optional look as HTML and CSS snippets and it has to exchange the user data as JSON.

RGen is a lightweight Ruby framework which supports working with models and metamodels, model transformation and code generation. RGen is not required on the backend, but it is very handy. With RGen, the JSON data can be turned into Ruby objects with an interface similar to that of Rail's ActiveRecord models. There is also support for turning EMF/ECore metamodels into Concrete abstract syntax definitions. RGen's model transformation feature can be used to derive the abstract syntax definition from sources like UML class diagrams. Even if you don't use Ruby for the backend, RGen might be useful for deriving the abstract syntax definition with a separate Ruby script.

If Concrete is used with Rails, it is probably a good idea to represent the DSL data by means of RGen instead of ActiveRecord. In many cases, descriptions written in a DSL are self contained and can be stored in a single database field as JSON text. When the web application needs to access the description, RGen can turn the JSON into Ruby objects for easy navigation. This way, the database schema doesn't have to change when the DSL changes and even different versions of a DSL could be supported at the same time. The online registration example shows how RGen models and ActiveRecord models can be used together in one Rails application.

A special use of Concrete is to provide input data for code generation. In this case however, the generation step has to be done by an external tool like RGen as Concrete is not a code generator or transformation tool.

InfoQ: What approach to modelling does Concrete use?

Both Concrete and RGen are meant to be used with custom, domain specific metamodels (in contrast to using UML for example). RGen metamodels are compatible with the Eclipse EMF metamodels as both use ECore (or EMOF) as the meta-metamodel. Concrete metamodels are similar but a bit simpler as they just express what's relevant for the editor.

InfoQ: Can Concrete interpret metamodels to allow for it's editing? Are there limitations?

The whole purpose of metamodels (or abstract syntax definition) in Concrete is to configure the editor: the metamodel describes the auto completion options for creating new elements, it describes the possible target elements of references, the possible values of attribues, upper and lower limits of property value multiplicity and so on. Constraint violations are indicated at model elements and the error description appears in a hover.

Constraints which can not be expressed by means of the metamodel can be added using a Javascript API. The structure of metamodels in Concrete is very similar to that of EMF/ECore metamodels. Thus the abstract syntax of a DSL in Concrete must be expressible with ECore concepts.

InfoQ: Where does Concrete fit in the modelling world? Ie. if I already use a modelling solution/stack, can Concrete help?

If you already use a modeling solution you could use Concrete to add a web interface to it or to improve the user support and graphical representation of an existing editor. You would need to convert your existing metamodel into a JSON based metamodel and you would have to write a converter which transforms your models to JSON an back. Concrete comes with a RGen based ECore-to-JSON transformer and RGen is able to read and write JSON models.

InfoQ: What's the target audience for Concrete?

Web developers get a tool to make their internal or external DSLs more user friendly and web-safe and give them a nice look. Modellers get a lightweight, web-based, projectional model editor with the metamodels being EMF/ECore compatible. Any developer supplying a DSL to users gets a DSL editor which is simple and can easily be added to most existing applications.

References:

• The Concrete website
• Mmedit
• Online registration demo
• Screencast with an overview of Concrete and some basic examples
• Rubyconf X in New Orleans