mobl: a DSL for Mobile Web Development

Introduction

Application development for mobile devices such as smart phones and tablets is popular. Apple's AppStore (for iPhone, iPod and iPad) has over 350,000 applications and Android's marketplace is quickly catching up counting over 200,000 applications. Android and iOS are not the only two mobile platforms, however. BlackBerry is a big player in business, as is Nokia. Microsoft recently launched Windows Phone 7, and HP announced new WebOS devices. While choice is great for users, it is a nightmare for us, developers. What platform should we target with our applications?

Sharing code between mobile platforms is extremely difficult. Each platform has chosen its own development stack with its own languages and own set of APIs. For iOS development you have to use Objective-C and CocoaTouch APIs; for Android you use Java and Android's APIs; for Windows Phone 7 you use .NET and the Silverlight APIs.

However, there is a solution: the web, specifically: Webkit. As it turns out, all major mobile platform providers (except Microsoft) based their mobile browsers on Webkit -- the modern, fast open source browser engine. Webkit supports many HTML5 features that mobile applications need, including detection of touch gestures (taps, swipes and pinches), geolocation APIs (to determine the user's location) and local database support (a SQLite database in the browser, to cache data locally). These features are currently natively supported by Android, iOS, WebOS and BlackBerry OS 6 browsers. For devices that do not include a Webkit-based browser, there is PhoneGap. PhoneGap enables development of native applications using web technologies (including HTML5), and wraps the application as a native application -- ready to be distributed to users (e.g. through the platform's application marketplace). If a platform does not already have a WebKit browser built-in, PhoneGap provides it. PhoneGap applications can run on 6 different mobile platforms.

JavaScript framework vendors noticed the opportunity and are building variants of their frameworks targeted at the mobile web. Examples include jQuery Mobile and Sencha Touch. These frameworks are very impressive and are a great way for current web developers to start developing for the mobile web. However, they are still based on JavaScript, HTML and CSS, which were never intended to develop applications, but instead to develop webs of hyperlinked documents. While frameworks attempt to shape these languages to fit their new role, it makes you wonder... what would a language look like that was designed specifically for developing mobile applications?

If we were to design such a language, what are the problems we would like it to address?

• The first is tool support. Developers coming from an enterprise environment (e.g. Java and .NET developers) are used to Eclipse-like IDE features, such as as-you-type error-highlighting, code completion, reference resolving, code outlines and refactoring. JavaScript and HTML are very dynamic in nature, which is powerful, but also makes it difficult for tool vendors to build IDEs at Eclipse and IntelliJ's Java level. Good IDE support is a precondition for any language these days.
• A second problem to address is conciseness. For instance, user interface frameworks often require a lot of code involving copying data back and forth from the database to the user interface. Our new language should reduce the amount of boiler plate code a developer needs to write.
• A third problem is JavaScript's asynchronous programming model. JavaScript in the browser is single-threaded, developers have to use callback mechanisms to perform expensive operations, such as database queries, e.g. instead of writing synchronous code like:

  var results = tx.executeQuery("SELECT * FROM User");
  for(var i = 0; i < results.length; i++) {
    ...
  }

  code has to has to be written using callback functions:

  tx.executeQuery("SELECT * FROM User", function(results) {
    for(var i = 0; i < results.length; i++) {
      ...
    });

  which, when used on a larger scale, can lead to hard to read spaghetti code. Our new language should support a more developer-friendly synchronous programming model.

Our research group (the Software Engineering Research Group, of Delt University of Technology, the Netherlands), focuses on programming transformation and implementation. More recently we have focused on domain-specific languages. The current developments in mobile seemed like a good opportunity to develop a domain-specific language for the mobile web. We asked ourselves: starting from a clean slate, what would a language look like that is dedicated to developing mobile web applications? The result is mobl.

mobl: a 10,000 Feet View

mobl is a textual, statically typed, compiled language that is primarily used through its Eclipse plug-in. The plug-in offers syntax highlighting, inline error-highlighting, reference resolving and code completion. The mobl compiler (integrated into the IDE) compiles mobl modules to a combination of HTML, JavaScript and CSS on every save. mobl applications do not rely on any particular server-side technology, only handling the client-side aspect of your application. Calls can be made to (JSON) web services using AJAX (Asynchronous JavaScript and XML) calls.

The mobl language has a number of features that aim to increase the productivity of mobile developers:

1. Declarative user interfaces: user interfaces defined using mobl are specified in a declarative manner and react to changes in application state, unlike other approaches where state and view are strictly separated.
2. Transparent data persistence: mobl has language constructs to define data models using entity declarations. Changes to entity objects are automatically persisted to the database. Querying of data is also done at the level of entities and does not required string-embedded SQL queries, like many other frameworks.
3. Static in principle, dynamic when required: the mobl language is statically typed, enabling IDE features such as error highlighting, reference resolving and code completion. Nevertheless, because of type inference, in many cases no types have to be explicitly specified. In cases where dynamic typing is more convenient, the Dynamic type can be used, enabling arbitrary access to properties and methods on Dynamic variables.
4. Synchronous scripting: application logic can be written in a scripting language that looks a lot like a typed version of JavaScript. Code is written in a synchronous style, and automatically translated to asynchronous JavaScript code by the compiler using the continuation-passing style transform.

A Simple To-do List

To get a flavor of what mobl looks like, this section will demonstrate the implementation of a simple to-do list manager.

After creating a new mobl project in Eclipse, we end up with a bare bones mobl project with a single application file, named todo.mobl:

application todo

import mobl::ui::generic

screen root() {
  header("todo")
}

The first line of this mobl module defines that this is an application module. mobl has three types of modules:

1. application modules: typically one per project, this is the main entry point to the application.
2. regular modules: a library of definitions typically imported by one or more other (application) modules.
3. configuration module (config.mobl): defines application configuration options.

Application and regular modules contain any number of definitions: user interfaces, data models, styles, web service interfaces and functions.

First, we define a data model using entity definitions. Instances of entities will be persisted to a local database on the mobile device itself. Our to-do application only requires a single entity named Task. For every task object we want to keep track of a name and whether the task has been completed or not:

entity Task {
  name : String (searchable)
  done : Bool
}

For every property we specify a name, type and optionally one or more annotations. mobl supports two types of annotations: inverse annotations (defining inverse relationships) and searchable (to include the field in full-text searches). While this application does not require it, it is possible to define multiple entities and relationships between them, including one-to-one, one-to-many and many-to-many relationships.

Next, we adapt the root screen to show a list of tasks that we can tick and untick. In mobl, user interfaces are defined using screens and controls. A screen typically defines an entire screen layout by composing a number of controls. Thus, a control defines a smaller user interface element, such as a button, label or tab set. In addition, a screen or control can define local state (using variables) and use control structures to iterate over collections and conditionally show parts of the user interface.

Here is an initial definition of our root screen (the first screen to be shown when the application launches). It uses a number of controls from the mobl::ui::generic library, including header (which renders a screen header), group (which groups one or more items), and checkBox.

screen root() {
  header("Tasks")
  group {
    list(t in Task.all()) {
      item { checkBox(t.done, label=t.name) }
    }
  }
}

The list control construct is similar to a for-each loop: it iterates over a collection and for each item in the collection it renders its body. The checkBox control binds to two Task object properties: done and name. Data binding creates a synchronization relationship between application state (e.g. local variables or entity properties) and the user interface. For instance, when the user checks or unchecks the checkbox, the value of t.done is updated accordingly. Similarly, when some other part of the application updates the name property of task t, the checkbox automatically updates its label. This is called reactive programming, and is one of the fundamental mobl features -- the user interface reacts to changes in application state.

Initially, the database will be empty and thus no tasks will appear on our list. How do we add tasks? For this purpose we define the addTask screen:

screen addTask() {
  var newTask = Task()
  header("Add") {
    button("Done", onclick={
      add(newTask);
      screen return;
    })
   }
   group {
     item { textField(newTask.name, placeholder="Task name") }
   }
}

Using the var construct it is possible to create state local to a particular screen. In this case we define a variable newTask that we initialize with a new Task entity instance. We bind the textField control to newTasks's name property. When the user is done entering the task's name, he or she will push the "Done" button that appears in the header of the application. A button has an argument named onclick whose value is a Callback, a snippet of imperative application logic to be executed when the event occurs. In this particular case two things happen:

1. The newTask object is added to the database. Implicitly, this results in a change to the Task.all() collection, resulting in the new task to be added to the list in the root screen automatically.
2. The user is sent back to the screen where it came from (screen return is analogous to return in functions).

However, while we defined a root and addTask screen, there is no way to get from one to the other. What we still have to do is add an "Add" button to the root screen, which navigates to the addTask screen. Therefore, we adapt the call to the header control in root to the following:

header("Tasks") {
  button("Add", onclick={ addTask(); })
}

As you can see, a screen is called similarly to a regular function, in fact, screens, like functions, can return values.

Our application is now fully functional. We add one last feature: search. In our data model we used the (searchable) annotation for the name property of Task. Let us take advantage of that by using search to filter our task list (to more quickly find the task we are looking for).

We adapt the root screen as follows:

screen root(){
  var phrase = ""
  header("Tasks") {
    button("Add", onclick={ addTask(); })
  }
  searchBox(phrase)
  group {
    list(t in Task.searchPrefix(phrase)) {
      item { checkBox(t.done, label=t.name) }
    }
  }
}

We added a new local variable phrase to our screen that holds the search phrase. We use the searchBox and bind it to phrase. Then, instead of iterating overTask.all() in our list, we iterate over the search collection Task.searchPrefix(phrase). When run, the list of search results updates while we type in our search query. Again, the user interface adapts to application state (in this case the phrase variable) automatically.

We have now finished building a basic to-do list application including search functionality, ready to be deployed. As mobl modules are saved, they are simultaneously compiled to JavaScript, HTML and CSS in the www/ directory of the Eclipse project. These generated files can deployed to a any webserver able to serve static files (e.g. Apache, IIS or Tomcat), mobl is completely back-end agnostic.

Beyond To-do Lists

Of course, a to-do list is a toy example, using only simple controls such as group, item, button and textField. The mobl standard library also provides higher-level controls such as tab sets, master-detail views, context menus and expandable lists.

In addition to language constructs for defining user interfaces, data models, and application logic, mobl also has constructs for:

• Defining styling, using a language very similar to CSS.
• Web service access. To pull in data from the server and cache it locally. In the future mobl will also support transparent data synchronization.

To learn more about these features, look through the tutorials on the mobl website.

To DSL or not to DSL

mobl can be described as a domain-specific language (DSL), a language aimed at a particular application domain. Traditionally, DSLs have had fairly limited domains. For instance, HTML is a DSL for defining structured web pages. SQL is a DSL for expressing database queries. The domain of mobile applications is much larger. It is large enough, in fact, to require a lot of features you would typically only find in general-purpose languages, or GPLs, such as Java, Python and Ruby. Such typical GPL features include object-oriented programming, if-statements and for-loops. Is mobl still a DSL, considering the fact that it has GPL features?

We feel it is, because it has language constructs that are geared specifically at the domain of data-driven mobile web applications. For instance, a general-purpose language with styling support would not make much sense if its programs deal with scientific calculations. Nor would screen constructs make much sense in server-side computing. Language features like entities, screens, controls, styles and web services are not general purpose -- they are domain specific.

mobl is not the only DSL for mobile development. Others include Applause and the derived Applitude. However, these DSLs have limited flexibility. They have a set of built-in controls, built in functions and once those no longer suffice, you have to fall back to Objective-C of Java coding. A goal of mobl is to be flexible as well as expressive.

For this purpose, we kept the language relatively small, and added functionality through libraries written in mobl itself. For instance, the controls we used to build the to-do list application were not built into the language. Instead, they were imported from a mobl library, itself defined in mobl. At the lowest level, implementations of controls use low-level HTML tags and CSS styles. While regular users will only deal with high-level conceptual controls such as headers and buttons, expert developers can tweak exactly what a button looks like by adapting the underlying HTML code in a library.

Beside defining controls in libraries, mobl also comes with libraries exposing many HTML5 APIs, including geolocation, 2D drawing using Canvas and WebSockets. These libraries simply wrap the existing JavaScript APIs as mobl APIs using mobl's native interface, which enables calling of "native" JavaScript code. Thus, mobl is extensible through its library mechanism that allows users to extend the platform without having to extend the language or compiler itself.

In certain cases, mobl adds syntactic sugar for particular libraries. For instance, for querying, mobl exposes the Collection<T> type, which has methods to filter, sort and page the contents of a collection of entity objects. These methods would can be called as follows:

var doneTasks = Task.all().filter("done", "=", true)
                .order("date", false).limit(10);

Clearly, this is a somewhat cumbersome syntax. Therefore, mobl adds syntactic sugar for queries, allowing the above query to be written as:

var doneTasks = Task.all() where done == true 
                order by date desc limit 10;

Not only is this more concise, the IDE now checks whether Task in fact has the properties done and date, and offers appropriate code completion.

Conclusion

Rather than building a framework on top of an existing language, mobl started with a clean slate as an external DSL. This approach has advantages and disadvantages. A disadvantage is that its users have to learn a new language, new libraries and new tools. An advantage is that language design decisions can be made that significantly decrease the amount of code a developer has to write. In mobl we tried to make the language feel familiar by keeping its syntax and semantics close to JavaScript's. In addition, mobl integrates into the existing Eclipse IDE and the tooling around it, providing error checking as-you-type, reference resolving, code completion and compile-on-save.

mobl is still a young language. The first public release was in January of 2011. Its compiler, tools and documentation are still maturing. Nevertheless, we feel it does already show the potential of using DSLs in the mobile space.

About the Author

Zef Hemel is currently a Ph.D. student at Delft University of Technology in The Netherlands where he works on the design and implementation of domain-specific languages. He has a particular interest in web technology. Previously he has worked on WebDSL and PIL. He blogs regularly on his blog, and tweets on his twitter account.