Domain Specific Languages: A summary of recent ideas & debates

A number of parallel discussions in blogspace have introduced some new distinctions useful for understanding the proper use cases for domain specific languages. Joel Spolsky explained how the use of a DSL avoided large porting costs and simplified deployment/maintenance of their infrastructure into client data centers.  Mark Dominus made the case that design patterns are a sign of language deficiency and that we should seek to improve the language, not build pattern frameworks.  Buko Obele introduces the variable of "change" into the context of discussion, and explains how DSLs are a bad idea because they do not do a good job controlling change over time.  The rest of this entry summarizes some of the key points from these sources.

Idea: Use a DSL to preserve existing code investments and simplify deployment and tech support. Joel Spolsky's company sells FogBugz, a bug tracking and feature scheduling software system built in VBScript over a 6 year period. The software is now written in Wasabi, an internally built language that is backwards compatible with VBScript but supports many higher level features such as "closures, active records, lambdas, embedded SQL a la LINQ." Joel's firm implemented a custom compiler to support their customers target platforms. Joel's customers want to deploy locally instead of trust a hosted model, leading Joels firm to build their own language and compiler instead of program to existing platforms such as PHP, Java, or .NET. This set off  a number of criticisms in blogspace, with the most active discussion over at codinghorror.com where Joel himself defended his decision by presenting the technical requirements that lead to the decision:

1. Six years of code already written in VBScript
2. Needs to run on customers' Windows and Unix servers
3. Minimize tech support costs
4. Many customers refuse to install new runtimes on their servers, either because of IT policies or out of stubborness

Spolsky concludes:  

We could port the VBScript to PHP/.NET/Java by hand, or we could spend two months writing a compiler. The latter decision took less work (a couple of months) and produced better working code that runs on all our customers' machines without modification.

On the DSL con-side, .NET Guru Dare Obesanjo also responded to Joel's blog with a criticism of DSLs, stating that they they present larger problems when you consider maintaining the language in the face of constant evolution of other major languages, as well as recruitment of developers and attrition of your main compiler gurus. 

Idea: Design Patterns are a sign of language deficiency.   Mark Dominus examined the evolution of software languages since the 70's, and taking a number of common definitions of what a design pattern is, and used them to show that subroutines, Structs, Objects, and Iterators were at one point patterns themselves which later became implemented directly into modern programming languages. Mark argues that:

Instead of seeing the use of design patterns as valuable in itself, it should be widely recognized that each design pattern is an expression of the failure of the source language.

Continuing to program with patterns and re-implementing patterns is inefficient and time wasting. Once a pattern is identified, "the correct place to implement a common solution to a recurring design problem is in the programming language" [not in multiple implementations of pattern frameworks]. Mark was also careful to suggest in his conclusion that improving the language should be a "long term goal."   Update (Sept 19): Ralph Johnson, original member of the GoF (Gang of Four) who wrote the most popular patterns book, responded that "this is wrong":

Patterns might be a sign of weakness, but they might be a sign of simplicity. There is a trade-off between putting something in your programming language and making it be a convention, or perhaps putting it in the library...Many of the patterns will get subsumed by future languages, but probably not all of them. Patterns should be subsumed into languages when doing so makes programs simpler and more reliable.

Idea: DSLs are a bad idea because they do not do a good job controlling change over time. Buko Obele was an early DSL convert who says he has implemented dozens of languages and is now against the idea of writing DSLs. Buko interpretted Mark Dominus' blog as suggesting that DSLs should be built instead of using patterns, but says it can be very time consuming; "everytime I found myself writing some "boilerplate" code...I went back to redesigning the whole language."  Buko introduces the concept of change and explains how DSLs don't respond well to it:

What I discovered during my time of inventing and re-inventing DSLs over and over is that I couldn't handle change well. Often the smallest change would lead to me changing the very language of my solution; essentially a total rewrite. This was the reality of software maitenance...And so I began to question the wisdom of creating DSLs.

Buko goes on to suggest that if you have a "mechanical view" of programming languages (just syntax and semantics; a simplistic tool that was good only for solving a set of problems"), then DSLs make sense, they suggest using the right tool for the right job.  However, the missing variable is change:

Because of change nobody ever really understands the [business] problem in the first place... Programs always seem to require ongoing human guidance or maintenance...because of the nature of change and it's counterpart in the human psyche -- choice, the idea that programming languages merely solve problems is flawed....

Looking further at the impact of change, Buko suggests that programming languages are really a framework for dealing with change, and change is "a structural problem of software development." Making his main point:

a programming language is not simply a means of solving some particular problem. A programming language doesn't define the solution to the problem, it defines how the solution will change over time. This is the categorical mistake that so many make when criticizing languages: the expressive power of a language is not the measure of a language's ability to model a problem domain, it's rather the ability of the language to control changes in the problem domain. This is ultimately why DSLs and their ilk may be more expressive than a general solution, and still possess less expressive power than a general solution.

Buko concludes, based on his own experiences, that DSLs are "pretty much never a good idea except for trivial, static problem domains."

As modern dynamic languages (which make writing DSLs easier) become more popular in the Java and .NET communities (and of course Ruby is it's own), the idea of DSLs will certainly become a topic of greater discussion. Where do you stand?