People are talking about Erlang. This talk is disproportionately about concurrency rather than any of the other powerful properties of the platform, such as reliability. This article is about a property of Erlang which does not get the credit it deserves - Erlang is a fantastic workbench for Domain Specific Languages. The domain I have chosen is investment finance. You will see how easy it is to translate plain English at runtime and execute it in the Erlang Runtime System. You may also learn a thing or two about functional programming. The Erlang reference manual is a good place to start if you get lost along the way.
Let's begin with a high level run through of how this DSL can be used and move into a step by step detailed discussion of the implementation.
$ # First, specify the business rules
$ echo "buy 9000 shares of GOOG when price is less than 500" > biz_rules.txt
$ echo "sell 400 shares of MSFT when price is greater than 30" >> biz_rules.txt
$ echo "buy 7000 shares of AAPL when price is less than 160" >> biz_rules.txt
$ erl # start the Erlang emulator (similar to irb or beanshell)
1> c(dsl). % compile and load, assumes dsl.erl is in the current directory
2> Pid = spawn(fun() -> dsl:broker() end). % start a broker in parallel
3> Functions = dsl:load_biz_rules(Pid, "biz_rules.txt").
4> MarketData = [{'GOOG', 498}, {'MSFT', 30}, {'AAPL', 158}].
5> dsl:apply_biz_rules(Functions, MarketData).
Order placed: buying 9000 shares of 'GOOG'
Order placed: buying 7000 shares of 'AAPL'
Implementation
The first three echo commands create the file biz_rules.txt, populating it with three rules. The logic of these rules is very straight forward; each is expressed no different than if it were coming from the mouth of a user who depends on it.
buy 9000 shares of GOOG when price is less than 500
sell 400 shares of MSFT when price is greater than 30
buy 7000 shares of AAPL when price is less than 160
Our DSL exists in an Erlang module called "dsl", consisting of a single file called dsl.erl. The first command in our erl session compiles and loads this module with the built in c function.
1>c(dsl). % compiles and loads, assumes dsl.erl is in the current directory
Spawning a Broker
The dsl module has a public function named broker.
broker() ->
receive
{buy, Quantity, Ticker} ->
% place order to an external system here
%
Msg = "Order placed: buying ~p shares of ~p",
io:format(Msg, [Quantity, Ticker]),
broker();
{sell, Quantity, Ticker} ->
% place order to an external system here
%
Msg = "Order placed: selling ~p shares of ~p",
io:format(Msg, [Quantity, Ticker]),
broker()
end.
The broker function simply waits for messages with iterative use of a receive block. It can only receive two kinds of messages: a message to sell a stock or a message to buy a stock. The actual implementation of placing an order has been ignored for demonstration purposes.
Notice that broker is tail recursive. In an imperative programming language one might accomplish this with a loop. In Erlang there is no need for loops because tail recursive functions are transparently optimized to run in constant space. An Erlang developer is not tasked with the responsibility of manual memory management with keywords like "for" or "while" or "do". These three keywords are to the stack segment just as "malloc" and "dealloc" are to the heap segment ... unnecessary.
The second command given to the emulator spawns an anonymous function as an Erlang process and returns the process id. The process id value is bound to the variable Pid.
2> Pid = spawn(fun() -> dsl:broker() end). % call broker in parallel
Anonymous functions begin and end with keywords 'fun' and 'end' in Erlang. Pay attention when you see this because there will be lots of anonymous functions in this article. This particular anonymous function is pretty simple: it wraps broker.
We aren't going to go into detail about the many interesting properties of an Erlang process. Think of this as a separate independent path of execution for the time being. This will be known as the broker process. We will be placing orders to it via the process id returned by the built in spawn function.
One Way to Load Business Rules
Next we call load_biz_rules, another public function of the dsl module.
3> Functions = dsl:load_biz_rules(Pid, "biz_rules.txt").
The arguments applied to the load_biz_rules are the broker process id and the file name containing business rules; the return value is a list of Erlang functions. You will see many examples of functions which return functions in this article. This can be a little bit of speed bump for anyone new to functional programming. If you can relate to this just remember that in an object oriented world it is normal for objects to create other objects and return them from methods - there are even design patterns for it, such as the Abstract Factory.
Each function element of the list returned by the load_biz_rules represents one of the loaded business rules echoed to biz_rules.txt. In an object oriented programming language these rules would most likely be modeled by a list of instances; in Erlang we use functions.
load_biz_rules(Pid, File) ->
{ok, Bin} = file:read_file(File),
Rules = string:tokens(erlang:binary_to_list(Bin), "\n"),
[rule_to_function(Pid, Rule) || Rule <- Rules].
The load_biz_rules function begins by reading the file into memory. The contents of this file are tokenized into a list of strings and bound to a list called Rules. In Erlang the last line of a function is the implied return value (just like Ruby) and it always ends in a period. The last line of load_biz_rules performs list comprehension to build and return a list of functions.
Readers familiar with list comprehension will recognize Rule >- Rules as the generator and rule_to_function(Pid, Rule) as the expression template. What does this mean? It means we are creating a new list and populating it with transformed elements of the Rules list. The rule_to_function function is the actual transformation. In other words, the last line says "pass the broker process id and each Rule in Rules to rule_to_function ... give me a list of functions returned by rule_to_function".
rule_to_function(Pid, Rule) ->
{ok, Scanned, _} = erl_scan:string(Rule),
[{_,_,Action},{_,_,Quantity},_,_|Tail] = Scanned,
[{_,_,Ticker},_,_,_,{_,_,Operator},_,{_,_,Limit}] = Tail,
to_function(Pid, Action, Quantity, Ticker, Operator, Limit).
The Rule string applied to rule_to_function is scanned into a tuple with the first line of code. The next two lines use pattern matching to pick off the values required to enforce this rule when the returned function is actually executed. Those values are bound to variables such Quantity or Ticker. The broker process id and these five values are then applied to the to_function function. A functional representation of the business rule is built and returned by to_function.
We'll be looking at two implementations of to_function, starting with the more practical version.
to_function(Pid, Action, Quantity, Ticker, Operator, Limit) ->
fun(Ticker_, Price) ->
if
Ticker =:= Ticker_ andalso
( ( Price < Limit andalso Operator =:= less ) orelse
( Price > Limit andalso Operator =:= greater ) ) ->
Pid ! {Action, Quantity, Ticker}; % place an order
true ->
erlang:display("no rule applied")
end
end.
This version of to_function does one thing - it returns an anonymous function. The anonymous function takes two market data as arguments: a ticker symbol and a price. The ticker symbol and price that are applied to it (when it is executed) will be compared to the ticker symbol and price limit specified by the business rules (when it is created). If it finds a match, it uses the send operator (the ! symbol) to send a message to the broker process telling it to place an order.
Two Ways to Load Business Rules
The second version of to_function is a little academic. It builds an Erlang expression in abstract form and returns an anonymous function which can dynamically evaluate it later.
to_function(Pid, Action, Quantity, Ticker, Operator, Limit) ->
Abstract = rule_to_abstract(Action, Quantity, Ticker, Operator, Limit),
fun(Ticker_, Price) ->
TickerBinding = erl_eval:add_binding('Ticker', Ticker_, erl_eval:new_bindings()),
PriceBindings = erl_eval:add_binding('Price', Price, TickerBinding),
Bindings = erl_eval:add_binding('Pid', Pid, PriceBindings),
erl_eval:exprs(Abstract, Bindings)
end.
The first line of this function delegates its dirty work to the rule_to_abstract function. You probably shouldn't invest much time reading rule_to_abstract unless you are also a person who enjoys reading Perl.
rule_to_abstract(Action, Quantity, Ticker, Operator, Limit) ->
Comparison = if Operator =:= greater -> '>'; true -> '<' end,
[{'if',1,
[{clause,1,[],
[[{op,1,
'andalso',
{op,1,'=:=',{atom,1,Ticker},{var,1,'Ticker'}},
{op,1,Comparison,{var,1,'Price'},{integer,1,Limit}}}]],
[{op,1,
'!',
{var,1,'Pid'},
{tuple,1,[{atom,1,Action},
{integer,1,Quantity},
{atom,1,Ticker}]}}]},
{clause,1,[],
[[{atom,1,true}]],
[{call,1,
{remote,1,{atom,1,erlang},{atom,1,display}},
[{string,1,"no rule applied"}]}]}]}].
The rule_to_abstract function builds and returns an Erlang control structure in abstract form. This control structure is an 'if' statement represented by a concrete list of plain Erlang terms. As a side note, notice the abstract form uses postfix operator notation as opposed to infix notation used by regular Erlang syntax. To conceptualize this for an individual rule, we are programmatically building the abstract form of the following:
if
Ticker =:= ‘GOOG’ andalso Price < 500 ->
Pid ! {sell, 9000, ‘GOOG’}; % place order with broker
true ->
erlang:display("no rule applied")
end
Once the business logic is obtained in abstract form within this version of to_function, it returns an anonymous function (just like the first version of to_function). Within this anonymous function three variables are pulled from execution scope and dynamically bound to the constructed expression via the built in function erl_eval:add_binding. These values are the ticker symbol, price limit and process id. The expression is finally executed with the built in erl_eval:exprs library function.
Applying Business Rules
Now that we have loaded our business rules and built Erlang functions for each of them, it is time to apply market data arguments to them. A list of tuples represents these data. Each tuple represents a pair of stock tickers and stock prices.
4> MarketData = [{'GOOG', 498}, {'MSFT', 30}, {'AAPL', 158}].
5> dsl:apply_biz_rules(Functions, MarketData).
We apply the business rules in function form and the market data to the apply_biz_rules function.
apply_biz_rules(Functions, MarketData) ->
lists:map(fun({Ticker,Price}) ->
lists:map(fun(Function) ->
Function(Ticker, Price)
end, Functions)
end, MarketData).
It's a very good thing we only have three business rules because the runtime for apply_biz_rules is exponential. This algorithm can be a little hard to read for anyone who is unfamiliar with Erlang syntax and/or not paying attention. The apply_biz_rules function maps an inner function to each ticker/price pair within the market data. The inner function maps a second inner function to each business rule function. The second inner function applies the ticker and price to the business rule function!
As apply_biz_rules evaluates the broker process indicates it has received orders to buy 9000 shares of Google and 7000 shares of Apple.
5> dsl:apply_biz_rules(Functions, MarketData).
Order placed: buying 9000 shares of 'GOOG'
Order placed: buying 7000 shares of 'AAPL'
It has not placed an order to buy or sell Microsoft. A quick look back at the business rules and the market data indicate the program is behaving as expected.
buy 9000 shares of GOOG when price is less than 500
sell 400 shares of MSFT when price is greater than 30
buy 7000 shares of AAPL when price is less than 160
4> MarketData = [{'GOOG', 498}, {'MSFT', 30}, {'AAPL', 158}].
If the price of Google goes up seven dollars and we change our sell criteria for Microsoft, we observe different behavior.
sell 400 shares of MSFT when price is greater than 27
6> UpdatedFunctions = dsl:load_biz_rules(Pid, "new_biz_rules.txt").
7> UpdatedMarketData = [{'GOOG', 505}, {'MSFT', 30}, {'AAPL', 158}].
8> dsl:apply_biz_rules(UpdatedFunctions, UpdatedMarketData).
Order placed: selling 400 shares of 'MSFT'
Order placed: buying 7000 shares of 'AAPL'
Conclusion
I'd like to reiterate my original point about Erlang. This is a fantastic workbench for DSLs. Anonymous functions, regular expression support and pattern matching are only the beginning. Erlang also gives us programmatic access to the tokenized, parsed and abstract forms of an expression. Observe Debasish Ghosh's string lambdas in Erlang as another example. I hope this article has helped some of you get out of your comfort zones with a new syntax and programming paradigm. I also hope people with think twice before labeling Erlang a specialist language.
About the Author
Dennis Byrne works for ThoughtWorks, a global consultancy with a focus on end-to-end agile software development of mission critical systems. Dennis is an active member of the open source community and he will present "Using Jinterface to bridge Erlang and Java" at the Erlang eXchange this year in June.