Tom Baeyens has released a preview of RockScript, an open source scripting language and engine for integrating and orchestrating microservices in the style of Event-Driven Architecture (EDA). RockScript uses a language that looks much like JavaScript, which when executed by the accompanying engine allows the coordination of transactional activities within microservice systems in a similar fashion to Business Process Execution Language (BPEL) and Business Process Modelling (BPM).
The challenge of coordinating service interaction within a distributed system is not a new issue, and the choice between "orchestration and choreography" has been discussed extensively within the traditional Service Oriented Architecture (SOA) community. Service orchestration refers to the automated execution of services on a pre-defined business process workflow by a centralised engine (for example using BPEL). Service choreography generally involves no centralised governance for business processes, and instead the coordination of service interaction is built into the services themselves; this has been referred to as "smart endpoints and dumb pipes" by microservice luminaries James Lewis and Martin Fowler.
The discussion of orchestration and choreography has continued within the microservice community, with the generally accepted best practice in this architectural style being the choreographing of services. However, the increasing number of enterprise organisations that are adopting the microservice architectural style (and have significant existing investment in orchestrated process), in combination with the rise in popularity of fine-grained services being implemented using serverless / Function-as-a-Service (FaaS) frameworks, has resulted in increased focus once again on orchestration frameworks. RockScript is one example of this type of orchestration framework that is specifically targeting microservice implementations.
The RockScript wiki states that "interactions with microservices are mostly done in JSON over HTTP", and therefore these interactions cannot easily participate in a transaction. At any point during an interaction the service being called may crash, or some component of the underlying infrastructure fail. In order to keep consistency when combining multiple microservice interactions for a business process, a record of the call graph and completions and failures must be kept and managed. A microservice best practice example of this type of multi-phase transaction protocol is captured within the Saga Pattern.
The RockScripts language is designed to capture what must happen in response to an event. Typically an external event is used as the input trigger, and this results in commands being sent to other microservice APIs, potentially with data transformations in between. RockScript is designed to be connected with both an HTTP API and a message exchange, although currently only the HTTP API is implemented. Baeyens, the project creator and founder of the Activiti and jBPM platforms, stated that the vision of RockScript is "to make it easy to write small scripts that are connected to events being fired in enterprise systems, and we want to make it easy to establish those type of trigger connections to scripts." This is similar to the concept of triggers in the workflow automation frameworks IFTTT or Zapier.
RockScript executions are resilient, and the runtime engine ensures that script executions can recover from application and infrastructure crashes. The runtime state of each script execution is stored across a cluster using an event sourcing approach, and therefore script executions don't consume resources like threads and memory when they are waiting for asynchronous activities to complete. The use of event sourcing also allows the inspection of what happened when something goes wrong, such as a server crash, and it is possible to attempt recovery from the last saved execution position.
InfoQ sat down with Baeyens and discussed his motivation for creating RockScript.
InfoQ: Welcome to InfoQ Tom! You have been involved workflow and Business Process Management (BPM) for a long time (having founded jBPM, Activiti and Effektif in the past), and so why did you start this new project?
Baeyens: Those projects gave me a ton of background in implementing engines that deal with asynchronous continuations. But on the product side, I realized that developers had to work in 2 different worlds: Workflow are based on a diagram. Data transformations had to be put in between the activities which were typically written in JavaScript or some other programming language.
Developers then had to work across these two distinct worlds. I believe that is an important reason why workflow did not yet breakthrough as a mainstream developer technology. By bringing the workflow activities concept to a scripting language like JavaScript developers can can work in the single world which they like. My ambition is to bring microservice integration to a much broader developer community and make integrating microservices a lot easier.
InfoQ: What are other technologies that are comparable with RockScript?
Baeyens: Recently cloud platform vendors launched similar services like AWS step functions and Microsoft logic apps. These solutions present themselves as the glue between API's, custom apps and serverless cloud functions (aka lambda's). Service interactions nowadays are typically based on JSON, with transport over HTTP and shifting towards asynchronous messaging. Historically BPEL aimed to address integration XML based services although that attempt was horrendously complex. But it shows that the need for an integration programming language is constant. With RockScript, the focus is to be the integration script language that fits best with today's API and microservice ecosystem.
InfoQ: We note that you have a strong pedigree in Java, but have chosen JavaScript as the basis for RockScript. Why was this?
Baeyens: There are 3 reasons for it:
- Because I envision a world where complete applications can be build on serverless cloud infrastructure with online IDE's and only JavaScript knowledge.
- It's important for CTO's and dev team leads to know that developers will be up and running in no time and that new developers will be quick to read and learn the technologies adopted by their company. Because RockScript is a subset of JavaScript, it will be very easy for most developers to learn it.
- JavaScript syntax is also ideal to perform the data transformations in between the microservice interactions. Most microservices are based on JSON and JavaScript is the best language to manipulate that type of data.
The RockScript wiki states that the project is currently at proof-of-concept stage, and as the implementation is volatile no stability guarantees are offered at this point in time. The current implementation is focused on developer experience, and as such persistence is currently offered only in-memory (with a plan to build persistence to a datastore like SQL or Kafka), and coverage of the JavaScript language is still minimal. Feedback is being sought, and developers can interact with the project via GitHub issues, Twitter, or mailing list.
Several new service orchestration frameworks are emerging in this space, and interested readers can find additional details on each project's website:
Existing integration frameworks, such as Apache Camel and Spring Integration, can also be used to emulate much of the orchestration behaviour. There is also similarity shared with existing Business Process Modelling (BPM) tooling, such as Activiti, jBPM, and MS BizTalk BPM.
Additional information on RockScript can be found on the project's website and GitHub repository, and a hands-on tutorial can be found on the project Wiki.
