Amazon Web Services have launched AWS Step Functions, a service that enables the coordination of distributed applications and microservices using visual workflows. The AWS Step Functions console allows the JSON specification of a state machine to be defined that executes coordinated ‘steps’ within an application, for example, by calling out to an AWS Lambda function or containerised microservice application running on AWS EC2 Container Service (ECS).
Each state machine created via the AWS Step Function service defines a set of states and the transitions between them. States can be activated sequentially or in parallel, and the service will ensure that all parallel states run to completion before moving forward. States perform work, make decisions, and control progress through the state machine. Multiple copies of each state machine can be running independently at the same time; each copy is called an “execution”. According to the AWS Blog, the AWS Step Functions service will allow the running of thousands of execution concurrently.
All work undertaken within the AWS Step Function state machine is conducted by “tasks”. A task can be:
- An AWS Lambda function, which is a completely cloud-based task that runs on the Lambda service. Lambda functions can be written in JavaScript (which can be written using the AWS Management Console or uploaded to the AWS Lambda service), or in Java or Python (which must uploaded to the AWS Lambda service).
- An Activity, which is a concept that refers to a task to be performed by a 'worker' application or microservce that is hosted on a platform such as AWS EC2 or AWS ECS.
- Workers can be implemented in any language that can make AWS Step Functions API calls.
- Workers must poll AWS Step Functions using the ‘GetActivityTask’ and ‘SendTask*’ API calls. The AWS Blog states that ultimately an activity can even be a ‘human task’ that waits for a human to perform some action and then continues.
- If an activity has a heartbeat timeout value, the worker which implements it must send heartbeat updates using the ‘SendTaskHeartbeat’ action.
- Activities are not versioned and are expected to always be backwards compatible. If backwards-incompatible changes must be made to an activity definition, then a new activity should be created with Step Functions using a unique name.
As part of a state machine, error handling behavior and retry logic can be specified within the JSON specification. This allows you to build “robust multi-step applications” that will run successfully even if transient issues in one part of the application cause a momentary failure.
When creating the state machine specification, the new AWS Labs ‘statelint’ Ruby gem can be used to validate hand or machine-generated JSON for common errors including unreachable states and the omission of a terminal state.
The design of any state machines can be visualised within the AWS Step Functions console, and during execution a colour-coded diagram can be enabled to debug the current progress (and success/failure of steps) within a state machine:
Although AWS Step Functions has only recently been released, Simon Wardley, a researcher for the Leading Edge Forum, suggests that this in combination with AWS Lambda-like 'serverless' functionality could support the future of the codification of business practices in order to increase agility, something that began with the earlier attempts to create Business Process Execution Language (BPEL).
When you think about AWS Lambda, AWS Step Functions et al then you need to view this through the lens of automating basic doctrine i.e. not just saying it and codifying in maps and related systems but embedding it everywhere. At scale and at the speed of competition that I expect us to reach then this is going to be essential.
The AWS Blog states that AWS Step Functions is available now within the US East (Northern Virginia), US East (Ohio), US West (Oregon), EU (Ireland), and Asia Pacific (Tokyo) Regions. Up to 4,000 state transitions per month can be run at no charge as part of the AWS Free Tier. Outside of the free tier, the cost is $0.025 for every 1,000 state transitions.
Additional information about AWS Step Functions can be found on the product page, the AWS Blog, and in a recent AWS webinar on YouTube, “Announcing AWS Step Functions”.
