Azure Orchestrators Simplify the Creation of Stateful Serverless Workflows

Azure Durable Functions aim to extend the paradigm of serverless computing by introducing the concept of orchestrator functions, enabling the definition of more complex workflows. If you have ever fancied using them, Microsoft has just published a useful walk-through to help developers start their journey in serverless computing and orchestrator functions.

Azure Functions are Microsoft’s take at serverless computing, and this service provides a basic mechanism that takes an input, processes it, and returns a result when actioned by a trigger. Durable Functions extends it in an interesting way in that they add state to something that is at its heart stateless. Durable functions are a set of functions that a so called orchestrator function wraps into a single logical transaction. They allow developers to write complex workflows in code and execute them.

The two fundamental concepts behind Azure Functions are functions and triggers. A function is just a single method from a static class. A trigger is an event that some code responds to. Typical triggers are Http, that responds to an HTTP request from a user, and Timer, that is able to schedule the execution of some code. On top of that, Durable Functions use two new concepts:

• Orchestrator functions, which can be seen as a sort of Cloud-based coroutine. In other words, an orchestrator function is able to set a checkpoint during its execution, exit while waiting for other functions to complete their execution, and then resume from where they stopped.

• Activity functions, which are functions that can be used inside an orchestrator. They can only respond to an orchestrator calling them. Activity function results are cached and are executed only once for the same orchestrator instance. This is an example of an activity function that retrieves a list of GitHub repositories:

  [FunctionName("GetAllRepositoriesForOrganization")]
  public static async Task<List> GetAllRepositoriesForOrganization([ActivityTrigger] DurableActivityContext context)
  {
  // retrieves the organization name from the Orchestrator function
  var organizationName = context.GetInput();
  // invoke the API to retrieve the list of repositories of a specific organization
  var repositories = (await github.Repository.GetAllForOrg(organizationName)).Select(x => (x.Id, x.Name)).ToList();
  return repositories;
  }
  

Note the use of an ActivityTrigger attribute with a DurableActivityContext parameter: this is what qualifies a function as an Activity function.

With this in mind, this is how you can define an orchestrator function that retrieves all repos from an organization, then sums all open issues count for each repo, and stores the result somewhere:

[FunctionName("Orchestrator")]
public static async Task RunOrchestrator(
    [OrchestrationTrigger] DurableOrchestrationContext context)
{
    // retrieves the organization name from the Orchestrator_HttpStart function
    var organizationName = context.GetInput();
    // retrieves the list of repositories for an organization by invoking a separate Activity Function.
    var repositories = await context.CallActivityAsync<List>("GetAllRepositoriesForOrganization", organizationName);

    // Creates an array of task to store the result of each functions
    var tasks = new Task[repositories.Count];
    for (int i = 0; i < repositories.Count; i++)
    {
        // Starting a `GetOpenedIssues` activity WITHOUT `async`
        // This will starts Activity Functions in parallel instead of sequentially.
        tasks[i] = context.CallActivityAsync("GetOpenedIssues", (repositories[i]));
    }

    // Wait for all Activity Functions to complete execution
    await Task.WhenAll(tasks);

    // Retrieve the result of each Activity Function and return them in a list
    var openedIssues = tasks.Select(x => x.Result).ToList();

    // Send the list to an Activity Function to save them to Blob Storage.
    await context.CallActivityAsync("SaveRepositories", openedIssues);

    return context.InstanceId;
}

In the example above, note the use of the OrchestrationTrigger attribute with a DurableOrchestrationContext parameter that qualifies RunOrchestrator as an orchestrator function.

What is interesting in the code above is that each call to context methods are themselves Azure Functions. This means that the workflow execution will benefit from the scalability and the reliability of Azure functions.

You can find the workflow above on GitHub, and of course do not miss Microsoft tutorial for the full detail.