Google has announced Cloud Tasks, a task queue service for Google Cloud Platform's App Engine, the web framework and fully managed cloud computing platform. Cloud Tasks enables asynchronous execution of tasks from applications, decoupling services and supporting the implementation of long-running and background activities.
With Google Cloud Tasks, applications can place tasks, pieces of work defined by an application developer, on a distributed queue which then pushes these to worker services. The concept of task queues is a proven one provided by all the large cloud providers like AWS with SQS and Azure with Service Bus. Actually, Cloud Tasks is an advancement of Google's Task Queue service, which also provided this same concept. With this announcement, management of all existing and new task queues are consolidated under Cloud Tasks, as explained in the announcement by Morgan Hallmon, product manager on Google Cloud Platform:
For many years, App Engine customers have leveraged Task Queue, an App Engine service that enables asynchronous task execution. With today's release, all management of task queues is now centralized in the Cloud Tasks console, command line and API.
There are several advantages to the use of task queues, like making systems loosely coupled, offloading of work to background services, and improving reliability by providing storage of tasks passed between systems.
Source: https://en.wikipedia.org/wiki/Scheduling_(computing)#task_queue
Initially, Cloud Tasks may appear to have much overlap with Google's other queueing service Pub/Sub. While similarities are noticeable, the use cases between the two do certainly differ. Google positions Pub/Sub as a service focused on big data and network operations, providing high throughput while remaining agnostic to content and target applications. On the other hand, Cloud Tasks provides a managed execution product, handling many explicit commands, which specific background workers then process.
Moreover, unlike Task Queues which only had limited incorporation in Google Cloud Platform, Cloud Tasks will integrate into the whole spectrum of GCP services. The documentation around Cloud Tasks describes typical use cases to include speeding up the user's experience, providing durability during incidents, and smoothing out traffic spikes.
- Speeding user response times by delegating potentially slow background operations like database updates to a worker
- Preserving requests in the context of unexpected production incidents
- Helping smooth traffic spikes by removing non-user-facing tasks from the main user flow
As Cloud Tasks is a managed service, it takes responsibility for provisioning and management of resources, allowing developers to focus on creating and the handling of the tasks. Furthermore, Cloud Tasks bases usage billing only on the actual execution time. Client libraries are provided by Google for several of the popular languages, like Java, Python, Node.js, PHP, and Go, while also giving the option to enqueue tasks through the REST API, allowing to add these from any application.
Working with Cloud Tasks requires the execution of several steps. It all starts with creating the task queue. Once the creation of the queue completes, the application publishing the tasks pushes these to the queue, which then stores the messages and returns an OK. Subsequently, the Cloud Tasks service sends the task to a background worker application, which then processes it. Once the task processing finishes, the worker sends an OK as an HTTP status code between 200 and 299 back to the queue, which then deletes the task. Additionally, in case the worker does not finish processing in a timely fashion, Cloud Tasks will automatically send the tasks out again, providing retries and at least once delivery.