HashiCorp has moved Consul-Terraform-Sync (CTS) into full general availability. CTS allows for the definition of tasks as Terraform modules that can be run as services are added or removed from Consul.
CTS is part of the Network Infrastructure Automation (NIA) solution which focuses on automating day two network tasks such as updating load balancer pools or firewall policies. It is also possible to create custom tasks to automate other activities.
CTS monitors the Consul service catalog for updates using a watcher. Where supported, Consul Blocking Queries are used with falling back to polling where they are not available. For each value being monitored a separate thread is maintained by the watcher. When a change is detected, any tasks that depend on the watched value are run.
Tasks consist of runbook automation written as a Terraform module. They are actions that take the dynamic service data and translate it into an infrastructure change. A driver is used to push out the changes, with the initial supported driver being a local Terraform run.
To simplify creating the Terraform module, one of the existing Terraform providers can be used. Providers exist for a number of infrastructure platforms including AWS, GCP, Azure, Heroku, DNSimple, CloudFlare, and vSphere. The full list is available from the Terraform registry. Once the appropriate provider is selected, the template repository can be cloned to be used as a starting point.
Once cloned, two project elements are required. The first is the root module, recommended to be named main.tf, which should contain the needed providers and define any additional resources. The second piece is to declare a services variable. This is done within a file named variables.tf. It contains a map of objects representing the response object from the Consul catalog API that will be consumed by the root module. Unused attributes from the response object can be omitted.
The following example makes use of the services variable. When used within the CTS automation, a local file (consul_services.txt) will be updated as the Consul service discovery information changes. The following would be within the main.tf file:
# Create a file with service names and their node addresses
resource "local_file" "consul_services" {
content = join("\n", [
for _, service in var.services : "${service.name} ${service.id} ${service.node_address}"
])
filename = "consul_services.txt"
}
As only the service name, ID, and node address are needed, the following would be the contents of the variable.tf file:
variable "services" {
description = "Consul services monitored by Consul-Terraform-Sync"
type = map(
object({
id = string
name = string
node_address = string
})
)
}
For information that is sensitive, CTS supports passing information from Vault, HashiCorp's secret management solution. It is also possible to leverage the recently released sensitive attribute on input variables and module outputs. When this is set, Terraform will redact the value from the CLI output.
There are CTS modules available from a number of partners of HashiCorp in the Terraform registry. This includes A10 Networks, AWS, Cisco, F5, and VMWare. For example, this module cts-alb_listener-nia, creates a listener rule and target group for an AWS Application Load Balancer when a server is registered within Consul.
At the time of release, HashiCorp is still validating that CTS can handle the large network architectures that many enterprise companies have. CTS is currently only compatible with Terraform OSS. It is expected to be integrated with Terraform Enterprise and Terraform Cloud in a future release. More information on CTS can be found within the documentation or HashiCorp Learn guides.