Terraspace, the new Terraform standard?

Over the past years, Terraform became one the reference tools to deploy infrastructure on public and private clouds. In order to organize the Terraform code inside a git repository, best practices and convention naming can be applied. For example, you can find an excellent Google article about Terraform good practices here.

However, when a single git repository hosts multi regions, environments and providers code, it can be really difficult to manage and automate the Terraform workflows. People created Terragrunt to answer this need, but it forces you to have a complex terraform project file tree (more detail at the end of this article). More recently, a new tool called Terraspace has been created by the  Boltops company.

In February 2022, Tekos Interactive discovered Terraspace and, after a lot of successful tests, decided to define Terraspace as its new standard for deploying infrastructures. In this article, we will introduce you to the main features of Terraspace and explain to you how it brings value to our teams everyday. Then we will give you some concrete examples on how we leveraged success with this tool, and why we chose it over Terragrunt.

Terraspace: A Terraform wrapper

Terraspace has many interesting features. In this part, we will only focus on the most important ones.

Ruby wrapper

The first and the most important feature of Terraspace is that you can wrap your Terraform code with ERB ruby templating. It means that you can generate Terraform code based on Terraform and Ruby variables. It is also important to note that you can also wrap your Terraform “provider” block for a dynamic configuration (which is a native limit of Terraform). Terraform generates the “real” code (according to your variables) in your cache folder when you launch the plan or apply command.

In this example, we use a Terraspace variable to fill in our description: 

description = "KMS encryption key for <%= expansion(':TS_PROJECT') %>"

The result will be:

description = "KMS encryption key for project_name"

You can also define loops to fill in your lists:

data "aws_iam_policy_document" "policy" {
  statement {
    sid    = "ARandomExample"
    effect = "Allow"
    principals {
      type = "AWS"
      identifiers = [
<%- account_ids_list.each_with_index do |account_id, index| -%>
        "<%= account_id %>",
<%- end -%>
      ]
    }
    actions = [
      "SomePermissions…"
    ]
  }
}

For each element of account_ids_list ruby variable, we write it in our policy:

data "aws_iam_policy_document" "policy" {
  statement {
    sid    = "ARandomExample"
    effect = "Allow"
    principals {
      type = "AWS"
      identifiers = [
        "AccountID1", 
        "AccountID2",
        …
      ]
    }
    actions = [
      "SomePermissions..."
    ]
  }
}

Standardized project folder

A key question that we could ask when we want to optimize our Terraform workflows is “How do I organize my project”. Terraspace answers this question by bringing its own standardized file tree. This is how does a Terraspace project look like : 

├── app
│   └── stacks
│   └── modules
├── config
│   ├── app.rb
│   ├── boot.rb
│   ├── helpers
│   └── terraform
├── Gemfile
├── Gemfile.lock
├── Makefile
├── README.md
├── Terrafile
└── Terrafile.lock

In this project file tree, each file and folder has its purpose :

• config: This folder gathers configurations that will impact the whole Terraspace project. These configurations control how Terraspace behaves, manage the backend, or define a Terraform file (like a local.tf file) that gets added to each stack.
• app: This folder will host your Terraform infrastructure code shared in the stack. A stack is a Terraform module that represents a part of your infrastructure. This module will be fed with values (called seed in Terraspace) when you deploy them into your environments. In the app folder we can also define modules. The key difference between a module and a stack is that we reuse modules in stacks. Of course, this must be used only in specific cases (for example : the development of a module that will not be reused in other Terraspace projects). If you want to deploy terraform modules into your stack, the best practices will be to use modules hosted in other repositories that you manage or to use community modules hosted on Terraform registry.
• Terrafile: As explained above, we will want to use as many remote Terraform modules as possible. However, it can be painful and repetitive to find every occurrence of one Terraform module that you use to update them. That is why the Terrafile has been added to Terraspace projects. With the Terrafile, you will be able to define in one single place, all modules (and their specific versions) that you use in your Terraspace projects. Bundling your project downloads these modules into a vendor folder, making them available for your stacks. When you will need to update one of your modules, update the Terrafile, and apply your stacks.

org "tekos-interactive"
mod "vpc", source: "git@github.com:TekosInteractive/terraform-aws-vpc.git" , version: "v1.0.0"
mod "vpc_endpoints", source: "git@github.com:TekosInteractive/terraform-aws-vpc-endpoints.git", version: "v1.0.0"

A multi-level layers

Thanks to its standardized file tree, Terraspace allows you to define a multi-layer configurations : 

• Project layer: This will be added to all stacks and all environments
• Stack layer: This will be added only to the stack and to all environments
• Seed layer: This will be added only to the stack and to only one environment (same as a standard .tfvars file)

Having this multiple layer strategy will help you to be flexible in your environment management.

Helpers

The last key feature of Terraspace is the possibility to add helpers. Helpers are “raw” ruby code that will help you to define variables values or will execute tasks for you. You can also define these helpers at any Terraspace layer. Moreover, you can also launch them in the early or late stage of your Terraform code build/plan/apply.

Helpers are really important because they bring some “intelligence” to your Terraform code and workflows. 

module Terraspace::Project::AccountIdHelper
    # Map of AWS account IDs used in provider configuration to automatically assume the role according to the environment
    def account_ids_map
        map = {
            prod: "111111111111",
            dev:  "222222222222"
        }
        map[Terraspace.env.to_sym] || ""
    end

Now that we presented the key features of Terraspace, let’s explain how it brings value to our team.

Key values

Terraspace brings a lot of value in Tekos Interactive daily work. Let’s review them together. 

• Automation at “ease”: The main value of Terraspace is its capability to automate a lot of tasks without any third party system. With this tool, you will be able to define stack dependencies (and then deploy them in orders through launches called “batches”), automatically create Terraform backends (for example S3 buckets), centralize your modules versions and variable values. All these aspects are stored and managed within your repository. You could achieve this same level of automation by adding a lot of bash (or any other language) scripts. However, Terraspace offers this automation experience out of the box and in a simple and readable workflow. Thanks to that, our DevOps department can only focus their attention on deploying stacks without thinking about the rest.
•  Flexibility: Thanks to its flexibility, mostly provided by its templating system and multi-layers variables, Terraspace can answer every need. You will be able to configure multiple providers, fine-tune your configurations and manage your variables. After more than 1 year of experience, Tekos Interactive didn’t find any use-case not possible to achieve with Terraspace.
• Centralization: The last main value that Terraspace provides is a high level of centralization. It can be centralization about variables but also about configurations and Terraform modules versioning. When a repository needs to be updated, we know where the configuration is , and we know that it will (if we want) impact the whole repository.

Some examples of success

In this part, we will describe some scenarios that we successfully solved thanks to Terraspace and its design.

AWS account creation and provisioning

A challenge that we face when we manage an AWS organization with Terraform is to create a new AWS account and automatically provision it. With only Terraform, we would have launched a first terraform command to deploy the aws_organizations_account resource, read the new account information (like the account ID), configure your provider and then launch a second Terraform that will provision the account. We could summarize this workflow with the schema below :

With Terraspace, we succeeded in automating the full process with only one command. The workflow could be summarized like  the schema below: 

These are the steps (launched with only one command “terraspace all up”):

1. The account stack deploys the new AWS account
2. Thanks to stack dependencies and helpers, we retrieve information from the account stack and we automatically configure the provider of the stack provisioning.
3. The stack provisioning is applied.

💡 In this Terraspace workflow, thanks to personalized helpers, we can also add some “intelligence” (through ruby code) to create or modify variables before applying the provisioning stack.

Multi account and backend management at ease

At Tekos, a single client infrastructure can have multiple environments. Each environment is separated into its own AWS account. Moreover, to allow sharing common resources into each environment account, we also provide a shared AWS account. It means that per client we could have up to 4 different AWS accounts (with dev, stage and prod). Managing all these Terraform providers could be inconvenient. However, thanks to Terraspace project configuration, we can generate these Terraform providers with a little bit of templating. For example: 

provider "aws" {
 region = "eu-west-3"
 assume_role {
   role_arn = "arn:aws:iam::<%= account_ids_map %>:role/yourrole"
 }
}

provider "aws" {
 region = "eu-west-3"
 alias  = "shared"
 assume_role {
   role_arn = "arn:aws:iam::<%= shared_account_id %>:role/yourrole"
 }
}

In this example we can see two providers, the first one is the provider of the environment account and the second one is the provider of the “shared” account. Both of ERB templating automatically generates them.Helpers generate these variables (account_ids_map and shared_account_id).

This same approach is used to generate the backend configuration:

terraform {
 backend "s3" {
   bucket         = "<%= expansion('prefix-:PROJECT-:ENV') %>"
   key            = "<%= expansion('yourpath/:REGION/terraform.tfstate') %>"
   region         = "<%= expansion(':REGION') %>"
   encrypt        = true
   dynamodb_table = "terraform_locks"
   role_arn       = "arn:aws:iam::<%= account_ids_map %>:role/yourrole"
 }
}

These examples are a few wins that we achieved with Terraspace.

Terraspace drawbacks

No tool is perfect, so does Terraspace. In this part we will quickly enumerate Terraspace drawbacks:

• A complex tool:  Terraspace is a complex tool, there are many features and commands. Before using Terraspace in a production environment, you will need to get familiar with all (or at least a lot) of its features. Moreover, if you are not familiar with Ruby syntax and code, you will need to learn it. Do some tests, and create a git repository template that answers your Terraform workflows
• Lack of documentation:  The main drawbacks of Terraspace is its documentation. The only documentation available as we write these lines is the one on the official website:Terraspace documentation. This documentation lacks details, and examples. Moreover, their video examples are only available if you pay for the Boltops course. That makes Terraspace hard to test. You will spend a lot of time testing and setting your workflows.
• No integration with Atlantis: Unfortunately, Terraspace is not compatible with the Atlantis project. That’s why, at Tekos, we have implemented our own Pull Request interactions system to work with Terraspace.

Terraspace VS Terragrunt

When we think about Terraform wrappers, the first solution that comes to mind is Terragrunt. According to their documentation, Terragrunt is a thin wrapper that provides extra tools for keeping your configurations DRY (Don’t repeat yourself), working with multiple Terraform modules and managing remote state.

If we want to compare Terraspace and Terragrunt:

In Tekos Interactive, we have decided to use Terraspace instead of Terragrunt to better manage our environments without having the headache of managing the Terragrunt file tree system. Moreover, Terraspace templating is more flexible than the simple Terragrunt dependency outputs values.

Conclusion

Terraspace is a great tool that provides a lot of flexibility to handle your Terraform workflows. Its main advantages are its ERB templating system, its multi-layers variables and its helpers (by default or custom).

Thanks to this tool, Tekos Interactive automated a lot of their Terraform workflows.

However, this tool requires a long learning curve and must be fully tested before putting it in your production workflows. But luckily, we can do that for you, just drop us a line at ronan@tekos-france.com

Would you like to read more articles by Tekos’s Team? Everything’s here.

Author

Laurent Vasseur

• LinkedIn
• Facebook