How to Automate AWS with Backstage

Streamline your AWS cloud operations and enhance efficiency with our practical, easy-to-follow guide. Ideal for DevOps and cloud professionals!

Subscribe

Subscribe

Backstage is a compelling solution for engineering organizations, especially as they scale out to multiple teams. With Backstage, teams can create new projects quickly, it is easy to view dependencies between software components, and integrations with third parties provide quick access to important information.

The Key Features of Backstage

  1. Rapid Project Creation: Backstage's core prowess is its robust support for templates. Engineers have the liberty to design templates that manifest as forms within the Backstage environment. Initiating a project is now reduced to a form-fill exercise. Upon submission, a tailor-made repository, influenced by the form's inputs, is instantly generated.

  2. Seamless Integration: While embedding CICD files, be it Codebuild Buildspecs or GitHub Action files, into your template repositories is straightforward, the real challenge is: how do we orchestrate the infrastructure? This piece aims to highlight the potential of the Backstage catalog as the definitive source of truth for your infrastructure.

How to Connect Backstage To AWS Infrastructure

The heart of Backstage’s ability to create new projects is its support for templates . Engineers can author templates that become viewable as forms within the Backstage portal. Team members can simply fill out the form, click submit, and a new repository is created which is scaffolded based on the values of the form fields.

How to Automate AWS with Backstage

You can, of course, include CICD files such as Codebuild Buildspecs or GitHub Action files within your template repositories, but there is still a missing piece. How does the infrastructure get created? This post will explain a solution for managing your infrastructure using the Backstage catalog as the source of truth.

The Solution Breakdown

Utilizing Terraform in tandem with the Backstage provider , we can query the Backstage API to retrieve a list of entities and their associated metadata. The magic happens when Terraform iterates over each entity, provisioning resources based on the metadata content.

Understanding the Backstage Catalog

When working with the Backstage catalog, it is important to understand the difference between kinds and types.

Kinds vs. Types: In the Backstage ecosystem, 'Kinds' are predefined categories like Users, Locations, and Templates. For our purposes, we're zoning in on the "Component" kind, synonymous with software components.

'Types', however, offer more flexibility. They allow operators to categorize software components into distinct groups, such as Services, Websites, or Libraries. But remember, these classifications are not set in stone; they can be customized to fit your needs.

In addition to Kinds and Types, you can add arbitrary keys and values to metadata annotations.

Metadata Annotations: Backstage offers the ability to append custom keys and values to metadata annotations. This feature can be invaluable, especially when you want to tag software with specific resource requirements like ''rds" or "s3" .

The solution outlined here uses the Backstage Terraform provider, and more specifically the backstage entities data source , which will allow us to query the Backstage API for a list of entities and filter the lists based upon types and annotations.

The Backstage Workflow

AWS Infrastructure- How to Automate AWS with Backstage2 (1)

  • Software Component Creation: Initiate new software components within Backstage. This platform will then auto-generate code repositories using the predefined templates.

  • Resource Provisioning with Terraform: By harnessing the Backstage provider, Terraform provisions resources for the software components fetched from the Backstage API. We use the entity’s data source to create lists of different types of software components. This is useful because, for example, we may want to provision S3 buckets for websites and API Gateway for services.

  • Automation at its Best: Post the creation or modification of a software component in Backstage, there's no need for manual Terraform tweaks. Just apply the Terraform project post-service creation in Backstage and watch as the resources are auto-generated. For those looking to up the ante, consider integrating Terraform Cloud , Atlantis , or CodePipeline for even smoother operations.

backstage_entities in Terraform

terraform {
  required_providers {
    backstage = {
      source = "tdabasinskas/backstage"
      version = "2.0.0"
    }
  }
}
provider "backstage" {
  base_url = "https://backstage-url.fqdn.com/"
}
provider "aws" {
  region = "us-east-1"
  default_tags {
    tags = local.common_tags
  }
}
data "backstage_entities" "services" {
  filters = [
    "kind=Component,spec.type=service",
  ]
}
locals {
  entities_map = { for entity in data.backstage_entities.services.entities : entity.metadata.name => entity }
  common_tags = {
    Team  = "Platform"
  }
}
resource "aws_s3_bucket" "backstage" {
  for_each = local.entities_map
  bucket   = each.key
}

In the above example, we use the backstage_entities data source to pull a list of software components of type service, and then we loop through these and create an S3 bucket for each service.

The flexibility of this approach lies in the filter. For example, in Backstage you could accept a form input for RDS, and write that to the metadata annotations, then create a list of all software components that selected RDS.

terraform {
  required_providers {
    backstage = {
      source = "tdabasinskas/backstage"
      version = "2.0.0"
    }
  }
}
provider "backstage" {
  base_url = "https://backstage-url.fqdn.com/"
}
provider "aws" {
  region = "us-east-1"
  default_tags {
    tags = local.common_tags
  }
}
data "backstage_entities" "services" {
  filters = [
    "kind=Component,spec.type=service",
  ]
}
locals {
  entities_map = { for entity in data.backstage_entities.services.entities : entity.metadata.name => entity }
  common_tags = {
    Team  = "Platform"
  }
}
resource "aws_s3_bucket" "backstage" {
  for_each = local.entities_map
  bucket   = each.key
}

 

Backstage Should Be The Source of Truth for Your Infrastructure

The solution described in this post, and the example Terraform, will help you to use Backstage as the source of truth for your infrastructure. This pattern can be extended to provide different types of resources for services, websites, or libraries.
You can use annotations to allow developers to customize the infrastructure their software requires. For help with Backstage or any other aspect of your Cloud journey, reach out to StratusGrid .

Similar posts