Modernization
Lambda Redis SlowLog to CW Logs Module on Terraform
Streamline Redis performance with StratusGrid's Lambda module, automating slow log analysis to AWS CloudWatch for optimal efficiency.
Streamline your AWS cloud operations and enhance efficiency with our practical, easy-to-follow guide. Ideal for DevOps and cloud professionals!
Backstage is a compelling solution for engineering organizations, especially as they scale out to multiple teams. With Backstage, teams can create new projects quickly, it is easy to view dependencies between software components, and integrations with third parties provide quick access to important information.
2 Key Features of Backstage
- Rapid project creation: Backstage's core prowess is its robust support for templates. Engineers have the liberty to design templates that manifest as forms within the Backstage environment.
Initiating a project is now reduced to a form-fill exercise. Upon submission, a tailor-made repository, influenced by the form's inputs, is instantly generated. - Seamless integration: While embedding CICD files, be it Codebuild Buildspecs or GitHub Action files, into your template repositories is straightforward, the real challenge is: how do we orchestrate the infrastructure? This piece aims to highlight the potential of the Backstage catalog as the definitive source of truth for your infrastructure.
How to Connect Backstage To AWS Infrastructure
The heart of Backstage’s ability to create new projects is its support for templates. Engineers can author templates that become viewable as forms within the Backstage portal. Team members can simply fill out the form, click submit, and a new repository is created which is scaffolded based on the values of the form fields.
You can, of course, include CICD files such as Codebuild Buildspecs or GitHub Action files within your template repositories, but there is still a missing piece. How does the infrastructure get created? This post will explain a solution for managing your infrastructure using the Backstage catalog as the source of truth.
The Solution Breakdown
Utilizing Terraform in tandem with the Backstage provider, we can query the Backstage API to retrieve a list of entities and their associated metadata. The magic happens when Terraform iterates over each entity, provisioning resources based on the metadata content.
Understanding the Backstage Catalog
When working with the Backstage catalog, it is important to understand the difference between kinds and types.
Kinds vs. Types: In the Backstage ecosystem, 'Kinds' are predefined categories like Users, Locations, and Templates. For our purposes, we're zoning in on the "Component" kind, synonymous with software components.
'Types', however, offer more flexibility. They allow operators to categorize software components into distinct groups, such as Services, Websites, or Libraries. But remember, these classifications are not set in stone; they can be customized to fit your needs.
In addition to Kinds and Types, you can add arbitrary keys and values to metadata annotations.
Metadata Annotations: Backstage offers the ability to append custom keys and values to metadata annotations. This feature can be invaluable, especially when you want to tag software with specific resource requirements like ''rds" or "s3" .
The solution outlined here uses the Backstage Terraform provider, and more specifically the backstage entities data source , which will allow us to query the Backstage API for a list of entities and filter the lists based upon types and annotations.
The Backstage Workflow
.webp?width=915&height=1084&name=AWS%20Infrastructure-%20How%20to%20Automate%20AWS%20with%20Backstage2%20(1).webp)
-
Software component creation: Initiate new software components within Backstage. This platform will then auto-generate code repositories using the predefined templates.
-
Resource provisioning with Terraform: By harnessing the Backstage provider, Terraform provisions resources for the software components fetched from the Backstage API. We use the entity’s data source to create lists of different types of software components. This is useful because, for example, we may want to provision S3 buckets for websites and API Gateway for services.
-
Automation at its best: Post the creation or modification of a software component in Backstage, there's no need for manual Terraform tweaks. Just apply the Terraform project post-service creation in Backstage and watch as the resources are auto-generated. For those looking to up the ante, consider integrating Terraform Cloud , Atlantis , or CodePipeline for even smoother operations.
backstage_entities in Terraform
terraform {
required_providers {
backstage = {
source = "tdabasinskas/backstage"
version = "2.0.0"
}
}
}
provider "backstage" {
base_url = "https://backstage-url.fqdn.com/"
}
provider "aws" {
region = "us-east-1"
default_tags {
tags = local.common_tags
}
}
data "backstage_entities" "services" {
filters = [
"kind=Component,spec.type=service",
]
}
locals {
entities_map = { for entity in data.backstage_entities.services.entities : entity.metadata.name => entity }
common_tags = {
Team = "Platform"
}
}
resource "aws_s3_bucket" "backstage" { for_each = local.entities_map bucket = each.key }
In the above example, we use the backstage_entities data source to pull a list of software components of type service, and then we loop through these and create an S3 bucket for each service.
The flexibility of this approach lies in the filter. For example, in Backstage you could accept a form input for RDS, and write that to the metadata annotations, then create a list of all software components that selected RDS.
terraform {
required_providers {
backstage = {
source = "tdabasinskas/backstage"
version = "2.0.0"
}
}
}
provider "backstage" {
base_url = "https://backstage-url.fqdn.com/"
}
provider "aws" {
region = "us-east-1"
default_tags {
tags = local.common_tags
}
}
data "backstage_entities" "services" {
filters = [
"kind=Component,spec.type=service",
]
}
locals {
entities_map = { for entity in data.backstage_entities.services.entities : entity.metadata.name => entity }
common_tags = {
Team = "Platform"
}
}
resource "aws_s3_bucket" "backstage" {
for_each = local.entities_map
bucket = each.key
}
Backstage Should Be The Source of Truth for Your Infrastructure
The solution described in this post, and the example Terraform, will help you to use Backstage as the source of truth for your infrastructure. This pattern can be extended to provide different types of resources for services, websites, or libraries.
You can use annotations to allow developers to customize the infrastructure their software requires. For help with Backstage or any other aspect of your Cloud journey, reach out to StratusGrid.
Elevate Your Engineering Workflow with StratusGrid and Backstage
Are you ready to transform your engineering processes and infrastructure management? StratusGrid offers a dynamic and efficient solution tailored for engineering organizations, especially those expanding across multiple teams.
With Backstage, you can swiftly initiate new projects, gain clear insights into software dependencies, and leverage seamless third-party integrations for enhanced accessibility to critical information.
Why Choose StratusGrid?
- Rapid project creation: Utilize Backstage's powerful template system to kickstart your projects with ease, ensuring a streamlined process from conception to execution.
- Seamless integration: Integrate effortlessly with a variety of CICD tools and manage your infrastructure effectively using Backstage as your single source of truth.
- Customizable infrastructure: With Terraform and Backstage, tailor your infrastructure provisioning to meet the unique needs of your software components, all within a unified platform.
Connect with StratusGrid Today!
Don't let complexity hinder your progress. For a personalized consultation or to learn more about our solutions, contact StratusGrid now.
BONUS: Find Out if Your Organization is Ready for The Cloud ⤵️
