SOPS Encryption: How to Manage Your Secrets In Git With SOPS

The Missing Piece

If you work on a team that deploys and operates distributed applications, you most likely need to work with secrets or other sensitive data. Even though there are excellent products available for storing and managing access to secrets, there is still a chicken-egg paradox when it comes to seeding a secrets manager with data.

Applications often require access to static secrets. An example is an API key provided by a vendor. While it is a best practice to store the secret in a secrets manager, such as Vault by Hashicorp or AWS Secrets Manager , you still have to figure out how to get the secret into the secrets manager in the first place.

If the vendor provides the key to you through their website, you might think about copying and pasting it directly into the secrets manager. However, if you are provisioning infrastructure automatically, you are now introducing a manual step every time you build a new environment.

Also, what if you require separate API keys for dev, staging, production, and ephemeral environments? You will quickly realize that manual copying and pasting of secrets is not sustainable. This is where Sops comes into play.

Sops Does One Thing Well

In 1978, Doug Mcllroy wrote that developers should "Make each program do one thing well," and Sops is an excellent example of this philosophy put into practice. Sops allows us to encrypt secrets, share them with our team, and version them alongside our code. It does not replace your secrets manager, but it does enable you to seed your secrets manager when you receive static secrets from a third party.

Given an input on the command line, Sops encrypts secrets and writes them out in plain text ASCII. Files are encrypted using a "key source", and there are many key sources supported, such as AWS KMS , PGP , and Age .

The files produced by Sops can be committed to your Git project alongside your code, and since they require access to the key source for decryption, your existing Identity Provider can be used to determine who is authorized to decrypt them.

Here is an example workflow for using Sops with Terraform and AWS Secrets Manager:

In the above example, using Terraform you can encrypt the values in your tfvars file. When Terraform runs in your CI/CD system, the CI/CD system is able to decrypt the file since it is authorized to retrieve the KMS key used as the key source. Once decrypted, Terraform can use the value to load the secret into AWS Secrets Manager.

But Wait, There’s More!

Sops offers some great features that make it a pleasure to work with. For one, when working with YAML, JSON, ENV, or INI files, Sops will encrypt the values of the files while keeping the keys unencrypted. This makes it easier to peruse and understand the codebase without having to decrypt the files.

Another nice feature is that you can add a sops.yaml file to your project, enabling you to specify different keys for different files, even using regex. A use case for this feature is to encrypt your development secrets using a dev key, and your production secrets using a prod key.

Lastly, something we really appreciate about Sops is that the information required to decrypt files lives within the encrypted file itself. So, you never need to tell Sops which key to use in order to decrypt a file; it already knows based upon the encrypted file's contents.

Next Steps

Secrets are critical to your applications. In an ideal world, all secrets would be dynamically provisioned to your applications and rotated at regular intervals. However, many times we are in the situation where secrets are provided to us by a third party, and we absolutely must incorporate them with the rest of our Infrastructure-as-Code in order to eliminate the risks associated with manual configuration. If you find yourself in this situation, then you should give Sops a try.

Please see this repository for an example of using Sops with Terraform to seed secrets in AWS Secrets Manager.