A Little Cloudformation Intro
Cloudformation is an AWS service where you declare cloud resources like an EC2 instance or an S3 bucket in JSON or YAML files. These JSON or YAML files are called templates. Here's a YAML template the describes an S3 bucket:
# saved as bucket.yaml
AWSTemplateFormatVersion: 2010-09-09
Resources:
S3Bucket:
Type: AWS::S3::Bucket
With this template, you provision an S3 bucket by creating a stack using the awscli.
aws cloudformation create-stack \
--stack-name mys3bucket \
--template-body file://bucket.yaml
This command takes a short amount of time to complete. You can check the status of the stack with the describe-stacks command.
aws cloudformation describe-stacks \
--stack-name mys3bucket
describe-stacks returns StackStatus: CREATE_IN_PROGRESS while the stack provisions the S3 bucket and StackStatus: CREATE_COMPLETE when the stack finishes provisioning the S3 bucket.
Once complete, the S3 bucket declared in the template is available. You can verify by listing your s3 buckets.
aws s3 ls
You should see a new bucket prefixed with the stack name (mys3bucket). Naming is hard, so Cloudformation handled naming the S3 bucket for you. In our template, we did not specify a name for a bucket. So by default, Cloudformation creates a pseudo-random name using the stack's name (mys3ubkcet) and logical resource name (S3Bucket).
To delete the S3 bucket, call the delete-stack command on the mys3bucket stack.
aws cloudformation delete-stack \
--stack-name mys3bucket
You've now successfully created and deleted an S3 bucket using Cloudformation!
What about updating configuration?
While creating and delete resources is an important step in managing your infrastructure with code, there are times you need to update existing resources created from stacks. Luckily, Cloudformation manages updates as well. For instance, say we forgot to add an environment tag to the provisioned S3 bucket in the mys3bucket stack from earlier. You can change the template to add a tag:
# saved as bucket-production.yaml
AWSTemplateFormatVersion: 2010-09-09
Resources:
S3Bucket:
Type: AWS::S3::Bucket
Properties:
Tags:
- Key: environment
Value: production
Now use the update-stack command to apply the change.
aws cloudformation update-stack \
--stack-name mys3bucket \
--template-body file://bucket-production.yaml
Similarly, if you removed the tags from the template and upload the template to the existing stack, Cloudformation will remove the tags from the S3 bucket.
Change Sets
A downside with update-stack is the lack of visibility into the steps Cloudformation takes to change an existing stack's resources. Sometimes cloudformation doesn't take the correct steps you had expected.
Let's look at RDS databases. Like S3 buckets, Cloudformation generates a pseudo-random name for an RDS database template that does not specify the DBName property. If we wanted to name the database, we could add the DBName property to the template and update the stack.
Unfortunately, a change for DBName recreates the database. Any data I had on the original instance is lost.
This situation is where change sets help.
Change sets allow you to preview what Cloudformation intends to change. The change set reports what resources will be created, modified, and removed. If you decide the changes are what you had expected, you can choose to execute the change set which applies the changes.
Let's use change sets to repeat updating the s3 bucket configuration from the prior section.
First, create the change set.
aws cloudformation create-change-set \
--stack-name mys3bucket \
--template-body file://bucket-production.yaml \
--change-set-name new-env-tags
create-change-set likes to take its time to create itself. Once done, you can describe the ChangeSet to see what's going to change.
aws cloudformation describe-change-set \
--stack-name mys3bucket \
--change-set-name new-env-tags
Here's the output from the command:
{
"Changes": [
{
"Type": "Resource",
"ResourceChange": {
"Action": "Modify",
"LogicalResourceId": "S3Bucket",
"PhysicalResourceId": "mys3bucket-s3bucket-XXXXXX",
"ResourceType": "AWS::S3::Bucket",
"Replacement": "False",
"Scope": [
"Properties",
"Tags"
],
"Details": [
{
"Target": {
"Attribute": "Properties",
"Name": "AccessControl",
"RequiresRecreation": "Never"
},
"Evaluation": "Static",
"ChangeSource": "DirectModification"
},
{
"Target": {
"Attribute": "Tags",
"RequiresRecreation": "Never"
},
"Evaluation": "Static",
"ChangeSource": "DirectModification"
}
]
}
}
],
// ... removed code ...
}
If the output blinds you, know you're not alone. Once you've finished rubbing your eyes, focus on the Replacement field.
Replacement determines whether Cloudformation will update the existing cloud resource or replace the cloud resource. In the output above, the value is False, which means the S3Bucket resource will not be replaced. Phew!
When it's early morning, and you're trying to fix a production incident, you'll want to know what this field says. Ignoring this field tends to lead to bloody feet.
You can find more examples of change sets in the cloudformation docs.
Assuming you want to apply the change, you can call execute-change-set.
aws cloudformation execute-change-set --stack-name mys3bucket --change-set-name new-env-tags
If you decided against the change, then delete the ChangeSet.
aws cloudformation delete-change-set --stack-name mys3bucket --change-set-name new-env-tags
ChangeSets help you preview changes before you make them. As your infrastructure grows, ChangeSets help you reduce deployment risks.
One Template Defines Many Resources and Creates Many Stacks
So far, I've discussed how a single template creates a single stack to showcase the basics of Cloudformation. You might believe that every time you want a new resource, you would create a new template. And if you felt your DRY detector firing, you aren't alone.
Luckily, this is not the case. A single template can create multiple stacks. Each stack created from the same template will have unique resources. For instance, if we used bucket.yaml to create two stacks, we will create two individual S3 buckets.
aws cloudformation create-stack \
--stack-name my-first-bucket \
--template-body file://bucket-production.yaml
aws cloudformation create-stack \
--stack-name my-second-bucket \
--template-body file://bucket-production.yaml
Creating many stacks from a single template allows you to create a staging environment for your small team or provide service components like databases and caches for a large engineering organization.
But to do that, I need to discuss another feature of templates: parameters.
Generalizing with Parameters
bucket-production.yaml is useful to create and manage S3 buckets for production. If production buckets are the only type of buckets you need, then you're all set. But, if you need a bucket for a staging environment, then we can use parameters. Parameters allow a template to take different arguments, similar to how a function in a programming language can take different arguments.
# saved as environmnet-bucket.yml
AWSTemplateFormatVersion: 2010-09-09
# Turn environment value into a parameter
Parameters:
Environment:
Type: String
Description: The environment
Resources:
S3Bucket:
Type: AWS::S3::Bucket
Properties:
Tags:
- Key: "environment"
# Use the Parameter as the value
Value: !Ref Environment
The environment-bucket.yml template is similar to bucket-production.yml but environment-bucket.yml uses a parameter to set the environment tag. In the template, the !Ref Environment snippet will be replaced with the value of the Environment parameter when the template is uploaded to Cloudformation. You can think of the code !Ref Environment snippet like an interpolated variable in ruby or a variable within an fstring in python.
With this new template, creating a stack now requires the Environment parameter which you can pass in with the --parameters flag. Creating a stack with the template with the Environment parameter set to "staging" will create a new S3 bucket tagged environment: staging .
aws cloudformation create-stack \
--stack-name my-prod-bucket \
--template-body file://environment-bucket.yaml \
--parameters Environment=production
aws cloudformation create-stack \
--stack-name my-stg-bucket \
--template-body file://environment-bucket.yaml \
--parameters Environment=staging
Now you have a generic S3 bucket template that adds a tag with an environment of your choice. Rejoice in reuse!
For more ideas on building templates, read the templates at awslabs/aws-cloudformation-templates .
A Review
We use templates to declare cloud resources using plain JSON or YAML. These templates are used to spawn stacks that provision the resources declared in the templates. When we want to change cloud resources, we modify the template and update the stack. When we no longer need the resources, we delete the stack, and all the resources are removed.
If we're unsure what a change to a stack or template will look like, we can rely on change sets. Creating a change set describes the changes Cloudformation plans to make to the cloud resources. Remember to look for the Replacement field!
We also know now that a single template can launch multiple stacks to create identically configured cloud resources. If we add parameters to templates, we can make generalized templates that are easier to reuse.