Terraform is an automation tool for infrastructure, and it allows you to define your infrastructure using code and manage it in an automated way. It has multiple implementations for different cloud providers to use Terraform to manage AWS, Azure, Google Cloud, Digital Ocean, etc. So it’s a versatile tool to have in your belt. I’m going to focus on AWS EC2 Servers for this tutorial. Sometimes when I’m working on tutorials or playing around with projects, I might need to set up a server. Still, I find that having to log in to the console and do all the steps manually can be time-consuming, and I need to run terraform apply to create it, then terraform destroy when I am done to delete it. Also, for Development, I am using AWS Spot instances, as they can be 70% to 90% cheaper than regular EC2 prices.
AWS Resources Needed to Run EC2
Before I start writing the code in Terraform for the AWS resources that we’re going to create, I’d like to explain what we will make first. You can see that it’s not as simple as some other cloud providers like DigitalOcean, Vultr or Linode. Still, once automated, you need to run one command to create the server and one command to delete it, as I mentioned earlier.
Network related resources
- VPC: A Virtual Private Cloud is an isolated virtual network specific to you within AWS, where you can launch your particular AWS services. You will have all the network setup, route tables, subnets, security groups, and network access control lists.
- Internet Gateway: for your VPC to access the Internet, you have to attach an Internet gateway to it.
- Public Subnet: A subnet is a logical group of a network; we assign a portion of the network IPs, and we can make it public or private so it’s not accessible directly from the Internet. In this case, I’m just creating one public subnet.
- VPC Route Table: a routing table contains a set of rules, called routes, used to determine where to direct your network traffic from your subnet or gateway.
Virtual Server Resources
- Security Group: A security group acts as a virtual firewall for your instance to control inbound and outbound traffic. Security groups operate at the instance level, not the subnet level. In this case, I am allowing traffic for SSH, HTTP and HTTPS.
- EC2 Server: This is the Virtual Private Server or VPS that we are interested in using.
- SSH Key: I am copying my public key when creating the server to log in using my SSH private key afterwards.
- Elastic IP: I am assigning an IP to the server to connect via this IP.
Setting up Dependencies
To use Terraform with AWS, you need to have an AWS account and the AWS CLI installed. After installing the CLI, run aws configure to configure it. Introduce your access key and account secret that you can find here.
You also need to have Terraform installed, and if you’re using OS X, install it using Homebrew:
$ brew tap hashicorp/tap
$ brew install hashicorp/tap/terraformYou can verify if Terraform is installed by running:
$ terraform -help
$ terraform -vInfrastructure as Code Using Terraform
The set of files used to describe infrastructure in Terraform is known as a Terraform configuration. Usually, terraform loads all files that end in .tf in the folder where you define your infrastructure; all files must be under the same root folder.
Here is the list of variables that I have defined for the infrastructure script:
variable "aws_region" {
type = string
default = "us-east-2"
}
variable "cidr_block" {
default = "10.0.0.0/16"
}
variable "aws_availability_zone" {
type = string
default = "us-east-2a"
}
variable "instance_type" {
type = string
default = "t4g.small"
}
variable "ssh_pub_path" {
type = string
default = "~/.ssh/id_rsa.pub"
description = "Path to public key to use to login to the server"
}
variable "instance_ami" {
type = string
default = "ami-03e1711813e5a07b1"
description = "Instance AMI image to use, by default Ubuntu 20.04 LTS"
}
variable "spot_price" {
type = string
default = "0.016"
description = "Maximum price to pay for spot instance"
}
variable "spot_type" {
type = string
default = "one-time"
description = "Spot instance type, this value only applies for spot instance type."
}
variable "spot_instance" {
type = string
default = "true"
description = "This value is true if we want to use a spot instance instead of a regular one"
}It is pretty self-explanatory. A few things to take into account:
- This script creates a spot instance of type t4g.small for a maximum of $0.016 per hour. If you wanted to create a regular instance, you could set the variable
spot_instanceto false, and all the values related to spot instances will not be used. - As a general rule of thumb, spot prices in the
us-east-2region are the cheapest. Not always true, but most of the time.
In the main.tf file, I’m defining that I want to use the AWS Terraform provider and configure the region and credentials to use. A provider is a Terraform plugin that allows users to manage an external API. Provider plugins like the AWS provider or the cloud-init provider act as a translation layer that enables Terraform to communicate with many different cloud providers, databases, and services.
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
}
}
}
provider "aws" {
region = var.aws_region
profile = "default"
}In the network.tf file, I declare the network resources I described earlier. As you can see, I’m using input variables, as they let you customize aspects of Terraform modules without altering the source code, making it easier to reuse the Terraform code.
resource "aws_vpc" "vps-env" {
cidr_block = var.cidr_block
enable_dns_hostnames = true
enable_dns_support = true
}
resource "aws_subnet" "subnet-uno" {
# creates a subnet
cidr_block = "${cidrsubnet(aws_vpc.vps-env.cidr_block, 3, 1)}"
vpc_id = "${aws_vpc.vps-env.id}"
availability_zone = var.aws_availability_zone
}
resource "aws_security_group" "ingress-ssh-vps" {
name = "allow-ssh-sg"
vpc_id = "${aws_vpc.vps-env.id}"
ingress {
cidr_blocks = [
"0.0.0.0/0"
]
from_port = 22
to_port = 22
protocol = "tcp"
}
egress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
}
resource "aws_security_group" "ingress-http-vps" {
name = "allow-http-sg"
vpc_id = "${aws_vpc.vps-env.id}"
ingress {
cidr_blocks = [
"0.0.0.0/0"
]
from_port = 80
to_port = 80
protocol = "tcp"
}
egress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
}
resource "aws_security_group" "ingress-https-vps" {
name = "allow-https-sg"
vpc_id = "${aws_vpc.vps-env.id}"
ingress {
cidr_blocks = [
"0.0.0.0/0"
]
from_port = 443
to_port = 443
protocol = "tcp"
}
egress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
}
resource "aws_internet_gateway" "vps-env-gw" {
vpc_id = "${aws_vpc.vps-env.id}"
}
resource "aws_route_table" "route-table-vps-env" {
vpc_id = "${aws_vpc.vps-env.id}"
route {
cidr_block = "0.0.0.0/0"
gateway_id = "${aws_internet_gateway.vps-env-gw.id}"
}
}
resource "aws_route_table_association" "subnet-association" {
subnet_id = "${aws_subnet.subnet-uno.id}"
route_table_id = "${aws_route_table.route-table-vps-env.id}"
}In the ec2.tf file, I’m declaring the regular or spot instance. I am using a trick to create one or the other conditionally. Using the count property from Terraform, I can use one or zero instance based on an input variable. You could also be creating more than one instance if you needed to.
resource "aws_eip" "ip-vps-env" {
instance = "${var.spot_instance == "true" ? "${aws_spot_instance_request.vps[0].spot_instance_id}" : "${aws_instance.web[0].id}"}"
vpc = true
}
resource "aws_key_pair" "ssh_key" {
key_name = "ssh_key"
public_key = "${file(var.ssh_pub_path)}"
}
resource "aws_spot_instance_request" "vps" {
ami = var.instance_ami
spot_price = var.spot_price
instance_type = var.instance_type
spot_type = var.spot_type
# block_duration_minutes = 120
wait_for_fulfillment = "true"
key_name = aws_key_pair.ssh_key.key_name
count = "${var.spot_instance == "true" ? 1 : 0}"
security_groups = ["${aws_security_group.ingress-ssh-vps.id}", "${aws_security_group.ingress-http-vps.id}",
"${aws_security_group.ingress-https-vps.id}"]
subnet_id = aws_subnet.subnet-uno.id
}
resource "aws_instance" "web" {
ami = var.instance_ami
instance_type = var.instance_type
key_name = aws_key_pair.ssh_key.key_name
subnet_id = aws_subnet.subnet-uno.id
associate_public_ip_address = true
vpc_security_group_ids = ["${aws_security_group.ingress-ssh-vps.id}", "${aws_security_group.ingress-http-vps.id}",
"${aws_security_group.ingress-https-vps.id}"]
count = "${var.spot_instance == "true" ? 0 : 1}"
}The last file that I am using is outputs.tf, output values make information about your infrastructure available on the command line and can expose information for other Terraform configurations to use. Output values are similar to return values in programming languages.
output "ubuntu_ip" {
value = aws_eip.ip-vps-env.public_ip
description = "Spot intstance IP"
}Executing our Terraform Script to Create the AWS Infrastructure
Now that we have all our scripts created, we should try to make that in AWS. We will start by initializing the project directory, running in a terminal.
$ terraform initTerraform downloads the AWS provider and installs it in a hidden subdirectory of the current working directory. The output shows which version of the plugin was installed. Another helpful command correctly formats and indent the Terraform configuration files:
$ terraform fmtTo verify that our scripts are syntactically correct:
$ terraform validateBefore we create the resources in AWS, we can see the Terraform plan by running:
$ terraform planIf we are happy with the results from terraform plan and want to go ahead and create those changes, we need to run:
terraform applyTerraform State
After we create the AWS resources, Terraform creates a file terraform.tfstate. This file now contains the IDs and properties of the resources Terraform created to manage or destroy those resources, as I will show you. The state file must be stored securely, and it can be shared among team members, although I’d recommend keeping Terraform state remotely, in S3 is a good idea, and it’s supported easily by Terraform. To see the current state of our managed resources, you can run:
$ terraform showAnother helpful command I used during the creation of this tutorial is “taint”, to mark a resource for deletion and force it to recreate when you run terraform apply.
Enhanced Resource Management and Best Practices
As our Terraform skills grow, so does our need for better resource management. Let’s explore some key practices that have become essential since the original post in 2022.
The Power of Tagging
Remember when you lost your keys and spent hours searching? That’s what managing AWS resources without tags feels like. Tags are like labels we stick on our AWS resources, making them easier to find, manage, and understand.
Let’s update our EC2 instance with some helpful tags:
resource "aws_instance" "web" {
# ... existing configuration ...
tags = {
Name = "WebServer-${var.environment}"
Environment = var.environment
Project = "CoolApp"
Owner = "DevOps-Team"
CostCenter = "CC-123"
}
}Now, imagine you’re trying to figure out why your AWS bill is so high. With these tags, you can easily see which projects or teams are using the most resources. It’s like having a detailed receipt instead of just a total at the end of your grocery shopping.
Here’s a real-world scenario: Your finance team asks, “How much are we spending on development environments?” With proper tagging, you can answer in minutes instead of days. You could even set up automated reports or alerts based on these tags.
Don’t forget to tag other resources too. Here’s how you might tag a VPC:
resource "aws_vpc" "main" {
# ... existing configuration ...
tags = {
Name = "MainVPC-${var.environment}"
Environment = var.environment
Project = "CoolApp"
ManagedBy = "Terraform"
}
}Input Variables: Teaching Terraform to Double-Check
We all make mistakes, especially when typing long, complex configurations. That’s where input variable validation comes in handy.
Let’s add some checks to our variables:
variable "environment" {
type = string
description = "Environment (e.g., dev, staging, prod)"
validation {
condition = contains(["dev", "staging", "prod"], var.environment)
error_message = "Oops! Environment must be dev, staging, or prod. Did you make a typo?"
}
}
variable "instance_type" {
type = string
default = "t3.micro"
description = "EC2 instance type"
validation {
condition = can(regex("^t(2|3|3a|4g)\\.", var.instance_type))
error_message = "Hold up! We only use t2, t3, t3a, or t4g instances here. Let's stick to the plan!"
}
}Now, if someone tries to create a “test” environment or use a c5.large instance, Terraform will politely refuse and explain why. It’s like having guardrails on your infrastructure – you can still drive fast, but you’re less likely to drive off a cliff.
Outputs: Sharing the Good News
After Terraform finishes its work, it’s nice to get a summary of what it did. That’s where outputs come in. They’re like a friendly note saying, “Here’s what I did for you!”
Let’s expand our outputs to give us more useful information:
output "instance_id" {
value = aws_instance.web.id
description = "ID of the EC2 instance"
}
output "instance_public_ip" {
value = aws_instance.web.public_ip
description = "Public IP address of the EC2 instance"
}
output "instance_public_dns" {
value = aws_instance.web.public_dns
description = "Public DNS name of the EC2 instance"
}
output "vpc_id" {
value = aws_vpc.main.id
description = "ID of the VPC"
}
output "ssh_command" {
value = "ssh -i ~/.ssh/mykey.pem ubuntu@${aws_instance.web.public_dns}"
description = "Command to SSH into the instance"
}Now, after running terraform apply, you’ll get a neat summary of your new resources.
Scaling Your Terraform Configuration: Modules and Remote State
As your infrastructure grows, you’ll want to keep things organized and collaborative. That’s where modules and remote state come in handy.
Modular Magic: Reusable Infrastructure
Think of modules as recipe cards in your infrastructure cookbook. Instead of writing out the whole recipe every time, you can just say “make the network stuff” and Terraform knows what to do.
Let’s create a module for our network configuration:
resource "aws_vpc" "main" {
cidr_block = var.vpc_cidr
tags = {
Name = "MainVPC-${var.environment}"
Environment = var.environment
}
}
resource "aws_subnet" "public" {
count = length(var.public_subnet_cidrs)
vpc_id = aws_vpc.main.id
cidr_block = var.public_subnet_cidrs[count.index]
availability_zone = var.availability_zones[count.index]
tags = {
Name = "PublicSubnet-${count.index + 1}"
Environment = var.environment
}
}
# ... other network resources ...
output "vpc_id" {
value = aws_vpc.main.id
}
output "public_subnet_ids" {
value = aws_subnet.public[*].id
}Now, in your main configuration, you can use this module like this:
module "network" {
source = "./modules/network"
vpc_cidr = "10.0.0.0/16"
public_subnet_cidrs = ["10.0.1.0/24", "10.0.2.0/24"]
availability_zones = ["us-east-2a", "us-east-2b"]
environment = var.environment
}
resource "aws_instance" "web" {
# ... existing configuration ...
subnet_id = module.network.public_subnet_ids[0]
}This approach makes your code more organized and reusable. It’s like having a set of LEGO blocks – you can quickly build complex structures by combining simple, well-defined pieces.
Remote State: Sharing is Caring
When you’re working in a team, or even just across multiple computers, keeping your Terraform state in sync is crucial. That’s where remote state comes in.
Think of remote state like a shared Google Doc for your infrastructure. Everyone sees the same thing, and changes are synchronized automatically.
Here’s how to set it up with AWS S3 and DynamoDB:
terraform {
backend "s3" {
bucket = "my-terraform-state-bucket"
key = "dev/network/terraform.tfstate"
region = "us-east-2"
encrypt = true
dynamodb_table = "terraform-state-lock"
}
}But before you can use this, you need to create the S3 bucket and DynamoDB table. Here’s a one-time setup you can run:
resource "aws_s3_bucket" "terraform_state" {
bucket = "my-terraform-state-bucket"
lifecycle {
prevent_destroy = true
}
}
resource "aws_s3_bucket_versioning" "terraform_state" {
bucket = aws_s3_bucket.terraform_state.id
versioning_configuration {
status = "Enabled"
}
}
resource "aws_dynamodb_table" "terraform_state_lock" {
name = "terraform-state-lock"
billing_mode = "PAY_PER_REQUEST"
hash_key = "LockID"
attribute {
name = "LockID"
type = "S"
}
}Run this once to set up your remote state infrastructure. After that, you can use the backend configuration in all your projects.
Using remote state has several benefits:
- Team Collaboration: Everyone works with the same state, avoiding conflicts.
- State Backup: S3 versioning keeps a history of your state, letting you roll back if needed.
- State Locking: DynamoDB prevents two people from changing things at the same time, avoiding conflicts.
It’s like having a shared, version-controlled, locked safe for your infrastructure blueprints. No more “But it works on my machine!”
By implementing these practices, you’re not just creating infrastructure – you’re building a scalable, manageable, and collaborative infrastructure development process. It’s like upgrading from a notebook to a smart project management system!
Conclusion
As you can see, Terraform is a powerful tool for automating and managing infrastructure across various cloud providers. We’ve covered the basics of setting up AWS resources, and advanced into best practices like tagging, input validation, and modular design. We’ve also explored how to scale your Terraform usage with remote state management, making it easier to collaborate in teams and manage complex infrastructures.
By leveraging these techniques, you’re not just creating infrastructure – you’re building a scalable, manageable, and collaborative infrastructure development process. This approach allows you to create repeatable environments, reduce human errors, and efficiently manage resources across multiple cloud providers.
Remember, the journey to infrastructure as code is ongoing. As you grow more comfortable with these concepts, continue to explore Terraform’s capabilities and stay updated with the latest best practices in the rapidly evolving world of DevOps and cloud infrastructure management.
