How to setup Amazon ECS Fargate using Terraform

Amazon ECS Fargate is one of the two options for running container-based applications in the AWS Cloud (often also called container orchestration). ECS Fargate is the more luxurious option, so to speak, since the complete management of the underlying infrastructure such as EC2 Instances or Autoscaling Groups is taken over by Amazon. ECS Fargate is therefore often referred to as serverless (although technically there is still a lot of backend infrastructure behind it).

We have already looked at the much more complex configuration of ECS on EC2 in another article and will now focus here on an equivalent, production-grade setup based on ECS Fargate. We therefore include all important security best practices, scalability as well as operational effort in our considerations. To enable working with modern and efficient development methods like trunk-based development and Continuous Integration and Continuous Deployment, we use without exception Infrastructure-as-Code with Terraform for provisioning infrastructure resources.

The Terraform code samples in this article are limited to the changes from the EC2-based version of ECS. It is therefore highly recommended that readers should study both articles to build a solid understanding of the technical background. You can find all code sources for the ECS Fargate setup in the corresponding GitHub repository as working example in production-grade quality. To clone the repository, run the following command and take a look into the README file:

git clone git@github.com:nexgeneerz/aws-scenario-ecs-fargate.git

There can be a cost to building infrastructure in AWS. All resources should therefore be removed again after trying them out, which can easily be done with a simple terraform destroy.

At nexgeneerz.io, we support startups and SME’s in building their cloud-native infrastructure to enable maximum scalability, security and flexibility with full cost control. In this context, ECS Fargate has become a popular functionality in recent years, and is being used more and more frequently for applications of various sizes. In this series of articles on cloud computing in AWS, we will therefore highlight all the different technologies available in the Amazon Cloud.

What is Amazon ECS?

Amazon ECS is a managed platform for orchestrating Docker containers. It runs on clusters of Virtual Machines (VM) within the AWS Cloud and manages (deploys, scales, and schedules) these instances within different Availability Zones and in the selected AWS Regions. The management of EC2 Instances can either be done independently or completely abstracted with ECS Fargate. Your applications run on instances that come pre-installed with Docker.

ECS offers a programmatically manageable platform with further integratable extras such as log management via CloudWatch or a fine-grained permission system via IAM. Amazon ECS can be used for small, medium and large workloads and scales effortlessly and on-demand.

As the name suggests, Amazon ECS is a solution specialized for the AWS Cloud. So if you need a multi-cloud strategy for your application or consider portability to other cloud providers to be necessary, you will have to calculate with additional effort for ECS, which is lower with Kubernetes.

Terms and their meaning

In the universe of ECS, you come into contact with any number of terms specific to certain parts of this technology. The most important terms are the following:

ECS Cluster

An ECS Cluster is the superstructure for the entire setup in ECS. A cluster contains multiple EC2 Instances or Container Instances (also of different types like On-demand Instances or Spot Instances). An ECS Cluster contains one or more ECS Services.

ECS Service

An ECS Service is responsible for a certain number of ECS Tasks and manages their execution within an ECS Cluster on the associated Container Instances. In the simple case, an ECS Service can be seen as a (micro) service, i.e. an application or a web service that is executed with a defined redundancy.

ECS Task

An ECS Task is de facto equivalent to an executable Docker container, which typically runs an instance of an application. The associated ECS Service defines how many instances of an ECS Task should be executed. Each ECS Task uses a specific Task Definition, which in turn is linked to a tagged Docker Image.

ECS Task Definition

An ECS Task Definition is the definition of the associated Docker image with information about host port, container port, mounted drives, log configurations or CPU and memory. A new Task Revision is created with each deployment, which may include a revised Task Definition.

Container Instance

A Container Instance is the synonym for an EC2 Instance, i.e. a Virtual Machine that serves as a runtime environment for ECS Clusters and thus ECS Tasks. An EC2 Instance is only referred to as a Container Instance in the context of an ECS Cluster.

ECS Container Agent

The ECS Container Agent allows Container Instances to connect to the ECS Cluster. The ECS Container Agent is included in all ECS-optimized AMI’s, but can also be installed manually on any compatible operating system (OS).

Amazon ECS – the golden standard for Startups?

At this point, I would like to make a brief digression on the topic of overengineering. Particularly in the startup sector, it is not uncommon to observe how even the simplest technical solutions and applications are deployed and operated with a massive overflow of technology. A typical business model, which is based on either a single app or a setup consisting of one or two handfuls of services, can of course technically be operated with a technology like Kubernetes. The burning question, however, is whether it has to be. From the perspective of a technical leader, you should therefore also think very carefully about the medium- and long-term commitments and efforts involved, both in terms of personnel and costs.

Amazon ECS offers a cost-effective, scalable technology that can be operated with little to medium knowledge, especially for the first years, when a startup is primarily concerned with the (further) development of their business model. This technology enables a typically small development team to invest valuable human resources in the implementation of functionality instead of infrastructure that is invisible to the end user. The topic of cloud infrastructure for Startups is therefore far from being just a technological decision in the background, but can have enormous consequences in terms of personnel costs or development speed and time-to-market. The nasty thing is that the decisions are quite heavy and, for example, after the decision for Kubernetes, switching to another technology can be extremely time-consuming – but keeping it can be an equally drastic financial burden.

ECS Fargate as the serverless version of EC2

If you compare the required infrastructure between ECS on Fargate and ECS on EC2, you will immediately notice that all components such as EC2 Launch Templates, Autoscaling Groups, or Capacity Providers are completely omitted. The background is that these resources are abstracted when ECS Fargate is used by AWS and therefore do not need to be configured, provisioned and maintained by us.

So our target architecture looks much simpler compared to ECS EC2:

In our example scenario, we create the presentation tier for a multi-tier setup, but without going into components such as databases or the like. We create a service that is designed for high availability and should therefore be available in several Availability Zones (AZ). We will also ensure the scalability on ECS Service level with the target tracking method.

The technical differences to ECS on EC2

Since the vast majority of Terraform infrastructure resources are identical to our already described version of ECS on EC2, we will refrain from mentioning all components and code blocks again in this article. The complete code can of course be found in the corresponding GitHub repository for this example.

The changes are exactly the following (the complete code examples follow):

• The resource aws_ecs_service is extended by the launch type FARGATE.
• The resource aws_ecs_task_definition needs some additional information like CPU and memory.
• We need a different aws_security_group for the ECS container instance compared to ECS on EC2.
• The target_type for the ALB Target Group must be set to ip instead of instance
• The port of the ALB Target Group must match the container port 3000 instead of port 80

The following terraform resources are completely omitted:

• The key pair aws_key_pair.default is omitted because the management of the EC2 instances is handled by AWS and therefore we cannot get access via SSH.
• ECS Fargate does not allow to select the instance type. An AMI aws_ami.amazon_linux_2 as well as the launch template aws_launch_template.ecs_launch_template are therefore no longer needed.
• The user_data.sh script is also no longer needed due to the omission of the configuration of EC2 instances.
• The resources aws_iam_role.ec2_instance_role, aws_iam_role_policy_attachment.ec2_instance_role_policy, aws_iam_instance_profile.ec2_instance_role_profile and aws_iam_policy_document.ec2_instance_role_policy are not needed as they are only required for EC2 instances.
• The same is true for the ECS Service IAM Role as well as Policies: aws_iam_roleecs_service_role, aws_iam_policy_document.ecs_service_policy, aws_iam_role_policy.ecs_service_role_policy and aws_iam_policy_document.ecs_service_role_policy are maintained by AWS.
• The capacity provider resources aws_ecs_capacity_provider.cas and aws_ecs_cluster_capacity_providers.cas are not required.
• Similarly, the autoscaling group aws_autoscaling_group.ecs_autoscaling_group is not required as this part of the infrastructure is also maintained by AWS.

All resources not listed here can be used exactly as they are configured for the EC2-based version. Now let’s take a closer look at the resources that are changing.

The ECS Service

The obvious change to this resource is the launch_type for Fargate. Also, the network_configuration block is new. Since we use awsvpc network mode for ECS task definition, we need to configure this block so that these tasks get their own Elastic Network Interface (ENI). Here we also specify our Private Subnet IDs so that even though we let AWS manage the infrastructure underneath, we can be sure that the tasks are running on a network that is not accessible from the Internet. An optional but still important parameter is the Security Groups (SG) used for the ECS Task Container Instance. We will go into more detail about this SG, which has changed compared to EC2, in a moment.

## Creates an ECS Service running on Fargate

resource "aws_ecs_service" "service" {
  name                               = "${var.namespace}_ECS_Service_${var.environment}"
  cluster                            = aws_ecs_cluster.default.id
  task_definition                    = aws_ecs_task_definition.default.arn
  desired_count                      = var.ecs_task_desired_count
  deployment_minimum_healthy_percent = var.ecs_task_deployment_minimum_healthy_percent
  deployment_maximum_percent         = var.ecs_task_deployment_maximum_percent
  launch_type                        = "FARGATE"

  load_balancer {
    target_group_arn = aws_alb_target_group.service_target_group.arn
    container_name   = var.service_name
    container_port   = var.container_port
  }

  network_configuration {
    security_groups  = [aws_security_group.ecs_container_instance.id]
    subnets          = aws_subnet.private.*.id
    assign_public_ip = false
  }

  lifecycle {
    ignore_changes = [desired_count]
  }
}

The ECS Task Definition for the Fargate Environment

The ECS task definition also has many overlaps with the EC2 version. Nevertheless, the network_mode is important here on the one hand, as well as the compatibility with Fargate. The configurations for CPU and memory represent a change, which must be configured for Fargate not only within the container definition, but also at task definition level.

The host port for Fargate must use the same port as the container port. This is different from EC2, where one of the Ephemeral Ports is automatically selected by AWS for the Host Port. However, if we use ECS Fargate and our container port is port 3000, the host port must also be set to port 3000. This change is then also the transition to the change of the Security Group, which we will see in the next paragraph.

## Creates ECS Task Definition

resource "aws_ecs_task_definition" "default" {
  family                   = "${var.namespace}_ECS_TaskDefinition_${var.environment}"
  network_mode             = "awsvpc"
  requires_compatibilities = ["FARGATE"]
  execution_role_arn       = aws_iam_role.ecs_task_execution_role.arn
  task_role_arn            = aws_iam_role.ecs_task_iam_role.arn
  cpu                      = var.cpu_units
  memory                   = var.memory

  container_definitions = jsonencode([
    {
      name         = var.service_name
      image        = "${aws_ecr_repository.ecr.repository_url}:${var.hash}"
      cpu          = var.cpu_units
      memory       = var.memory
      essential    = true
      portMappings = [
        {
          containerPort = var.container_port
          hostPort      = var.container_port
          protocol      = "tcp"
        }
      ]
      logConfiguration = {
        logDriver = "awslogs",
        options   = {
          "awslogs-group"         = aws_cloudwatch_log_group.log_group.name,
          "awslogs-region"        = var.region,
          "awslogs-stream-prefix" = "${var.service_name}-log-stream-${var.environment}"
        }
      }
    }
  ])
}

The ECS Container Instance Security Group

To keep the network security of the container instances as high as possible, we restrict the allowed network traffic to the origin of the application load balancer on port 3000. This makes it impossible to call the ALB directly and bypass the CloudFront distribution. Here we are also helped by the Custom Origin Header that must be passed between CloudFront and the ALB.

## Security Group for ECS Task Container Instances (managed by Fargate)

resource "aws_security_group" "ecs_container_instance" {
  name        = "${var.namespace}_ECS_Task_SecurityGroup_${var.environment}"
  description = "Security group for ECS task running on Fargate"
  vpc_id      = aws_vpc.default.id

  ingress {
    description     = "Allow ingress traffic from ALB on HTTP only"
    from_port       = var.container_port
    to_port         = var.container_port
    protocol        = "tcp"
    security_groups = [aws_security_group.alb.id]
  }

  egress {
    description = "Allow all egress traffic"
    from_port   = 0
    to_port     = 0
    protocol    = -1
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags = {
    Name     = "${var.namespace}_ECS_Task_SecurityGroup_${var.environment}"
  }
}

Target Group configuration

When we create a Target Group for an ECS Service with Fargate, we need to set the target_type to ip. Remember: the Target Group is the link between the Application Load Balancer and the actual service and is the “output” of the ALB, while the ALB Listener receives the network traffic.

The second change is about the Target Group port, which is set to the port of the container (3000 in our example). With Fargate, both containerPort and hostPort in the ECS Task definition have to be set to the same value. In order to route our traffic to the target, we need to update the Target Group with the same port.

## Creates the Target Group for our service

resource "aws_alb_target_group" "service_target_group" {
  name                 = "${var.namespace}-TargetGroup-${var.environment}"
  port                 = var.container_port
  protocol             = "HTTP"
  vpc_id               = aws_vpc.default.id
  deregistration_delay = 5
  target_type          = "ip"

  health_check {
    healthy_threshold   = 2
    unhealthy_threshold = 2
    interval            = 60
    matcher             = var.healthcheck_matcher
    path                = var.healthcheck_endpoint
    port                = "traffic-port"
    protocol            = "HTTP"
    timeout             = 30
  }

  depends_on = [aws_alb.alb]
}

These are the only changes and additions that need to be made in the resources known from EC2. With this code base we can now start our tutorial service and get to know ECS Fargate practically.

When to prefer ECS Fargate over EC2

ECS Fargate and ECS EC2 are very similar in their setup. Both systems are professional cloud computing environments designed for scale and high availability that can be used without compromising security, extensibility or maintainability. However, there are a few differences that you should be aware of for professional use.

ECS Fargate is clearly the priority in terms of ease of use as well as operational effort. The entire effort for setting up and maintaining the EC2-based infrastructure as well as the autoscaling group and other components for scalability is completely eliminated. Since this substructure also brings a corresponding operational backpack, ECS Fargate is clearly recommended for use cases in which the time and/or knowledge around cloud computing technologies such as EC2 are not available.

There are also some factors to consider on the cost side. As a rule of thumb, it can be said that ECS Fargate is worthwhile for small to medium workloads that do not exceed a utilization of the entire ECS cluster of around 50%. This value is of course not fixed and depends on other factors. From a higher utilization, however, you should check whether a switch to ECS EC2 is not the more cost-effective option. However, don’t forget to also take into account the time savings due to the elimination of operational overhead.

At the same time, this fact is also a potential contra criterion, because the elimination of the EC2 infrastructure and the use of the serverless version also eliminates the flexibility and configurability of this layer. So anyone who is dependent on the use of GPUs, for example, or who has to integrate EBS volumes or wants to start a service as a daemon will not have the most suitable tool with ECS Fargate.

From a software engineer’s perspective, it should not be hidden that ECS Fargate offers limited capabilities to debug non-booting Docker images directly on the EC2 instance. While it is possible with ECS EC2 to log in to the instance in question via SSH and perform a precise error analysis, finding the error with Fargate can degenerate into a lengthy search.

Conclusion

For those looking for a simple, scalable solution for container-based cloud computing and where Kubernetes is not an option due to its much higher operational price, it is clear that AWS ECS Fargate is worth a closer look. Amazon has created a highly professional environment with this technology that can take a lot of headaches off teams while being fully compatible with modern software development techniques and tools, such as CI/CD, trunk-based development, infrastructure-as-code, and a setup designed for full automation.

It is such technologies as ECS that allow interdisciplinary teams to quickly and easily implement complex business models while spending as little time as possible on the operational running of the foundation.

And that’s it so far. 🎉

I hope you enjoyed this article and that it helped you understand ECS Fargate. Cloud infrastructure can sometimes be a bit overwhelming in its complexity, but now you should hopefully have a good overview of fully automated deployments of Amazon ECS Fargate with Terraform.

Remember that there are many other ways to configure ECS Fargate. We always try to find solutions that are as lean and use as few technologies as possible, but of course there are many alternatives.

Don’t hesitate to reach out to us if you have any questions or feedback. We’d love to hear from you! You can also connect with us on LinkedIn to stay up to date about further articles. Let’s stay in touch and learn from each other! 🚀