Containers

ECS Anywhere: Run AWS Containers on your Home Server On-premises!

Discover the power of AWS ECS Anywhere and learn how to spin up containers on top of on-premises VMs. Exploring this service allows you to use your own infrastructure architecture and run software clusters.

So many services are either “cloud only” or “only on-premises” technologies. However, what if you wanted a blend of both running in your home lab or in production when it comes to running your containers? If you haven’t heard about it before, AWS ECS Anywhere is just such a service that allows you to house the management plane for your containers in the AWS cloud while you have the actual containers running on-premises in a virtualization stack. Let’s take a look at ECS Anywhere and how we can use it to spin up containers on top of on-premises VMs and run software clusters using your own infrastructure architecture.

What is ECS Anywhere?

It is part of Amazon Elastic Container Service (Amazon ECS, including managed instance configurations and AWS Fargate) that is purpose-built to extend its capabilities beyond the confines of the AWS cloud and EC2 instances, all without AWS Outposts. If you remember Outposts (not that great of adoption), it is the service that allowed organizations to run AWS on-premises.

However, this is not needed with Amazon ECS Anywhere. Instead, it allows using the Amazon ECS control plane for ECS in the AWS cloud and have your container workloads running in your own environment as an external instance. This can be a great solution for businesses who may need containers to run adjacent to other backend on-premises servers like database servers on-premises, or who for various reasons, like compliance, efficiency, and the need to keep all the data locally, including data running in their containers.

Helps when AWS regions are lacking

Also, what if Amazon does not have an AWS region in the edge location that would be geographically optimal for your application? ECS Anywhere helps to solve this challenge since you can run it in your own data center.

Using ECS Anywhere, developers and DevOps alike can deploy applications to their on-premises container hosts from the cloud. This offsets the cost of compute capacity in AWS, since it runs on-premises on your own hardware.

Simplified Management Across Environments

One of the features of ECS Anywhere is the simplified management experience. Admins can monitor and manage their container instances across AWS and on-premises environments from a single pane of glass from the Amazon ECS Console.

1. Installing ECS Anywhere

One of the things I like about ECS Anywhere is getting started involves a few steps. The initial setup includes registering your external instances (such as virtual machines or physical servers) with the ECS control plane, installing the ECS agent, and configuring network connectivity.

Prerequisites

Before diving into the installation process, ensure that your environment meets the necessary prerequisites. This includes having the AWS CLI installed, an active AWS account, and the SSM Agent running on your external instances. Supported operating systems for ECS Anywhere include Amazon Linux 2, Ubuntu, and RHEL, among others.

Also, make sure you have updated your Linux distro. Here I am updating my Ubuntu Server 22.04 LTS instance:

sudo apt-get update && apt-get upgrade -y

Also, I am using a Windows workstation with WSL installed with the AWS CLI installed.

2. Export environment variables

The first thing we need to do is update the environment variables to work with the ECS cluster. This assumes you have already setup your AWS CLI tools and you have ran an aws configure which will ask for your AWS ID and secret key.

After connecting your AWS CLI to your AWS environment, run the following export commands. As a note, the ROLE_NAME, CLUSTER_NAME, and SERVICE_NAME can be named anything you want them to be. These don’t have to be anything specific. But as a best practice, make them intuitve.

export AWS_DEFAULT_REGION=us-east-1
export ROLE_NAME=ECSAnyWhereRole
export CLUSTER_NAME=cloudlocal-ecs-anywhere
export SERVICE_NAME=cloudlocal-ecs-anywhere-svc
Export the environment variables in linux
Export the environment variables in linux

3. Create the AWS IAM role

The next thing we need to do is create the AWS IAM role to work with the ECS environment. To create the IAM role, we need to create a file called ssm-trust-policy.json file with the following contents:

{
"Version": "2012-10-17",
"Statement": {
"Effect": "Allow",
"Principal": {"Service": [
"ssm.amazonaws.com"
]},
"Action": "sts:AssumeRole"
}
}
Creating an iam role
Creating an iam role

Next, we run the following aws cli commands:

aws iam create-role --role-name $ROLE_NAME --assume-role-policy-document file://ssm-trust-policy.json

aws iam attach-role-policy --role-name $ROLE_NAME --policy-arn arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore

aws iam attach-role-policy --role-name $ROLE_NAME --policy-arn arn:aws:iam::aws:policy/service-role/AmazonEC2ContainerServiceforEC2Role

# Verify
aws iam list-attached-role-policies --role-name $ROLE_NAME
Attaching a role policy to a role for ecs anywhere
Attaching a role policy to a role for ecs anywhere

4. Create the AWS ECS Cluster

Now that we have the AWS IAM roles in place, we can create the AWS ECS cluster.

aws ecs create-cluster --cluster-name $CLUSTER_NAME
Creating an ecs anywhere cluster
Creating an ecs anywhere cluster

If you log into your AWS console and head over to the ECS dashboard, you should see your cluster:

Viewing the newly created ecs anywhere cluster in the aws console
Viewing the newly created ecs anywhere cluster in the aws console

5. Create the cluster activation key

Now, we need to create the activation key pair that includes the activation ID and activation code.

aws ssm create-activation --iam-role $ROLE_NAME | tee ssm-activation.json
Create the cluster activation json
Create the cluster activation json

6. Install the ECS anywhere agent

Now that we have the cluster activation information, we can now install the ECS anywhere agent.

#Run the commands to install the agent

export ACTIVATION_ID=<your activation ID>
export ACTIVATION_CODE=<your activation code>

# Download the ecs-anywhere install Script
curl -o "ecs-anywhere-install.sh" "https://amazon-ecs-agent-packages-preview.s3.us-east-1.amazonaws.com/ecs-anywhere-install.sh" && sudo chmod +x ecs-anywhere-install.sh

# (Optional) Check integrity of the shell script
curl -o "ecs-anywhere-install.sh.sha256" "https://amazon-ecs-agent-packages-preview.s3.us-east-1.amazonaws.com/ecs-anywhere-install.sh.sha256" && sha256sum -c ecs-anywhere-install.sh.sha256

# Run the install script
sudo ./ecs-anywhere-install.sh \
--cluster <your ecs cluster name> \
--activation-id $ACTIVATION_ID \
--activation-code $ACTIVATION_CODE \
--region us-east-1
Installing the ecs anywhere agent
Installing the ecs anywhere agent

Below, you can see the External instance has been registered.

Viewing the external ecs anywhere container instances in the aws console
Viewing the external ecs anywhere container instances in the aws console

7. Check the AWS ECS agent installation

You can use the following commands to check the installation of the ECS agent:

aws ssm describe-instance-information
aws ecs list-container-instances --cluster $CLUSTER_NAME
Describing the ecs anywhere host information
Describing the ecs anywhere host information

8. Create a new AWS ECS Anywhere container task definition

Now that we have the infrastructure in place, we can create a new task definition. 

Task definitions are a JSON-formatted text file that outlines one to ten containers, making up the components of your application. It serves as a blueprint, detailing the configuration of your application. 

This includes specifying the containers that will be used, including the container image, the launch type, the ports, and the data volumes associated with the containers in the task. The range of parameters that can be defined within a task definition is determined by the chosen launch type.

You will note in the below, the task definition for ECS Anywhere is configured as EXTERNAL. Save the following on your management workstation that has the AWS CLI tools installed as task-definition.json. It creates a simple nginx container with port 8080 exposed on the container host.

{
  "requiresCompatibilities": [
    "EXTERNAL"
  ],
  "containerDefinitions": [
    {
      "name": "nginx",
      "image": "nginx:latest",
      "memory": 256,
      "cpu": 256,
      "essential": true,
      "portMappings": [
        {
          "containerPort": 80,
          "hostPort": 8080,
          "protocol": "tcp"
        }
      ]
    }
  ],
  "networkMode": "bridge",
  "family": "nginx"
}
Viewing the nginx task definition for the ecs anywhere cluster
Viewing the nginx task definition for the ecs anywhere cluster

9. Register the new task definition with the AWS ECS Anywhere cluster

#Register the task definition
aws ecs register-task-definition --cli-input-json file://external-task-definition.json

#Run the task
aws ecs run-task --cluster $CLUSTER_NAME --launch-type EXTERNAL --task-definition nginx

#Get the Task ID
TEST_TASKID=$(aws ecs list-tasks --cluster $CLUSTER_NAME | jq -r '.taskArns[0]')

#Verify Task is Running
aws ecs describe-tasks --cluster $CLUSTER_NAME --tasks $TEST_TASKID
Registering the ecs anywhere task definition
Registering the ecs anywhere task definition

10. Check the running containers on your Docker host

Now that we have created and registered the new task definition, we can check the Docker host and see if the nginx container has been spun.docker ps

Awesome! We see the new nginx container runnning and we also can see the Amazon ECS Anywhere agent container running.

Viewing the docker containers running on the ecs anywhere docker container host
Viewing the docker containers running on the ecs anywhere docker container host

11. Check the AWS ECS cluster tasks

We can jump back to the AWS console and check the AWS ECS Cluster tasks.

Viewing the running task in the ecs anywhere cluster
Viewing the running task in the ecs anywhere cluster

Other things to note

ECS Anywhere not only simplifies container management across diverse environments but also integrates seamlessly with various AWS services. This includes AWS Lambda for serverless computing, Amazon Elastic Container Registry (ECR) for storing container images, and Amazon CloudWatch for monitoring and logging. So in other words, it opens up a large part of the AWS catalog you can run on-premises that otherwise you wouldn’t have access to unless running natively in AWS.

Cost

What does it cost? It is important to note that ECS Anywhere does have a cost associated with it. However, in terms of playing around with it and running a single instance in your home lab, it isn’t much. First, what is an instance? An instance is a customer-managed instance, which really equates to your Docker host, that has been registered with Amazon and is running the ECS Anywhere agent. So it is not each container.

The price is calculated based on the number of hours ECS Anywhere is managing an on-premises instance, with a minimum charge of 1 minute per instance. The price is $0.01025 per managed instance.

Amazon notes the following as an example:

  • 10 on-premises instances that are connected to the Amazon ECS cluster for 30 days continuously
  • Total fee for ECS Anywhere = 10 on-premises instances x 30 days x 24 hours x $0.01025 instance hour = $73.80

Scalability and Security

With ECS Anywhere, it has tremendous scalability and security. AWS’s security model extends to protect your container workloads. You can use AWS IAM roles and policies to manage access and leverage AWS’s infrastructure for secure, scalable container deployments.

Home Lab Implementations

ECS Anywhere is exceptionally cool for a home lab environments, and allows tech enthusiasts and professionals to run AWS containers on their own hardware. If you are experimenting with new applications or testing deployments, ECS Anywhere brings AWS container orchestration into your home lab or on-premises production environment.

Key Benefits and Features

You can run containers on-premises with the same ease as in the cloud which has benefits to efficienty, flexibility, and consistency in management and operations. Many operating systems are supported and the ECS extended capabilities ensure a seamless transition of container workloads between environments.

Considerations

Using your own infrastructure means there are several considerations to be made in terms of backup and disaster recovery of compute, network, and storage infrastructure. Reliability and availability depend on the same mechanisms that must be considered with any other on-premises workloads. So, you will need to protect your Docker hosts as you would other virtual machines running in production. 

Wrapping up

Amazon ECS Anywhere is a great way to have a single management platform, without the need to run full-blown Kubernetes environments. You can keep your containers deployment close to the data they need to pull from, or if you are limited in an AWS region that makes sense, you can place the ECS hosts locally in the environment without the added latency and degraded performance of needing to place the containers in an AWS region far away. Also, there is just something really cool about having your containers running in your home lab show up in the AWS console in the cloud. 

Subscribe to VirtualizationHowto via Email 🔔

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Brandon Lee

Brandon Lee is the Senior Writer, Engineer and owner at Virtualizationhowto.com and has over two decades of experience in Information Technology. Having worked for numerous Fortune 500 companies as well as in various industries, Brandon has extensive experience in various IT segments and is a strong advocate for open source technologies. Brandon holds many industry certifications, loves the outdoors and spending time with family.

Related Articles

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.