How to monitor memory usage on AWS EC2 ??

If you open a panel with EC2 metrics, you’ll notice that there are no metrics for RAM usage. You won’t find them in CloudWatch either. If you don’t have your own monitoring system and you don’t want to log into the virtual machine every time to check memory usage, it is possible for AWS to show you these metrics.

By monitoring the use of RAM, we also gain one more very important thing. On their basis, AWS will be able to automatically determine the recommendations of EC2 machines sizes that we should use. Without it, he can only recommend machines based on CPU, disk or network, which is usually not enough.

In this article, I will show you step by step what needs to be done to see these memory usage metrics on the AWS platform. This is not a very simple process for AWS beginners as it assumes some familiarity with the platform. However, I will try to make it as understandable as possible. By following the steps in this article one by one, you will surely be able to enable these metrics.

1) The IAM role for EC2

If your EC2 machine has an IAM role assigned, you will have to find it and make sure it has the appropriate permissions – “CloudWatchFullAccess”. If you do not have any role assigned yet, below you will find a description of how to create it and correctly assign it to the EC2 machine.

First, create an IAM role. You will need it to add permissions to it and assign it to the EC2 machine. Search for IAM, select “Roles” from the menu and click on “Create Role”.

Great. Now select EC2 and click Next. Because you want it to be seen by your EC2 machines. From there, you can also select containers or other AWS resources that you want to allow to use this role.

Now search for the role “CloudWatchFullAccess”, select it and click next.

Here you can also add any additional permissions that you would like your EC2 machine with that role assigned to have. For example, you can allow the EC2 machine to access S3 without entering a password. If you are interested in how to do it, then you can skip to this article – Access to private AWS S3 bucket from EC2 without login and password in 5 steps.

Once you are done with the permissions, you can add some TAGs and proceed to the summary. In the last tab, give your role a name and you can add a description to it. I encourage you to add a description and do not have to wonder in a year what this role is for ?

If you have created the role correctly, now you only need to assign it to the EC2 machine. To do this, select the machine from the list of EC2 instances, select it and click on Action/ Security/ Modify IAM role. Then select the created role from the drop-down list and click Save.

2) Installing CloudWatch Agent

Below you will find a description of how to install CloudWatch Agent on an Ubuntu machine. If you have a different operating system, go to the AWS documentation and follow the instructions dedicated to your system. Here you will find documentation for CloudWatch. First, log in to the virtual machine, download the appropriate file and execute the following commands:

#Ubuntu
wget https://s3.amazonaws.com/amazoncloudwatch-agent/ubuntu/amd64/latest/amazon-cloudwatch-agent.deb

sudo dpkg -i -E ./amazon-cloudwatch-agent.deb

#Amazon Linux 2 
sudo yum install amazon-cloudwatch-agent

If you want CloudWatch Agent to send information about frame usage every 60 seconds. Then you need to use your favorite editor to create the file /opt/aws/amazon-cloudwatch-agent/bin/config.json. I will use nano for this. Of course, you can set the interval as needed.

CloudWatch Agent can send memory usage information every 60 seconds if you want. You just need to use your favorite editor to create the file /opt/aws/amazon-cloudwatch-agent/bin/config.json with the appropriate content. I will use nano for this. Of course, you can set the interval as needed.

nano /opt/aws/amazon-cloudwatch-agent/bin/config.json

The file should contain:

{
   "metrics":{
      "metrics_collected":{
         "mem":{
            "measurement":[
               "mem_used_percent"
            ],
            "metrics_collection_interval":60
         }
      },
      "append_dimensions": {
        "InstanceId": "${aws:InstanceId}"
      }
   }
}

Now you just need to start CloudWatch Agent with the command below and that’s it.

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c file:/opt/aws/amazon-cloudwatch-agent/bin/config.json -s

If you did everything correctly then you should see the new CWAgent namespace in the Cloudwatch console.

The important thing is that the namespace in CloudWatch must contain InstanceID!! If this is not the case, we will not receive a recommendation from AWS based on memory usage.

If there are problems then you can review the logs, which should be in /opt/aws/amazon-cloudwatch-agent/logs/. You can also use one of the commands below to check the status or stop the agent.

With this command you can check the status of the CloudWatch agent:

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a status

To stop the agent, use the command:

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -m ec2 -a stop

If everything is working properly, AWS should be able to make recommendations based on memory usage after a minimum of 24 hours.

Summary

All commands made in the article can be automated using IaC, chef puppet etc. automatically run on EC2. However, my aim was to show you a simple way to understand what is being done and why. I encourage you to automate these steps, because if you have a lot of EC2 instances and would like to monitor memory usage on all of them, then you will have a lot of work doing it manually.

Good luck, and finally a small request. If you liked the article, please share it with your friends on social media.

If you want, I can prepare a simple tutorial introducing automation in one of the upcoming posts. Let me know in the comment if you are interested in this topic. Meanwhile, I also encourage you to take a look at other articles in the AWS category.

5 ways to remove AWS S3 bucket

Automatically delete old files from AWS S3