Skip to content

How to monitor memory usage on AWS EC2 ??

How to monitor memory usage on AWS EC2 2022

If you open a panel with EC2 metrics, you’ll notice that there are no metrics for RAM usage. You won’t find them in CloudWatch either. If you don’t have your own monitoring system and you don’t want to log into the virtual machine every time to check memory usage, it is possible for AWS to show you these metrics.

aws memory utilization percent install

By monitoring the use of RAM, we also gain one more very important thing. On their basis, AWS will be able to automatically determine the recommendations of EC2 machines sizes that we should use. Without it, he can only recommend machines based on CPU, disk or network, which is usually not enough.

aws ec2 recommendation 2021

In this article, I will show you step by step what needs to be done to see these memory usage metrics on the AWS platform. This is not a very simple process for AWS beginners as it assumes some familiarity with the platform. However, I will try to make it as understandable as possible. By following the steps in this article one by one, you will surely be able to enable these metrics.

1) The IAM role for EC2

If your EC2 machine has an IAM role assigned, you will have to find it and make sure it has the appropriate permissions – “CloudWatchFullAccess”. If you do not have any role assigned yet, below you will find a description of how to create it and correctly assign it to the EC2 machine.

First, create an IAM role. You will need it to add permissions to it and assign it to the EC2 machine. Search for IAM, select “Roles” from the menu and click on “Create Role”.

AWS - IAM - create role

Great. Now select EC2 and click Next. Because you want it to be seen by your EC2 machines. From there, you can also select containers or other AWS resources that you want to allow to use this role.

AWS - IAM - create role - chose EC2

Now search for the role “CloudWatchFullAccess”, select it and click next.

AWS create role attach permissions

Here you can also add any additional permissions that you would like your EC2 machine with that role assigned to have. For example, you can allow the EC2 machine to access S3 without entering a password. If you are interested in how to do it, then you can skip to this article – Access to private AWS S3 bucket from EC2 without login and password in 5 steps.

Once you are done with the permissions, you can add some TAGs and proceed to the summary. In the last tab, give your role a name and you can add a description to it. I encourage you to add a description and do not have to wonder in a year what this role is for ?

aws create role review

If you have created the role correctly, now you only need to assign it to the EC2 machine. To do this, select the machine from the list of EC2 instances, select it and click on Action/ Security/ Modify IAM role. Then select the created role from the drop-down list and click Save.

AWS - EC2 modify IAM

2) Installing CloudWatch Agent

Below you will find a description of how to install CloudWatch Agent on an Ubuntu machine. If you have a different operating system, go to the AWS documentation and follow the instructions dedicated to your system. Here you will find documentation for CloudWatch. First, log in to the virtual machine, download the appropriate file and execute the following commands:

#Ubuntu
wget https://s3.amazonaws.com/amazoncloudwatch-agent/ubuntu/amd64/latest/amazon-cloudwatch-agent.deb

sudo dpkg -i -E ./amazon-cloudwatch-agent.deb

#Amazon Linux 2 
sudo yum install amazon-cloudwatch-agent

If you want CloudWatch Agent to send information about frame usage every 60 seconds. Then you need to use your favorite editor to create the file /opt/aws/amazon-cloudwatch-agent/bin/config.json. I will use nano for this. Of course, you can set the interval as needed.

CloudWatch Agent can send memory usage information every 60 seconds if you want. You just need to use your favorite editor to create the file /opt/aws/amazon-cloudwatch-agent/bin/config.json with the appropriate content. I will use nano for this. Of course, you can set the interval as needed.

nano /opt/aws/amazon-cloudwatch-agent/bin/config.json

The file should contain:

{
   "metrics":{
      "metrics_collected":{
         "mem":{
            "measurement":[
               "mem_used_percent"
            ],
            "metrics_collection_interval":60
         }
      },
      "append_dimensions": {
        "InstanceId": "${aws:InstanceId}"
      }
   }
}

Now you just need to start CloudWatch Agent with the command below and that’s it.

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c file:/opt/aws/amazon-cloudwatch-agent/bin/config.json -s

If you did everything correctly then you should see the new CWAgent namespace in the Cloudwatch console.

aws namespace cloudwatch cwagent

The important thing is that the namespace in CloudWatch must contain InstanceID!! If this is not the case, we will not receive a recommendation from AWS based on memory usage.

AWS MEMORY UTILIZATION

If there are problems then you can review the logs, which should be in /opt/aws/amazon-cloudwatch-agent/logs/. You can also use one of the commands below to check the status or stop the agent.

With this command you can check the status of the CloudWatch agent:

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a status
Ubuntu CloudWatch agent 2023

To stop the agent, use the command:

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -m ec2 -a stop

If everything is working properly, AWS should be able to make recommendations based on memory usage after a minimum of 24 hours.

AWS ec2 recommendation MEMORY UTILIZATION 2

Summary

All commands made in the article can be automated using IaC, chef puppet etc. automatically run on EC2. However, my aim was to show you a simple way to understand what is being done and why. I encourage you to automate these steps, because if you have a lot of EC2 instances and would like to monitor memory usage on all of them, then you will have a lot of work doing it manually.

Good luck, and finally a small request. If you liked the article, please share it with your friends on social media.

If you want, I can prepare a simple tutorial introducing automation in one of the upcoming posts. Let me know in the comment if you are interested in this topic. Meanwhile, I also encourage you to take a look at other articles in the AWS category.

26 thoughts on “How to monitor memory usage on AWS EC2 ??”

  1. Great post. Please more similiar & how to for automation will also be very usefull.

  2. This does not seem to work.

    Fails quite early on, since there is no CloudWatchFullAccess to select at the creation of a ‘role’ .

    1. Of course it works.
      CloudWatchFullAccess is a policy, not a role 🙂

      If you don’t see this role, make sure you create a role for EC2.

      If you follow the tips in this article carefully, you will definitely succeed. If you have any more problems, let me know.

    1. In a very similar way. It will be difficult to describe it in detail in the commentary, maybe I will write a separate article one day.
      In general, you also create IAM role and connect it to EC2. In the next step you download the agent files, change the configuration. When it’s ready, you install the agent and run it.

      You can also download and install Agent on your EC2 instance using SSM Run Command 😉

  3. The process of monitoring memory usage on AWS EC2 instances is relatively simple and straightforward. By following the steps outlined in this article, you should be able to get a good understanding of your instance’s memory usage and make changes as needed to ensure optimal performance.

  4. Hi,
    I’m facing a problem while starting the aws-cloud-agent. The error shown is:

    /opt/aws/amazon-cloudwatch-agent/bin/config-downloader –output-dir /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d –download-source file:/opt/aws/amazon-cloudwatch-agent/bin/config.json –mode ec2 –config /opt/aws/amazon-cloudwatch-agent/etc/common-config.toml –multi-config default
    2022/10/21 09:27:47 E! Failed to open Common Config: open /opt/aws/amazon-cloudwatch-agent/etc/common-config.toml: no such file or directory
    E! Failed to open Common Config: open /opt/aws/amazon-cloudwatch-agent/etc/common-config.toml: no such file or directory
    Fail to fetch the config!

    I’m using ubuntu 20.04.

  5. Hi I am able to monitor memory usage but I want to monitor multiple server how can I do that?

  6. You can watch multiple servers on CloudWatch. You can even create a dashboard and alerts.
    You can also use external programs like prometheus and grafana or datadog or something else

  7. Shahbaz Ahmad

    Hello Sir
    This Document Amazing but I have one question.
    Sir can you explain if i will create ASG and create Launch Template with running configured server then its required to put this command in user data in launch template

  8. Scripts can be run as you wrote.
    But you can also customize the AMI to your needs and save a new image. In the autoscaling, Launch Template configuration, you provide the image that you modified based on the base image from AWS.
    There are more possibilities, you can also add and configure CloudWatch Agent using Lambda function or SSM .

  9. I really need few live project in aws and devops so i can learn more as i have to start freelancing in devops

  10. How can i setup for multiple servers? Do not want to use grafana and prometheus.
    Need metrics of all servers in single alarm.

  11. Where could i find the concrete command/aws console window to get the memory usage details/AWS recommendations regarding your statement “AWS should be able to make recommendations based on memory usage after a minimum of 24 hours”?

  12. Hi,
    I can’t config CWAgent on EC2 instance, I follow the steps but somehow CWAgent didn’t get the credentials and never connect to CloudWatch.

    warn ec2tagger/ec2tagger.go:485 ec2tagger: Unable to describe ec2 tags for initial retrieval {“kind”: “processor”, “name”: “ec2tagger”, “pipeline”: “metrics/host”, “error”: “NoCredentialProviders: no valid providers in chain\ncaused by: EnvAccessKeyNotFound: failed to find credentials in the environment.\nSharedCredsLoad: failed to load profile

  13. I just repeated all the steps in my blog on a new clean ec2 machine with Ubuntu and it works.

    Please note that the commands will be different for different operating systems.
    The error you described looks like you have a permissions problem. Check if the IAM role you added to ec2 contains the appropriate permissions.

Comments are closed.