We usually store a lot of files during our work. Fast internet and data availability support this very much. Unfortunately, a very large part of the data we collect is unnecessary or out of date after some time. Some people view their datasets and periodically delete unnecessary things.
What about the cloud? Since the price for storage has become very low, many of us collect data not only at home, but in the cloud. Why?
Because it is cheap, convenient and we have access to it from anywhere in the world 🙂
Nowadays, we need to filter the data well to find something that is useful to us. If we don’t, we’ll be drowned in a flood of information. With the stubbornness of a maniac, we collect the necessary data, keep it, and after some time it becomes unnecessary and obsolete.
The same applies to logs stored in the cloud. Storing them is also cheap and convenient, only … only logs also have validity. Let’s not keep data longer than we have to. For some systems it will be 30 days, for others it will be 90 days, and for some systems it will be a year or several years, if we are required to do so by law and regulations.
If you have any data in AWS that you would like to automatically delete after a certain period of time, then this article is for you !!
Automatic deletion of data from the entire S3 bucket
On the AWS (Amazon Web Service) platform, we can easily automatically delete data from our S3 bucket. We open Amazon S3 and select one bucket from the list, on which we want to enable automatic deletion of files after a specified time.
Go to Management and click Create lifecycle rule.
We give the name of our rule. Select the option saying that our changes are to apply to all objects and select the checkbox that appears. We must be aware that all objects older than the number of days listed below will be deleted.
If we want the files to be deleted after 30 days, select the “Expire current versions of objects” option and enter the number of days after which the files will be deleted.
In summary, we should see something similar to the picture below. When everything is correct, click on Create rule and our automatic file deletion rule will be ready.
Automatic deletion of data from one folder from S3 bucket
Everything works nicely, but what if we wanted to have different retention set for each group of files? We won’t be creating a few buckets for every flu. Managing so many buckets would be a nightmare. There is an easier way. We can limit the deletion of files to a specific folder or subfolder only.
I strongly recommend that you check this option on a test bucket if you are just learning, or make a copy of the bucket you are implementing it on, just in case.
If you make a mistake, you could lose your data.
If you are sure of what you are doing then you can limit file deletion to a specific folder only.
The whole thing is very similar as before, only the difference is that this time you select the “Limit the scope of this rule using one or more filters” option and enter the name of the folder with / to which the rule is to be created for example ‘folder1/‘.
Summary of Lifecycle rules
These are the basics, we can extend our automation much more. The basics will help you avoid unnecessary expenses and keep order by automatically delete old logs or outdated data from AWS S3 .
If the article has given you some value, or you know someone who might need it, please share it on the Internet. Don’t let him idle on his blog and waste his potential.
I would also be pleased if you write a comment.
If you do not like to spend money unnecessarily, I invite you to read other articles on saving money in the cloud.