When we want to delete AWS S3 bucket, this task seems simple. The easiest way would be to select the bucket to be deleted and click delete. But if it contains files, we will first need to empty our bucket.
However, this is only a theory. In practice, if the bucket is really large, it contains millions of items, the task is not that simple anymore. Fortunately, we have several options for doing this.
1) Delete the content of AWS S3 via UI console
In short, we log in via the website and manually remove objects from S3. If we have several hundred files, that’s cool. Unfortunately, with a few thousand files the time of such an operation is very long, and the whole operation seems to be something terrible.
2) Delete the content of AWS S3 via CLI (S3 RB)
aws s3 rb s3://bucket-name --forceAnother way much better and faster than the previous one. One command and that’s it. The command will easily delete a few hundred or even several thousand files, and then an empty bucket. However, I know from experience that if it works longer, the operation will be interrupted. Often, after an hour, the command stopped working. We will have to repeat the command a few or a dozen times if we have a large number of files.
Command can be used to delete the non-versioned objects in the bucket.
3) Automatic delete the content of AWS S3 via CLI (S3 RM)
aws s3 rm s3://bucket-name/example --recursiveWe can do a little automation and add a S3 cleanup command to the script we run, or we can just use cron. This works much better because the command is run at certain times and subsequent files are deleted. Instead of clicking on the interface, everything is done automatically. When all objects are deleted, then we can delete the empty bucket.
4) Delete the content of S3 via AWS UI console nad use “EMPTY”
We can also select a bucket in our AWS S3 list and click Empty. Make no mistake, the operation cannot be canceled!! AWS itself suggests that if the bucket contains a lot of files, it is better to use the lifecycle rule to remove the content. You can find out how to do this in the next section.
5) Delete the content of S3 via lifecycle policy
A very simple and effective method that can be used without advanced knowledge of AWS. All you need to do is log in to your AWS account, go to the bucket you want to remove and create a lifecycle policy. We check ‘delete the current version of the object‘ and the previous one, if we used ‘Bucket Versioning’. I also mark ‘Delete expired delete markers or incomplete multipart uploads‘.
We specify “1” as the number of days after which AWS is to remove the objects. As a result, all objects that are older than 1 day will be deleted automatically by AWS.
Small note, removing large buckets may take some time as the operation is performed asynchronously. For me, it took 4 days to delete the contents of one large bucket. If you are impatient and have a lot of objects, check your Amazon S3 Bucket Size every day and you will see that it will decrease.
Summary
It is good to choose the method of removing a bucket according to its size and be aware of the advantages and disadvantages of individual solutions. There are actually more ways to remove Amazon S3 content. I chose and described the most practical for me. Let me know in the comment if you know of any other simple and quick way to remove a large bucket and its contents from AWS S3.
I deliberately omit IaC solutions such as Terraform or CloudFormation, because all you need to do there is change the code.
S3 documentation can be found at this link https://docs.aws.amazon.com/s3/index.html.
If you liked the post, check out the other AWS articles that you can find in the AWS category.