Amazon Simple Storage Service (Amazon S3) is Amazon’s object storage service. It has lots of uses and integrates nicely with other AWS services when you need to store data. One thing it’s not meant to be used is publicly sharing your files.
File sharing services such as Box.com, Dropbox.com or Google Drive make public folders very easy. You visit the folder and get a nice web-based user interface that you can use to browse the files and subfolders. Amazon S3, on the other hand, tries its hardest to disallow public access.
Why is public access blocked by default?
Even though it would be convenient to access files publicly in some scenarios, it’s a security vulnerability in most cases. A poorly configured S3 bucket may leak confidential and sensitive documents. It’s a common occurrence that leaky buckets are the root cause of data breaches,
It’s such a lucrative opportunity for hackers that many S3 bucket vulnerability scanners are out there. Take a look at this list to understand how popular it is: Amazon S3 bucket scanners.
So, unless you are absolutely sure you need public access, stick with the default settings and block public access.
How to share and browse files publicly with Amazon S3
First, turn off Block public access settings.
Disabling block public access setting does not automatically make objects public. It opens the possibility of making objects public. After you’ve disabled the block public access, you see the permissions overview changes as shown below:
As the information box tells us, the bucket is not public at the moment but can be made public. You can test this easily by simply uploading a file and trying to access the file URL.
The uploaded file looks like this:
Select the file and click the Copy URL button.
Then open a new browser tab and paste the URL. You should see an access denied error such as this:
The easiest way to make objects publicly readable and listable is to edit the bucket policy.
To do that, scroll down a bit and click the Edit button in the Bucket policy section.
In the Edit bucket policy window, paste the following policy JSON:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "PublicRead",
"Effect": "Allow",
"Principal": "*",
"Action": [
"s3:ListBucket",
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::YOUR_BUCKET_NAME",
"arn:aws:s3:::YOUR_BUCKET_NAME/*"
]
}
]
}
Before you proceed, replace YOUR_BUCKET_NAME with the actual name of your bucket.
After you’ve updated the policy, check the Permission overview section, and you should see this:
Upload a few test objects to your bucket. In my example, my bucket looks like this:
Select a file and click the Copy URL button. Now paste the URL in a new browser tab, and you should see your file’s contents (assuming it’s a file your browser can open like a text file in my example):
As you can see, it now lists the files (key being the file name), MD5 hashes, lengths and last modified dates.
If you create folders in the bucket, they are also listed in the object list. For example, I created a folder named folder-01 and uploaded the same files under that folder, and the refreshed file list looked like this:
This is the easiest way to give your clients access to your buckets. They can read/download the individual files and also get a list of the bucket contents in XML format.
AWS S3 Bucket Explorer
Raw XML is efficient but may not exactly work for you if you are dealing with external clients. They may need a simpler user interface to view the files. There are a few experimental open-source projects out there. Some of them are now defunct; some of them barely work or are very primitive. In my research, I grew fond of this open source project, particularly: AWS S3 Bucket Browser.
The usage is quite simple: You download the template index.html file and update a few settings. Make sure to update the bucket policy and CORS permissions, and you’re all set.
In the following example, the bucket name is test-directory-browsing and the relevant part of index.html looks like this:
My bucket policy is as shown below:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "PublicRead",
"Effect": "Allow",
"Principal": "*",
"Action": [
"s3:ListBucket",
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::test-directory-browsing",
"arn:aws:s3:::test-directory-browsing/*"
]
}
]
}
Finally, my CORS configuration looks like this:
[
{
"AllowedHeaders": [
"*"
],
"AllowedMethods": [
"GET"
],
"AllowedOrigins": [
"https://test-directory-browsing.s3.us-east-2.amazonaws.com"
],
"ExposeHeaders": [
"x-amz-server-side-encryption",
"x-amz-request-id",
"x-amz-id-2"
],
"MaxAgeSeconds": 3000
}
]
When implementing it for your bucket, make sure to replace all instances of test-directory-browsing with your bucket name. Also, change your region if you use a region other than us-east-2.
The contents of the example bucket look like this:
When I copy the URL of the index.html file and paste it into a new browser tab, I get this:
It’s nicely formatted. Since it’s open source, you have full control over the CSS and the images so you can modify them to your liking. When you click a folder, it also shows its contents with the folder name placed on top, such as this:
As you can see, it’s very intuitive and allows your users to browse your public content very easily.
Conclusion
AWS tries their best to lock the buckets and their contents, but in some cases, you still might want to give your users public access to your buckets’ contents. You can achieve this by updating your bucket policy without needing any external tools.
In some scenarios, users may need an easy-to-use user interface to browse through the folders and files in your bucket. If that’s your use case, try out this GitHub repository. After a few quick modifications to your bucket and uploading an index.html file, you can give your customers a good experience browsing your files.
