s3

List a S3 Bucket

An example that demonstrates the use of AWS Go SDK to list a public S3 bucket. There are some public S3 buckets, like those in AWS Open Data Sponsorship Program, that only allows access if you provide your access key id and access secret key when calling the AWS SDK.

These public S3 buckets are the ones I found that allow anonymous access.

• Snowflake Workshop Lab
• Radiant MLHub
• Open Observatory of Network Interference (OONI)

This program's output tries to emulate the output of aws s3 ls as closely as possible.

Notes

While S3 bucket looks like a filesystem, it is rally a flat object storage system. Folders don't exist in S3. Each object, usually a blob object like a file, has a key, which you can namespace it by appending a prefix to the key name. So appending say folder/subfolder/file emulates a typical filesystem.

Setup

1. Run the program.

   $ make run
   

2. We can also compare the result with the output of aws s3 ls s3://snowflake-workshop-lab --no-sign-request if awscli is installed in your system.

Reference

• AWS Open Data Sponsorship Program
• Github: aws-sdk-go-v2
• Wikipedia: multi-byte unit