storage-benchmark

Storage Benchmark

(Originally forked from dvassallo/s3-benchmark) (then forked from iternity-dotcom/s3-benchmark)

The performance of a storage system depends on 3 things:

1. Your distance to the storage endpoint.
2. The size of your objects.
3. The number of parallel transfers you can make.

With this tool you can measure the performance of different file and object (S3) operations using different thread counts and object sizes.

Usage

Build

curl https://dl.google.com/go/go1.20.4.linux-amd64.tar.gz -o go1.20.4.linux-amd64.tar.gz rm -rf /usr/local/go && tar -C /usr/local -xzf go1.20.4.linux-amd64.tar.gz export PATH=$PATH:/usr/local/go/bin

go get ... ./go-build-all

Running

e.g. ./build/linux-amd64/storage-benchmark -bucket-name nick-sandbox -region us-east-1

Download

macOS

curl -OL https://github.com/lumafield/storage-benchmark/raw/master/build/darwin-amd64/storage-benchmark

Linux 64-bit x86

curl -OL https://github.com/lumafield/storage-benchmark/raw/master/build/linux-amd64/storage-benchmark

Linux 64-bit ARM

curl -OL https://github.com/lumafield/storage-benchmark/raw/master/build/linux-arm64/storage-benchmark

Windows 64-bit x86

curl -OL https://github.com/lumafield/storage-benchmark/raw/master/build/windows-amd64/storage-benchmark

S3 Credentials

For testing S3 endpoints the tool needs credentials with full S3 permissions. The tool will try to find the credentials from the usual places.

Run

Make the file executable:

chmod +x storage-benchmark

Run the --help command to get an overview of the possible parameters

./storage-benchmark --help

Build

1. Install Go

   sudo apt-get install golang-go
   

   or

   sudo yum install go
   

   may work too.

2. Setup Go environment variables (Usually GOPATH and GOBIN) and test Go installation

3. Clone the repo

4. Install dep

   go get -u github.com/golang/dep/cmd/dep
   

5. Go to source directory and run dep ensure

6. Run go run main.go

License

This project is released under the MIT License.