bosh-gcscli
A Golang CLI for uploading, fetching and deleting content to/from Google Cloud Storage.
This tool exists to work with the bosh-cli and director.
This is not an official Google Product.
Installation
go get github.com/cloudfoundry/bosh-gcscli
Commands
Usage
bosh-gcscli --help
Upload an object
bosh-gcscli -c config.json put <path/to/file> <remote-blob>
Fetch an object
bosh-gcscli -c config.json get <remote-blob> <path/to/file>
Delete an object
bosh-gcscli -c config.json delete <remote-blob>
Check if an object exists
bosh-gcscli -c config.json exists <remote-blob>
Generate a signed url for an object
If there is an encryption key present in the config, then an additional header is sent
bosh-gcscli -c config.json sign <remote-blob> <http action> <expiry>
Where:
<http action>
is GET, PUT, or DELETE
<expiry>
is a duration string less than 7 days (e.g. "6h")
Configuration
The command line tool expects a JSON configuration file. Run bosh-gcscli --help
for details.
Authentication Methods (credentials_source
)
static
: A service account key will be provided via the json_key
field.
none
: No credentials are provided. The client is reading from a public bucket.
- <empty>: Application Default Credentials
will be used if they exist (either through
gcloud auth application-default login
or a service account).
If they don't exist the client will fall back to none
behavior.
Running Integration Tests
- Ensure gcloud is installed and you have authenticated (
gcloud auth login
).
These credentials will be used by the Makefile to create/destroy Google Cloud Storage buckets for testing.
- Set the Google Cloud project:
gcloud config set project <your project>
- Generate a service account with the
Storage Admin
role for your project and set the contents as
the environment variable GOOGLE_APPLICATION_CREDENTIALS
, for example:
export project_id=$(gcloud config get-value project)
export service_account_name=bosh-gcscli-integration-tests
export service_account_email=${service_account_name}@${project_id}.iam.gserviceaccount.com
credentials_file=$(mktemp)
gcloud config set project ${project_id}
gcloud iam service-accounts create ${service_account_name} --display-name "Integration Test Access for bosh-gcscli"
gcloud iam service-accounts keys create ${credentials_file} --iam-account ${service_account_email}
gcloud project add-iam-policy-binding ${project_id} --member serviceAccount:${service_account_email} --role roles/storage.admin
export GOOGLE_SERVICE_ACCOUNT="$(cat ${credentials_file})"
export GOOGLE_APPLICATION_CREDENTIALS="$(cat ${credentials_file})"
export LC_ALL=C # fix `tr` complaining about "illegal byte sequence" on OSX
- Run the unit and fast integration tests:
make test-fast-int
- Clean up buckets:
make clean-gcs
Development
- A Makefile is provided that automates integration testing. Try
make help
to get started.
- gvt is used for vendoring.
Contributing
For details on how to contribute to this project - including filing bug reports and contributing code changes - please see CONTRIBUTING.md.
License
This tool is licensed under Apache 2.0. Full license text is available in LICENSE.