gocloud.dev: gocloud.dev/blob/gcsblob Index | Examples | Files

package gcsblob

import "gocloud.dev/blob/gcsblob"

Package gcsblob provides a blob implementation that uses GCS. Use OpenBucket to construct a *blob.Bucket.


For blob.OpenBucket, gcsblob registers for the scheme "gs". The default URL opener will creating a connection using use default credentials from the environment, as described in https://cloud.google.com/docs/authentication/production. To customize the URL opener, or for more details on the URL format, see URLOpener. See https://godoc.org/gocloud.dev#hdr-URLs for background information.


Go CDK supports all UTF-8 strings; to make this work with providers lacking full UTF-8 support, strings must be escaped (during writes) and unescaped (during reads). The following escapes are performed for gcsblob:

- Blob keys: ASCII characters 10 and 13 are escaped to "__0x<hex>__".
  Additionally, the "/" in "../" is escaped in the same way.


gcsblob exposes the following types for As:

- Bucket: *storage.Client
- Error: *googleapi.Error
- ListObject: storage.ObjectAttrs
- ListOptions.BeforeList: *storage.Query
- Reader: storage.Reader
- Attributes: storage.ObjectAttrs
- WriterOptions.BeforeWrite: *storage.Writer


ctx := context.Background()

// OpenBucket creates a *blob.Bucket from a URL.
// This URL will open the bucket "my-bucket" using default credentials.
b, err := blob.OpenBucket(ctx, "gs://my-bucket")
defer b.Close()
_, _ = b, err


// Your GCP credentials.
// See https://cloud.google.com/docs/authentication/production
// for more info on alternatives.
ctx := context.Background()
creds, err := gcp.DefaultCredentials(ctx)
if err != nil {

// Create an HTTP client.
// This example uses the default HTTP transport and the credentials created
// above.
client, err := gcp.NewHTTPClient(gcp.DefaultTransport(), gcp.CredentialsTokenSource(creds))
if err != nil {

// Create a *blob.Bucket.
b, err := gcsblob.OpenBucket(ctx, client, "my-bucket", nil)
if err != nil {
defer b.Close()

// Now we can use b to read or write files to the container.
data, err := b.ReadAll(ctx, "my-key")
if err != nil {
_ = data



Package Files



const Scheme = "gs"

Scheme is the URL scheme gcsblob registers its URLOpener under on blob.DefaultMux.

func OpenBucket Uses

func OpenBucket(ctx context.Context, client *gcp.HTTPClient, bucketName string, opts *Options) (*blob.Bucket, error)

OpenBucket returns a *blob.Bucket backed by an existing GCS bucket. See the package documentation for an example.

type Options Uses

type Options struct {
    // GoogleAccessID represents the authorizer for SignedURL.
    // Required to use SignedURL.
    // See https://godoc.org/cloud.google.com/go/storage#SignedURLOptions.
    GoogleAccessID string

    // PrivateKey is the Google service account private key.
    // Exactly one of PrivateKey or SignBytes must be non-nil to use SignedURL.
    // See https://godoc.org/cloud.google.com/go/storage#SignedURLOptions.
    PrivateKey []byte

    // SignBytes is a function for implementing custom signing.
    // Exactly one of PrivateKey or SignBytes must be non-nil to use SignedURL.
    // See https://godoc.org/cloud.google.com/go/storage#SignedURLOptions.
    SignBytes func([]byte) ([]byte, error)

Options sets options for constructing a *blob.Bucket backed by GCS.

type URLOpener Uses

type URLOpener struct {
    // Client must be set to a non-nil HTTP client authenticated with
    // Cloud Storage scope or equivalent.
    Client *gcp.HTTPClient

    // Options specifies the default options to pass to OpenBucket.
    Options Options

URLOpener opens GCS URLs like "gs://mybucket".

The URL host is used as the bucket name.

The following query parameters are supported:

- access_id: sets Options.GoogleAccessID
- private_key_path: path to read for Options.PrivateKey

func (*URLOpener) OpenBucketURL Uses

func (o *URLOpener) OpenBucketURL(ctx context.Context, u *url.URL) (*blob.Bucket, error)

OpenBucketURL opens the GCS bucket with the same name as the URL's host.

Package gcsblob imports 20 packages (graph) and is imported by 5 packages. Updated 2019-03-14. Refresh now. Tools for package owners.