beam: github.com/apache/beam/sdks/go/pkg/beam/util/gcsx Index | Examples | Files

package gcsx

import "github.com/apache/beam/sdks/go/pkg/beam/util/gcsx"

Package gcsx contains utilities for working with Google Cloud Storage (GCS).

Code:

ctx := context.Background()
c, err := gcsx.NewClient(ctx, storage.ScopeReadOnly)
if err != nil {
    // do something
}

buckets, object, err := gcsx.ParseObject("gs://some-bucket/some-object")
if err != nil {
    // do something
}

ctx, cancel := context.WithTimeout(ctx, 30*time.Second)
defer cancel()

bytes, err := gcsx.ReadObject(ctx, c, buckets, object)
if err != nil {
    // do something
}

_ = bytes

Index

Examples

Package Files

gcs.go

func BucketExists Uses

func BucketExists(ctx context.Context, client *storage.Client, bucket string) (bool, error)

BucketExists returns true iff the given bucket exists.

func CreateBucket Uses

func CreateBucket(ctx context.Context, client *storage.Client, project, bucket string) error

CreateBucket creates a bucket in GCS.

func Join Uses

func Join(object string, elms ...string) string

Join joins a GCS path with an element. Preserves the gs:// prefix.

func MakeObject Uses

func MakeObject(bucket, path string) string

MakeObject creates a object location from bucket and path. For example, MakeObject("foo", "bar/baz") returns "gs://foo/bar/baz". The bucket must be non-empty.

func NewClient Uses

func NewClient(ctx context.Context, scope string) (*storage.Client, error)

NewClient creates a new GCS client with default application credentials, and supplied OAuth scope. The OAuth scopes are defined in https://godoc.org/cloud.google.com/go/storage#pkg-constants.

func NewUnauthenticatedClient Uses

func NewUnauthenticatedClient(ctx context.Context) (*storage.Client, error)

NewUnauthenticatedClient creates a new GCS client without authentication.

func ParseObject Uses

func ParseObject(object string) (bucket, path string, err error)

ParseObject deconstructs a GCS object name into (bucket, name).

func ReadObject Uses

func ReadObject(ctx context.Context, client *storage.Client, bucket, object string) ([]byte, error)

ReadObject reads the content of the given object in full.

func Upload Uses

func Upload(ctx context.Context, client *storage.Client, project, bucket, object string, r io.Reader) (string, error)

Upload writes the given content to GCS. If the specified bucket does not exist, it is created first. Returns the full path of the object.

func WriteObject Uses

func WriteObject(ctx context.Context, client *storage.Client, bucket, object string, r io.Reader) error

WriteObject writes the given content to the specified object. If the object already exist, it is overwritten.

Package gcsx imports 9 packages (graph) and is imported by 4 packages. Updated 2019-05-10. Refresh now. Tools for package owners.