Documentation
¶
Overview ¶
Package storage wraps Google Storage routines into a simpler interface.
Index ¶
- type Metadata
- func (m *Metadata) Add(md Metadatum)
- func (m *Metadata) Assemble() map[string]string
- func (m *Metadata) Clone() *Metadata
- func (m *Metadata) Empty() bool
- func (m *Metadata) Equal(o *Metadata) bool
- func (m *Metadata) Keys() []string
- func (m *Metadata) ToPretty(now time.Time, limit int) string
- func (m *Metadata) TrimUnimportant(keep int)
- func (m *Metadata) Values(k string) []Metadatum
- type Metadatum
- type Object
- type Storage
- type Timestamp
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Metadata ¶
type Metadata struct {
// contains filtered or unexported fields
}
Metadata is an interpreted metadata dict.
Cloud Storage object metadata is nominally key=>value map, but we need a multi-map. This is accomplished by embedding timestamps inside keys (as "<key>@<microsec timestamp>"). Metadata struct handled such representation.
func ParseMetadata ¶
ParseMetadata convert key[@timestamp]=>value map into Metadata object.
func (*Metadata) TrimUnimportant ¶
TrimUnimportant for each individual key removes values until only 'keep' entries remain.
Always keeps the oldest entry and most recent ones, removing only ones in the middle.
Helps to keep metadata small by "forgetting" uninteresting entries.
type Metadatum ¶
type Metadatum struct {
Key string // key with the timestamp stripped, if it had any
Timestamp Timestamp // timestamp extracted from the key or 0 if it had none
Value string // value associated with the key
}
Metadatum is one parsed key-value metadata pair.
type Object ¶
type Object struct {
Bucket string // e.g. "bucket-name"
Name string // e.g. "dir/dir/123456.tar.gz"
Generation int64 // e.g. 12312423452345
Metageneration int64 // e.g. 12312423452345
Created time.Time // when it was created
Owner string // FYI who uploaded it
MD5 string // FYI for logs only
Metadata *Metadata // custom metadata
// contains filtered or unexported fields
}
Object is a pointer to a concrete version of a file in Google Storage, along with its custom metadata.
type Storage ¶
type Storage struct {
// contains filtered or unexported fields
}
Storage knows how to upload tarballs to a predefined Google Storage prefix.
func New ¶
New returns a Storage that uploads tarballs Google Storage.
'location' should have form "gs://<bucket>[/<path>]".
func (*Storage) Check ¶
Check fetches information about existing Google Storage object.
Returns:
(*Object, nil) if such object already exists. (nil, nil) if such object doesn't exist. (nil, error) on errors.
func (*Storage) UpdateMetadata ¶
func (s *Storage) UpdateMetadata(ctx context.Context, obj *Object, cb func(m *Metadata) error) error
UpdateMetadata fetches existing metadata, calls 'cb' to mutate it, and pushes it back if it has changed.
'obj' must have come from Check or Upload. Panics otherwise.
May call 'cb' multiple times if someone is updating the metadata concurrently with us. If 'cb' returns an error, returns exact same error as is.
func (*Storage) Upload ¶
Upload uploads the tarball with the given hex SHA256 digest to the storage.
Its body is produced by 'r' and the Storage will double check that the body matches the digest. On mismatch the upload operation is abandoned (i.e. the bucket is left unchanged).
The object is named as "<prefix>/<name>", where <prefix> is taken from <location> passed to New(...) when the Storage was initialized.
Unconditionally overwrites an existing object, if any. Doesn't try to set any ACLs. The bucket must exist already.
Respects context deadlines and cancellation. On success returns a pointer to the uploaded object (including its generation number).
type Timestamp ¶
type Timestamp int64
Timestamp is a unix timestamp in microsecond precision.
func TimestampFromTime ¶
TimestampFromTime converts `t` to Timestamp (trimming precision accordingly).
Source Files