blobrepo

package

v0.0.0-...-90deddd Latest Latest Go to latest Published: Oct 18, 2023 License: Apache-2.0 Imports: 17 Imported by: 0

Details

Valid go.mod file

The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.
Redistributable license

Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.
Tagged version

Modules with tagged versions give importers more predictable builds.
Stable version

When a project reaches major version v1 it is considered stable.
Learn more about best practices

Repository

github.com/grailbio/reflow

Links

Open Source Insights

Documentation ¶

Overview ¶

Package blobrepo implements a generic reflow.Repository on top of a blob.Bucket.

Index ¶

func Dial(u *url.URL) (reflow.Repository, error)
func Register(scheme string, store blob.Store)
type Repository

Constants ¶

This section is empty.

Variables ¶

This section is empty.

Functions ¶

func Dial ¶

func Dial(u *url.URL) (reflow.Repository, error)

Dial dials a blob repository. The URL must have the form:

type://bucket/prefix

TODO(marius): we should support shipping authentication information in the URL also.

func Register ¶

func Register(scheme string, store blob.Store)

Register registers a blob store implementation used to dial repositories for the provided scheme.

Types ¶

type Repository ¶

type Repository struct {
	Bucket blob.Bucket
	Prefix string
}

Repository implements an blob-backed Repository. Objects are stored in the given bucket under the given prefix, followed by "objects":

type://bucket/<prefix>/objects/sha256:<hex>...

In-progress uploads are stored under "uploads":

type://bucket/<prefix>/uploads/<hex>

Prefix may be empty.

func (*Repository) Collect ¶

func (r *Repository) Collect(ctx context.Context, live liveset.Liveset) error

Collect is not supported on S3.

func (*Repository) CollectWithThreshold ¶

func (r *Repository) CollectWithThreshold(ctx context.Context, live liveset.Liveset, dead liveset.Liveset, threshold time.Time, dryRun bool) error

CollectWithThreshold removes from this repository any objects which are not in the liveset and which have not been accessed more recently than the liveset's threshold time

func (*Repository) Get ¶

func (r *Repository) Get(ctx context.Context, id digest.Digest) (io.ReadCloser, error)

Get retrieves an object from the repository.

func (*Repository) GetFile ¶

func (r *Repository) GetFile(ctx context.Context, id digest.Digest, w io.WriterAt) (int64, error)

GetFile retrieves an object from the repository directly to the a io.WriterAt. This uses the S3 download manager to download chunks concurrently.

func (*Repository) Put ¶

func (r *Repository) Put(ctx context.Context, body io.Reader) (digest.Digest, error)

Put installs an object into the repository; its digest ID is returned.

func (*Repository) PutFile ¶

func (r *Repository) PutFile(ctx context.Context, file reflow.File, body io.Reader) error

PutFile installs a file into the repository. PutFile uses the S3 upload manager directly.

func (*Repository) ReadFrom ¶

func (r *Repository) ReadFrom(_ context.Context, id digest.Digest, u *url.URL) error

ReadFrom is unsupported by the blob repository.

TODO(marius): we can support other s3r here by performing CopyObjects.

func (*Repository) Stat ¶

func (r *Repository) Stat(ctx context.Context, id digest.Digest) (reflow.File, error)

Stat queries the repository for object metadata.

func (*Repository) String ¶

func (r *Repository) String() string

String returns the repository URL.

func (*Repository) URL ¶

func (r *Repository) URL() *url.URL

URL returns the URL for this repository. It is of the form:

<type>://bucket/prefix

func (*Repository) WriteTo ¶

func (r *Repository) WriteTo(_ context.Context, id digest.Digest, u *url.URL) error

WriteTo is unsupported by the blob repository.

TODO(marius): we can support other s3r here by performing CopyObjects.

Source Files ¶

View all Source files

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL