blobstore

package module
v0.0.0-...-0ecd2b1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 21, 2014 License: LGPL-3.0 Imports: 17 Imported by: 0

README

juju/blobstore

This package provides a Mongo GridFS-backed blob storage engine.

Documentation

Index

Constants

This section is empty.

Variables

View Source
var ErrRequestExpired = fmt.Errorf("request expired")

ErrRequestExpired is used to indicate that a put request has already expired when an attempt is made to supply a response.

View Source
var ErrResourceDeleted = fmt.Errorf("resource was deleted")

ErrResourceDeleted is used to indicate that a resource was deleted before the put response could be acted on.

View Source
var ErrResponseMismatch = fmt.Errorf("response checksums do not match")

ErrResponseMismatch is used to indicate that a put response did not contain the expected checksums.

View Source
var ErrUploadPending = fmt.Errorf("Resource not available because upload is not yet complete")

ErrUploadPending is used to indicate that the underlying resource for a catalog entry is not yet fully uploaded.

Functions

func NewPutResponse

func NewPutResponse(requestId int64, md5Hash, sha256Hash string) putResponse

NewPutResponse creates a new putResponse for the given requestId and hashes.

Types

type ManagedResource

type ManagedResource struct {
	EnvUUID string
	User    string
	Path    string
}

ManagedResource is a catalog entry for stored data. The data may be associated with a specified environment and/or user. The data is logically considered to be stored at the specified path.

type ManagedStorage

type ManagedStorage interface {
	// GetForEnvironment returns a reader for data at path, namespaced to the environment.
	// If the data is still being uploaded and is not fully written yet,
	// an ErrUploadPending error is returned. This means the path is valid but the caller
	// should try again to retrieve the data.
	GetForEnvironment(envUUID, path string) (r io.ReadCloser, length int64, err error)

	// PutForEnvironment stores data from reader at path, namespaced to the environment.
	PutForEnvironment(envUUID, path string, r io.Reader, length int64) error

	// RemoveForEnvironment deletes data at path, namespaced to the environment.
	RemoveForEnvironment(envUUID, path string) error

	// PutForEnvironmentRequest requests that data, which may already exist in storage,
	// be saved at path, namespaced to the environment. It allows callers who can
	// demonstrate proof of ownership of the data to store a reference to it without
	// having to upload it all. If no such data exists, a NotFound error is returned
	// and a call to EnvironmentPut is required. If matching data is found, the caller
	// is returned a response indicating the random byte range to for which they must
	// provide checksums to complete the process.
	PutForEnvironmentRequest(envUUID, path string, hash ResourceHash) (*RequestResponse, error)

	// ProofOfAccessResponse is called to respond to a Put..Request call in order to
	// prove ownership of data for which a storage reference is created.
	ProofOfAccessResponse(putResponse) error
}

ManagedStorage instances persist data for an environment, for a user, or globally. (Only environment storage is currently implemented).

func NewManagedStorage

func NewManagedStorage(db *mgo.Database, rs ResourceStorage) ManagedStorage

NewManagedStorage creates a new ManagedStorage using the transaction runner, storing resource entries in the specified database, and resource data in the specified resource storage.

type PutRequest

type PutRequest struct {
	// contains filtered or unexported fields
}

PutRequest is to record a request to put a file pending proof of access.

type RequestResponse

type RequestResponse struct {
	RequestId   int64
	RangeStart  int64
	RangeLength int64
}

RequestResponse is returned by a put request to inform the caller the data range over which to calculate the hashes for the response.

type Resource

type Resource struct {
	ResourceHash
	Path   string
	Length int64
}

Resource is a catalog entry for stored data. It contains the path where the data is stored as well as hashes of the data which are used for de-duping.

type ResourceCatalog

type ResourceCatalog interface {
	// Get fetches a Resource with the given id.
	Get(id string) (*Resource, error)

	// Find returns the resource id for the Resource with the given hashes.
	Find(rh *ResourceHash) (id string, err error)

	// Put ensures a Resource entry exists for the given ResourceHash, returning the id, path,
	// and a flag indicating if this is a new record.
	// If the Resource exists, its reference count is incremented, otherwise a new entry is created.
	Put(rh *ResourceHash, length int64) (id, path string, isNew bool, err error)

	// UploadComplete records that the underlying resource described by the Resource entry with id
	// is now fully uploaded and the resource is available for use.
	UploadComplete(id string) error

	// Remove decrements the reference count for a Resource with the given id, deleting it
	// if the reference count reaches zero. The path of the Resource is returned.
	// If the Resource is deleted, wasDeleted is returned as true.
	Remove(id string) (wasDeleted bool, path string, err error)
}

ResourceCatalog instances persist Resources. Resources with the same hash values are not duplicated; instead a reference count is incremented. Similarly, when a Resource is removed, the reference count is decremented. When the reference count reaches zero, the Resource is deleted.

type ResourceHash

type ResourceHash struct {
	MD5Hash    string
	SHA256Hash string
}

ResourceHash contains hashes which are used to unambiguously identify stored data.

type ResourceStorage

type ResourceStorage interface {
	// Get returns a reader for the resource located at path.
	Get(path string) (io.ReadCloser, error)

	// Put writes data from the specified reader to path and returns a checksum of the data written.
	Put(path string, r io.Reader, length int64) (checksum string, err error)

	// Remove deletes the data at the specified path.
	Remove(path string) error
}

ResourceStorage instances save and retrieve data from an underlying storage implementation.

func NewGridFS

func NewGridFS(dbName, namespace string, session *mgo.Session) ResourceStorage

NewGridFS returns a ResourceStorage instance backed by a mongo GridFS. namespace is used to segregate different sets of data.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL