blobstore.v2: Index | Files

package blobstore

import ""


Package Files

gridfs.go interface.go managedstorage.go resourcecatalog.go


var ErrRequestExpired = fmt.Errorf("request expired")

ErrRequestExpired is used to indicate that a put request has already expired when an attempt is made to supply a response.

var ErrResourceDeleted = fmt.Errorf("resource was deleted")

ErrResourceDeleted is used to indicate that a resource was deleted before the put response could be acted on.

var ErrResponseMismatch = fmt.Errorf("response checksums do not match")

ErrResponseMismatch is used to indicate that a put response did not contain the expected checksums.

var (
    // ErrUploadPending is used to indicate that the underlying resource for a catalog entry
    // is not yet fully uploaded.
    ErrUploadPending = errors.New("Resource not available because upload is not yet complete")

func NewPutResponse Uses

func NewPutResponse(requestId int64, sha384hash string) putResponse

NewPutResponse creates a new putResponse for the given requestId and hashes.

type ManagedResource Uses

type ManagedResource struct {
    BucketUUID string
    User       string
    Path       string

ManagedResource is a catalog entry for stored data. The data may be associated with a specified bucket and/or user. The data is logically considered to be stored at the specified path.

type ManagedStorage Uses

type ManagedStorage interface {
    // GetForBucket returns a reader for data at path, namespaced to the bucket.
    // If the data is still being uploaded and is not fully written yet,
    // an ErrUploadPending error is returned. This means the path is valid but the caller
    // should try again to retrieve the data.
    GetForBucket(bucketUUID, path string) (r io.ReadCloser, length int64, err error)

    // PutForBucket stores data from reader at path, namespaced to the bucket.
    // PutForBucket is equivalent to PutForBucketAndCheckHash with an empty
    // hash string.
    PutForBucket(bucketUUID, path string, r io.Reader, length int64) error

    // PutForBucketAndCheckHash is the same as PutForBucket
    // except that it also checks that the content matches the provided
    // hash. The hash must be hex-encoded SHA-384.
    // If checkHash is empty, then the hash check is elided.
    // If length is < 0, then the reader will be consumed until EOF.
    PutForBucketAndCheckHash(bucketUUID, path string, r io.Reader, length int64, checkHash string) error

    // RemoveForBucket deletes data at path, namespaced to the bucket.
    RemoveForBucket(bucketUUID, path string) error

    // PutForBucketRequest requests that data, which may already exist in storage,
    // be saved at path, namespaced to the bucket. It allows callers who can
    // demonstrate proof of ownership of the data to store a reference to it without
    // having to upload it all. If no such data exists, a NotFound error is returned
    // and a call to bucket is required. If matching data is found, the caller
    // is returned a response indicating the random byte range to for which they must
    // provide a checksum to complete the process.
    PutForBucketRequest(bucketUUID, path string, hash string) (*RequestResponse, error)

    // ProofOfAccessResponse is called to respond to a Put..Request call in order to
    // prove ownership of data for which a storage reference is created.
    ProofOfAccessResponse(putResponse) error

ManagedStorage instances persist data for a bucket, for a user, or globally. (Only bucket storage is currently implemented).

func NewManagedStorage Uses

func NewManagedStorage(db *mgo.Database, rs ResourceStorage) ManagedStorage

NewManagedStorage creates a new ManagedStorage using the transaction runner, storing resource entries in the specified database, and resource data in the specified resource storage.

type PutRequest Uses

type PutRequest struct {
    // contains filtered or unexported fields

PutRequest is to record a request to put a file pending proof of access.

type RequestResponse Uses

type RequestResponse struct {
    RequestId   int64
    RangeStart  int64
    RangeLength int64

RequestResponse is returned by a put request to inform the caller the data range over which to calculate the hashes for the response.

type Resource Uses

type Resource struct {
    SHA384Hash string
    Path       string
    Length     int64

Resource is a catalog entry for stored data. It contains the path where the data is stored as well as a hash of the data which are used for de-duping.

type ResourceCatalog Uses

type ResourceCatalog interface {
    // Get fetches a Resource with the given id.
    Get(id string) (*Resource, error)

    // Find returns the resource id for the Resource with the given hash.
    Find(hash string) (id string, err error)

    // Put ensures a Resource entry exists for the given hash,
    // returning the id and path recorded by UploadComplete.
    // If UploadComplete has not been called, path will be empty.
    // If the Resource entry exists, its reference count is incremented,
    // otherwise a new entry is created with a reference count of 1.
    Put(hash string, length int64) (id, path string, err error)

    // UploadComplete records that the underlying resource described by
    // the Resource entry with id is now fully uploaded to the specified
    // storage path, and the resource is available for use. If another
    // uploader already recorded a path, then UploadComplete will return
    // an error satisfiying juju/errors.IsAlreadyExists.
    UploadComplete(id, path string) error

    // Remove decrements the reference count for a Resource with the given id, deleting it
    // if the reference count reaches zero. The path of the Resource is returned.
    // If the Resource is deleted, wasDeleted is returned as true.
    Remove(id string) (wasDeleted bool, path string, err error)

ResourceCatalog instances persist Resources. Resources with the same hash values are not duplicated; instead a reference count is incremented. Similarly, when a Resource is removed, the reference count is decremented. When the reference count reaches zero, the Resource is deleted.

type ResourceStorage Uses

type ResourceStorage interface {
    // Get returns a reader for the resource located at path.
    Get(path string) (io.ReadCloser, error)

    // Put writes data from the specified reader to path and returns a checksum of the data written.
    Put(path string, r io.Reader, length int64) (checksum string, err error)

    // Remove deletes the data at the specified path.
    Remove(path string) error

ResourceStorage instances save and retrieve data from an underlying storage implementation.

func NewGridFS Uses

func NewGridFS(dbName, namespace string, session *mgo.Session) ResourceStorage

NewGridFS returns a ResourceStorage instance backed by a mongo GridFS. namespace is used to segregate different sets of data.

Package blobstore imports 17 packages (graph) and is imported by 114 packages. Updated 2016-07-16. Refresh now. Tools for package owners.