camlistore.org: camlistore.org/pkg/blob Index | Files

package blob

import "camlistore.org/pkg/blob"

Package blob defines types to refer to and retrieve low-level Camlistore blobs.

Index

Package Files

blob.go chanpeek.go fetcher.go ref.go

Constants

const Pattern = `\b([a-z][a-z0-9]*)-([a-f0-9]+)\b`

Pattern is the regular expression which matches a blobref. It does not contain ^ or $.

Variables

var (
    ErrNegativeSubFetch         = errors.New("invalid negative subfetch parameters")
    ErrOutOfRangeOffsetSubFetch = errors.New("subfetch offset greater than blob size")
)

func HashFuncs Uses

func HashFuncs() []string

HashFuncs returns the names of the supported hash functions.

func NewHash Uses

func NewHash() hash.Hash

NewHash returns a new hash.Hash of the currently recommended hash type. Currently this is just SHA-1, but will likely change within the next year or so.

func NewLazyReadSeekCloser Uses

func NewLazyReadSeekCloser(src Fetcher, br Ref) readerutil.ReadSeekCloser

NewLazyReadSeekCloser returns a ReadSeekCloser that does no work until one of its Read, Seek, or Close methods is called, but then fetches the ref from src. Any fetch error is returned in the Read, Seek, or Close call.

func ReaderAt Uses

func ReaderAt(sf SubFetcher, br Ref) io.ReaderAt

ReaderAt returns an io.ReaderAt of br, fetching against sf.

func TypeAlphabet Uses

func TypeAlphabet(typ string) string

TypeAlphabet returns the valid characters in the given blobref type. It returns the empty string if the typ is unknown.

func ValidRefString Uses

func ValidRefString(s string) bool

type Blob Uses

type Blob struct {
    // contains filtered or unexported fields
}

Blob represents a blob. Use the methods Size, SizedRef and Open to query and get data from Blob.

func FromFetcher Uses

func FromFetcher(fetcher Fetcher, br Ref) (*Blob, error)

FromFetcher fetches br from fetcher and slurps its contents to memory. It does not validate the blob's digest. Use the Blob.ValidContents method for that.

func FromReader Uses

func FromReader(br Ref, r io.Reader, size uint32) (*Blob, error)

FromReader slurps the given blob from r to memory. It does not validate the blob's digest. Use the Blob.ValidContents method for that.

func NewBlob Uses

func NewBlob(ref Ref, size uint32, newReader func() readerutil.ReadSeekCloser) *Blob

NewBlob constructs a Blob from its Ref, size and a function that returns an io.ReadCloser from which the blob can be read. Any error in the function newReader when constructing the io.ReadCloser should be returned upon the first call to Read or Close.

func (*Blob) IsUTF8 Uses

func (b *Blob) IsUTF8() bool

IsUTF8 reports whether the blob is entirely UTF-8.

func (*Blob) Open Uses

func (b *Blob) Open() readerutil.ReadSeekCloser

Open returns an io.ReadCloser that can be used to read the blob data. The caller must close the io.ReadCloser when finished.

func (*Blob) Ref Uses

func (b *Blob) Ref() Ref

Ref returns the blob's reference.

func (*Blob) Size Uses

func (b *Blob) Size() uint32

Size returns the size of the blob (in bytes).

func (*Blob) SizedRef Uses

func (b *Blob) SizedRef() SizedRef

SizedRef returns the SizedRef corresponding to the blob.

func (*Blob) ValidContents Uses

func (b *Blob) ValidContents() bool

ValidContents reports whether the hash of blob's content matches its reference.

type ByRef Uses

type ByRef []Ref

ByRef sorts blob references.

func (ByRef) Len Uses

func (s ByRef) Len() int

func (ByRef) Less Uses

func (s ByRef) Less(i, j int) bool

func (ByRef) Swap Uses

func (s ByRef) Swap(i, j int)

type ChanPeeker Uses

type ChanPeeker struct {
    Ch <-chan SizedRef
    // contains filtered or unexported fields
}

ChanPeeker wraps a channel receiving SizedRefs and adds a 1 element buffer on it, with Peek and Take methods.

func (*ChanPeeker) Closed Uses

func (cp *ChanPeeker) Closed() bool

Closed reports true if no more SizedRef values are available.

func (*ChanPeeker) ConsumeAll Uses

func (cp *ChanPeeker) ConsumeAll()

ConsumeAll drains the channel of all items.

func (*ChanPeeker) MustPeek Uses

func (cp *ChanPeeker) MustPeek() SizedRef

MustPeek returns the next SizedRef or panics if none is available.

func (*ChanPeeker) MustTake Uses

func (cp *ChanPeeker) MustTake() SizedRef

MustTake returns the next SizedRef, else panics if none is available.

func (*ChanPeeker) Peek Uses

func (cp *ChanPeeker) Peek() (sr SizedRef, ok bool)

Peek returns the next SizedRef and whether one was available.

func (*ChanPeeker) Take Uses

func (cp *ChanPeeker) Take() (sr SizedRef, ok bool)

Take returns the next SizedRef and whether one was available for the taking.

type DirFetcher Uses

type DirFetcher struct {
    // contains filtered or unexported fields
}

func NewSimpleDirectoryFetcher Uses

func NewSimpleDirectoryFetcher(dir string) *DirFetcher

func (*DirFetcher) Fetch Uses

func (df *DirFetcher) Fetch(r Ref) (file io.ReadCloser, size uint32, err error)

type Fetcher Uses

type Fetcher interface {
    // Fetch returns a blob.  If the blob is not found then
    // os.ErrNotExist should be returned for the error (not a wrapped
    // error with a ErrNotExist inside)
    //
    // The contents are not guaranteed to match the digest of the
    // provided Ref (e.g. when streamed over HTTP). Paranoid
    // callers should verify them.
    //
    // The caller must close blob.
    Fetch(Ref) (blob io.ReadCloser, size uint32, err error)
}

Fetcher is the minimal interface for retrieving a blob from storage. The full storage interface is blobserver.Storage.

func NewSerialFetcher Uses

func NewSerialFetcher(fetchers ...Fetcher) Fetcher

type Ref Uses

type Ref struct {
    // contains filtered or unexported fields
}

Ref is a reference to a Camlistore blob. It is used as a value type and supports equality (with ==) and the ability to use it as a map key.

func MustParse Uses

func MustParse(s string) Ref

MustParse parse s as a blobref and panics on failure.

func Parse Uses

func Parse(s string) (ref Ref, ok bool)

Parse parse s as a blobref and returns the ref and whether it was parsed successfully.

func ParseBytes Uses

func ParseBytes(s []byte) (ref Ref, ok bool)

ParseBytes is like Parse, but parses from a byte slice.

func ParseKnown Uses

func ParseKnown(s string) (ref Ref, ok bool)

ParseKnown is like Parse, but only parse blobrefs known to this server. It returns ok == false for well-formed but unsupported blobrefs.

func ParseOrZero Uses

func ParseOrZero(s string) Ref

Parse parse s as a blobref. If s is invalid, a zero Ref is returned which can be tested with the Valid method.

func RefFromHash Uses

func RefFromHash(h hash.Hash) Ref

RefFromHash returns a blobref representing the given hash. It panics if the hash isn't of a known type.

func RefFromString Uses

func RefFromString(s string) Ref

RefFromString returns a blobref from the given string, for the currently recommended hash function

func SHA1FromBytes Uses

func SHA1FromBytes(b []byte) Ref

SHA1FromBytes returns a SHA-1 blobref of the provided bytes.

func SHA1FromString Uses

func SHA1FromString(s string) Ref

SHA1FromString returns a SHA-1 blobref of the provided string.

func (Ref) Digest Uses

func (r Ref) Digest() string

Digest returns the lower hex digest of the blobref, without the e.g. "sha1-" prefix. It panics if r is zero.

func (Ref) DigestPrefix Uses

func (r Ref) DigestPrefix(digits int) string

func (Ref) DomID Uses

func (r Ref) DomID() string

func (Ref) EqualString Uses

func (r Ref) EqualString(s string) bool

EqualString reports whether r.String() is equal to s. It does not allocate.

func (Ref) Hash Uses

func (r Ref) Hash() hash.Hash

Hash returns a new hash.Hash of r's type. It panics if r is zero.

func (Ref) HashMatches Uses

func (r Ref) HashMatches(h hash.Hash) bool

func (Ref) HashName Uses

func (r Ref) HashName() string

HashName returns the lowercase hash function name of the reference. It panics if r is zero.

func (Ref) IsSupported Uses

func (r Ref) IsSupported() bool

func (Ref) Less Uses

func (r Ref) Less(o Ref) bool

Less reports whether r sorts before o. Invalid references blobs sort first.

func (Ref) MarshalBinary Uses

func (r Ref) MarshalBinary() (data []byte, err error)

MarshalBinary implements Go's encoding.BinaryMarshaler interface.

func (Ref) MarshalJSON Uses

func (r Ref) MarshalJSON() ([]byte, error)

func (Ref) String Uses

func (r Ref) String() string

func (Ref) StringMinusOne Uses

func (r Ref) StringMinusOne() string

StringMinusOne returns the first string that's before String.

func (Ref) Sum32 Uses

func (r Ref) Sum32() uint32

func (Ref) Sum64 Uses

func (r Ref) Sum64() uint64

func (*Ref) UnmarshalBinary Uses

func (r *Ref) UnmarshalBinary(data []byte) error

UnmarshalBinary implements Go's encoding.BinaryUnmarshaler interface.

func (*Ref) UnmarshalJSON Uses

func (r *Ref) UnmarshalJSON(d []byte) error

func (Ref) Valid Uses

func (r Ref) Valid() bool

type SizedByRef Uses

type SizedByRef []SizedRef

SizedByRef sorts SizedRefs by their blobref.

func (SizedByRef) Len Uses

func (s SizedByRef) Len() int

func (SizedByRef) Less Uses

func (s SizedByRef) Less(i, j int) bool

func (SizedByRef) Swap Uses

func (s SizedByRef) Swap(i, j int)

type SizedRef Uses

type SizedRef struct {
    Ref  Ref    `json:"blobRef"`
    Size uint32 `json:"size"`
}

SizedRef is like a Ref but includes a size. It should also be used as a value type and supports equality.

func (SizedRef) HashMatches Uses

func (sr SizedRef) HashMatches(h hash.Hash) bool

func (SizedRef) Less Uses

func (sr SizedRef) Less(o SizedRef) bool

Less reports whether sr sorts before o. Invalid references blobs sort first.

func (SizedRef) String Uses

func (sr SizedRef) String() string

func (SizedRef) Valid Uses

func (sr SizedRef) Valid() bool

type SubFetcher Uses

type SubFetcher interface {
    // SubFetch returns part of a blob.
    // The caller must close the returned io.ReadCloser.
    // The Reader may return fewer than 'length' bytes. Callers should
    // check. The returned error should be: ErrNegativeSubFetch if any of
    // offset or length is negative, or os.ErrNotExist if the blob
    // doesn't exist, or ErrOutOfRangeOffsetSubFetch if offset goes over
    // the size of the blob.
    SubFetch(ref Ref, offset, length int64) (io.ReadCloser, error)
}

A SubFetcher is a Fetcher that can retrieve part of a blob.

Package blob imports 15 packages (graph) and is imported by 1105 packages. Updated 2018-01-15. Refresh now. Tools for package owners.