stash

package module
v2.0.0+incompatible Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 14, 2019 License: BSD-3-Clause Imports: 13 Imported by: 0

README

Stash

Improved Go package for disk-based blob cache for files and chunks of files.

Installation

Install Stash using the go get command:

$ go get github.com/RstorLabs/stash

The only dependency is the Go distribution.

Motivation

This package allows us to reduce calls to our blob storage by caching the most recently used blobs to disk.

Contributing

Contributions are welcome.

License

Stash is available under the BSD (3-Clause) License.

Disclaimer

The package is a work in progress. It is functional, but does not claim to be production-ready. Please use it at your own risk.

The icon, made by Smashicons from www.flaticon.com, is licensed by CC 3.0 BY.

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var (
	ErrNotFound      = errors.New("not found")
	ErrBadDir        = errors.New("invalid directory")
	ErrBadSize       = errors.New("storage size must be greater then zero")
	ErrBadCap        = errors.New("file number must be greater then zero")
	ErrTooLarge      = errors.New("file size must be less or equal storage size")
	ErrUntagged      = errors.New("file is not tagged")
	ErrAlreadyTagged = errors.New("file is already tagged")
	ErrChunkNotFound = errors.New("chunk not found")
	ErrIncoherentTag = errors.New("incoherent tag")
)

Functions

This section is empty.

Types

type Cache 

type Cache struct {
	// contains filtered or unexported fields
}

func NewCache 

func NewCache(dir string, sz, c int64) (*Cache, error)

New creates a Cache backed by dir on disk. The cache allows at most c files of total size sz.

func (*Cache) Clear added in v1.3.0 

func (c *Cache) Clear() error

Clear resets the cache and erases the files from the cache directory.

func (*Cache) Delete 

func (c *Cache) Delete(key string) error

Delete a key from the cache, return error in case of key not present.

func (*Cache) DeleteIf added in v1.5.0 

func (c *Cache) DeleteIf(key string, removeTest func(tag []byte) bool) (bool, error)

Delete a key from the cache if the given lambda returns true, do nothing otherwise.

func (*Cache) Dump added in v1.6.5 

func (c *Cache) Dump()

Dump the content on the cache for debugging.

func (*Cache) Empty 

func (c *Cache) Empty() bool

Empty returns true if the cache is empty.

func (*Cache) Get 

func (c *Cache) Get(key string) (ReadSeekCloser, int64, error)

Get returns a reader for a blob in the cache, or ErrNotFound otherwise.

func (*Cache) GetStats added in v1.6.2 

func (c *Cache) GetStats() Stats

GetStats returns the Cache stats.

func (*Cache) GetTag added in v1.4.0 

func (c *Cache) GetTag(key string) ([]byte, error)

GetTag retrieves the tag associated with the key.

func (*Cache) GetWithTag added in v1.4.0 

func (c *Cache) GetWithTag(key string) (ReadSeekCloser, []byte, int64, error)

GetWithTag returns a reader for a blob in the cache along with the associated tag, or ErrNotFound otherwise.

func (*Cache) Keys 

func (c *Cache) Keys() []string

Keys returns a list of keys in the cache.

func (*Cache) NumEntries added in v1.5.0 

func (c *Cache) NumEntries() int64

numEntries returns the number of entries in the cache.

func (*Cache) Put 

func (c *Cache) Put(key string, val []byte) error

Put adds a byte slice as a blob to the cache against the given key.

func (*Cache) PutReader 

func (c *Cache) PutReader(key string, r LazyReader) (io.ReadCloser, error)

PutReader adds the contents of a lazy reader as a blob to the cache against the given key.

func (*Cache) PutReaderWithTag added in v1.4.0 

func (c *Cache) PutReaderWithTag(key string, tag []byte, lr LazyReader) (io.ReadCloser, error)

PutReaderWithTag like PutReader, adds the contents of a reader as blog along with a tag annotation against the given key.

func (*Cache) PutWithTag added in v1.4.0 

func (c *Cache) PutWithTag(key string, tag, val []byte) error

Put like Put, adds a byte slice as a blob along with a tag annotation.

func (*Cache) ResetStats added in v1.3.0 

func (c *Cache) ResetStats()

ResetStats resets the statistics of the cache.

func (*Cache) SetTag added in v1.4.0 

func (c *Cache) SetTag(key string, tag []byte) error

SetTag simple sets a binary Tag to the cached key element.

func (*Cache) Size 

func (c *Cache) Size() int64

Size returns the size of the cache in bytes.

type ChunkCache 

type ChunkCache struct {
	Cache  Cache
	Chunks map[string]*FileEntry
	L      sync.Mutex
}

func NewChunkCache 

func NewChunkCache(dir string, sz, c int64) (*ChunkCache, error)

func (*ChunkCache) Clear 

func (c *ChunkCache) Clear() error

func (*ChunkCache) Delete 

func (c *ChunkCache) Delete(key string) error

func (*ChunkCache) DeleteChunk 

func (c *ChunkCache) DeleteChunk(key string, offset int64) error

func (*ChunkCache) Empty 

func (c *ChunkCache) Empty() bool

func (*ChunkCache) Get 

func (c *ChunkCache) Get(key string, offset int64) (ReadSeekCloser, int64, error)

func (*ChunkCache) GetCommonTag 

func (c *ChunkCache) GetCommonTag(key string) ([]byte, error)

func (*ChunkCache) GetStats 

func (c *ChunkCache) GetStats() Stats

func (*ChunkCache) GetTag 

func (c *ChunkCache) GetTag(key string, offset int64) ([]byte, error)

func (*ChunkCache) GetWithTag 

func (c *ChunkCache) GetWithTag(key string, offset int64) (ReadSeekCloser, []byte, int64, error)

func (*ChunkCache) Keys 

func (c *ChunkCache) Keys() []string

func (*ChunkCache) NumChunksOf 

func (c *ChunkCache) NumChunksOf(key string) (int64, error)

func (*ChunkCache) NumEntries 

func (c *ChunkCache) NumEntries() int64

func (*ChunkCache) NumTotalChunks 

func (c *ChunkCache) NumTotalChunks() int64

func (*ChunkCache) Put 

func (c *ChunkCache) Put(key string, offset int64, val []byte) error

func (*ChunkCache) PutReader 

func (c *ChunkCache) PutReader(key string, offset int64, r LazyReader) (io.ReadCloser, error)

func (*ChunkCache) PutReaderWithTag 

func (c *ChunkCache) PutReaderWithTag(key string, offset int64, tag []byte, lz LazyReader) (io.ReadCloser, error)

func (*ChunkCache) PutWithTag 

func (c *ChunkCache) PutWithTag(key string, offset int64, tag []byte, val []byte) error

func (*ChunkCache) ResetStats 

func (c *ChunkCache) ResetStats()

func (*ChunkCache) SetTag 

func (c *ChunkCache) SetTag(key string, offset int64, tag []byte) error

func (*ChunkCache) SetUntaggedChunks 

func (c *ChunkCache) SetUntaggedChunks(key string, tag []byte) (int, error)

func (*ChunkCache) Shrink 

func (c *ChunkCache) Shrink()

func (*ChunkCache) Size 

func (c *ChunkCache) Size() int64

func (*ChunkCache) WaitPut 

func (c *ChunkCache) WaitPut(key string)

type EntryStatus 

type EntryStatus int64
const (
	EntryBusy EntryStatus = iota
	EntryReady
	EntryDeleted
)

type FileEntry 

type FileEntry struct {
	Offset    map[int64]struct{}
	CommonTag []byte
	// contains filtered or unexported fields
}

type FileError 

type FileError struct {
	Dir string
	Key string
	Err error
}

FileError records the storage directory name and key of the that failed to cached.

func (*FileError) Error 

func (e *FileError) Error() string

type LazyReader 

type LazyReader func() (io.ReadCloser, error)

func NewLazyReader 

func NewLazyReader(r io.ReadCloser) LazyReader

func NewLazyReaderFromBuffer 

func NewLazyReaderFromBuffer(buf []byte) LazyReader

type Meta 

type Meta struct {
	Key    string
	Size   int64
	Path   string
	Tag    []byte // user annotation
	Status EntryStatus
}

Meta is the information about the cache entry.

type ReadSeekCloser added in v1.6.4 

type ReadSeekCloser interface {
	io.Reader
	io.Seeker
	io.Closer
}

type Stats added in v1.6.2 

type Stats struct {
	Size    int64
	Entries int64
	Hit     int64
	Miss    int64
}

Stats is the cache stats.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.