filecache

package module
v0.0.0-...-3be0f1f Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 16, 2023 License: Apache-2.0 Imports: 20 Imported by: 0

README

filecache-go

Documentation

Index

Constants

This section is empty.

Variables

View Source
var (
	// ErrQueueClosed is returned when enqueue/dequeue is being called
	// after Close/Destroy.
	ErrQueueClosed = errors.New("queue doesn't exist")
	// ErrNilType is returned when the encoder is not enabled/invalid.
	ErrNilType = errors.New("cannot use nil type")
	// ErrExceedsSize represents the error when the file size exceeds the given size.
	ErrExceedsSize = errors.New("exceeds max size")
)
View Source
var ErrTooLarge = errors.New("Data too large")

ErrTooLarge can be returned from `Encode` to indicate that data is large and EncodeToReader should be used

Functions

func ConfigureDBPath

func ConfigureDBPath(path string)

ConfigureDBPath updates the db path

func DequeueAll

func DequeueAll(q Queue, dest interface{}) error

DequeueAll dequeues all elements of type dest from q and stores it into dest.

func Tar

func Tar(w io.Writer, fileCaches ...Queue) error

Tar gzips and tars encoded data retrieved from the filecaches and writes it to w.

func Untar

func Untar(destDir string, r io.Reader) error

Untar reads data from the reader r and writes it to the destDir as independent files. It is upto the caller to make sure directory exists.

Types

type DropMode

type DropMode int

DropMode represensts the supported drop modes.

const (
	HeadDrop DropMode = iota
	TailDrop
)

Drop modes.

type EncodingType

type EncodingType int

EncodingType represents the supported encoding types.

const (
	Gob EncodingType = iota
	JSON
)

Encoding types.

type FileCache

type FileCache struct {
	sync.Mutex
	// contains filtered or unexported fields
}

FileCache implements Queue interface.

func NewFileCache

func NewFileCache(name string, options ...Option) (*FileCache, error)

NewFileCache creates a new filecache handler to handle arbitrary types to be stored in on disk. It uses goque which is backend by go's port of levelDB. It implements Queue interface and has tools to encode/decode using msgpack. This package also runs a cleaner job which every 24Hrs, will remove the existing DB completely and create a new DB instance. name - used as the filename in the db for this type. All methods are thread-safe.

func (*FileCache) Close

func (f *FileCache) Close() error

Close closes the db.

func (*FileCache) Decode

func (f *FileCache) Decode(data []byte) ([]interface{}, error)

Decode decodes the given data and returns the list as []interface. Caller has to type assert the interface to the respected types.

func (*FileCache) Dequeue

func (f *FileCache) Dequeue(obj interface{}) error

Dequeue deletes removes the first entry in the queue.

func (*FileCache) Destroy

func (f *FileCache) Destroy() error

Destroy drops the db, removes any leftover files/directories and frees the queue.

func (*FileCache) Encode

func (f *FileCache) Encode() ([]byte, error)

Encode dequeues all the entries and encodes them with the reflect.Type.

func (*FileCache) EncodeToReader

func (f *FileCache) EncodeToReader() (io.Reader, int64, error)

EncodeToReader functions like Encode but returns a reader.

func (*FileCache) Enqueue

func (f *FileCache) Enqueue(obj interface{}) error

Enqueue inserts a new entry to the queue. Non-blocking.

func (*FileCache) Errors

func (f *FileCache) Errors() chan error

Errors returns the error pipe.

func (*FileCache) Name

func (f *FileCache) Name() string

Name returns name used by the caller to identify the filecache.

func (*FileCache) Run

func (f *FileCache) Run(ctx context.Context) error

Run runs cleaner job and starts listener for new entries.

type Option

type Option func(*FileCache)

An Option represents a Filecache option.

func OptionDropMode

func OptionDropMode(mode DropMode) Option

OptionDropMode used to set the drop mode when maxElements limit has reached. Default: HeadDrop.

func OptionEncoderConfig

func OptionEncoderConfig(obj interface{}, bufferSize int) Option

OptionEncoderConfig used to set the type and the buffer size. obj - the object type to use with Encode/Decode. bufferSize - when Encode is called, we will limit the encoded data size with the provided bytes size. [0 - infinite entries] Default: Disabled.

func OptionEncodingType

func OptionEncodingType(e EncodingType) Option

OptionEncodingType used to set encoding type. Supported options are JSON and Gob. Default: Gob.

func OptionMaxElements

func OptionMaxElements(maxElements uint64) Option

OptionMaxElements used to set the max elements. maxElements - the db will be restricted to not exceed this limit and any enqueues after this will call dequeue and enqueues the new entry. It is validated before OptionMaxSize. Default: Infinite.

func OptionMaxSize

func OptionMaxSize(maxSize int64) Option

OptionMaxSize used to set the max data size on disk. maxSize - the db will be restricted to not exceed this limit and any enqueues after this will be dropped with ErrExceedsSize error. In most cases, you need to use OptionMaxElements over this. Default: Disabled.

func OptionTTL

func OptionTTL(duration time.Duration) Option

OptionTTL used to set the TTL of this DB. Default: 24h.

type Queue

type Queue interface {
	Name() string
	Run(ctx context.Context) error
	Enqueue(element interface{}) error
	Dequeue(element interface{}) error
	Close() error
	Destroy() error
	Errors() chan error

	Encode() ([]byte, error)
	Decode(data []byte) ([]interface{}, error)

	EncodeToReader() (io.Reader, int64, error)
}

Queue interface has all necessary methods to enqueue, dequeue, encode, decode entries.

Directories

Path Synopsis
Package mockfilecache is a generated GoMock package.
Package mockfilecache is a generated GoMock package.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL