filecache

package module
v2.1.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 4, 2024 License: MIT Imports: 15 Imported by: 0

README

📦 FileCache v2

Make GoDoc GoReport

Store data from io.Reader or bytes to cache files with TTL and metadata.

Installation

go get github.com/kukymbr/filecache/v2 

Usage

Initializing the cache instance
// With target dir specified
fc, err := filecache.New("/path/to/cache/dir")
// With temp dir as a target
fc, err := filecache.NewInTemp()
// With options
fc, err := filecache.New(
    "/path/to/cache/dir",
    filecache.InstanceOptions{
        PathGenerator: filecache.FilteredKeyPath,
        DefaultTTL:    time.Hour,
        GC:            filecache.NewIntervalGarbageCollector("/path/to/cache/dir", time.Hour),
    },
)

See the InstanceOptions godoc for the instance configuration values.

Saving data to the cache
// From the io.Reader
_, err := fc.Write(context.Background(), "key1", strings.NewReader("value1"))
// From the byte array
_, err := fc.WriteData(context.Background(), "key2", []byte("value2"))
// With the item options
_, err := fc.Write(
    context.Background(), 
    "key3", 
    strings.NewReader("value3"),
    filecache.ItemOptions{
        Name:   "Key 3",
        TTL:    time.Hour * 24,
        Fields: filecache.NewValues("field1", "val1", "field2", "val2"),
    },
)

See the ItemOptions godoc for the instance configuration values.

Reading from cache
// Opening the cache file reader
res, err := fc.Open(context.Background(), "key1")
if err != nil { 
    // Handle the error...
}

if res.Hit() {
    reader := res.Reader()
    // Read the data...
}
// Read all the data
res, err := fc.Read(context.Background(), "key2")
if err != nil && res.Hit() {
    data := res.Data()
}
// Read options
res, err := fc.Read(context.Background(), "key3")
if err != nil && res.Hit() {
    name := res.Options().Name
}

The Open() and Read() functions return an error only if context is canceled or if the file open operation has failed. If there is no error, this doesn't mean the result is found, the res.Hit() function should be called.

Iterate through the cached items

To iterate through the cached items, use the Scanner tool:

Removing the expired items

The expired cache items are removed by the GarbageCollector, assigned to the FileCache instance.

There are three types of the GarbageCollector adapters:

  • filecache.NewNopGarbageCollector() — the GarbageCollector doing nothing, all the files are kept;
  • filecache.NewProbabilityGarbageCollector() — the GarbageCollector running with the defined probability, used by default;
  • filecache.NewIntervalGarbageCollector() — the GarbageCollector running by the time interval.

See the gc.go's godocs for more info.

// Initialize the scanner
scanner := filecache.NewScanner(fc.GetPath())

// Iterate
err = scanner.Scan(func(entry filecache.ScanEntry) error {
    // Do some nice things...
    return nil
})

License

MIT.

Documentation

Index

Constants

View Source
const (
	// TTLEternal is a TTL value for eternal cache.
	TTLEternal = time.Duration(-1)
)

Variables

View Source
var (
	ErrDirNotExists = errors.New("directory does not exist")
	ErrNotADir      = errors.New("not a directory")
)

Functions

func FilteredKeyPath

func FilteredKeyPath(key string) string

FilteredKeyPath uses a key without path separators as a path.

func HashedKeyPath

func HashedKeyPath(key string) string

HashedKeyPath return hashes key and uses it as a file name.

func HashedKeySplitPath

func HashedKeySplitPath(key string) string

HashedKeySplitPath return hashes key, splits it on the parts which are directories and a file name.

Types

type FileCache

type FileCache interface {
	// GetPath returns the target path of the FileCache instance.
	GetPath() string

	// Write writes data from the reader to the cache file.
	Write(ctx context.Context, key string, reader io.Reader, options ...ItemOptions) (written int64, err error)

	// WriteData writes data to the cache file.
	WriteData(ctx context.Context, key string, data []byte, options ...ItemOptions) (written int64, err error)

	// Open opens the reader with cached data.
	// Returns no error on successful cache hit, on no hit, on invalid cache files.
	// Returns an error if failed to open an existing cache file or if context is done.
	Open(ctx context.Context, key string) (result *OpenResult, err error)

	// Read reads data from the cache file.
	// Returns no error on successful cache hit, on no hit, on invalid cache files.
	// Returns an error if failed to open or read an existing cache file or if context is done.
	Read(ctx context.Context, key string) (result *ReadResult, err error)

	// Invalidate removes data associated with a key from a cache.
	Invalidate(ctx context.Context, key string) error

	// Close closes the FileCache instance.
	Close() error
}

FileCache is a tool to cache data from any io.Reader to the file.

func New

func New(targetDir string, options ...InstanceOptions) (FileCache, error)

New creates a new FileCache instance with a specified target dir & options.

func NewInTemp added in v2.1.2

func NewInTemp(options ...InstanceOptions) (FileCache, error)

NewInTemp creates a new FileCache instance with files stored in the system's temp dir.

func NewNop

func NewNop() FileCache

NewNop creates no-operation file cache instance.

type GarbageCollector added in v2.1.0

type GarbageCollector interface {
	// OnInstanceInit is executed on the initialization of the FileCache instance.
	OnInstanceInit()

	// OnOperation is executed on the every item's operation in the FileCache instance.
	OnOperation()

	// Close closes the GarbageCollector.
	Close() error
}

GarbageCollector is a tool to remove expired cache items.

func NewIntervalGarbageCollector added in v2.1.0

func NewIntervalGarbageCollector(dir string, interval time.Duration) GarbageCollector

NewIntervalGarbageCollector returns the GarbageCollector running by the interval.

func NewNopGarbageCollector added in v2.1.0

func NewNopGarbageCollector() GarbageCollector

NewNopGarbageCollector returns the GarbageCollector doing nothing.

func NewProbabilityGarbageCollector added in v2.1.0

func NewProbabilityGarbageCollector(dir string, onInitDivisor uint, onOpDivisor uint) GarbageCollector

NewProbabilityGarbageCollector returns the GarbageCollector running with the defined probability. Divisor is a run probability divisor (e.g., divisor equals 100 is a 1/100 probability).

Function arguments: * dir - the directory with the FileCache's instance files; * onInitDivisor - divisor for the probability on the OnInstanceInit() function call; * onOpDivisor - divisor for the probability on the OnOperation() function call.

type InstanceOptions

type InstanceOptions struct {
	// PathGenerator is a function to generate cache item's file path.
	PathGenerator PathGeneratorFn

	// DefaultTTL is a TTL value for the items without it.
	DefaultTTL time.Duration

	// GC is a GarbageCollector instance for the cache instance.
	GC GarbageCollector

	// GCDivisor is a garbage collector run probability divisor
	// (e.g., 100 is 1/100 probability).
	//
	// Deprecated: use the GC property instead.
	GCDivisor uint
}

InstanceOptions are a cache instance options.

type ItemOptions

type ItemOptions struct {
	// Name is a human-readable item name.
	Name string

	// TTL is an item's time-to-live value.
	TTL time.Duration

	// Fields is a map of any other metadata fields.
	Fields Values
}

ItemOptions are a cache item options

type OpenResult

type OpenResult struct {
	// contains filtered or unexported fields
}

OpenResult is a result of the file cache's Open operation.

func (*OpenResult) Hit

func (r *OpenResult) Hit() bool

Hit returns true, if requested key found in cache.

func (*OpenResult) Options

func (r *OpenResult) Options() *ItemOptions

Options returns a found cache item options.

func (*OpenResult) Reader

func (r *OpenResult) Reader() io.ReadCloser

Reader returns the cached data reader.

type PathGeneratorFn

type PathGeneratorFn func(key string) string

PathGeneratorFn is a function to generate cache item's file path.

func WithExt

func WithExt(fn PathGeneratorFn, ext string) PathGeneratorFn

WithExt returns new PathGeneratorWithExt instance.

type ReadResult

type ReadResult struct {
	// contains filtered or unexported fields
}

ReadResult is a result of the file cache's Read operation.

func (*ReadResult) Data

func (r *ReadResult) Data() []byte

Data returns the cached data.

func (*ReadResult) Hit

func (r *ReadResult) Hit() bool

Hit returns true, if requested key found in cache.

func (*ReadResult) Options

func (r *ReadResult) Options() *ItemOptions

Options returns a found cache item options.

type ScanEntry

type ScanEntry struct {
	Key       string
	CreatedAt time.Time
	Options   *ItemOptions
	// contains filtered or unexported fields
}

ScanEntry is a scanner hit entry.

type Scanner

type Scanner interface {
	Scan(onHit ScannerHitFn) error
}

Scanner is a tool to scan cache items inside the specified directory.

func NewScanner

func NewScanner(dir string) Scanner

NewScanner creates a Scanner looking for the valid cache items.

type ScannerHitFn

type ScannerHitFn func(entry ScanEntry) error

ScannerHitFn is a function called on every scanner's hit. Function receives the ScanEntry, describing the found cache item. If the function returns an error, the iteration will be stopped.

type Values

type Values map[string]any

Values are a custom values map

func NewValues

func NewValues(keysAndValues ...any) Values

NewValues creates a new custom Values map. The keysAndValues parameters expect a list of keys and values in format: key1, value1, key2, value2...

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL