filecache

package module
v1.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Dec 18, 2019 License: MIT Imports: 10 Imported by: 0

README

Go FileCache

Build Status Coverage GoDoc GoReport

Store data from any io.Reader to cache files with TTL and metadata.

Installation

go get -u gitlab.com/kukymbrgo/filecache

Usage

package main

import (
    "gitlab.com/kukymbrgo/filecache"
    "io/ioutil"
)

func main()  {
	
    // Set defaults for all instances (optional of course):
    // items namespace
    filecache.NamespaceDefault = "dft"
    
    // default cache files extension
    filecache.ExtDefault = ".cache"
    
    // default time-to-live in seconds; set -1 to eternal
    filecache.TTLDefault = -1
    
    // Set garbage collector run probability divisor
    // (e.g. 10 is 1/10 probability), optional
    filecache.GCDivisor = 10
	
    // Initialize cache instance
    fc, err := filecache.New("/path/to/cache/dir")
    if err != nil {
    	panic(err)
    } 
    
    // Set instance defaults:
    fc.NamespaceDefault = "wiki"
    fc.TTLDefault = 3600
    fc.Ext = ".html"
    
    // Read and write some data 
    
    pageUrl := "https://en.wikipedia.org/wiki/Main_Page"
    
    item, err := fc.Read(pageUrl, "")
    
    if err != nil {
        // Get some reader to read from
        downloader := getPageDownloaderReader()
        // Read from the reader to the cache
        item, _, err = fc.WriteOpen(&filecache.Meta{Key: pageUrl}, downloader)
        if err != nil {
            // If failed to cache, handle the error       
            panic(err)
        }
    }
    
    // Do some stuff
    _, _ = ioutil.ReadAll(item.File)
}
Scanner

If you need to iterate through existing cache files, you can use the filecache.Scanner tool.

License

MIT. See the LICENSE file.

Documentation

Overview

Package filecache is a tool to write data from any io.Reader to cache files with TTL and metadata.

Index

Constants

View Source
const MetaPostfix = "--meta"

MetaPostfix is a metadata files name postfix

Variables

View Source
var (
	// NamespaceDefault is a default cache files namespace
	NamespaceDefault = "dft"

	// ExtDefault is a default files extension string
	ExtDefault = ".cache"

	// TTLDefault is a default value
	// in seconds of cache items' Time-To-Live
	TTLDefault = int64(-1)

	// GCDivisor is a garbage collector run probability divisor
	// (e.g. 100 is 1/100 probability)
	GCDivisor uint = 100
)

Functions

This section is empty.

Types

type FileCache

type FileCache struct {

	// NamespaceDefault is a default namespace if left empty in functions params
	NamespaceDefault string

	// Ext is a cache files extension string
	Ext string

	// TTLDefault is a default TTL of cache instance items
	TTLDefault int64
	// contains filtered or unexported fields
}

FileCache is a file-based cache structure

func New

func New(path string) (*FileCache, error)

New creates new file cache instance

func (*FileCache) Create added in v1.2.0

func (fc *FileCache) Create(meta *Meta) (item *Item, err error)

Create cache file by metadata and open it. Returns cache Item and error if occurs.

func (*FileCache) Invalidate added in v1.0.0

func (fc *FileCache) Invalidate(key string, namespace string) error

Invalidate deletes cache item by its key & namespace

func (*FileCache) Path

func (fc *FileCache) Path() string

Path returns the cache directory root path

func (*FileCache) Read added in v1.0.0

func (fc *FileCache) Read(key string, namespace string) (item *Item, err error)

Read returns cache Item if exists

func (*FileCache) Write added in v1.0.0

func (fc *FileCache) Write(meta *Meta, src io.Reader) (written int64, err error)

Write copies data from src Reader to cache file Returns the count of written bytes

func (*FileCache) WriteOpen added in v1.1.0

func (fc *FileCache) WriteOpen(meta *Meta, src io.Reader) (item *Item, written int64, err error)

WriteOpen copies data from src Reader to cache file and returns opened cache Item and count of written bytes

type Item added in v1.0.0

type Item struct {
	File *os.File
	Meta *Meta
	Path string
}

Item is a cached item structure

func (*Item) Close added in v1.1.0

func (i *Item) Close() error

Close file descriptor

type Meta added in v1.0.0

type Meta struct {
	// Key is a non-hashed unique for namespace item id
	// Required
	Key string `json:"k"`

	// Namespace is a cache item category folder name
	// If empty FileCache.NamespaceDefault will be set
	Namespace string `json:"n"`

	// OriginalName is an original file name; optional
	OriginalName string `json:"o,omitempty"`

	// TTL is a item's time-to-live value in seconds
	TTL int64 `json:"t"`

	// Created is a time when cache file was written
	// Do not set it by yourself
	Created int64 `json:"c"`

	// Fields is a map of any others metadata fields
	Fields map[string]interface{} `json:"f,omitempty"`
}

Meta is a cache item metadata structure Short json keys are used to reduce file size

func MetaFromFile added in v1.3.0

func MetaFromFile(path string) (meta *Meta, err error)

MetaFromFile reads cache metadata from file

func (*Meta) IsExpired added in v1.3.0

func (m *Meta) IsExpired() bool

IsExpired returns true if file is expired or if its TTL is 0

func (*Meta) SaveToFile added in v1.3.0

func (m *Meta) SaveToFile(path string) error

SaveToFile saves JSON-encoded metadata to file

type Scanner added in v1.3.0

type Scanner struct {
	// contains filtered or unexported fields
}

Scanner is a tool to walk through existing cache files

func NewScanner added in v1.3.0

func NewScanner(fc *FileCache) *Scanner

NewScanner creates new Scanner instance

func (*Scanner) Scan added in v1.3.0

func (s *Scanner) Scan(hitFunc ScannerHitFunc, skipExpired bool, ignoreLStatErrors bool) error

Scan walks through existing cache files and executes the hit function on every cache file found.

type ScannerHitFunc added in v1.3.0

type ScannerHitFunc = func(meta *Meta, itemPath string, metaPath string) error

ScannerHitFunc is a function called on every cache file hit while scanning. Receives found cache item meta, path of cached content file & its info.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL