fcache

package module
v0.5.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 2, 2022 License: MIT Imports: 14 Imported by: 0

README

fcache Go codecov go report card Go Reference

Package fcache introduces file cache implementation for caching files.

s3

Note: s3 file cache doesn't expire files by its own, for doing that you have to set lifecycle policy for the bucket, that will be used for caching

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var ErrNotFound = errors.New("not found")

ErrNotFound represents a not found error.

Functions

This section is empty.

Types

type CacheStats added in v0.2.0 

type CacheStats struct {
	Hits   int64
	Misses int64
	Errors int64
	StoreStats
}

CacheStats represent stat values.

type FileMeta added in v0.1.1 

type FileMeta struct {
	Name string
	Mime string

	// Meta maintains additional data about the file.
	// Note: some metadata keys with leading underscore are reserved by fcache.
	Meta map[string]string

	// Size might not be provided when loading file, though it might be useful
	// for some cache implementations, like S3 as it runs streaming multipart
	// method, if size is provided
	Size int64

	// store stat fields
	Key       string
	CreatedAt time.Time
}

FileMeta represent information about the file.

type GetRequest added in v0.5.0 

type GetRequest struct {
	Key string
	TTL time.Duration
	Loader
}

GetRequest defines parameters to get file.

type GetURLParams added in v0.3.0 

type GetURLParams struct {
	Filename string
	Expires  time.Duration
}

GetURLParams describes additional parameters, besides key, to form a signed URL.

type Loader added in v0.1.1 

type Loader func(ctx context.Context) (io.ReadCloser, FileMeta, error)

Loader is a function to load a file in case if it's missing in cache.

type LoadingCache added in v0.2.0 

type LoadingCache struct {
	Store
	Options
	CacheStats
	// contains filtered or unexported fields
}

LoadingCache is a wrapper for Store, which removes file at their TTL. Only files, added by GetFile and GetURL methods will be removed.

func NewLoadingCache added in v0.2.0 

func NewLoadingCache(backend Store, opts ...Option) *LoadingCache

NewLoadingCache makes new instance of LoadingCache.

func (*LoadingCache) GetFile added in v0.2.0 

func (l *LoadingCache) GetFile(ctx context.Context, req GetRequest) (rd io.ReadCloser, meta FileMeta, err error)

GetFile gets the file from cache or loads it, if absent.

func (*LoadingCache) GetURL added in v0.2.0 

func (l *LoadingCache) GetURL(ctx context.Context, req GetRequest, params GetURLParams) (url string, meta FileMeta, err error)

GetURL returns the URL from the cache backend.

func (*LoadingCache) Invalidate added in v0.5.0 

func (l *LoadingCache) Invalidate(ctx context.Context) (invalidated int64, err error)

Invalidate invalidates expired cache items. Used for tests.

func (*LoadingCache) Run added in v0.2.0 

func (l *LoadingCache) Run(ctx context.Context) error

Run runs invalidation goroutine. It will check for files TTL expiration and, if it expires, removes it manually.

func (*LoadingCache) Stat added in v0.2.0 

func (l *LoadingCache) Stat(ctx context.Context) (CacheStats, error)

Stat returns cache stats

type Logger 

type Logger interface {
	Printf(format string, args ...interface{})
}

Logger defines a single method for logging in caches.

func NopLogger added in v0.2.0 

func NopLogger() Logger

NopLogger returns a no-op logger.

type Option 

type Option func(*Options)

Option is a function to apply options.

func WithInvalidationPeriod 

func WithInvalidationPeriod(period time.Duration) Option

WithInvalidationPeriod sets the period between cache's checks for expired items. Useful for caches, like S3. No manual invalidation by default.

func WithLogger 

func WithLogger(log Logger) Option

WithLogger sets logger for cache. `log` package is used by default.

type Options 

type Options struct {
	Log Logger
	// InvalidatePeriod sets the time for checking cache for expired items.
	// Zero means "no invalidation", i.e. backend invalidates items by its own.
	InvalidatePeriod time.Duration
	// ExtendTTL sets whether cache should extend TTL of cached items on hit.
	ExtendTTL bool
}

Options defines cache options.

type S3 

type S3 struct {
	// contains filtered or unexported fields
}

S3 implements Cache for S3.

func NewS3 

func NewS3(cl *minio.Client, bucket, prefix string, log Logger) *S3

NewS3 makes new instance of S3.

func (*S3) Get added in v0.2.0 

func (s *S3) Get(ctx context.Context, key string) (io.ReadCloser, error)

Get gets the file from cache or loads it, if absent. NOTE: if file under this key is not present in S3, this method WILL NOT return a NotFound error, instead, it will return a reader, which will return error when there will be an attempt to read.

func (*S3) GetURL 

func (s *S3) GetURL(ctx context.Context, key string, params GetURLParams) (string, error)

GetURL returns the URL from the cache backend.

func (*S3) Keys 

func (s *S3) Keys(ctx context.Context) ([]string, error)

Keys returns all keys, present in cache.

func (*S3) List added in v0.2.0 

func (s *S3) List(ctx context.Context) ([]FileMeta, error)

List lists object in S3 bucket.

func (*S3) Meta added in v0.2.0 

func (s *S3) Meta(ctx context.Context, key string) (FileMeta, error)

Meta returns meta information about the file at underlying key.

func (*S3) Put added in v0.2.0 

func (s *S3) Put(ctx context.Context, key string, meta FileMeta, rd io.ReadCloser) error

Put puts file into S3.

func (*S3) Remove added in v0.2.0 

func (s *S3) Remove(ctx context.Context, key string) error

Remove removes file by its key.

func (*S3) Stat 

func (s *S3) Stat(ctx context.Context) (res StoreStats, err error)

Stat returns cache stats.

func (*S3) UpdateMeta added in v0.5.0 

func (s *S3) UpdateMeta(ctx context.Context, key string, meta FileMeta) error

UpdateMeta updates meta information about the file at underlying key.

type Store added in v0.2.0 

type Store interface {
	Meta(ctx context.Context, key string) (FileMeta, error)
	UpdateMeta(ctx context.Context, key string, meta FileMeta) error
	Get(ctx context.Context, key string) (rd io.ReadCloser, err error)
	GetURL(ctx context.Context, key string, params GetURLParams) (url string, err error)
	Put(ctx context.Context, key string, meta FileMeta, rd io.ReadCloser) error
	Remove(ctx context.Context, key string) error
	Stat(ctx context.Context) (StoreStats, error)
	Keys(ctx context.Context) ([]string, error)
	List(ctx context.Context) ([]FileMeta, error)
}

Store defines methods that the backend store should implement

type StoreMock added in v0.2.0 

type StoreMock struct {
	// GetFunc mocks the Get method.
	GetFunc func(ctx context.Context, key string) (io.ReadCloser, error)

	// GetURLFunc mocks the GetURL method.
	GetURLFunc func(ctx context.Context, key string, params GetURLParams) (string, error)

	// KeysFunc mocks the Keys method.
	KeysFunc func(ctx context.Context) ([]string, error)

	// ListFunc mocks the List method.
	ListFunc func(ctx context.Context) ([]FileMeta, error)

	// MetaFunc mocks the Meta method.
	MetaFunc func(ctx context.Context, key string) (FileMeta, error)

	// PutFunc mocks the Put method.
	PutFunc func(ctx context.Context, key string, meta FileMeta, rd io.ReadCloser) error

	// RemoveFunc mocks the Remove method.
	RemoveFunc func(ctx context.Context, key string) error

	// StatFunc mocks the Stat method.
	StatFunc func(ctx context.Context) (StoreStats, error)

	// UpdateMetaFunc mocks the UpdateMeta method.
	UpdateMetaFunc func(ctx context.Context, key string, meta FileMeta) error
	// contains filtered or unexported fields
}

StoreMock is a mock implementation of Store. 

func TestSomethingThatUsesStore(t *testing.T) {

	// make and configure a mocked Store
	mockedStore := &StoreMock{
		GetFunc: func(ctx context.Context, key string) (io.ReadCloser, error) {
			panic("mock out the Get method")
		},
		GetURLFunc: func(ctx context.Context, key string, params GetURLParams) (string, error) {
			panic("mock out the GetURL method")
		},
		KeysFunc: func(ctx context.Context) ([]string, error) {
			panic("mock out the Keys method")
		},
		ListFunc: func(ctx context.Context) ([]FileMeta, error) {
			panic("mock out the List method")
		},
		MetaFunc: func(ctx context.Context, key string) (FileMeta, error) {
			panic("mock out the Meta method")
		},
		PutFunc: func(ctx context.Context, key string, meta FileMeta, rd io.ReadCloser) error {
			panic("mock out the Put method")
		},
		RemoveFunc: func(ctx context.Context, key string) error {
			panic("mock out the Remove method")
		},
		StatFunc: func(ctx context.Context) (StoreStats, error) {
			panic("mock out the Stat method")
		},
		UpdateMetaFunc: func(ctx context.Context, key string, meta FileMeta) error {
			panic("mock out the UpdateMeta method")
		},
	}

	// use mockedStore in code that requires Store
	// and then make assertions.

}

func (*StoreMock) Get added in v0.2.0 

func (mock *StoreMock) Get(ctx context.Context, key string) (io.ReadCloser, error)

Get calls GetFunc.

func (*StoreMock) GetCalls added in v0.2.0 

func (mock *StoreMock) GetCalls() []struct {
	Ctx context.Context
	Key string
}

GetCalls gets all the calls that were made to Get. Check the length with: 

len(mockedStore.GetCalls())

func (*StoreMock) GetURL added in v0.2.0 

func (mock *StoreMock) GetURL(ctx context.Context, key string, params GetURLParams) (string, error)

GetURL calls GetURLFunc.

func (*StoreMock) GetURLCalls added in v0.2.0 

func (mock *StoreMock) GetURLCalls() []struct {
	Ctx    context.Context
	Key    string
	Params GetURLParams
}

GetURLCalls gets all the calls that were made to GetURL. Check the length with: 

len(mockedStore.GetURLCalls())

func (*StoreMock) Keys added in v0.2.0 

func (mock *StoreMock) Keys(ctx context.Context) ([]string, error)

Keys calls KeysFunc.

func (*StoreMock) KeysCalls added in v0.2.0 

func (mock *StoreMock) KeysCalls() []struct {
	Ctx context.Context
}

KeysCalls gets all the calls that were made to Keys. Check the length with: 

len(mockedStore.KeysCalls())

func (*StoreMock) List added in v0.2.0 

func (mock *StoreMock) List(ctx context.Context) ([]FileMeta, error)

List calls ListFunc.

func (*StoreMock) ListCalls added in v0.2.0 

func (mock *StoreMock) ListCalls() []struct {
	Ctx context.Context
}

ListCalls gets all the calls that were made to List. Check the length with: 

len(mockedStore.ListCalls())

func (*StoreMock) Meta added in v0.2.0 

func (mock *StoreMock) Meta(ctx context.Context, key string) (FileMeta, error)

Meta calls MetaFunc.

func (*StoreMock) MetaCalls added in v0.2.0 

func (mock *StoreMock) MetaCalls() []struct {
	Ctx context.Context
	Key string
}

MetaCalls gets all the calls that were made to Meta. Check the length with: 

len(mockedStore.MetaCalls())

func (*StoreMock) Put added in v0.2.0 

func (mock *StoreMock) Put(ctx context.Context, key string, meta FileMeta, rd io.ReadCloser) error

Put calls PutFunc.

func (*StoreMock) PutCalls added in v0.2.0 

func (mock *StoreMock) PutCalls() []struct {
	Ctx  context.Context
	Key  string
	Meta FileMeta
	Rd   io.ReadCloser
}

PutCalls gets all the calls that were made to Put. Check the length with: 

len(mockedStore.PutCalls())

func (*StoreMock) Remove added in v0.2.0 

func (mock *StoreMock) Remove(ctx context.Context, key string) error

Remove calls RemoveFunc.

func (*StoreMock) RemoveCalls added in v0.2.0 

func (mock *StoreMock) RemoveCalls() []struct {
	Ctx context.Context
	Key string
}

RemoveCalls gets all the calls that were made to Remove. Check the length with: 

len(mockedStore.RemoveCalls())

func (*StoreMock) Stat added in v0.2.0 

func (mock *StoreMock) Stat(ctx context.Context) (StoreStats, error)

Stat calls StatFunc.

func (*StoreMock) StatCalls added in v0.2.0 

func (mock *StoreMock) StatCalls() []struct {
	Ctx context.Context
}

StatCalls gets all the calls that were made to Stat. Check the length with: 

len(mockedStore.StatCalls())

func (*StoreMock) UpdateMeta added in v0.5.0 

func (mock *StoreMock) UpdateMeta(ctx context.Context, key string, meta FileMeta) error

UpdateMeta calls UpdateMetaFunc.

func (*StoreMock) UpdateMetaCalls added in v0.5.0 

func (mock *StoreMock) UpdateMetaCalls() []struct {
	Ctx  context.Context
	Key  string
	Meta FileMeta
}

UpdateMetaCalls gets all the calls that were made to UpdateMeta. Check the length with: 

len(mockedStore.UpdateMetaCalls())

type StoreStats added in v0.2.0 

type StoreStats struct {
	Keys int
	Size int64
}

StoreStats represents stats of the backend store.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.