Documentation
¶
Overview ¶
Package writecache implements write-cache for objects.
Write-cache has 2 components: 1. Key-value (bbolt) database for storing small objects. 2. Filesystem tree for storing big objects.
Flushing from the writecache to the main storage is done in the background. To make it possible to serve Read requests after the object was flushed, we maintain an LRU cache containing addresses of all the objects that could be safely deleted. The actual deletion is done during eviction from this cache.
Index ¶
- Variables
- func Get(db *bbolt.DB, key []byte) ([]byte, error)
- func IterateDB(db *bbolt.DB, f func(oid.Address) error) error
- func OpenDB(p string, ro bool) (*bbolt.DB, error)
- type Cache
- type Info
- type IterationPrm
- type ObjectStatus
- type Option
- func WithBlobstor(bs *blobstor.BlobStor) Option
- func WithFlushWorkersCount(c int) Option
- func WithLogger(log *zap.Logger) Option
- func WithMaxBatchDelay(d time.Duration) Option
- func WithMaxBatchSize(sz int) Option
- func WithMaxCacheSize(sz uint64) Option
- func WithMaxObjectSize(sz uint64) Option
- func WithMetabase(db *meta.DB) Option
- func WithNoSync(noSync bool) Option
- func WithPath(path string) Option
- func WithReportErrorFunc(f func(string, error)) Option
- func WithSmallObjectSize(sz uint64) Option
Constants ¶
This section is empty.
Variables ¶
var ( // ErrBigObject is returned when object is too big to be placed in cache. ErrBigObject = errors.New("too big object") // ErrOutOfSpace is returned when there is no space left to put a new object. ErrOutOfSpace = errors.New("no space left in the write cache") )
var ErrNoDefaultBucket = errors.New("no default bucket")
ErrNoDefaultBucket is returned by IterateDB when default bucket for objects is missing.
var ErrReadOnly = logicerr.New("write-cache is in read-only mode")
ErrReadOnly is returned when Put/Write is performed in a read-only mode.
Functions ¶
func Get ¶ added in v0.26.0
Get fetches object from the underlying database. Key should be a stringified address.
Returns an error of type apistatus.ObjectNotFound if the requested object is missing in db.
func IterateDB ¶ added in v0.26.0
IterateDB iterates over all objects stored in bbolt.DB instance and passes them to f until error return. It is assumed that db is an underlying database of some WriteCache instance.
Returns ErrNoDefaultBucket if there is no default bucket in db.
DB must not be nil and should be opened.
Types ¶
type Cache ¶
type Cache interface {
Get(address oid.Address) (*object.Object, error)
// GetBytes reads object from the Cache by address into memory buffer in a
// canonical NeoFS binary format. Returns [apistatus.ObjectNotFound] if object
// is missing.
GetBytes(oid.Address) ([]byte, error)
Head(oid.Address) (*object.Object, error)
// Delete removes object referenced by the given oid.Address from the
// Cache. Returns any error encountered that prevented the object to be
// removed.
//
// Returns apistatus.ObjectNotFound if object is missing in the Cache.
// Returns ErrReadOnly if the Cache is currently in the read-only mode.
Delete(oid.Address) error
Iterate(IterationPrm) error
Put(common.PutPrm) (common.PutRes, error)
SetMode(mode.Mode) error
SetLogger(*zap.Logger)
DumpInfo() Info
Flush(bool) error
Init() error
Open(readOnly bool) error
Close() error
ObjectStatus(address oid.Address) (ObjectStatus, error)
}
Cache represents write-cache for objects.
type Info ¶ added in v0.27.2
type Info struct {
// Full path to the write-cache.
Path string
}
Info groups the information about write-cache.
type IterationPrm ¶ added in v0.27.5
type IterationPrm struct {
// contains filtered or unexported fields
}
IterationPrm contains iteration parameters.
func (*IterationPrm) WithHandler ¶ added in v0.27.5
func (p *IterationPrm) WithHandler(f func([]byte) error)
WithHandler sets a callback to be executed on every object.
func (*IterationPrm) WithIgnoreErrors ¶ added in v0.27.5
func (p *IterationPrm) WithIgnoreErrors(ignore bool)
WithIgnoreErrors sets a flag indicating that errors should be ignored.
type ObjectStatus ¶ added in v0.39.0
ObjectStatus represents the status of the object in the Writecache.
type Option ¶
type Option func(*options)
Option represents write-cache configuration option.
func WithBlobstor ¶
WithBlobstor sets main object storage.
func WithFlushWorkersCount ¶
func WithMaxBatchDelay ¶ added in v0.29.0
WithMaxBatchDelay sets max batch delay for the small object database.
func WithMaxBatchSize ¶ added in v0.29.0
WithMaxBatchSize sets max batch size for the small object database.
func WithMaxCacheSize ¶ added in v0.25.0
WithMaxCacheSize sets maximum write-cache size in bytes.
func WithMaxObjectSize ¶
WithMaxObjectSize sets maximum object size to be stored in write-cache.
func WithNoSync ¶ added in v0.36.0
WithNoSync sets an option to allow returning to caller on PUT before write is persisted. Note, that we use this flag for FSTree only and DO NOT use it for a bolt DB because we cannot yet properly handle the corrupted database during the startup. This SHOULD NOT be relied upon and may be changed in future.
func WithReportErrorFunc ¶ added in v0.36.0
WithReportErrorFunc sets error reporting function.
func WithSmallObjectSize ¶
WithSmallObjectSize sets maximum object size to be stored in write-cache.
Source Files