meta

package
v0.0.0-...-79ba347 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 1, 2023 License: GPL-3.0 Imports: 31 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source
var (
	ErrUnknownObjectType        = errors.New("unknown object type")
	ErrIncorrectSplitInfoUpdate = errors.New("updating split info on object without it")
	ErrIncorrectRootObject      = errors.New("invalid root object")
)
View Source
var ErrDegradedMode = logicerr.New("metabase is in a degraded mode")

ErrDegradedMode is returned when metabase is in a degraded mode.

View Source
var ErrEndOfListing = logicerr.New("end of object listing")

ErrEndOfListing is returned from object listing with cursor when storage can't return any more objects after provided cursor. Use nil cursor object to start listing again.

View Source
var ErrInterruptIterator = logicerr.New("iterator is interrupted")

ErrInterruptIterator is returned by iteration handlers as a "break" keyword.

View Source
var ErrLackSplitInfo = logicerr.New("no split info on parent object")
View Source
var ErrLockObjectRemoval = logicerr.New("lock object removal")

ErrLockObjectRemoval is returned when inhume operation is being performed on lock object, and it is not a forced object removal.

View Source
var ErrObjectIsExpired = logicerr.New("object is expired")

ErrObjectIsExpired is returned when the requested object's epoch is less than the current one. Such objects are considered as removed and should not be returned from the Storage Engine.

View Source
var ErrOutdatedVersion = logicerr.New("invalid version, resynchronization is required")

ErrOutdatedVersion is returned on initializing an existing metabase that is not compatible with the current code version.

View Source
var ErrReadOnlyMode = logicerr.New("metabase is in a read-only mode")

ErrReadOnlyMode is returned when metabase is in a read-only mode.

Functions

func IsErrRemoved

func IsErrRemoved(err error) bool

IsErrRemoved checks if error returned by Shard Exists/Get/Put method corresponds to removed object.

Types

type Cursor

type Cursor struct {
	// contains filtered or unexported fields
}

Cursor is a type for continuous object listing.

type DB

type DB struct {
	// contains filtered or unexported fields
}

DB represents local metabase of storage node.

func New

func New(opts ...Option) *DB

New creates and returns new Metabase instance.

func (*DB) Close

func (db *DB) Close() error

Close closes boltDB instance.

func (*DB) ContainerSize

func (db *DB) ContainerSize(id cid.ID) (size uint64, err error)

func (*DB) Containers

func (db *DB) Containers() (list []cid.ID, err error)

func (*DB) Delete

func (db *DB) Delete(prm DeletePrm) (DeleteRes, error)

Delete removed object records from metabase indexes.

func (*DB) DoNotMove

func (db *DB) DoNotMove(prm DoNotMovePrm) (res DoNotMoveRes, err error)

DoNotMove removes `MoveIt` mark from the object.

func (*DB) DropGraves

func (db *DB) DropGraves(tss []TombstonedObject) error

DropGraves deletes tombstoned objects from the graveyard bucket.

Returns any error appeared during deletion process.

func (*DB) DumpInfo

func (db *DB) DumpInfo() Info

DumpInfo returns information about the DB.

func (*DB) Exists

func (db *DB) Exists(prm ExistsPrm) (res ExistsRes, err error)

Exists returns ErrAlreadyRemoved if addr was marked as removed. Otherwise it returns true if addr is in primary index or false if it is not.

Returns an error of type apistatus.ObjectAlreadyRemoved if object has been placed in graveyard. Returns the object.ErrObjectIsExpired if the object is presented but already expired.

func (*DB) FreeLockedBy

func (db *DB) FreeLockedBy(lockers []oid.Address) error

FreeLockedBy unlocks all objects in DB which are locked by lockers.

func (*DB) Get

func (db *DB) Get(prm GetPrm) (res GetRes, err error)

Get returns object header for specified address.

Returns an error of type apistatus.ObjectNotFound if object is missing in DB. Returns an error of type apistatus.ObjectAlreadyRemoved if object has been placed in graveyard. Returns the object.ErrObjectIsExpired if the object is presented but already expired.

func (*DB) Inhume

func (db *DB) Inhume(prm InhumePrm) (res InhumeRes, err error)

Inhume marks objects as removed but not removes it from metabase.

Allows inhuming non-locked objects only. Returns apistatus.ObjectLocked if at least one object is locked. Returns ErrLockObjectRemoval if inhuming is being performed on lock (not locked) object.

NOTE: Marks any object with GC mark (despite any prohibitions on operations with that object) if WithForceGCMark option has been provided.

func (*DB) Init

func (db *DB) Init() error

Init initializes metabase. It creates static (CID-independent) buckets in underlying BoltDB instance.

Returns ErrOutdatedVersion if a database at the provided path is outdated.

Does nothing if metabase has already been initialized and filled. To roll back the database to its initial state, use Reset.

func (*DB) IsLocked

func (db *DB) IsLocked(prm IsLockedPrm) (res IsLockedRes, err error)

IsLocked checks is the provided object is locked by any `LOCK`. Not found object is considered as non-locked.

Returns only non-logical errors related to underlying database.

func (*DB) IterateCoveredByTombstones

func (db *DB) IterateCoveredByTombstones(tss map[string]oid.Address, h func(oid.Address) error) error

IterateCoveredByTombstones iterates over all objects in DB which are covered by tombstone with string address from tss. Locked objects are not included (do not confuse with objects of type LOCK).

If h returns ErrInterruptIterator, nil returns immediately. Returns other errors of h directly.

Does not modify tss.

func (*DB) IterateExpired

func (db *DB) IterateExpired(epoch uint64, h ExpiredObjectHandler) error

IterateExpired iterates over all objects in DB which are out of date relative to epoch. Locked objects are not included (do not confuse with objects of type LOCK).

If h returns ErrInterruptIterator, nil returns immediately. Returns other errors of h directly.

func (*DB) IterateOverGarbage

func (db *DB) IterateOverGarbage(p GarbageIterationPrm) error

IterateOverGarbage iterates over all objects marked with GC mark.

If h returns ErrInterruptIterator, nil returns immediately. Returns other errors of h directly.

func (*DB) IterateOverGraveyard

func (db *DB) IterateOverGraveyard(p GraveyardIterationPrm) error

IterateOverGraveyard iterates over all graves in DB.

If h returns ErrInterruptIterator, nil returns immediately. Returns other errors of h directly.

func (*DB) ListWithCursor

func (db *DB) ListWithCursor(prm ListPrm) (res ListRes, err error)

ListWithCursor lists physical objects available in metabase starting from cursor. Includes objects of all types. Does not include inhumed objects. Use cursor value from response for consecutive requests.

Returns ErrEndOfListing if there are no more objects to return or count parameter set to zero.

func (*DB) Lock

func (db *DB) Lock(cnr cid.ID, locker oid.ID, locked []oid.ID) error

Lock marks objects as locked with another object. All objects are from the specified container.

Allows locking regular objects only (otherwise returns apistatus.LockNonRegularObject).

Locked list should be unique. Panics if it is empty.

func (*DB) Movable

func (db *DB) Movable(_ MovablePrm) (MovableRes, error)

Movable returns list of marked objects to move into other shard.

func (*DB) ObjectCounters

func (db *DB) ObjectCounters() (cc ObjectCounters, err error)

ObjectCounters returns object counters that metabase has tracked since it was opened and initialized.

Returns only the errors that do not allow reading counter in Bolt database.

func (*DB) Open

func (db *DB) Open(readOnly bool) error

Open boltDB instance for metabase.

func (*DB) Put

func (db *DB) Put(prm PutPrm) (res PutRes, err error)

Put saves object header in metabase. Object payload expected to be cut.

Returns an error of type apistatus.ObjectAlreadyRemoved if object has been placed in graveyard. Returns the object.ErrObjectIsExpired if the object is presented but already expired.

func (*DB) ReadShardID

func (db *DB) ReadShardID() ([]byte, error)

ReadShardID reads shard id from db. If id is missing, returns nil, nil.

func (*DB) Reload

func (db *DB) Reload(opts ...Option) (bool, error)

Reload reloads part of the configuration. It returns true iff database was reopened. If a config option is invalid, it logs an error and returns nil. If there was a problem with applying new configuration, an error is returned.

If a metabase was couldn't be reopened because of an error, ErrDegradedMode is returned.

func (*DB) Reset

func (db *DB) Reset() error

Reset resets metabase. Works similar to Init but cleans up all static buckets and removes all dynamic (CID-dependent) ones in non-blank BoltDB instances.

func (*DB) Select

func (db *DB) Select(prm SelectPrm) (res SelectRes, err error)

Select returns list of addresses of objects that match search filters.

func (*DB) SetLogger

func (db *DB) SetLogger(l *logger.Logger)

SetLogger sets logger. It is used after the shard ID was generated to use it in logs.

func (*DB) SetMode

func (db *DB) SetMode(m mode.Mode) error

SetMode sets the metabase mode of operation. If the mode assumes no operation metabase, the database is closed.

func (*DB) StorageID

func (db *DB) StorageID(prm StorageIDPrm) (res StorageIDRes, err error)

StorageID returns storage descriptor for objects from the blobstor. It is put together with the object can makes get/delete operation faster.

func (*DB) SyncCounters

func (db *DB) SyncCounters() error

SyncCounters forces to synchronize the object counters.

func (*DB) ToMoveIt

func (db *DB) ToMoveIt(prm ToMoveItPrm) (res ToMoveItRes, err error)

ToMoveIt marks objects to move it into another shard. This useful for faster HRW fetching.

func (*DB) UpdateStorageID

func (db *DB) UpdateStorageID(prm UpdateStorageIDPrm) (res UpdateStorageIDRes, err error)

UpdateStorageID updates storage descriptor for objects from the blobstor.

func (*DB) WriteShardID

func (db *DB) WriteShardID(id []byte) error

WriteShardID writes shard it to db.

type DeletePrm

type DeletePrm struct {
	// contains filtered or unexported fields
}

DeletePrm groups the parameters of Delete operation.

func (*DeletePrm) SetAddresses

func (p *DeletePrm) SetAddresses(addrs ...oid.Address)

SetAddresses is a Delete option to set the addresses of the objects to delete.

Option is required.

type DeleteRes

type DeleteRes struct {
	// contains filtered or unexported fields
}

DeleteRes groups the resulting values of Delete operation.

func (DeleteRes) AvailableObjectsRemoved

func (d DeleteRes) AvailableObjectsRemoved() uint64

AvailableObjectsRemoved returns the number of removed available objects.

func (DeleteRes) RawObjectsRemoved

func (d DeleteRes) RawObjectsRemoved() uint64

RawObjectsRemoved returns the number of removed raw objects.

func (DeleteRes) RemovedLogicalObjectSizes

func (d DeleteRes) RemovedLogicalObjectSizes() []uint64

RemovedLogicalObjectSizes returns the sizes of removed logical objects.

func (DeleteRes) RemovedPhysicalObjectSizes

func (d DeleteRes) RemovedPhysicalObjectSizes() []uint64

RemovedPhysicalObjectSizes returns the sizes of removed physical objects.

type DeletionInfo

type DeletionInfo struct {
	Size uint64
	CID  cid.ID
}

DeletionInfo contains details on deleted object.

type DoNotMovePrm

type DoNotMovePrm struct {
	// contains filtered or unexported fields
}

DoNotMovePrm groups the parameters of DoNotMove operation.

func (*DoNotMovePrm) SetAddress

func (p *DoNotMovePrm) SetAddress(addr oid.Address)

SetAddress sets address of the object to prevent moving into another shard.

type DoNotMoveRes

type DoNotMoveRes struct{}

DoNotMoveRes groups the resulting values of DoNotMove operation.

type EpochState

type EpochState interface {
	// CurrentEpoch must return current epoch height.
	CurrentEpoch() uint64
}

EpochState is an interface that provides access to the current epoch number.

type ExistsPrm

type ExistsPrm struct {
	// contains filtered or unexported fields
}

ExistsPrm groups the parameters of Exists operation.

func (*ExistsPrm) SetAddress

func (p *ExistsPrm) SetAddress(addr oid.Address)

SetAddress is an Exists option to set object checked for existence.

type ExistsRes

type ExistsRes struct {
	// contains filtered or unexported fields
}

ExistsRes groups the resulting values of Exists operation.

func (ExistsRes) Exists

func (p ExistsRes) Exists() bool

Exists returns the fact that the object is in the metabase.

type ExpiredObject

type ExpiredObject struct {
	// contains filtered or unexported fields
}

ExpiredObject is a descriptor of expired object from DB.

func (*ExpiredObject) Address

func (e *ExpiredObject) Address() oid.Address

Address returns address of the expired object.

func (*ExpiredObject) Type

func (e *ExpiredObject) Type() object.Type

Type returns type of the expired object.

type ExpiredObjectHandler

type ExpiredObjectHandler func(*ExpiredObject) error

ExpiredObjectHandler is an ExpiredObject handling function.

type GarbageHandler

type GarbageHandler func(GarbageObject) error

GarbageHandler is a GarbageObject handling function.

type GarbageIterationPrm

type GarbageIterationPrm struct {
	// contains filtered or unexported fields
}

GarbageIterationPrm groups parameters of the garbage iteration process.

func (*GarbageIterationPrm) SetHandler

func (g *GarbageIterationPrm) SetHandler(h GarbageHandler)

SetHandler sets a handler that will be called on every GarbageObject.

func (*GarbageIterationPrm) SetOffset

func (g *GarbageIterationPrm) SetOffset(offset oid.Address)

SetOffset sets an offset of the iteration operation. The handler will be applied to the next after the specified offset if any are left.

Note: if offset is not found in db, iteration starts from the element that WOULD BE the following after the offset if offset was presented. That means that it is safe to delete offset element and pass if to the iteration once again: iteration would start from the next element.

Nil offset means start an integration from the beginning.

type GarbageObject

type GarbageObject struct {
	// contains filtered or unexported fields
}

GarbageObject represents descriptor of the object that has been marked with GC.

func (GarbageObject) Address

func (g GarbageObject) Address() oid.Address

Address returns garbage object address.

type GetPrm

type GetPrm struct {
	// contains filtered or unexported fields
}

GetPrm groups the parameters of Get operation.

func (*GetPrm) SetAddress

func (p *GetPrm) SetAddress(addr oid.Address)

SetAddress is a Get option to set the address of the requested object.

Option is required.

func (*GetPrm) SetRaw

func (p *GetPrm) SetRaw(raw bool)

SetRaw is a Get option to set raw flag value. If flag is unset, then Get returns header of virtual object, otherwise it returns SplitInfo of virtual object.

type GetRes

type GetRes struct {
	// contains filtered or unexported fields
}

GetRes groups the resulting values of Get operation.

func (GetRes) Header

func (r GetRes) Header() *objectSDK.Object

Header returns the requested object header.

type GraveyardIterationPrm

type GraveyardIterationPrm struct {
	// contains filtered or unexported fields
}

GraveyardIterationPrm groups parameters of the graveyard iteration process.

func (*GraveyardIterationPrm) SetHandler

func (g *GraveyardIterationPrm) SetHandler(h TombstonedHandler)

SetHandler sets a handler that will be called on every TombstonedObject.

func (*GraveyardIterationPrm) SetOffset

func (g *GraveyardIterationPrm) SetOffset(offset oid.Address)

SetOffset sets an offset of the iteration operation. The handler will be applied to the next after the specified offset if any are left.

Note: if offset is not found in db, iteration starts from the element that WOULD BE the following after the offset if offset was presented. That means that it is safe to delete offset element and pass if to the iteration once again: iteration would start from the next element.

Nil offset means start an integration from the beginning.

type Info

type Info struct {
	// Full path to the metabase.
	Path string

	// Permission of database file.
	Permission fs.FileMode
}

Info groups the information about DB.

type InhumePrm

type InhumePrm struct {
	// contains filtered or unexported fields
}

InhumePrm encapsulates parameters for Inhume operation.

func (*InhumePrm) SetAddresses

func (p *InhumePrm) SetAddresses(addrs ...oid.Address)

SetAddresses sets a list of object addresses that should be inhumed.

func (*InhumePrm) SetForceGCMark

func (p *InhumePrm) SetForceGCMark()

SetForceGCMark allows removal any object. Expected to be called only in control service.

func (*InhumePrm) SetGCMark

func (p *InhumePrm) SetGCMark()

SetGCMark marks the object to be physically removed.

Should not be called along with SetTombstoneAddress.

func (*InhumePrm) SetLockObjectHandling

func (p *InhumePrm) SetLockObjectHandling()

SetLockObjectHandling checks if there were any LOCK object among the targets set via WithAddresses.

func (*InhumePrm) SetTombstoneAddress

func (p *InhumePrm) SetTombstoneAddress(addr oid.Address)

SetTombstoneAddress sets tombstone address as the reason for inhume operation.

addr should not be nil. Should not be called along with SetGCMark.

type InhumeRes

type InhumeRes struct {
	// contains filtered or unexported fields
}

InhumeRes encapsulates results of Inhume operation.

func (InhumeRes) AvailableInhumed

func (i InhumeRes) AvailableInhumed() uint64

AvailableInhumed return number of available object that have been inhumed.

func (InhumeRes) DeletedLockObjects

func (i InhumeRes) DeletedLockObjects() []oid.Address

DeletedLockObjects returns deleted object of LOCK type. Returns always nil if WithoutLockObjectHandling was provided to the InhumePrm.

func (InhumeRes) GetDeletionInfoByIndex

func (i InhumeRes) GetDeletionInfoByIndex(target int) DeletionInfo

GetDeletionInfoByIndex returns both deleted object sizes and associated container ID by index.

func (InhumeRes) GetDeletionInfoLength

func (i InhumeRes) GetDeletionInfoLength() int

GetDeletionInfoLength returns amount of stored elements in deleted sizes array.

type IsLockedPrm

type IsLockedPrm struct {
	// contains filtered or unexported fields
}

IsLockedPrm groups the parameters of IsLocked operation.

func (*IsLockedPrm) SetAddress

func (i *IsLockedPrm) SetAddress(addr oid.Address)

SetAddress sets object address that will be checked for lock relations.

type IsLockedRes

type IsLockedRes struct {
	// contains filtered or unexported fields
}

IsLockedRes groups the resulting values of IsLocked operation.

func (IsLockedRes) Locked

func (i IsLockedRes) Locked() bool

Locked describes the requested object status according to the metabase current state.

type ListPrm

type ListPrm struct {
	// contains filtered or unexported fields
}

ListPrm contains parameters for ListWithCursor operation.

func (*ListPrm) SetCount

func (l *ListPrm) SetCount(count uint32)

SetCount sets maximum amount of addresses that ListWithCursor should return.

func (*ListPrm) SetCursor

func (l *ListPrm) SetCursor(cursor *Cursor)

SetCursor sets cursor for ListWithCursor operation. For initial request ignore this param or use nil value. For consecutive requests, use value from ListRes.

type ListRes

type ListRes struct {
	// contains filtered or unexported fields
}

ListRes contains values returned from ListWithCursor operation.

func (ListRes) AddressList

func (l ListRes) AddressList() []objectcore.AddressWithType

AddressList returns addresses selected by ListWithCursor operation.

func (ListRes) Cursor

func (l ListRes) Cursor() *Cursor

Cursor returns cursor for consecutive listing requests.

type MovablePrm

type MovablePrm struct{}

MovablePrm groups the parameters of Movable operation.

type MovableRes

type MovableRes struct {
	// contains filtered or unexported fields
}

MovableRes groups the resulting values of Movable operation.

func (MovableRes) AddressList

func (p MovableRes) AddressList() []oid.Address

AddressList returns resulting addresses of Movable operation.

type ObjectCounters

type ObjectCounters struct {
	// contains filtered or unexported fields
}

ObjectCounters groups object counter according to metabase state.

func (ObjectCounters) Logic

func (o ObjectCounters) Logic() uint64

Logic returns logical object counter.

func (ObjectCounters) Phy

func (o ObjectCounters) Phy() uint64

Phy returns physical object counter.

type Option

type Option func(*cfg)

Option is an option of DB constructor.

func WithBoltDBOptions

func WithBoltDBOptions(opts *bbolt.Options) Option

WithBoltDBOptions returns option to specify BoltDB options.

func WithEpochState

func WithEpochState(s EpochState) Option

WithEpochState return option to specify a source of current epoch height.

func WithLogger

func WithLogger(l *logger.Logger) Option

WithLogger returns option to set logger of DB.

func WithMaxBatchDelay

func WithMaxBatchDelay(d time.Duration) Option

WithMaxBatchDelay returns option to specify maximum time to wait before the batch of concurrent transactions is processed. This option is missing from `bbolt.Options` but is set right after DB is open.

func WithMaxBatchSize

func WithMaxBatchSize(s int) Option

WithMaxBatchSize returns option to specify maximum concurrent operations to be processed in a single transactions. This option is missing from `bbolt.Options` but is set right after DB is open.

func WithPath

func WithPath(path string) Option

WithPath returns option to set system path to Metabase.

func WithPermissions

func WithPermissions(perm fs.FileMode) Option

WithPermissions returns option to specify permission bits of Metabase system path.

type PutPrm

type PutPrm struct {
	// contains filtered or unexported fields
}

PutPrm groups the parameters of Put operation.

func (*PutPrm) SetObject

func (p *PutPrm) SetObject(obj *objectSDK.Object)

SetObject is a Put option to set object to save.

func (*PutPrm) SetStorageID

func (p *PutPrm) SetStorageID(id []byte)

SetStorageID is a Put option to set storage ID to save.

type PutRes

type PutRes struct{}

PutRes groups the resulting values of Put operation.

type SelectPrm

type SelectPrm struct {
	// contains filtered or unexported fields
}

SelectPrm groups the parameters of Select operation.

func (*SelectPrm) SetContainerID

func (p *SelectPrm) SetContainerID(cnr cid.ID)

SetContainerID is a Select option to set the container id to search in.

func (*SelectPrm) SetFilters

func (p *SelectPrm) SetFilters(fs object.SearchFilters)

SetFilters is a Select option to set the object filters.

type SelectRes

type SelectRes struct {
	// contains filtered or unexported fields
}

SelectRes groups the resulting values of Select operation.

func (SelectRes) AddressList

func (r SelectRes) AddressList() []oid.Address

AddressList returns list of addresses of the selected objects.

type StorageIDPrm

type StorageIDPrm struct {
	// contains filtered or unexported fields
}

StorageIDPrm groups the parameters of StorageID operation.

func (*StorageIDPrm) SetAddress

func (p *StorageIDPrm) SetAddress(addr oid.Address)

SetAddress is a StorageID option to set the object address to check.

type StorageIDRes

type StorageIDRes struct {
	// contains filtered or unexported fields
}

StorageIDRes groups the resulting values of StorageID operation.

func (StorageIDRes) StorageID

func (r StorageIDRes) StorageID() []byte

StorageID returns storage ID.

type ToMoveItPrm

type ToMoveItPrm struct {
	// contains filtered or unexported fields
}

ToMoveItPrm groups the parameters of ToMoveIt operation.

func (*ToMoveItPrm) SetAddress

func (p *ToMoveItPrm) SetAddress(addr oid.Address)

SetAddress sets address of the object to move into another shard.

type ToMoveItRes

type ToMoveItRes struct{}

ToMoveItRes groups the resulting values of ToMoveIt operation.

type TombstonedHandler

type TombstonedHandler func(object TombstonedObject) error

TombstonedHandler is a TombstonedObject handling function.

type TombstonedObject

type TombstonedObject struct {
	// contains filtered or unexported fields
}

TombstonedObject represents descriptor of the object that has been covered with tombstone.

func (TombstonedObject) Address

func (g TombstonedObject) Address() oid.Address

Address returns tombstoned object address.

func (TombstonedObject) Tombstone

func (g TombstonedObject) Tombstone() oid.Address

Tombstone returns address of a tombstone that covers object.

type UpdateStorageIDPrm

type UpdateStorageIDPrm struct {
	// contains filtered or unexported fields
}

UpdateStorageIDPrm groups the parameters of UpdateStorageID operation.

func (*UpdateStorageIDPrm) SetAddress

func (p *UpdateStorageIDPrm) SetAddress(addr oid.Address)

SetAddress is an UpdateStorageID option to set the object address to check.

func (*UpdateStorageIDPrm) SetStorageID

func (p *UpdateStorageIDPrm) SetStorageID(id []byte)

SetStorageID is an UpdateStorageID option to set the storage ID.

type UpdateStorageIDRes

type UpdateStorageIDRes struct{}

UpdateStorageIDRes groups the resulting values of UpdateStorageID operation.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL