engine

package
v0.38.5 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 12, 2024 License: GPL-3.0 Imports: 40 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source
var ErrEndOfListing = shard.ErrEndOfListing

ErrEndOfListing is returned from an object listing with cursor when the storage can't return any more objects after the provided cursor. Use nil cursor object to start listing again.

View Source
var (
	ErrMustBeReadOnly = logicerr.New("shard must be in read-only mode")
)

Functions

func ContainerSize

func ContainerSize(e *StorageEngine, id cid.ID) (uint64, error)

ContainerSize calls ContainerSize method on engine to calculate sum of estimation container sizes among all shards.

func Get

func Get(ctx context.Context, storage *StorageEngine, addr oid.Address) (*objectSDK.Object, error)

Get reads object from local storage by provided address.

func GetRange

func GetRange(ctx context.Context, storage *StorageEngine, addr oid.Address, rng *objectSDK.Range) ([]byte, error)

GetRange reads object payload range from local storage by provided address.

func Head(ctx context.Context, storage *StorageEngine, addr oid.Address) (*objectSDK.Object, error)

Head reads object header from local storage by provided address.

func HeadRaw

func HeadRaw(ctx context.Context, storage *StorageEngine, addr oid.Address, raw bool) (*objectSDK.Object, error)

HeadRaw reads object header from local storage by provided address and raw flag.

func List

func List(ctx context.Context, storage *StorageEngine, limit uint64) ([]oid.Address, error)

List returns `limit` available physically storage object addresses in engine. If limit is zero, then returns all available object addresses.

func ListContainers

func ListContainers(ctx context.Context, e *StorageEngine) ([]cid.ID, error)

ListContainers calls ListContainers method on engine to get a unique container IDs presented in the engine objects.

func Put

func Put(ctx context.Context, storage *StorageEngine, obj *objectSDK.Object) error

Put writes provided object to local storage.

func Select

func Select(ctx context.Context, storage *StorageEngine, cnr cid.ID, fs objectSDK.SearchFilters) ([]oid.Address, error)

Select selects objects from local storage using provided filters.

Types

type ContainerSizePrm

type ContainerSizePrm struct {
	// contains filtered or unexported fields
}

ContainerSizePrm groups parameters of ContainerSize operation.

func (*ContainerSizePrm) SetContainerID

func (p *ContainerSizePrm) SetContainerID(cnr cid.ID)

SetContainerID sets the identifier of the container to estimate the size.

type ContainerSizeRes

type ContainerSizeRes struct {
	// contains filtered or unexported fields
}

ContainerSizeRes resulting values of ContainerSize operation.

func (ContainerSizeRes) Size

func (r ContainerSizeRes) Size() uint64

Size returns calculated estimation of the container size.

type Cursor

type Cursor struct {
	// contains filtered or unexported fields
}

Cursor is a type for continuous object listing. Cursor contains shard IDs to read and shard cursors that contain state from previous read.

type DeletePrm

type DeletePrm struct {
	// contains filtered or unexported fields
}

DeletePrm groups the parameters of Delete operation.

func (*DeletePrm) WithAddress

func (p *DeletePrm) WithAddress(addr oid.Address)

WithAddress is a Delete option to set the addresses of the objects to delete.

Option is required.

func (*DeletePrm) WithForceRemoval

func (p *DeletePrm) WithForceRemoval()

WithForceRemoval is a Delete option to remove an object despite any restrictions imposed on deleting that object. Expected to be used only in control service.

type DeleteRes

type DeleteRes struct{}

DeleteRes groups the resulting values of Delete operation.

type EvacuateProcessState added in v0.37.0

type EvacuateProcessState int
const (
	EvacuateProcessStateUndefined EvacuateProcessState = iota
	EvacuateProcessStateRunning
	EvacuateProcessStateCompleted
)

type EvacuateScope added in v0.38.0

type EvacuateScope uint32

EvacuateScope is an evacuation scope. Keep in sync with pkg/services/control/service.proto.

var (
	EvacuateScopeObjects EvacuateScope = 1
	EvacuateScopeTrees   EvacuateScope = 2
)

func (EvacuateScope) String added in v0.38.0

func (s EvacuateScope) String() string

func (EvacuateScope) TreesOnly added in v0.38.0

func (s EvacuateScope) TreesOnly() bool

func (EvacuateScope) WithObjects added in v0.38.0

func (s EvacuateScope) WithObjects() bool

func (EvacuateScope) WithTrees added in v0.38.0

func (s EvacuateScope) WithTrees() bool

type EvacuateShardPrm

type EvacuateShardPrm struct {
	ShardID        []*shard.ID
	ObjectsHandler func(context.Context, oid.Address, *objectSDK.Object) error
	TreeHandler    func(context.Context, cid.ID, string, pilorama.Forest) (string, error)
	IgnoreErrors   bool
	Async          bool
	Scope          EvacuateScope
}

EvacuateShardPrm represents parameters for the EvacuateShard operation.

type EvacuateShardRes

type EvacuateShardRes struct {
	// contains filtered or unexported fields
}

EvacuateShardRes represents result of the EvacuateShard operation.

func NewEvacuateShardRes added in v0.37.0

func NewEvacuateShardRes() *EvacuateShardRes

NewEvacuateShardRes creates new EvacuateShardRes instance.

func (*EvacuateShardRes) DeepCopy added in v0.37.0

func (p *EvacuateShardRes) DeepCopy() *EvacuateShardRes

DeepCopy returns deep copy of result instance.

func (*EvacuateShardRes) ObjectsEvacuated added in v0.38.0

func (p *EvacuateShardRes) ObjectsEvacuated() uint64

ObjectsEvacuated returns amount of evacuated objects. Objects for which handler returned no error are also assumed evacuated.

func (*EvacuateShardRes) ObjectsFailed added in v0.38.0

func (p *EvacuateShardRes) ObjectsFailed() uint64

ObjectsFailed returns count of failed objects to evacuate.

func (*EvacuateShardRes) ObjectsSkipped added in v0.38.0

func (p *EvacuateShardRes) ObjectsSkipped() uint64

ObjectsSkipped returns count of skipped objects.

func (*EvacuateShardRes) ObjectsTotal added in v0.38.0

func (p *EvacuateShardRes) ObjectsTotal() uint64

ObjectsTotal returns total count objects to evacuate.

func (*EvacuateShardRes) TreesEvacuated added in v0.38.0

func (p *EvacuateShardRes) TreesEvacuated() uint64

TreesEvacuated returns amount of evacuated trees.

func (*EvacuateShardRes) TreesFailed added in v0.38.0

func (p *EvacuateShardRes) TreesFailed() uint64

TreesFailed returns count of failed trees to evacuate.

func (*EvacuateShardRes) TreesTotal added in v0.38.0

func (p *EvacuateShardRes) TreesTotal() uint64

TreesTotal returns total count trees to evacuate.

type EvacuationState added in v0.37.0

type EvacuationState struct {
	// contains filtered or unexported fields
}

func (*EvacuationState) DeepCopy added in v0.37.0

func (s *EvacuationState) DeepCopy() *EvacuationState

func (*EvacuationState) ErrorMessage added in v0.37.0

func (s *EvacuationState) ErrorMessage() string

func (*EvacuationState) FinishedAt added in v0.37.0

func (s *EvacuationState) FinishedAt() *time.Time

func (*EvacuationState) ObjectsEvacuated added in v0.38.0

func (s *EvacuationState) ObjectsEvacuated() uint64

func (*EvacuationState) ObjectsFailed added in v0.38.0

func (s *EvacuationState) ObjectsFailed() uint64

func (*EvacuationState) ObjectsSkipped added in v0.38.0

func (s *EvacuationState) ObjectsSkipped() uint64

func (*EvacuationState) ObjectsTotal added in v0.38.0

func (s *EvacuationState) ObjectsTotal() uint64

func (*EvacuationState) ProcessingStatus added in v0.37.0

func (s *EvacuationState) ProcessingStatus() EvacuateProcessState

func (*EvacuationState) ShardIDs added in v0.37.0

func (s *EvacuationState) ShardIDs() []string

func (*EvacuationState) StartedAt added in v0.37.0

func (s *EvacuationState) StartedAt() *time.Time

func (*EvacuationState) TreesEvacuated added in v0.38.0

func (s *EvacuationState) TreesEvacuated() uint64

func (*EvacuationState) TreesFailed added in v0.38.0

func (s *EvacuationState) TreesFailed() uint64

func (*EvacuationState) TreesTotal added in v0.38.0

func (s *EvacuationState) TreesTotal() uint64

type FlushWriteCachePrm

type FlushWriteCachePrm struct {
	// contains filtered or unexported fields
}

FlushWriteCachePrm groups the parameters of FlushWriteCache operation.

func (*FlushWriteCachePrm) SetIgnoreErrors

func (p *FlushWriteCachePrm) SetIgnoreErrors(ignore bool)

SetIgnoreErrors sets errors ignore flag.

func (*FlushWriteCachePrm) SetSeal added in v0.38.0

func (p *FlushWriteCachePrm) SetSeal(v bool)

SetSeal sets seal flag.

func (*FlushWriteCachePrm) SetShardID

func (p *FlushWriteCachePrm) SetShardID(id *shard.ID)

SetShardID is an option to set shard ID.

Option is required.

type FlushWriteCacheRes

type FlushWriteCacheRes struct{}

FlushWriteCacheRes groups the resulting values of FlushWriteCache operation.

type GetPrm

type GetPrm struct {
	// contains filtered or unexported fields
}

GetPrm groups the parameters of Get operation.

func (*GetPrm) WithAddress

func (p *GetPrm) WithAddress(addr oid.Address)

WithAddress is a Get option to set the address of the requested object.

Option is required.

type GetRes

type GetRes struct {
	// contains filtered or unexported fields
}

GetRes groups the resulting values of Get operation.

func (GetRes) Object

func (r GetRes) Object() *objectSDK.Object

Object returns the requested object.

type HeadPrm

type HeadPrm struct {
	// contains filtered or unexported fields
}

HeadPrm groups the parameters of Head operation.

func (*HeadPrm) WithAddress

func (p *HeadPrm) WithAddress(addr oid.Address)

WithAddress is a Head option to set the address of the requested object.

Option is required.

func (*HeadPrm) WithRaw

func (p *HeadPrm) WithRaw(raw bool)

WithRaw is a Head option to set raw flag value. If flag is unset, then Head returns the header of the virtual object, otherwise it returns SplitInfo of the virtual object.

type HeadRes

type HeadRes struct {
	// contains filtered or unexported fields
}

HeadRes groups the resulting values of Head operation.

func (HeadRes) Header

func (r HeadRes) Header() *objectSDK.Object

Header returns the requested object header.

Instance has empty payload.

type Info

type Info struct {
	Shards []shard.Info
}

Info groups the information about StorageEngine.

type InhumePrm

type InhumePrm struct {
	// contains filtered or unexported fields
}

InhumePrm encapsulates parameters for inhume operation.

func (*InhumePrm) MarkAsGarbage

func (p *InhumePrm) MarkAsGarbage(addrs ...oid.Address)

MarkAsGarbage marks an object to be physically removed from local storage.

Should not be called along with WithTarget.

func (*InhumePrm) WithForceRemoval

func (p *InhumePrm) WithForceRemoval()

WithForceRemoval inhumes objects specified via MarkAsGarbage with GC mark without any object restrictions checks.

func (*InhumePrm) WithTarget

func (p *InhumePrm) WithTarget(tombstone oid.Address, addrs ...oid.Address)

WithTarget sets a list of objects that should be inhumed and tombstone address as the reason for inhume operation.

tombstone should not be nil, addr should not be empty. Should not be called along with MarkAsGarbage.

type InhumeRes

type InhumeRes struct{}

InhumeRes encapsulates results of inhume operation.

type ListContainersPrm

type ListContainersPrm struct{}

ListContainersPrm groups parameters of ListContainers operation.

type ListContainersRes

type ListContainersRes struct {
	// contains filtered or unexported fields
}

ListContainersRes groups the resulting values of ListContainers operation.

func (ListContainersRes) Containers

func (r ListContainersRes) Containers() []cid.ID

Containers returns a list of identifiers of the containers in which local objects are stored.

type ListWithCursorPrm

type ListWithCursorPrm struct {
	// contains filtered or unexported fields
}

ListWithCursorPrm contains parameters for ListWithCursor operation.

func (*ListWithCursorPrm) WithCount

func (p *ListWithCursorPrm) WithCount(count uint32)

WithCount sets the maximum amount of addresses that ListWithCursor should return.

func (*ListWithCursorPrm) WithCursor

func (p *ListWithCursorPrm) WithCursor(cursor *Cursor)

WithCursor sets a cursor for ListWithCursor operation. For initial request ignore this param or use nil value. For consecutive requests, use value from ListWithCursorRes.

type ListWithCursorRes

type ListWithCursorRes struct {
	// contains filtered or unexported fields
}

ListWithCursorRes contains values returned from ListWithCursor operation.

func (ListWithCursorRes) AddressList

func (l ListWithCursorRes) AddressList() []objectcore.AddressWithType

AddressList returns addresses selected by ListWithCursor operation.

func (ListWithCursorRes) Cursor

func (l ListWithCursorRes) Cursor() *Cursor

Cursor returns cursor for consecutive listing requests.

type MetricRegister

type MetricRegister interface {
	AddMethodDuration(method string, d time.Duration)

	SetObjectCounter(shardID, objectType string, v uint64)
	AddToObjectCounter(shardID, objectType string, delta int)

	SetMode(shardID string, mode mode.Mode)

	AddToContainerSize(cnrID string, size int64)
	DeleteContainerSize(cnrID string)
	DeleteContainerCount(cnrID string)
	AddToPayloadCounter(shardID string, size int64)
	IncErrorCounter(shardID string)
	ClearErrorCounter(shardID string)
	DeleteShardMetrics(shardID string)

	SetContainerObjectCounter(shardID, contID, objectType string, v uint64)
	IncContainerObjectCounter(shardID, contID, objectType string)
	SubContainerObjectCounter(shardID, contID, objectType string, v uint64)

	WriteCache() metrics.WriteCacheMetrics
	GC() metrics.GCMetrics
}

type Option

type Option func(*cfg)

Option represents StorageEngine's constructor option.

func WithErrorThreshold

func WithErrorThreshold(sz uint32) Option

WithErrorThreshold returns an option to specify size amount of errors after which shard is moved to read-only mode.

func WithLogger

func WithLogger(l *logger.Logger) Option

WithLogger returns option to set StorageEngine's logger.

func WithLowMemoryConsumption added in v0.37.0

func WithLowMemoryConsumption(lowMemCons bool) Option

WithLowMemoryConsumption returns an option to set the flag to reduce memory consumption by reducing performance.

func WithMetrics

func WithMetrics(v MetricRegister) Option

func WithRebuildWorkersCount added in v0.38.0

func WithRebuildWorkersCount(count uint32) Option

WithRebuildWorkersCount returns an option to set the count of concurrent rebuild workers.

func WithShardPoolSize

func WithShardPoolSize(sz uint32) Option

WithShardPoolSize returns option to specify size of worker pool for each shard.

type PutPrm

type PutPrm struct {
	// contains filtered or unexported fields
}

PutPrm groups the parameters of Put operation.

func (*PutPrm) WithObject

func (p *PutPrm) WithObject(obj *objectSDK.Object)

WithObject is a Put option to set object to save.

Option is required.

type ReConfiguration

type ReConfiguration struct {
	// contains filtered or unexported fields
}

func (*ReConfiguration) AddShard

func (rCfg *ReConfiguration) AddShard(id string, opts []shard.Option)

AddShard adds a shard for the reconfiguration. Shard identifier is calculated from paths used in blobstor.

func (*ReConfiguration) SetErrorsThreshold

func (rCfg *ReConfiguration) SetErrorsThreshold(errorsThreshold uint32)

SetErrorsThreshold sets a size amount of errors after which shard is moved to read-only mode.

func (*ReConfiguration) SetShardPoolSize

func (rCfg *ReConfiguration) SetShardPoolSize(shardPoolSize uint32)

SetShardPoolSize sets a size of worker pool for each shard.

type RemoveDuplicatesPrm

type RemoveDuplicatesPrm struct {
	Concurrency int
}

type RngPrm

type RngPrm struct {
	// contains filtered or unexported fields
}

RngPrm groups the parameters of GetRange operation.

func (*RngPrm) WithAddress

func (p *RngPrm) WithAddress(addr oid.Address)

WithAddress is a GetRng option to set the address of the requested object.

Option is required.

func (*RngPrm) WithPayloadRange

func (p *RngPrm) WithPayloadRange(rng *objectSDK.Range)

WithPayloadRange is a GetRange option to set range of requested payload data.

Missing an option or calling with zero length is equivalent to getting the full payload range.

type RngRes

type RngRes struct {
	// contains filtered or unexported fields
}

RngRes groups the resulting values of GetRange operation.

func (RngRes) Object

func (r RngRes) Object() *objectSDK.Object

Object returns the requested object part.

Instance payload contains the requested range of the original object.

type SealWriteCachePrm added in v0.38.0

type SealWriteCachePrm struct {
	ShardIDs     []*shard.ID
	IgnoreErrors bool
}

type SealWriteCacheRes added in v0.38.0

type SealWriteCacheRes struct {
	ShardResults []ShardSealResult
}

type SelectPrm

type SelectPrm struct {
	// contains filtered or unexported fields
}

SelectPrm groups the parameters of Select operation.

func (*SelectPrm) WithContainerID

func (p *SelectPrm) WithContainerID(cnr cid.ID)

WithContainerID is a Select option to set the container id to search in.

func (*SelectPrm) WithFilters

func (p *SelectPrm) WithFilters(fs objectSDK.SearchFilters)

WithFilters is a Select option to set the object filters.

type SelectRes

type SelectRes struct {
	// contains filtered or unexported fields
}

SelectRes groups the resulting values of Select operation.

func (SelectRes) AddressList

func (r SelectRes) AddressList() []oid.Address

AddressList returns list of addresses of the selected objects.

type ShardSealResult added in v0.38.0

type ShardSealResult struct {
	ShardID  *shard.ID
	Success  bool
	ErrorMsg string
}

type StorageEngine

type StorageEngine struct {
	// contains filtered or unexported fields
}

StorageEngine represents FrostFS local storage engine.

func New

func New(opts ...Option) *StorageEngine

New creates, initializes and returns new StorageEngine instance.

func (*StorageEngine) AddShard

func (e *StorageEngine) AddShard(ctx context.Context, opts ...shard.Option) (*shard.ID, error)

AddShard adds a new shard to the storage engine.

Returns any error encountered that did not allow adding a shard. Otherwise returns the ID of the added shard.

func (*StorageEngine) BlockExecution

func (e *StorageEngine) BlockExecution(err error) error

BlockExecution blocks the execution of any data-related operation. All blocked ops will return err. To resume the execution, use ResumeExecution method.

Сan be called regardless of the fact of the previous blocking. If execution wasn't blocked, releases all resources similar to Close. Can be called concurrently with Close and any data related method (waits for all executions to complete). Returns error if any Close has been called before.

Must not be called concurrently with either Open or Init.

Note: technically passing nil error will resume the execution, otherwise, it is recommended to call ResumeExecution for this.

func (*StorageEngine) Close

func (e *StorageEngine) Close(ctx context.Context) error

Close releases all StorageEngine's components. Waits for all data-related operations to complete. After the call, all the next ones will fail.

The method MUST only be called when the application exits.

func (*StorageEngine) ContainerSize

func (e *StorageEngine) ContainerSize(prm ContainerSizePrm) (res ContainerSizeRes, err error)

ContainerSize returns the sum of estimation container sizes among all shards.

Returns an error if executions are blocked (see BlockExecution).

func (*StorageEngine) Delete

func (e *StorageEngine) Delete(ctx context.Context, prm DeletePrm) (res DeleteRes, err error)

Delete marks the objects to be removed.

Returns an error if executions are blocked (see BlockExecution).

Returns apistatus.ObjectLocked if at least one object is locked. In this case no object from the list is marked to be deleted.

NOTE: Marks any object to be deleted (despite any prohibitions on operations with that object) if WithForceRemoval option has been provided.

func (*StorageEngine) DetachShards added in v0.38.0

func (e *StorageEngine) DetachShards(ids []*shard.ID) error

func (*StorageEngine) DumpInfo

func (e *StorageEngine) DumpInfo() (i Info)

DumpInfo returns information about the StorageEngine.

func (*StorageEngine) EnqueRunningEvacuationStop added in v0.37.0

func (e *StorageEngine) EnqueRunningEvacuationStop(ctx context.Context) error

func (*StorageEngine) Evacuate

Evacuate moves data from one shard to the others. The shard being moved must be in read-only mode.

func (*StorageEngine) FlushWriteCache

FlushWriteCache flushes write-cache on a single shard.

func (*StorageEngine) Get

func (e *StorageEngine) Get(ctx context.Context, prm GetPrm) (res GetRes, err error)

Get reads an object from local storage.

Returns any error encountered that did not allow to completely read the object part.

Returns an error of type apistatus.ObjectNotFound if the requested object is missing in local storage. Returns an error of type apistatus.ObjectAlreadyRemoved if the object has been marked as removed.

Returns an error if executions are blocked (see BlockExecution).

func (*StorageEngine) GetEvacuationState added in v0.37.0

func (e *StorageEngine) GetEvacuationState(ctx context.Context) (*EvacuationState, error)

func (*StorageEngine) GetRange

func (e *StorageEngine) GetRange(ctx context.Context, prm RngPrm) (res RngRes, err error)

GetRange reads part of an object from local storage.

Returns any error encountered that did not allow to completely read the object part.

Returns an error of type apistatus.ObjectNotFound if the requested object is missing in local storage. Returns an error of type apistatus.ObjectAlreadyRemoved if the requested object is inhumed. Returns ErrRangeOutOfBounds if the requested object range is out of bounds.

Returns an error if executions are blocked (see BlockExecution).

func (*StorageEngine) HandleNewEpoch

func (e *StorageEngine) HandleNewEpoch(ctx context.Context, epoch uint64)

HandleNewEpoch notifies every shard about NewEpoch event.

func (*StorageEngine) Head

func (e *StorageEngine) Head(ctx context.Context, prm HeadPrm) (res HeadRes, err error)

Head reads object header from local storage.

Returns any error encountered that did not allow to completely read the object header.

Returns an error of type apistatus.ObjectNotFound if the requested object is missing in local storage. Returns an error of type apistatus.ObjectAlreadyRemoved if the requested object was inhumed.

Returns an error if executions are blocked (see BlockExecution).

func (*StorageEngine) Inhume

func (e *StorageEngine) Inhume(ctx context.Context, prm InhumePrm) (res InhumeRes, err error)

Inhume calls metabase. Inhume method to mark an object as removed. It won't be removed physically from the shard until `Delete` operation.

Allows inhuming non-locked objects only. Returns apistatus.ObjectLocked if at least one object is locked.

NOTE: Marks any object as removed (despite any prohibitions on operations with that object) if WithForceRemoval option has been provided.

Returns an error if executions are blocked (see BlockExecution).

func (*StorageEngine) Init

func (e *StorageEngine) Init(ctx context.Context) error

Init initializes all StorageEngine's components.

func (*StorageEngine) IsLocked

func (e *StorageEngine) IsLocked(ctx context.Context, addr oid.Address) (bool, error)

IsLocked checks whether an object is locked according to StorageEngine's state.

func (*StorageEngine) List

func (e *StorageEngine) List(ctx context.Context, limit uint64) (res SelectRes, err error)

List returns `limit` available physically storage object addresses in engine. If limit is zero, then returns all available object addresses.

Returns an error if executions are blocked (see BlockExecution).

func (*StorageEngine) ListContainers

func (e *StorageEngine) ListContainers(ctx context.Context, _ ListContainersPrm) (res ListContainersRes, err error)

ListContainers returns a unique container IDs presented in the engine objects.

Returns an error if executions are blocked (see BlockExecution).

func (*StorageEngine) ListWithCursor

func (e *StorageEngine) ListWithCursor(ctx context.Context, prm ListWithCursorPrm) (ListWithCursorRes, error)

ListWithCursor lists physical objects available in the engine starting from the cursor. It includes regular, tombstone and storage group objects. Does not include inhumed objects. Use cursor value from the response for consecutive requests.

If count param is big enough, then the method reads objects from different shards by portions. In this case shards are chosen randomly, if they're not read out yet.

Adding a shard between ListWithCursor does not invalidate the cursor but new shard won't be listed. Removing a shard between ListWithCursor leads to the undefined behavior (e.g. usage of the objects from the removed shard).

Returns ErrEndOfListing if there are no more objects to return or count parameter set to zero.

func (*StorageEngine) Lock

func (e *StorageEngine) Lock(ctx context.Context, idCnr cid.ID, locker oid.ID, locked []oid.ID) error

Lock marks objects as locked with another object. All objects from the specified container.

Allows locking regular objects only (otherwise returns apistatus.LockNonRegularObject).

Locked list should be unique. Panics if it is empty.

func (*StorageEngine) Open

func (e *StorageEngine) Open(ctx context.Context) error

Open opens all StorageEngine's components.

func (*StorageEngine) Put

func (e *StorageEngine) Put(ctx context.Context, prm PutPrm) (err error)

Put saves the object to local storage.

Returns any error encountered that did not allow to completely save the object.

Returns an error if executions are blocked (see BlockExecution).

Returns an error of type apistatus.ObjectAlreadyRemoved if the object has been marked as removed.

func (*StorageEngine) Reload

func (e *StorageEngine) Reload(ctx context.Context, rcfg ReConfiguration) error

Reload reloads StorageEngine's configuration in runtime.

func (*StorageEngine) RemoveDuplicates

func (e *StorageEngine) RemoveDuplicates(ctx context.Context, prm RemoveDuplicatesPrm) error

RemoveDuplicates iterates over all objects and removes duplicate object copies from shards which are worse as defined by HRW sort. Safety:

  1. Concurrent execution is prohibited, thus 1 object copy should always be left.
  2. If we delete an object from another thread, this is not a problem. Currently, we have 2 thread that can remove "valid" (non-expired and logically non-removed) objects: policer and rebalance. For rebalance see (1). If policer removes something, we do not care if both copies are removed or one of them is left, as the remaining copy will be removed during the next policer iteration.

func (*StorageEngine) ResetEvacuationStatus added in v0.38.0

func (e *StorageEngine) ResetEvacuationStatus(ctx context.Context) error

func (*StorageEngine) ResumeExecution

func (e *StorageEngine) ResumeExecution() error

ResumeExecution resumes the execution of any data-related operation. To block the execution, use BlockExecution method.

Сan be called regardless of the fact of the previous blocking. If execution was blocked, prepares all resources similar to Open. Can be called concurrently with Close and any data related method (waits for all executions to complete). Returns error if any Close has been called before.

Must not be called concurrently with either Open or Init.

func (*StorageEngine) SealWriteCache added in v0.38.0

func (e *StorageEngine) SealWriteCache(ctx context.Context, prm SealWriteCachePrm) (SealWriteCacheRes, error)

SealWriteCache flushed all data to blobstore and moves write-cache to degraded read only mode.

func (*StorageEngine) Select

func (e *StorageEngine) Select(ctx context.Context, prm SelectPrm) (res SelectRes, err error)

Select selects the objects from local storage that match select parameters.

Returns any error encountered that did not allow to completely select the objects.

Returns an error if executions are blocked (see BlockExecution).

func (*StorageEngine) SetContainerSource added in v0.38.0

func (e *StorageEngine) SetContainerSource(cs container.Source)

SetContainerSource sets container source.

func (*StorageEngine) SetShardMode

func (e *StorageEngine) SetShardMode(id *shard.ID, m mode.Mode, resetErrorCounter bool) error

SetShardMode sets mode of the shard with provided identifier.

Returns an error if shard mode was not set, or shard was not found in storage engine.

func (*StorageEngine) TreeAddByPath

func (e *StorageEngine) TreeAddByPath(ctx context.Context, d pilorama.CIDDescriptor, treeID string, attr string, path []string, m []pilorama.KeyValue) ([]pilorama.Move, error)

TreeAddByPath implements the pilorama.Forest interface.

func (*StorageEngine) TreeApply

func (e *StorageEngine) TreeApply(ctx context.Context, cnr cidSDK.ID, treeID string, m *pilorama.Move, backgroundSync bool) error

TreeApply implements the pilorama.Forest interface.

func (*StorageEngine) TreeDrop

func (e *StorageEngine) TreeDrop(ctx context.Context, cid cidSDK.ID, treeID string) error

TreeDrop implements the pilorama.Forest interface.

func (*StorageEngine) TreeExists

func (e *StorageEngine) TreeExists(ctx context.Context, cid cidSDK.ID, treeID string) (bool, error)

TreeExists implements the pilorama.Forest interface.

func (*StorageEngine) TreeGetByPath

func (e *StorageEngine) TreeGetByPath(ctx context.Context, cid cidSDK.ID, treeID string, attr string, path []string, latest bool) ([]pilorama.Node, error)

TreeGetByPath implements the pilorama.Forest interface.

func (*StorageEngine) TreeGetChildren

func (e *StorageEngine) TreeGetChildren(ctx context.Context, cid cidSDK.ID, treeID string, nodeID pilorama.Node) ([]pilorama.NodeInfo, error)

TreeGetChildren implements the pilorama.Forest interface.

func (*StorageEngine) TreeGetMeta

func (e *StorageEngine) TreeGetMeta(ctx context.Context, cid cidSDK.ID, treeID string, nodeID pilorama.Node) (pilorama.Meta, uint64, error)

TreeGetMeta implements the pilorama.Forest interface.

func (*StorageEngine) TreeGetOpLog

func (e *StorageEngine) TreeGetOpLog(ctx context.Context, cid cidSDK.ID, treeID string, height uint64) (pilorama.Move, error)

TreeGetOpLog implements the pilorama.Forest interface.

func (*StorageEngine) TreeHeight

func (e *StorageEngine) TreeHeight(ctx context.Context, cid cidSDK.ID, treeID string) (uint64, error)

func (*StorageEngine) TreeLastSyncHeight

func (e *StorageEngine) TreeLastSyncHeight(ctx context.Context, cid cidSDK.ID, treeID string) (uint64, error)

TreeLastSyncHeight implements the pilorama.Forest interface.

func (*StorageEngine) TreeList

func (e *StorageEngine) TreeList(ctx context.Context, cid cidSDK.ID) ([]string, error)

TreeList implements the pilorama.Forest interface.

func (*StorageEngine) TreeMove

TreeMove implements the pilorama.Forest interface.

func (*StorageEngine) TreeSortedByFilename added in v0.38.2

func (e *StorageEngine) TreeSortedByFilename(ctx context.Context, cid cidSDK.ID, treeID string, nodeID pilorama.Node, last *string, count int) ([]pilorama.NodeInfo, *string, error)

TreeSortedByFilename implements the pilorama.Forest interface.

func (*StorageEngine) TreeUpdateLastSyncHeight

func (e *StorageEngine) TreeUpdateLastSyncHeight(ctx context.Context, cid cidSDK.ID, treeID string, height uint64) error

TreeUpdateLastSyncHeight implements the pilorama.Forest interface.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL