blobstor

package
v0.41.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 27, 2024 License: GPL-3.0 Imports: 13 Imported by: 1

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var ErrNoPlaceFound = logicerr.New("couldn't find a place to store an object")

ErrNoPlaceFound is returned when object can't be saved to any sub-storage component because of the policy.

Functions

func IterateBinaryObjects added in v0.25.0 

func IterateBinaryObjects(blz *BlobStor, f func(addr oid.Address, data []byte, descriptor []byte) error) error

IterateBinaryObjects is a helper function which iterates over BlobStor and passes binary objects to f. Errors related to object reading and unmarshaling are logged and skipped.

Types

type BlobStor 

type BlobStor struct {
	// contains filtered or unexported fields
}

BlobStor represents NeoFS local BLOB storage.

func New 

func New(opts ...Option) *BlobStor

New creates, initializes and returns new BlobStor instance.

func (*BlobStor) Close 

func (b *BlobStor) Close() error

Close releases all internal resources of BlobStor.

func (*BlobStor) Delete added in v0.32.0 

func (b *BlobStor) Delete(prm common.DeletePrm) (common.DeleteRes, error)

func (*BlobStor) DumpInfo 

func (b *BlobStor) DumpInfo() Info

DumpInfo returns information about blob stor.

func (*BlobStor) Exists 

func (b *BlobStor) Exists(prm common.ExistsPrm) (common.ExistsRes, error)

Exists checks if the object is presented in BLOB storage.

Returns any error encountered that did not allow to completely check object existence.

func (*BlobStor) Get added in v0.32.0 

func (b *BlobStor) Get(prm common.GetPrm) (common.GetRes, error)

Get reads the object from b. If the descriptor is present, only one sub-storage is tried, Otherwise, each sub-storage is tried in order.

func (*BlobStor) GetBytes added in v0.41.0 

func (b *BlobStor) GetBytes(addr oid.Address, subStorageID []byte) ([]byte, error)

GetBytes reads object from the BlobStor by address into memory buffer in a canonical NeoFS binary format. Returns apistatus.ObjectNotFound if object is missing.

func (*BlobStor) GetRange added in v0.32.0 

func (b *BlobStor) GetRange(prm common.GetRangePrm) (common.GetRangeRes, error)

GetRange reads object payload data from b. If the descriptor is present, only one sub-storage is tried, Otherwise, each sub-storage is tried in order.

func (*BlobStor) Init 

func (b *BlobStor) Init() error

Init initializes internal data structures and system resources.

If BlobStor is already initialized, no action is taken.

func (*BlobStor) Iterate 

func (b *BlobStor) Iterate(prm common.IteratePrm) (common.IterateRes, error)

Iterate traverses the storage over the stored objects and calls the handler on each element.

Returns any error encountered that did not allow to completely iterate over the storage.

If handler returns an error, method wraps and returns it immediately.

func (*BlobStor) NeedsCompression added in v0.27.4 

func (b *BlobStor) NeedsCompression(obj *objectSDK.Object) bool

NeedsCompression returns true if the object should be compressed. For an object to be compressed 2 conditions must hold: 1. Compression is enabled in settings. 2. Object MIME Content-Type is allowed for compression.

func (*BlobStor) ObjectStatus added in v0.39.0 

func (b *BlobStor) ObjectStatus(address oid.Address) (ObjectStatus, error)

ObjectStatus returns the status of the object in the Blob storage. It contains status of the object in all blob substorages.

func (*BlobStor) Open 

func (b *BlobStor) Open(readOnly bool) error

Open opens BlobStor.

func (*BlobStor) Put 

func (b *BlobStor) Put(prm common.PutPrm) (common.PutRes, error)

Put saves the object in BLOB storage.

If object is "big", BlobStor saves the object in shallow dir. Otherwise, BlobStor saves the object in peapod.

Returns any error encountered that did not allow to completely save the object.

func (*BlobStor) SetLogger added in v0.30.0 

func (b *BlobStor) SetLogger(l *zap.Logger)

SetLogger sets logger. It is used after the shard ID was generated to use it in logs.

func (*BlobStor) SetMode added in v0.30.0 

func (b *BlobStor) SetMode(m mode.Mode) error

SetMode sets the blobstor mode of operation.

func (*BlobStor) SetReportErrorFunc added in v0.36.0 

func (b *BlobStor) SetReportErrorFunc(f func(string, error))

SetReportErrorFunc allows to provide a function to be called on disk errors. This function MUST be called before Open.

type Info 

type Info struct {
	SubStorages []SubStorageInfo
}

Info contains information about blobstor.

type ObjectStatus added in v0.39.0 

type ObjectStatus struct {
	Substorages []ObjectSubstorageStatus
}

ObjectStatus represents the status of the object in the Blob storage.

type ObjectSubstorageStatus added in v0.39.0 

type ObjectSubstorageStatus struct {
	Type  string
	Path  string
	Error error
}

ObjectSubstorageStatus represents the status of the object in the Blob storage, containing the type and path of the storage and an error if it occurred.

type Option 

type Option func(*cfg)

Option represents BlobStor's constructor option.

func WithCompressObjects 

func WithCompressObjects(comp bool) Option

WithCompressObjects returns option to toggle compression of the stored objects.

If true, Zstandard algorithm is used for data compression.

If compressor (decompressor) creation failed, the uncompressed option will be used, and the error is recorded in the provided log.

func WithLogger 

func WithLogger(l *zap.Logger) Option

WithLogger returns option to specify BlobStor's logger.

func WithStorages added in v0.32.0 

func WithStorages(st []SubStorage) Option

WithStorages provides sub-blobstors.

func WithUncompressableContentTypes added in v0.27.4 

func WithUncompressableContentTypes(values []string) Option

WithUncompressableContentTypes returns option to disable decompression for specific content types as seen by object.AttributeContentType attribute.

type SubStorage added in v0.32.0 

type SubStorage struct {
	Storage common.Storage
	Policy  func(*objectSDK.Object, []byte) bool
}

SubStorage represents single storage component with some storage policy.

type SubStorageInfo added in v0.33.0 

type SubStorageInfo struct {
	Type string
	Path string
}

SubStorageInfo contains information about blobstor storage component.

Source Files

View all Source files

Directories

Path Synopsis
internal
blobstortest

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.