packer

package
v0.0.0-...-269d6e4 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 24, 2022 License: MIT Imports: 24 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type CARUploader

type CARUploader interface {
	Store(context.Context, string, io.Reader) (string, error)
}

CARUploader provides blob storage for CAR files.

type Option

type Option func(*config) error

Option applies a configuration change.

func WithBatchMinSize

func WithBatchMinSize(minSize int64) Option

WithBatchMinSize configures the minimum batch size that can be considered for preparation.

func WithBatchMinWaiting

func WithBatchMinWaiting(minWaiting time.Duration) Option

WithBatchMinWaiting configure the minimum waiting duration for a batch t obe considered for preparation.

func WithBatchWaitScalingFactor

func WithBatchWaitScalingFactor(scalingFactor int64) Option

WithBatchWaitScalingFactor is used to scale waiting duration through 1MiB, 100MiB and 1GiB size ranges: - [1MiB, 100MiB] = minWaitingTime * scalingFactor^2 (default: 1m * 5^2 = 25m) - [100MiB, 1GiB] = minWaitingTime * scalingFactor (default: 1m * 5 = 5m) - [1GiB, inf] = minWaitingTime (default: 1m).

func WithCARExportURL

func WithCARExportURL(rawURL string) Option

WithCARExportURL configures the frequency of exporting the pin count metric.

func WithCARUploader

func WithCARUploader(uploader CARUploader) Option

WithCARUploader configures a file uploader for CAR files.

func WithDaemonFrequency

func WithDaemonFrequency(frequency time.Duration) Option

WithDaemonFrequency indicates the frequency in which ready batches are processed.

func WithExportMetricsFrequency

func WithExportMetricsFrequency(frequency time.Duration) Option

WithExportMetricsFrequency indicates the exporting metrics frequency of open batches.

func WithRetryDelay

func WithRetryDelay(delay time.Duration) Option

WithRetryDelay indicates how many time erorred batch processing should be delayed before retrying.

func WithSectorSize

func WithSectorSize(sectorSize int64) Option

WithSectorSize configures the sector size that will be considered for the maximum size of batches.

type Packer

type Packer struct {
	// contains filtered or unexported fields
}

Packer provides batching strategies to bundle multiple StorageRequest into a Batch.

func New

func New(
	postgresURI string,
	pinnerClient *httpapi.HttpApi,
	ipfsEndpoints []multiaddr.Multiaddr,
	mb mbroker.MsgBroker,
	opts ...Option) (*Packer, error)

New returns a new Packer.

func (*Packer) Close

func (p *Packer) Close() error

Close closes the packer.

func (*Packer) ReadyToBatch

func (p *Packer) ReadyToBatch(
	ctx context.Context,
	opID mbroker.OperationID,
	rtbds []mbroker.ReadyToBatchData) (err error)

ReadyToBatch includes a StorageRequest in an open batch.

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL