lmdbbs

package module
v1.0.5 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 6, 2021 License: Apache-2.0, MIT Imports: 14 Imported by: 3

README

go-bs-lmdb

An LMDB-backed IPFS Blockstore.

License

Dual-licensed: MIT, Apache Software License v2, by way of the Permissive License Stack.

Documentation

Index

Constants

This section is empty.

Variables

View Source
var (
	// DefaultInitialMmapSize is the default initial mmap size to be used if the
	// supplied value is zero or invalid. Unless modified, this value is 1GiB.
	DefaultInitialMmapSize = int64(1 << 30) // 1GiB.

	// DefaultMmapGrowthStepFactor is the default mmap growth step factor to be
	// used if the supplied value is zero or invalid. Unless modified, this
	// value is 1.5, which multiplies the mmap size by 1.5 every time we
	// encounter an MDB_MAP_FULL error.
	DefaultMmapGrowthStepFactor = 1.5 // 1.5x the mmap every time.

	// DefaultMmapGrowthStepMax is the default mmap growth maximum step to be
	// used if the supplied value is zero or invalid. Unless modified, this
	// value is 4GiB.
	DefaultMmapGrowthStepMax = int64(4 << 30) // maximum step size is 4GiB at a time.

	// DefaultMaxReaders is the default number of max readers if one is not
	// provided. By default it is 254, not 256, following the note from the LMDB
	// author that indicates that the original default was 126 because it fit
	// exactly into 8KiB along with a couple of mutexes.
	//
	// http://www.lmdb.tech/doc/group__readers.html#gadff1f7b4d4626610a8d616e0c6dbbea4
	DefaultMaxReaders = 254

	// DefaultRetryDelay is the default retry delay for reattempting transactions
	// that errored with MDB_READERS_FULL.
	DefaultRetryDelay = 10 * time.Millisecond

	// RetryJitter is the jitter to apply to the delay interval. Default: 20%.
	RetryJitter = 0.2
)

Functions

This section is empty.

Types

type Blockstore

type Blockstore struct {
	// contains filtered or unexported fields
}

func Open

func Open(opts *Options) (*Blockstore, error)

func (*Blockstore) AllKeysChan

func (b *Blockstore) AllKeysChan(ctx context.Context) (<-chan cid.Cid, error)

AllKeysChan starts a cursor to return all keys from the underlying MDB store. The cursor could be preempted at any time by a mmap grow operation. When that happens, the cursor yields, and the grow operation resumes it after the mmap expansion is completed.

Consistency is not guaranteed. That is, keys returned are not a snapshot taken when this method is called. The set of returned keys will vary with concurrent writes.

func (*Blockstore) Close

func (b *Blockstore) Close() error

func (*Blockstore) DeleteBlock

func (b *Blockstore) DeleteBlock(cid cid.Cid) error

func (*Blockstore) DeleteMany added in v1.0.5

func (b *Blockstore) DeleteMany(cids []cid.Cid) error

func (*Blockstore) Get

func (b *Blockstore) Get(cid cid.Cid) (blocks.Block, error)

func (*Blockstore) GetSize

func (b *Blockstore) GetSize(cid cid.Cid) (int, error)

func (*Blockstore) Has

func (b *Blockstore) Has(cid cid.Cid) (bool, error)

func (*Blockstore) HashOnRead

func (b *Blockstore) HashOnRead(_ bool)

func (*Blockstore) Put

func (b *Blockstore) Put(block blocks.Block) error

func (*Blockstore) PutMany

func (b *Blockstore) PutMany(blocks []blocks.Block) error

func (*Blockstore) Stat added in v1.0.5

func (b *Blockstore) Stat() (*lmdb.Stat, error)

func (*Blockstore) View

func (b *Blockstore) View(cid cid.Cid, callback func([]byte) error) error

type Options added in v1.0.0

type Options struct {
	// Path is the directory where the LMDB blockstore resides. If it doesn't
	// exist, it will be created.
	Path string

	// ReadOnly, if true, opens this blockstore in read-only mode.
	ReadOnly bool

	// NoSync disables flushing system buffers to disk immediately when
	// committing transactions.
	NoSync bool

	// +++ DB sizing fields. +++ //
	// InitialMmapSize is the initial mmap size passed to LMDB when
	// creating/opening the environment.
	InitialMmapSize int64

	// MmapGrowthStepFactor determines the next map size when a write fails. The
	// current size is multiplied by the factor, and rounded up to the next
	// value divisible by os.Getpagesize(), to obtain the new map size, which is
	// applied with mdb_env_set_mapsize.
	MmapGrowthStepFactor float64

	// MmapGrowthStepMax is the maximum step size by which we'll grow the mmap.
	MmapGrowthStepMax int64

	// MaxReaders is the maximum amount of concurrent reader slots that exist
	// in the lock table. By default 254.
	MaxReaders int

	// RetryDelay is a fixed delay to wait before a transaction that errored
	// with MDB_READERS_FULL will be reattempted. Contention due to incorrect
	// sizing of MaxReaders will thus lead to a system slowdown via
	// backpressure, instead of a straight out error.
	// Jittered by RetryJitter (default: +/-20%).
	RetryDelay time.Duration
}

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL