lsmt

package
v1.8.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 12, 2022 License: BSD-3-Clause Imports: 18 Imported by: 1

README 

The 'lsmtree' package was going to be the production one, but now I am making this
package (the lsmt) package, in order to tidy everything up. I believe it will end
up being a bit more clean this way.

Documentation

Index

Constants

View Source 
const (
	SizeKB   = 1<<10 - 1
	SizeMB   = 1<<20 - 1
	Size64KB = math.MaxUint16
	Size4GB  = math.MaxUint32
)
View Source 
const (
	LevelDebug logLevel = iota
	LevelInfo
	LevelWarn
	LevelError
	LevelFatal
	LevelOff
)
View Source 
const (
	ScanNewToOld int = sstable.ScanNewToOld
	ScanOldToNew int = sstable.ScanOldToNew
)

Variables

View Source 
var (
	ErrKeyNotFound    = errors.New("lsmt: key not found")
	ErrFoundTombstone = errors.New("lsmt: found tombstone or empty value")
	ErrDeleted        = ErrFoundTombstone

	ErrNotFound       = errors.New("lsmt: not found")
	ErrIncompleteSet  = errors.New("lsmt: incomplete batch or set")
	ErrFlushThreshold = errors.New("lsmt: flush threshold has been reached")

	ErrBadKey        = errors.New("lsmt: bad key")
	ErrKeyTooLarge   = errors.New("lsmt: key too large")
	ErrBadValue      = errors.New("lsmt: bad value")
	ErrValueTooLarge = errors.New("lsmt: value too large")

	ErrBadChecksum = errors.New("lsmt: bad checksum")
)
View Source 
var Tombstone = []byte(nil)

Functions

func CalcCRC added in v1.7.0 

func CalcCRC(d []byte) uint32

func LevelText added in v1.7.0 

func LevelText(level logLevel) string
func Search(pattern string, input []byte) (bool, error)

Types

type DebugEntry added in v1.7.0 

type DebugEntry struct {
	Key string
	Val []byte
	CRC uint32
}

func NewDebugEntry added in v1.7.0 

func NewDebugEntry(k string, v []byte) *DebugEntry

func (*DebugEntry) IsOK added in v1.7.0 

func (de *DebugEntry) IsOK(entry *DebugEntry) bool

type LSMConfig added in v1.3.1 

type LSMConfig struct {
	BaseDir         string   // base directory
	SyncOnWrite     bool     // perform sync every time an entry is written
	LoggingLevel    logLevel // enable logging
	FlushThreshold  int64    // mem-table flush threshold
	BloomFilterSize uint     // specify the bloom filter size
	MaxKeySize      int64    // the max allowed key size
	MaxValueSize    int64    // the maximum allowed value size
}

LSMConfig holds configuration settings for an LSMTree instance

func DefaultConfig added in v1.7.0 

func DefaultConfig(path string) *LSMConfig

func (*LSMConfig) String added in v1.7.0 

func (conf *LSMConfig) String() string

type LSMTree 

type LSMTree struct {
	// contains filtered or unexported fields
}

LSMTree is an LSMTree

func OpenLSMTree 

func OpenLSMTree(c *LSMConfig) (*LSMTree, error)

OpenLSMTree opens or creates an LSMTree instance.

func (*LSMTree) Close 

func (lsm *LSMTree) Close() error

func (*LSMTree) Del 

func (lsm *LSMTree) Del(k string) error

Del takes a key and overwrites the record with a tomstone or a 'deleted' or nil entry. It leaves the key in the LSMTree so that future table versions can properly merge.

func (*LSMTree) FlushToSSTableAndCycleWAL added in v1.7.0 

func (lsm *LSMTree) FlushToSSTableAndCycleWAL(memt *mtbl.RBTree) error

func (*LSMTree) Get 

func (lsm *LSMTree) Get(k string) ([]byte, error)

Get takes a key and attempts to find a match in the LSMTree. If a match cannot be found Get returns a nil value and ErrNotFound. Get first checks the bloom filter, then the mem-table. If it is still not found it attempts to do a binary search on the for the key in the ss-index and if that yields no result it will try to find the entry by doing a linear search of the ss-table itself.

func (*LSMTree) GetBatch added in v1.5.0 

func (lsm *LSMTree) GetBatch(keys ...string) (*binary.Batch, error)

GetBatch attempts to find entries matching the keys provided. If a matching entry is found, it is added to the batch that is returned. If a matching entry cannot be found it is simply skipped and not added to the batch. GetBatch will return a nil error if all the matching entries were found. If it found some but not all, GetBatch will return ErrIncompleteSet along with the batch of entries that it could find. If it could not find any matches at all, the batch will be nil and GetBatch will return an ErrNotFound

func (*LSMTree) GetLinear added in v1.7.0 

func (lsm *LSMTree) GetLinear(k string) ([]byte, error)

GetLinear takes a key and attempts to find a match in the LSMTree. If a match cannot be found Get returns a nil value and ErrNotFound. Get first checks the bloom filter, then the mem-table. If it is still not found [this is where it differs from Get] it attempts to do a linear search directly of the ss-table itself. It can be a bit quicker [if you know that your data is not memory resident.]

func (*LSMTree) Has added in v1.4.0 

func (lsm *LSMTree) Has(k string) bool

Has returns a boolean signaling weather or not the key is in the LSMTree. It should be noted that in some cases this may return a false positive, but it should never return a false negative.

func (*LSMTree) Put 

func (lsm *LSMTree) Put(k string, v []byte) error

Put takes a key and a value and adds them to the LSMTree. If the entry already exists, it should overwrite the old entry.

func (*LSMTree) PutBatch added in v1.3.1 

func (lsm *LSMTree) PutBatch(batch *binary.Batch) error

PutBatch takes a batch of entries and adds all of them at one time. It acts a bit like a transaction. If you have a configuration option of SyncOnWrite: true it will be disabled temporarily and the batch will sync at the end of all the writes. This is to give a slight performance advantage. It should be worth noting that very large batches may have an impact on performance and may also cause frequent ss-table flushes which may result in fragmentation.

func (*LSMTree) Scan added in v1.7.0 

func (lsm *LSMTree) Scan(direction int, iter func(e *binary.Entry) bool) error

Scan takes a scan direction and an iteration function and scans the ss-tables in the provided direction (young to old, or old to young) and provides you with a pointer to each entry during iteration. *It should be noted that modification of the entry pointer has unknown effects.

func (*LSMTree) Stats added in v1.6.0 

func (lsm *LSMTree) Stats() (*LSMTreeStats, error)

func (*LSMTree) Sync added in v1.3.1 

func (lsm *LSMTree) Sync() error

Sync forces a sync

type LSMTreeStats added in v1.6.0 

type LSMTreeStats struct {
	Config    *LSMConfig `json:"config,omitempty"`
	MtEntries int        `json:"mt_entries,omitempty"`
	MtSize    int64      `json:"mt_size,omitempty"`
	BfEntries int        `json:"bf_entries,omitempty"`
	BfSize    int64      `json:"bf_size,omitempty"`
}

func (*LSMTreeStats) JSON added in v1.6.0 

func (s *LSMTreeStats) JSON() (string, error)

func (*LSMTreeStats) String added in v1.6.0 

func (s *LSMTreeStats) String() string

type Logger added in v1.7.0 

type Logger struct {
	*log.Logger
	// contains filtered or unexported fields
}

func NewLogger added in v1.7.0 

func NewLogger(level logLevel) *Logger

func (*Logger) Debug added in v1.7.0 

func (l *Logger) Debug(s string, a ...interface{})

func (*Logger) Error added in v1.7.0 

func (l *Logger) Error(s string, a ...interface{})

func (*Logger) Fatal added in v1.7.0 

func (l *Logger) Fatal(s string, a ...interface{})

func (*Logger) Info added in v1.7.0 

func (l *Logger) Info(s string, a ...interface{})

func (*Logger) Warn added in v1.7.0 

func (l *Logger) Warn(s string, a ...interface{})

type ScanDirection added in v1.7.0 

type ScanDirection = sstable.ScanDirection

Source Files

View all Source files

Directories

Path Synopsis
trees
bptree
rbtree

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.