bogn

package
v0.0.0-...-b47ea92 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 10, 2021 License: MIT Imports: 26 Imported by: 2

README

Indexing data in Random Access Memory and Solid State Drive.

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var Compacttick = time.Duration(1 * time.Second)

Compacttick timer tick to check for memory overflow, persistance, flushing, compacting memory stores and/or disk stores. Purger routine uses this tick to periodically purge un-referenced storage instance.

Functions

func CompactIndex 

func CompactIndex(name, diskstore string, diskpaths []string, merge bool)

CompactIndex will remove older versions of disk level snapshots and if merge is true, will merge all disk-levels into single level.

func Defaultsettings 

func Defaultsettings() s.Settings

Defaultsettings for bogn instances. Applications can get the default settings and tune settings parameter for desired behaviour. Default settings include llrb.Defaultsettings().

"logpath" (string, default: "") 

Directory path to store log files. If not supplied, and durable
is true, then one of the diskpath from diskstore will be used.

"memstore" (string, default: "llrb") 

Type of index for in memory storage, can be "llrb" or "mvcc".

"diskstore" (string, default: "bubt") 

Type of index for in disk storage, can be "bubt".

"durable" (bool, default:false) 

Persist index on disk.

"dgm" (bool, default:false) 

		Disk-Greater-than-Memory, configure bogn-index whose size won't
     fit in memory.

"workingset" (bool, default: false) 

This configuration is valid only when `dgm` is set to true.
Set this as true only when a subset of keys in bogn-index will
be actived accessed, either for read or write.

"autocommit" (int64, default: 100) 

Time is seconds to periodically persist transient writes onto disk.
If set to ZERO, then it is upto the application to issue a Commit()
call to persist the transient writes.

"flushratio" (floating, default: .25) 

     This configuration is valid only when `dgm` is set to true.
     When ratio between memory footprint, for the latest batch of
		mutations, and latest level's disk-footprint falls below flushratio,
		then a newer level will be created to flush the latest batch of
		mutations in memory.

"compactratio" (floating, default: .50) 

     This configuration is valid only when `dgm` is set to true.
		Two succesive disk levels will be merged together into a single
		snapshot if their ratio exceed compactratio.

"compactperiod" (int64, default: 3600) 

     If the lifetime, measured in seconds, of a disk snapshot exceeds
		compactperiod, then it will be merged with next disk level snapshot.

"bubt.mblocksize" (int64, default: 4096) 

BottomsUpBTree, size of intermediate node, m-nodes, on disk.

"bubt.zblocksize" (int64, default: same as mblocksize) 

BottomsUpBTree, size of leaf node, z-nodes, on disk.

"bubt.vblocksize" (int64, default: same as mblocksize) 

BottomsUpBTree, size of value log blocsk, on disk.

"bubt.mmap" (bool, default: true) 

BottomsUpBTree, whether to memory-map leaf node, intermediate
nodes are always memory-mapped.

"bubt.diskpaths" (string, default: "/opt/bogn/") 

BottomsUpBTree, comma separated list of path to persist intermediate
nodes and leaf nodes.

func LogComponents 

func LogComponents(components ...string)

LogComponents enable logging. By default logging is disabled, if applications want log information for bogn components call this function with "self" or "bogn" as argument. To enable logging for bogn and all of its components call this function with "all" or "bogn","bubt","llrb" as argument.

func PurgeIndex 

func PurgeIndex(name, logpath, diskstore string, diskpaths []string)

PurgeIndex will purge all the disk level snapshots for index `name` founder under `diskpaths`.

Types

type Bogn 

type Bogn struct {
	// contains filtered or unexported fields
}

Bogn instance to index key,value pairs.

func New 

func New(name string, setts s.Settings) (*Bogn, error)

New create a new bogn instance.

func (*Bogn) BeginTxn 

func (bogn *Bogn) BeginTxn(id uint64) api.Transactor

BeginTxn starts a read-write transaction. All transactions should either be committed or aborted. If transactions are not released for long time it might increase the memory pressure on the system. Concurrent transactions are allowed, and serialized internally.

func (*Bogn) Close 

func (bogn *Bogn) Close()

Close this instance, no calls allowed after Close.

func (*Bogn) Commit 

func (bogn *Bogn) Commit(appdata []byte)

Commit will trigger a memory to disk flush and/or disk compaction. Applications can supply appdata that will be stored as part of the lastest snapshot.

func (*Bogn) Delete 

func (bogn *Bogn) Delete(key, oldvalue []byte, lsm bool) ([]byte, uint64)

Delete key from index. Key should not be nil, if key found return its value. If lsm is true, then don't delete the node instead mark the node as deleted. Again, if lsm is true but key is not found in index, a new entry will inserted.

func (*Bogn) Destroy 

func (bogn *Bogn) Destroy()

Destroy the disk snapshots of this instance, no calls allowed after Destroy.

func (*Bogn) Disksnapshots 

func (bogn *Bogn) Disksnapshots() func(fin bool) api.Disksnapshot

Disksnapshots return a iterator to iterate on disk snapshots. Until the iterator is closed, by calling diskiterator(true /*fin*/), all write operations will be blocked. Caller can iterate until a nil is return for api.Disksnapshot

func (*Bogn) Get 

func (bogn *Bogn) Get(key, value []byte) (v []byte, cas uint64, del, ok bool)

Get value for key, if value argument points to valid buffer it will, be used to copy the entry's value. Also returns entry's cas, whether entry is marked as deleted by LSM. If ok is false, then key is not found.

func (*Bogn) Getseqno 

func (bogn *Bogn) Getseqno() uint64

Getseqno return current mutation seqno on write path.

func (*Bogn) ID 

func (bogn *Bogn) ID() string

ID is same as the name of the instance used when creating it.

func (*Bogn) Log 

func (bogn *Bogn) Log()

Log vital statistics for all active bogn levels.

func (*Bogn) Scan 

func (bogn *Bogn) Scan() api.Iterator

Scan return a full table iterator, if iteration is stopped before reaching end of table (io.EOF), application should call iterator with fin as true. EG: iter(true)

func (*Bogn) ScanEntries 

func (bogn *Bogn) ScanEntries() api.EntryIterator

ScanEntries is not supported by Bogn.

func (*Bogn) Set 

func (bogn *Bogn) Set(key, value, oldvalue []byte) (ov []byte, cas uint64)

Set a key, value pair in the index, if key is already present, its value will be over-written. Make sure key is not nil. Return old value if oldvalue points to valid buffer.

func (*Bogn) SetCAS 

func (bogn *Bogn) SetCAS(
	key, value, oldvalue []byte, cas uint64) ([]byte, uint64, error)

SetCAS a key, value pair in the index, if CAS is ZERO then key should not be present in the index, otherwise existing CAS should match the supplied CAS. Value will be over-written. Make sure key is not nil. Return old value if oldvalue points to valid buffer.

func (*Bogn) Start 

func (bogn *Bogn) Start() *Bogn

Start bogn service. Typically bogn instances are created and started as: 

inst := NewBogn("storage", setts).Start()

func (*Bogn) TombstonePurge 

func (bogn *Bogn) TombstonePurge()

TombstonePurge call will remove all entries marked as deleted from the oldest and top-most disk level, provided the highest seqno stored in that level is less that `seqno`.

func (*Bogn) Validate 

func (bogn *Bogn) Validate()

Validate active bogn levels.

func (*Bogn) View 

func (bogn *Bogn) View(id uint64) api.Transactor

View starts a read-only transaction. Other than that it is similar to BeginTxn. All view transactions should be aborted.

type Cursor 

type Cursor struct {
	// contains filtered or unexported fields
}

Cursor object maintains an active pointer into index. Use OpenCursor on Txn object to create a new cursor.

func (*Cursor) Delcursor 

func (cur *Cursor) Delcursor(lsm bool)

Delcursor deletes the entry at the cursor.

func (*Cursor) Delete 

func (cur *Cursor) Delete(key, oldvalue []byte, lsm bool) []byte

Delete is an alias to txn.Delete call. The current position of the cursor does not affect the delete operation.

func (*Cursor) GetNext 

func (cur *Cursor) GetNext() (key, value []byte, deleted bool, err error)

GetNext move cursor to next entry in snapshot and return its key and value. Returned byte slices will be a reference to index entry, hence must not be used after transaction is committed or aborted.

func (*Cursor) Key 

func (cur *Cursor) Key() (key []byte, deleted bool)

Key return current key under the cursor. Returned byte slice will be a reference to index-key, hence must not be used after transaction is commited or aborted.

func (*Cursor) Set 

func (cur *Cursor) Set(key, value, oldvalue []byte) []byte

Set is an alias to txn.Set call. The current position of the cursor does not affect the set operation.

func (*Cursor) Value 

func (cur *Cursor) Value() []byte

Value return current value under the cursor. Returned byte slice will be a reference to value in index, hence must not be used after transaction is commited or aborted.

func (*Cursor) YNext 

func (cur *Cursor) YNext(
	fin bool) (key, value []byte, cas uint64, deleted bool, err error)

YNext can be used for lsm-sort.

type Txn 

type Txn struct {
	// contains filtered or unexported fields
}

Txn transaction definition. Transaction gives a gaurantee of isolation and atomicity on the latest snapshot.

func (*Txn) Abort 

func (txn *Txn) Abort()

Abort transaction, underlying index won't be touched.

func (*Txn) Commit 

func (txn *Txn) Commit() error

Commit transaction, commit will block until all write operations under the transaction are successfully applied. Return ErrorRollback if ACID properties are not met while applying the write operations. Transactions are never partially committed.

func (*Txn) Delete 

func (txn *Txn) Delete(key, oldvalue []byte, lsm bool) []byte

Delete key from index. The Delete operation will be remembered as a log entry and applied on the underlying structure during commit.

func (*Txn) Get 

func (txn *Txn) Get(
	key, value []byte) (v []byte, cas uint64, deleted, ok bool)

Get value for key from snapshot.

func (*Txn) ID 

func (txn *Txn) ID() uint64

ID return transaction id.

func (*Txn) OpenCursor 

func (txn *Txn) OpenCursor(key []byte) (api.Cursor, error)

OpenCursor open an active cursor inside the index.

func (*Txn) Set 

func (txn *Txn) Set(key, value, oldvalue []byte) []byte

Set an entry of key, value pair. The set operation will be remembered as a log entry and applied on the underlying structure during Commit.

type View 

type View struct {
	// contains filtered or unexported fields
}

View transaction definition. Read only version of Txn.

func (*View) Abort 

func (view *View) Abort()

Abort view, must be called once done with the view.

func (*View) Commit 

func (view *View) Commit() error

Commit not allowed.

func (*View) Delete 

func (view *View) Delete(key, oldvalue []byte, lsm bool) []byte

Delete is not allowed.

func (*View) Get 

func (view *View) Get(
	key, value []byte) (v []byte, cas uint64, deleted, ok bool)

Get value for key from snapshot.

func (*View) ID 

func (view *View) ID() uint64

ID return transaction id.

func (*View) OpenCursor 

func (view *View) OpenCursor(key []byte) (api.Cursor, error)

OpenCursor open an active cursor inside the index.

func (*View) Set 

func (view *View) Set(key, value, oldvalue []byte) []byte

Set is not allowed

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.