README
¶
yojoudb
An embeddable, fast and persistent key-value storage engine based on Bitcask.
Getting Started
Basic Example
package main
import (
"fmt"
"github.com/berylyvos/yojoudb"
)
func main() {
// specify the options
options := yojoudb.DefaultOptions
// open a database
db, err := yojoudb.Open(options)
if err != nil {
panic(err)
}
defer func() {
_ = db.Close()
}()
// put a key
key := []byte("hello")
if err = db.Put(key, []byte("yojoudb")); err != nil {
panic(err)
}
// get a key
val, err := db.Get(key)
if err != nil {
panic(err)
}
println(string(val))
// delete a key
if err = db.Delete(key); err != nil {
panic(err)
}
// create a batch
batch := db.NewBatch(yojoudb.DefaultBatchOptions)
// batch put keys/values
for i := 0; i < 100; i++ {
_ = batch.Put([]byte(fmt.Sprintf("#%d", i)), []byte(fmt.Sprintf("yojoudb-%d", i)))
}
// commit the batch
_ = batch.Commit()
// create an iterator
iter := db.NewIterator(yojoudb.DefaultIteratorOptions)
defer iter.Close()
// iterate over all data
for ; iter.Valid(); iter.Next() {
v, _ := iter.Value()
println(string(v))
}
}
Benchmarks
We compared how well yojoudb performs in random writes and random point lookups against several high-performing Golang-based key-value stores using the benchmarking tool pogreb-bench.
Performance Metrics
| Engine | PUT | GET | put + get | file size | peak sys mem |
|---|---|---|---|---|---|
| yojoudb | 13.771s 145229 ops/s | 2.163s 924817 ops/s | 15.934s | 782.15MB | 1.31GB |
| badger | 8.813s 226930 ops/s | 4.939s 404921 ops/s | 13.752s | 250.95MB | 3.60GB |
| pebble | 13.594s 147125 ops/s | 4.844s 412882 ops/s | 18.438s | 229.16MB | 446.60MB |
| goleveldb | 25.199s 79367 ops/s | 6.956s 287539 ops/s | 32.155s | 714.31MB | 529.79MB |
| bbolt | 84.245s 23740 ops/s | 1.555s 1286247 ops/s | 85.800s | 1.03GB | 481.17MB |
Parameters
| key nums | key size | val size | concurrency |
|---|---|---|---|
| 2000000 | 16 ~ 64 | 128 ~ 512 | 5 |
In-memory Table
For yojoudb, we use ART(Adaptive Radix Tree) as the default in-memory table. Alternatively, other index types (B-tree, skiplist) can be specified by yojoudb.DefaultOptions.IndexType (IndexBTree, IndexART, IndexSKL).
Documentation
¶
Index ¶
- Constants
- Variables
- type Batch
- type BatchOptions
- type DB
- func (db *DB) Close() error
- func (db *DB) Delete(key K) error
- func (db *DB) Exist(key K) (bool, error)
- func (db *DB) Fold(fn func(key []byte, value []byte) bool) error
- func (db *DB) Get(key K) (V, error)
- func (db *DB) GetDirPath() string
- func (db *DB) ListKeys() [][]byte
- func (db *DB) Merge() error
- func (db *DB) NewBatch(options BatchOptions) *Batch
- func (db *DB) NewIterator(options IteratorOptions) *Iterator
- func (db *DB) Put(key K, val V) error
- func (db *DB) Stat() *Stat
- func (db *DB) Sync() error
- type IndexRecord
- type IndexType
- type Iterator
- type IteratorOptions
- type K
- type LR
- type LRType
- type Loc
- type LogRecord
- type Options
- type Stat
- type V
Constants ¶
const ( B = 1 KB = 1024 * B MB = 1024 * KB GB = 1024 * MB )
Variables ¶
var ( ErrKeyEmpty = errors.New("the key is empty") ErrKeyNotFound = errors.New("key is not found in database") ErrDirPathIsEmpty = errors.New("database dir path is empty") ErrDataFileSizeNotPositive = errors.New("database data file size must be greater than 0") ErrDatabaseIsUsing = errors.New("the database directory is used by another process") ErrDBClosed = errors.New("the database is closed") ErrReadOnlyBatch = errors.New("the batch is read only") ErrMergeIsRunning = errors.New("merge is in progress, try again later") )
var DefaultBatchOptions = BatchOptions{ Sync: true, ReadOnly: false, }
var DefaultIteratorOptions = IteratorOptions{ Prefix: nil, Reverse: false, }
Functions ¶
This section is empty.
Types ¶
type Batch ¶
type Batch struct {
// contains filtered or unexported fields
}
Batch is a batch operation of the database. A batch can be read-only, calling Get() to get data. Otherwise, calling Delete() or Put() to put data. You must call Commit() to commit the batch, otherwise the deadlock happens.
Batch is not a transaction, for it does not guarantee isolation. But it can guarantee atomicity, consistency and durability.
func (*Batch) Commit ¶
Commit commits the batch, if the batch is readonly or empty, return. It will iterate the pendingWrites and write the data to the db, then write a record to indicate the end of batch to guarantee atomicity. Finally, index will be updated.
func (*Batch) Get ¶
Get gets the value of the given key. The key could still be in pendingWrites and haven't committed yet, return the value if the record type is not LRDeleted. Otherwise, get the value from WAL.
type BatchOptions ¶
type DB ¶
type DB struct {
// contains filtered or unexported fields
}
DB is a database instance. It's built on a log-structured model, the Bitcask. Read and write data based on WAL(Write-Ahead Log). An in-memory index is for holding the keys and the corresponding locations. The index is rebuilt each time the database is restarted.
func Open ¶
Open opens a db instance with specified options. It will open the WAL files and build the index.
func (*DB) GetDirPath ¶
func (*DB) Merge ¶
Merge merges all the data files. It will iterate all over the WAL, filtering valid data, then writing them to a new WAL in the merge directory.
Calling this method is up to the users, it may be very time-consuming when the database is large.
func (*DB) NewBatch ¶
func (db *DB) NewBatch(options BatchOptions) *Batch
func (*DB) NewIterator ¶
func (db *DB) NewIterator(options IteratorOptions) *Iterator
type IndexRecord ¶
type IndexRecord struct {
// contains filtered or unexported fields
}
IndexRecord is the index record of the key. Only used in start up to build in-mem index.
type Iterator ¶
Iterator is the iterator of db based on IndexIter.
func (*Iterator) Seek ¶
Seek moves the iterator to the key which is greater(or less when reverse) than or equal to the specified key.
Source Files
Directories