Documentation ¶
Overview ¶
Copyright (c) 2014-2015 Rkv Authors. Released under MIT license. This is based on https://code.google.com/p/gocask/ initial work by andrebq. Implementation of the bitcask key-value store from Riak. Paper available at: http://downloads.basho.com/papers/bitcask-intro.pdf The key-value pairs is stored in a log file, which only writes to the append, so a write never include a disk seek. Each record is stored in the following format: |-------------------------------------------------------------------------------------------------------------------| |crc (int32) | tstamp (int32) | key length (int32) | value length (int32) | key data ([]byte) | value data ([]byte) | |-------------------------------------------------------------------------------------------------------------------| We use mostly same format but diverge in few aspects: 1. If []byte value contains no data and is empty array, then it is deleted key, no data. 2. tstamp contains days or 0. If tstamp is not 0 and it is less than todays day, key record has expired. Use PutForDays to take advantage of automatic record expiration. 3. Compact and AutoCompact reads database and compacts it. 4. Internally structs stored as JSON. This is decent format for databases up to 50K records.
Index ¶
- Constants
- Variables
- func ConvertToString(buff []byte) string
- type GFile
- type Interface
- type Keydir
- type KeydirEntry
- type Rkv
- func (kv *Rkv) AutoCompact(fillRatio float64) error
- func (kv *Rkv) Close()
- func (kv *Rkv) Compact() error
- func (kv *Rkv) Delete(key string) error
- func (kv *Rkv) DeleteAllKeys(with string) error
- func (kv *Rkv) Exist(key string) bool
- func (kv *Rkv) ExportJSON(w io.Writer) error
- func (kv *Rkv) ExportKeyJSON(w io.Writer, key string) error
- func (kv *Rkv) ExportKeysJSON(w io.Writer, with string) error
- func (kv *Rkv) Get(key string, value interface{}) error
- func (kv *Rkv) GetBytes(key string) ([]byte, error)
- func (kv *Rkv) GetKeys(with string, limit int) []string
- func (kv *Rkv) ImportCSV(r io.Reader, key int) error
- func (kv *Rkv) ImportJSON(r io.Reader) error
- func (kv *Rkv) Iterator(with string) <-chan string
- func (kv *Rkv) Put(key string, value interface{}) error
- func (kv *Rkv) PutForDays(key string, value interface{}, days int32) error
- func (kv *Rkv) Reopen() error
- type SafeRkv
- func (kv *SafeRkv) Compact() error
- func (kv *SafeRkv) Delete(key string) error
- func (kv *SafeRkv) DeleteAllKeys(with string) error
- func (kv *SafeRkv) Exist(key string) bool
- func (kv *SafeRkv) ExportJSON(w io.Writer) error
- func (kv *SafeRkv) Get(key string, value interface{}) error
- func (kv *SafeRkv) GetBytes(key string) ([]byte, error)
- func (kv *SafeRkv) GetKeys(with string, limit int) []string
- func (kv *SafeRkv) Iterator(with string) <-chan string
- func (kv *SafeRkv) Put(key string, value interface{}) error
- func (kv *SafeRkv) PutForDays(key string, value interface{}, days int32) error
Constants ¶
const ( /* Header offset for each record in the store. This offset contains the following information (in the given order) | -------------------------------------------------------------------------| | crc (int32) | tstamp (int32) | key length (int32) | value length (int32) | | -------------------------------------------------------------------------| */ RecordHeaderSize int32 = 16 MinCapKeys = 1000 )
Variables ¶
Functions ¶
func ConvertToString ¶
ConvertToString convert the given byte buffer using the default Go encoding (utf-8).
Types ¶
type GFile ¶
type GFile struct {
// contains filtered or unexported fields
}
GFile wrap a os.file and provide some convenient methods.
type Interface ¶
type Interface interface { Reopen() error Close() Compact() error GetKeys(with string, limit int) []string Get(key string, value interface{}) error GetBytes(key string) ([]byte, error) Put(key string, value interface{}) error PutForDays(key string, value interface{}, days int32) error Exist(key string) bool Delete(key string) error DeleteAllKeys(with string) error ExportJSON(w io.Writer) error ImportJSON(r io.Reader) error }
Interface only contains functions applicable to both Rkv and SafeRkv. Rkv has few more functions that are not goroutine friendly and not part of this common interface.
type Keydir ¶
type Keydir struct {
// contains filtered or unexported fields
}
Keydir in memory structure that holds the location of all the keys in the key-value store.
type KeydirEntry ¶
type KeydirEntry struct {
// contains filtered or unexported fields
}
KeydirEntry entries in the keydir, which holds the location of any key in the key-store.
type Rkv ¶
type Rkv struct { // values below are calculated only when store is open, they are not updated on Delete or Put FillRatio float64 // active records divided by dead-removed records, used for AutoCompact CapKeys int // total number of keys = alive + dead LenKeys int // number of keys = alive // contains filtered or unexported fields }
Main structure for any Rkv file. Holds the current directory and the active file
func New ¶
NewRkv open the key-value store at the given file. If the file doesn't exist one will be created. Always try to open for read-write, but if someone is already using this file, open in read-only mode. Populate the KeyDir structure with the information obtained from the data file.
func (*Rkv) AutoCompact ¶
AutoCompact auto compacts database once active records divided by dead-removed records (fill ratio) drops below fillRatio and there are enough alive and dead keys expressed as MinCapKeys.
func (*Rkv) DeleteAllKeys ¶
DeleteAllKeys that match.
func (*Rkv) ExportJSON ¶
ExportJSON export all data from KV store as mixed JSON.
func (*Rkv) ExportKeyJSON ¶
ExportKeyJSON export single key data from KV store as mixed JSON.
func (*Rkv) ExportKeysJSON ¶
ExportKeysJSON export keys data from KV store as mixed JSON. Will return KeyNotFound error if can not find such key in datastore.
func (*Rkv) Get ¶
Get retrieves the value for the given key from the keystore. May return ErrKeyNotFound error if can not find such key in datastore.
func (*Rkv) GetKeys ¶
GetKeys returns limited number of keys matching criterio, if limit is negative then returns all.
func (*Rkv) ImportJSON ¶
ImportJSON imports files produced with ExportJSON function, may use os.Stdin.
func (*Rkv) Iterator ¶
Iterator returns iterator object (channel) of key values, do not use in more than one goroutine.
func (*Rkv) PutForDays ¶
PutForDays save the key-value pair in the current file with expiration in future date. Checking for expiration happens on database load, so only when database is reopen records become expired.
type SafeRkv ¶
type SafeRkv struct { Rkv // contains filtered or unexported fields }
SafeRkv wraps Rkv to provide goroutine safe access to KV store.
func (*SafeRkv) DeleteAllKeys ¶
DeleteAllKeys same as Rkv function but goroutine friendly.
func (*SafeRkv) ExportJSON ¶
ExportJSON same as Rkv function but goroutine friendly.