Documentation
¶
Index ¶
- Constants
- Variables
- func AssertTrue(b bool)
- func AssertTruef(b bool, format string, args ...interface{})
- func Check(err error)
- func Check2(_ interface{}, err error)
- func CompareKeys(key1, key2 []byte) int
- func Copy(a []byte) []byte
- func CreateSyncedFile(filename string, sync bool) (*os.File, error)
- func FixedDuration(d time.Duration) string
- func KeyWithTs(key []byte, ts uint64) []byte
- func Lock(b []byte) error
- func Madvise(b []byte) error
- func Mmap(fd *os.File, offset, length int64, writable bool) ([]byte, error)
- func Munmap(b []byte) error
- func OpenExistingFile(filename string, flags uint32) (*os.File, error)
- func OpenSyncedFile(filename string, sync bool) (*os.File, error)
- func OpenTruncFile(filename string, sync bool) (*os.File, error)
- func ParseKey(key []byte) []byte
- func ParseTs(key []byte) uint64
- func SafeCopy(a, src []byte) []byte
- func SameKey(src, dst []byte) bool
- func Sync(b []byte) error
- func SyncDir(dir string) error
- func Unlock(b []byte) error
- func Wrap(err error) error
- func Wrapf(err error, format string, args ...interface{}) error
- type Allocator
- type Arena
- func (a *Arena) Acquire(offset, size uint32) []byte
- func (a *Arena) Allocate(n uint32) []byte
- func (a *Arena) AllocateAligned(n, align uint32) []byte
- func (a *Arena) AllocateSlot(n uint32) (begin, end uint32)
- func (a *Arena) AllocateSlotAligned(n, align uint32) (begin, end uint32)
- func (a *Arena) Reset()
- func (a *Arena) Size() int64
- type Iterator
- func (s *Iterator) Close() error
- func (s *Iterator) Key() []byte
- func (s *Iterator) Next()
- func (s *Iterator) Prev()
- func (s *Iterator) Seek(key []byte)
- func (s *Iterator) SeekForPrev(key []byte)
- func (s *Iterator) SeekToFirst()
- func (s *Iterator) SeekToLast()
- func (s *Iterator) Valid() bool
- func (s *Iterator) Value() []byte
- type Option
- type Skiplist
- func (s *Skiplist) DecrRef()
- func (s *Skiplist) Empty() bool
- func (s *Skiplist) Get(key []byte) []byte
- func (s *Skiplist) GetWithKey(key []byte) (nkey, value []byte)
- func (s *Skiplist) IncrRef()
- func (s *Skiplist) MemSize() int64
- func (s *Skiplist) NewIterator() *Iterator
- func (s *Skiplist) NewUniIterator(reversed bool) *UniIterator
- func (s *Skiplist) Put(key, value []byte)
- type Slice
- type Throttle
- type UniIterator
- type WaterMark
- func (w *WaterMark) Begin(index uint64)
- func (w *WaterMark) BeginMany(indices []uint64)
- func (w *WaterMark) Close() error
- func (w *WaterMark) Done(index uint64)
- func (w *WaterMark) DoneMany(indices []uint64)
- func (w *WaterMark) DoneUntil() uint64
- func (w *WaterMark) LastIndex() uint64
- func (w *WaterMark) SetDoneUntil(val uint64)
- func (w *WaterMark) WaitForMark(ctx context.Context, index uint64) error
Constants ¶
const ( // DSync indicates that O_DSYNC should be set on the underlying file, // ensuring that data writes do not return until the data is flushed // to disk. DSync = 1 << iota // ReadOnly opens the underlying file on a read-only basis. ReadOnly )
const ( // MaxNodeSize is the memory footprint of a node of maximum height. MaxNodeSize = int(unsafe.Sizeof(node{})) )
Variables ¶
var ( // LSMSize has size of the LSM in bytes LSMSize *expvar.Map // VlogSize has size of the value log in bytes VlogSize *expvar.Map // PendingWrites tracks the number of pending writes. PendingWrites *expvar.Map // NumReads has cumulative number of reads NumReads *expvar.Int // NumWrites has cumulative number of writes NumWrites *expvar.Int // NumBytesRead has cumulative number of bytes read NumBytesRead *expvar.Int // NumBytesWritten has cumulative number of bytes written NumBytesWritten *expvar.Int // NumLSMGets is number of LMS gets NumLSMGets *expvar.Map // NumLSMBloomHits is number of LMS bloom hits NumLSMBloomHits *expvar.Map // NumGets is number of gets NumGets *expvar.Int // NumPuts is number of puts NumPuts *expvar.Int // NumBlockedPuts is number of blocked puts NumBlockedPuts *expvar.Int // NumMemtableGets is number of memtable gets NumMemtableGets *expvar.Int )
var ( // CastagnoliCrcTable is a CRC32 polynomial table CastagnoliCrcTable = crc32.MakeTable(crc32.Castagnoli) )
var ErrEOF = errors.New("End of mapped region")
ErrEOF indicates an end of file when trying to read from a memory mapped file and encountering the end of slice.
Functions ¶
func AssertTrue ¶
func AssertTrue(b bool)
AssertTrue asserts that b is true. Otherwise, it would log fatal.
func AssertTruef ¶
AssertTruef is AssertTrue with extra info.
func Check2 ¶
func Check2(_ interface{}, err error)
Check2 acts as convenience wrapper around Check, using the 2nd argument as error.
func CompareKeys ¶
CompareKeys checks the key without timestamp and checks the timestamp if keyNoTs is same. a<timestamp> would be sorted higher than aa<timestamp> if we use bytes.compare All keys should have timestamp.
func CreateSyncedFile ¶
CreateSyncedFile creates a new file (using O_EXCL), errors if it already existed.
func FixedDuration ¶
FixedDuration returns a string representation of the given duration with the hours, minutes, and seconds.
func OpenExistingFile ¶
OpenExistingFile opens an existing file, errors if it doesn't exist.
func OpenSyncedFile ¶
OpenSyncedFile creates the file if one doesn't exist.
func OpenTruncFile ¶
OpenTruncFile opens the file with O_RDWR | O_CREATE | O_TRUNC
func SyncDir ¶
SyncDir When you create or delete a file, you have to ensure the directory entry for the file is synced in order to guarantee the file is visible (if the system crashes). (See the man page for fsync, or see https://github.com/coreos/etcd/issues/6368 for an example.)
Types ¶
type Allocator ¶
type Allocator interface {
Allocate(n uint32) []byte
AllocateAligned(n, align uint32) []byte
AllocateSlot(n uint32) (begin, end uint32)
AllocateSlotAligned(n, align uint32) (begin, end uint32)
Acquire(offset, size uint32) []byte
Size() int64
Reset()
}
Allocator the interface of allocator
type Arena ¶
type Arena struct {
// contains filtered or unexported fields
}
Arena should be lock-free.
func (*Arena) Acquire ¶
Acquire get the byte by given offset and size. it will painc when index out of slice.
func (*Arena) Allocate ¶
Allocate allocs n bytes of slice from the buffer. If the free space isn't enough, it will panic.
func (*Arena) AllocateAligned ¶
AllocateAligned align allocs n bytes of slice from the buffer. If the free space isn't enough, it will panic.
func (*Arena) AllocateSlot ¶
AllocateSlot allocate n bytes return the pos begin and end. If the free space isn't enough, it will panic.
func (*Arena) AllocateSlotAligned ¶
AllocateSlotAligned allocate aligned n bytes return the pos begin and end. If the free space isn't enough, it will panic.
type Iterator ¶
type Iterator struct {
// contains filtered or unexported fields
}
Iterator is an iterator over skiplist object. For new objects, you just need to initialize Iterator.list.
func (*Iterator) SeekForPrev ¶
SeekForPrev finds an entry with key <= target.
func (*Iterator) SeekToFirst ¶
func (s *Iterator) SeekToFirst()
SeekToFirst seeks position at the first entry in list. Final state of iterator is Valid() iff list is not empty.
func (*Iterator) SeekToLast ¶
func (s *Iterator) SeekToLast()
SeekToLast seeks position at the last entry in list. Final state of iterator is Valid() iff list is not empty.
type Option ¶
type Option func(s *Skiplist)
Option used to init user defined params
func KeyComparator ¶
KeyComparator user defined key comparator
func KeyEqualizer ¶
KeyEqualizer user defined key euqalizer used to get.
type Skiplist ¶
type Skiplist struct {
KeyComparator func(key1, key2 []byte) int
KeyEqualizer func(key1, key2 []byte) bool
// contains filtered or unexported fields
}
Skiplist maps keys to values (in memory)
func NewSkiplist ¶
NewSkiplist makes a new empty skiplist, with a given arena size
func (*Skiplist) DecrRef ¶
func (s *Skiplist) DecrRef()
DecrRef decrements the refcount, deallocating the Skiplist when done using it
func (*Skiplist) Get ¶
Get gets the value associated with the key. It returns a valid value if it finds equal or earlier version of the same key.
func (*Skiplist) GetWithKey ¶
GetWithKey gets the value associated with the key. It returns a valid key and value if it finds equal or earlier version of the same key, it can be used impl multi version key.
func (*Skiplist) MemSize ¶
MemSize returns the size of the Skiplist in terms of how much memory is used within its internal allocator.
func (*Skiplist) NewIterator ¶
NewIterator returns a skiplist iterator. You have to Close() the iterator.
func (*Skiplist) NewUniIterator ¶
func (s *Skiplist) NewUniIterator(reversed bool) *UniIterator
NewUniIterator returns a UniIterator.
type Slice ¶
type Slice struct {
// contains filtered or unexported fields
}
Slice holds a reusable buf, will reallocate if you request a larger size than ever before. One problem is with n distinct sizes in random order it'll reallocate log(n) times.
type Throttle ¶
type Throttle struct {
// contains filtered or unexported fields
}
Throttle allows a limited number of workers to run at a time. It also provides a mechanism to check for errors encountered by workers and wait for them to finish.
func NewThrottle ¶
NewThrottle creates a new throttle with a max number of workers.
func (*Throttle) Do ¶
Do should be called by workers before they start working. It blocks if there are already maximum number of workers working. If it detects an error from previously Done workers, it would return it.
func (*Throttle) Done ¶
Done should be called by workers when they finish working. They can also pass the error status of work done.
type UniIterator ¶
type UniIterator struct {
// contains filtered or unexported fields
}
UniIterator is a unidirectional memtable iterator. It is a thin wrapper around Iterator. We like to keep Iterator as before, because it is more powerful and we might support bidirectional iterators in the future.
func (*UniIterator) Close ¶
func (s *UniIterator) Close() error
Close implements y.Interface (and frees up the iter's resources)
type WaterMark ¶
type WaterMark struct {
Name string
// contains filtered or unexported fields
}
WaterMark is used to keep track of the minimum un-finished index. Typically, an index k becomes finished or "done" according to a WaterMark once Done(k) has been called
- as many times as Begin(k) has, AND
- a positive number of times.
An index may also become "done" by calling SetDoneUntil at a time such that it is not inter-mingled with Begin/Done calls.
Since doneUntil and lastIndex addresses are passed to sync/atomic packages, we ensure that they are 64-bit aligned by putting them at the beginning of the structure.
func NewWaterMark ¶
NewWaterMark return a instance of watermark
func (*WaterMark) DoneUntil ¶
DoneUntil returns the maximum index that has the property that all indices less than or equal to it are done.
func (*WaterMark) SetDoneUntil ¶
SetDoneUntil sets the maximum index that has the property that all indices less than or equal to it are done.
Source Files