Documentation ¶
Index ¶
- Constants
- Variables
- type Bucket
- func (b *Bucket) Bucket(name []byte) *Bucket
- func (b *Bucket) CreateBucket(key []byte) (*Bucket, error)
- func (b *Bucket) CreateBucketIfNotExists(key []byte) (*Bucket, error)
- func (b *Bucket) Cursor() *Cursor
- func (b *Bucket) Delete(key []byte) error
- func (b *Bucket) DeleteBucket(key []byte) error
- func (b *Bucket) ForEach(fn func(k, v []byte) error) error
- func (b *Bucket) Get(key []byte) []byte
- func (b *Bucket) NextSequence() (uint64, error)
- func (b *Bucket) Put(key []byte, value []byte) error
- func (b *Bucket) Root() pgid
- func (b *Bucket) Sequence() uint64
- func (b *Bucket) SetSequence(v uint64) error
- func (b *Bucket) Stats() BucketStats
- func (b *Bucket) Tx() *Tx
- func (b *Bucket) Writable() bool
- type BucketStats
- type Cursor
- func (c *Cursor) Bucket() *Bucket
- func (c *Cursor) Delete() error
- func (c *Cursor) First() (key []byte, value []byte)
- func (c *Cursor) Last() (key []byte, value []byte)
- func (c *Cursor) Next() (key []byte, value []byte)
- func (c *Cursor) Prev() (key []byte, value []byte)
- func (c *Cursor) Seek(seek []byte) (key []byte, value []byte)
- type DB
- func (db *DB) Batch(fn func(*Tx) error) error
- func (db *DB) Begin(writable bool) (*Tx, error)
- func (db *DB) Close() error
- func (db *DB) GoString() string
- func (db *DB) Info() *Info
- func (db *DB) IsReadOnly() bool
- func (db *DB) Path() string
- func (db *DB) Stats() Stats
- func (db *DB) String() string
- func (db *DB) Sync() error
- func (db *DB) Update(fn func(*Tx) error) error
- func (db *DB) View(fn func(*Tx) error) error
- type Info
- type Options
- type PageInfo
- type Stats
- type Tx
- func (tx *Tx) Bucket(name []byte) *Bucket
- func (tx *Tx) Check() <-chan error
- func (tx *Tx) Commit() error
- func (tx *Tx) Copy(w io.Writer) error
- func (tx *Tx) CopyFile(path string, mode os.FileMode) error
- func (tx *Tx) CreateBucket(name []byte) (*Bucket, error)
- func (tx *Tx) CreateBucketIfNotExists(name []byte) (*Bucket, error)
- func (tx *Tx) Cursor() *Cursor
- func (tx *Tx) DB() *DB
- func (tx *Tx) DeleteBucket(name []byte) error
- func (tx *Tx) ForEach(fn func(name []byte, b *Bucket) error) error
- func (tx *Tx) ID() int
- func (tx *Tx) OnCommit(fn func())
- func (tx *Tx) Page(id int) (*PageInfo, error)
- func (tx *Tx) Rollback() error
- func (tx *Tx) Size() int64
- func (tx *Tx) Writable() bool
- func (tx *Tx) WriteTo(w io.Writer) (n int64, err error)
- type TxStats
Constants ¶
const ( MaxKeySize = 32768 MaxValueSize = (1 << 31) - 2 )
const ( DefaultMaxBatchSize int = 1000 DefaultMaxBatchDelay = 10 * time.Millisecond // 16k DefaultAllocSize = 16 * 1024 * 1024 )
Default values if not set in a DB instance.
const DefaultFillPercent = 0.5
const IgnoreNoSync = runtime.GOOS == "openbsd"
IgnoreNoSync specifies whether the NoSync field of a DB is ignored when syncing changes to a file. This is required as some operating systems, such as OpenBSD, do not have a unified buffer cache (UBC) and writes must be synchronized using the msync(2) syscall.
Variables ¶
var ( // ErrDatabaseNotOpen is returned when a DB instance is accessed before it // is opened or after it is closed. ErrDatabaseNotOpen = errors.New("database not open") // ErrDatabaseOpen is returned when opening a database that is // already open. ErrDatabaseOpen = errors.New("database already open") // ErrInvalid is returned when both meta pages on a database are invalid. // This typically occurs when a file is not a bolt database. ErrInvalid = errors.New("invalid database") // ErrVersionMismatch is returned when the data file was created with a // different version of Bolt. ErrVersionMismatch = errors.New("version mismatch") // ErrChecksum is returned when either meta page checksum does not match. ErrChecksum = errors.New("checksum error") // ErrTimeout is returned when a database cannot obtain an exclusive lock // on the data file after the timeout passed to Open(). ErrTimeout = errors.New("timeout") )
These errors can be returned when opening or calling methods on a DB.
var ( // ErrTxNotWritable is returned when performing a write operation on a // read-only transaction. ErrTxNotWritable = errors.New("tx not writable") // ErrTxClosed is returned when committing or rolling back a transaction // that has already been committed or rolled back. ErrTxClosed = errors.New("tx closed") // ErrDatabaseReadOnly is returned when a mutating transaction is started on a // read-only database. ErrDatabaseReadOnly = errors.New("database is in read-only mode") )
These errors can occur when beginning or committing a Tx.
var ( // ErrBucketNotFound is returned when trying to access a bucket that has // not been created yet. ErrBucketNotFound = errors.New("bucket not found") // ErrBucketExists is returned when creating a bucket that already exists. ErrBucketExists = errors.New("bucket already exists") // ErrBucketNameRequired is returned when creating a bucket with a blank name. ErrBucketNameRequired = errors.New("bucket name required") // ErrKeyRequired is returned when inserting a zero-length key. ErrKeyRequired = errors.New("key required") // ErrKeyTooLarge is returned when inserting a key that is larger than MaxKeySize. ErrKeyTooLarge = errors.New("key too large") // ErrValueTooLarge is returned when inserting a value that is larger than MaxValueSize. ErrValueTooLarge = errors.New("value too large") // ErrIncompatibleValue is returned when trying create or delete a bucket // on an existing non-bucket key or when trying to create or delete a // non-bucket key on an existing bucket key. ErrIncompatibleValue = errors.New("incompatible value") )
These errors can occur when putting or deleting a value or a bucket.
var DefaultOptions = &Options{ Timeout: 0, NoGrowSync: false, }
DefaultOptions represent the options used if nil options are passed into Open(). No timeout is used which will cause Bolt to wait indefinitely for a lock.
var DefaultPageSize = os.Getpagesize()
default page size for db is set to the OS page size.
Functions ¶
This section is empty.
Types ¶
type Bucket ¶
type Bucket struct { FillPercent float64 // 填充率 // contains filtered or unexported fields }
一组key/value的集合 也就是一个b+树 单个Bucket的结构: [bucketElem | b_key | b_val ] b_val = val + page page : [pageHeader|leafElem1|leafElem2|...|leafElemn|k1|v1|k2|v2|...|kn|vn]
当一个桶同时满足下面两个条件的时候 视作内联 1. 当前的桶没有其他嵌套子桶 2. 当前桶内的元素所占的总字节数小于 1/4 pageSize(4K)
多个Bucket的结构:
[val1|child_inline_page1]
[bucketElem1 | bucketElem2 | bkey_1 | b_val1 | b_key2 | b_val2]
[val2|child_inline_page2]
func (*Bucket) CreateBucket ¶
根据 pgid 给指定的 parent 整个 child_node 用 key 创建 一个新桶
func (*Bucket) CreateBucketIfNotExists ¶
创建一个Bucket 根据指定的key来创建一个Bucket,如果指定key的Bucket已经存在,则会报错 如果指定的key之前有插入过元素,也会报错 否则的话,会在当前的Bucket中找到合适的位置,然后新建一个Bucket插入进去,最后返回给客户端
func (*Bucket) Cursor ¶
既然一个Bucket逻辑上是一颗b+树,那就意味着我们可以对其进行遍历 前面提到的set、get操作,无非是要在Bucket上先找到合适的位置,然后再进行操作 而“找”这个操作就是交由Cursor(游标)来完成的 简而言之对Bucket这颗b+树的遍历工作由Cursor来执行, 一个Bucket对象关联一个Cursor。
func (*Bucket) DeleteBucket ¶
TODO 能不能删掉自己 删除一个Bucket
func (*Bucket) ForEach ¶
遍历Bucket中所有的键值对 ForEach executes a function for each key/value pair in a bucket. If the provided function returns an error then the iteration is stopped and the error is returned to the caller. The provided function must not modify the bucket; this will result in undefined behavior.
func (*Bucket) NextSequence ¶
NextSequence 返回 Bucket 的自动递增整数
func (*Bucket) SetSequence ¶
SetSequence 更新 Bucket 的序列号
type BucketStats ¶
type BucketStats struct { // Page count statistics. BranchPageN int // number of logical branch pages BranchOverflowN int // number of physical branch overflow pages LeafPageN int // number of logical leaf pages LeafOverflowN int // number of physical leaf overflow pages // Tree statistics. KeyN int // number of keys/value pairs Depth int // number of levels in B+tree // Page size utilization. BranchAlloc int // bytes allocated for physical branch pages BranchInuse int // bytes actually used for branch data LeafAlloc int // bytes allocated for physical leaf pages LeafInuse int // bytes actually used for leaf data // Bucket statistics BucketN int // total number of buckets including the top bucket InlineBucketN int // total number on inlined buckets InlineBucketInuse int // bytes used for inlined buckets (also accounted for in LeafInuse) }
BucketStats records statistics about resources used by a bucket.
func (*BucketStats) Add ¶
func (s *BucketStats) Add(other BucketStats)
type Cursor ¶
type Cursor struct {
// contains filtered or unexported fields
}
func (*Cursor) Next ¶
定位到并返回 当前Cursor所在位置的 下一个 KV 再此我们从当前位置查找前一个或者下一个时,需要注意一个问题: 如果当前节点中元素已经完了,那么此时需要回退到遍历路径的上一个节点
type DB ¶
type DB struct { // When enabled, the database will perform a Check() after every commit. // A panic is issued if the database is in an inconsistent state. This // flag has a large performance impact so it should only be used for // debugging purposes. StrictMode bool // Setting the NoSync flag will cause the database to skip fsync() // calls after each commit. This can be useful when bulk loading data // into a database and you can restart the bulk load in the event of // a system failure or database corruption. Do not set this flag for // normal use. // // If the package global IgnoreNoSync constant is true, this value is // ignored. See the comment on that constant for more details. // // THIS IS UNSAFE. PLEASE USE WITH CAUTION. NoSync bool // When true, skips the truncate call when growing the database. // Setting this to true is only safe on non-ext3/ext4 systems. // Skipping truncation avoids preallocation of hard drive space and // bypasses a truncate() and fsync() syscall on remapping. // // https://github.com/boltdb/bolt/issues/284 NoGrowSync bool // If you want to read the entire database fast, you can set MmapFlag to // syscall.MAP_POPULATE on Linux 2.6.23+ for sequential read-ahead. MmapFlags int // MaxBatchSize is the maximum size of a batch. Default value is // copied from DefaultMaxBatchSize in Open. // // If <=0, disables batching. // // Do not change concurrently with calls to Batch. MaxBatchSize int // MaxBatchDelay is the maximum delay before a batch starts. // Default value is copied from DefaultMaxBatchDelay in Open. // // If <=0, effectively disables batching. // // Do not change concurrently with calls to Batch. MaxBatchDelay time.Duration // AllocSize is the amount of space allocated when the database // needs to create new pages. This is done to amortize the cost // of truncate() and fsync() when growing the data file. AllocSize int // contains filtered or unexported fields }
DB represents a collection of buckets persisted to a file on disk. All data access is performed through transactions which can be obtained through the DB. All the functions on DB will return a ErrDatabaseNotOpen if accessed before Open() is called.
func Open ¶
Open()创建数据库接口 Open()方法主要用来创建一个boltdb的DB对象,底层会执行新建或者打开存储数据的文件 当指定的文件不存在时, boltdb就会新建一个数据文件。否则的话,就直接加载指定的数据库文件内容 值的注意是,boltdb会根据Open时,options传递的参数来判断到底加互斥锁还是共享锁 新建时: 会调用init()方法,内部主要是新建一个文件
然后第0页、第1页写入元数据信息; 第2页写入freelist信息; 第3页写入bucket leaf信息。 并最终刷盘
加载时: 会读取第0页内容,也就是元信息。
然后对其进行校验和校验,当校验通过后获取pageSize。 否则的话,读取操作系统默认的pagesize(一般4k)
上述操作完成后,会通过mmap来映射数据。最后再根据磁盘页中的freelist数据初始化db的freelist字段
func (*DB) Info ¶
This is for internal access to the raw data bytes from the C cursor, use carefully, or not at all.
func (*DB) IsReadOnly ¶
func (*DB) Stats ¶
Stats retrieves ongoing performance stats for the database. This is only updated when a transaction closes.
type Options ¶
type Options struct { // Timeout is the amount of time to wait to obtain a file lock. // When set to zero it will wait indefinitely. This option is only // available on Darwin and Linux. Timeout time.Duration // Sets the DB.NoGrowSync flag before memory mapping the file. NoGrowSync bool // Open database in read-only mode. Uses flock(..., LOCK_SH |LOCK_NB) to // grab a shared lock (UNIX). ReadOnly bool // Sets the DB.MmapFlags flag before memory mapping the file. MmapFlags int // InitialMmapSize is the initial mmap size of the database // in bytes. Read transactions won't block write transaction // if the InitialMmapSize is large enough to hold database mmap // size. (See DB.Begin for more information) // // If <=0, the initial map size is 0. // If initialMmapSize is smaller than the previous database size, // it takes no effect. InitialMmapSize int }
type Stats ¶
type Stats struct { // Freelist stats FreePageN int // total number of free pages on the freelist PendingPageN int // total number of pending pages on the freelist FreeAlloc int // total bytes allocated in free pages FreelistInuse int // total bytes used by the freelist // Transaction stats TxN int // total number of started read transactions OpenTxN int // number of currently open read transactions TxStats TxStats // global, ongoing stats. }
Stats represents statistics about the database.
type Tx ¶
type Tx struct { // WriteFlag specifies the flag for write-related methods like WriteTo(). // Tx opens the database file with the specified flag to copy the data. // // By default, the flag is unset, which works well for mostly in-memory // workloads. For databases that are much larger than available RAM, // set the flag to syscall.O_DIRECT to avoid trashing the page cache. WriteFlag int // contains filtered or unexported fields }
Tx 主要封装了读事务和写事务。其中通过writable来区分是读事务还是写事务
func (*Tx) Commit ¶
Commit()方法内部实现中,总体思路是: 1. 先判定节点要不要合并、分裂 2. 对空闲列表的判断,是否存在溢出的情况,溢出的话,需要重新分配空间 3. 将事务中涉及改动的页进行排序(保证尽可能的顺序IO),排序后循环写入到磁盘中,最后再执行刷盘 4. 当数据写入成功后,再将元信息页写到磁盘中,刷盘以保证持久化 5. 上述操作中,但凡有失败,当前事务都会进行回滚
func (*Tx) CreateBucketIfNotExists ¶
func (*Tx) DeleteBucket ¶
func (*Tx) Page ¶
Page returns page information for a given page number. This is only safe for concurrent use when used by a writable transaction.
type TxStats ¶
type TxStats struct { // Page statistics. PageCount int // number of page allocations PageAlloc int // total bytes allocated // Cursor statistics. CursorCount int // number of cursors created // Node statistics NodeCount int // number of node allocations NodeDeref int // number of node dereferences // Rebalance statistics. Rebalance int // number of node rebalances RebalanceTime time.Duration // total time spent rebalancing // Split/Spill statistics. Split int // number of nodes split Spill int // number of nodes spilled SpillTime time.Duration // total time spent spilling // Write statistics. Write int // number of writes performed WriteTime time.Duration // total time spent writing to disk }
TxStats represents statistics about the actions performed by the transaction.