README
¶
txfile
Documentation
¶
Index ¶
- Constants
- type ErrKind
- type Error
- type File
- func (f *File) Begin() (*Tx, error)
- func (f *File) BeginReadonly() (*Tx, error)
- func (f *File) BeginWith(settings TxOptions) (*Tx, error)
- func (f *File) Close() error
- func (f *File) Offset(id PageID, offset uintptr) uintptr
- func (f *File) PageSize() int
- func (f *File) Readonly() bool
- func (f *File) SplitOffset(offset uintptr) (PageID, uintptr)
- type FileStats
- type Flag
- type Observer
- type Options
- type Page
- func (p *Page) Bytes() ([]byte, error)
- func (p *Page) Dirty() bool
- func (p *Page) Flush() error
- func (p *Page) Free() error
- func (p *Page) ID() PageID
- func (p *Page) Load() error
- func (p *Page) MarkDirty() error
- func (p *Page) Readonly() bool
- func (p *Page) SetBytes(contents []byte) error
- func (p *Page) Writable() bool
- type PageID
- type SyncMode
- type Tx
- func (tx *Tx) Active() bool
- func (tx *Tx) Alloc() (page *Page, err error)
- func (tx *Tx) AllocN(n int) (pages []*Page, err error)
- func (tx *Tx) CheckpointWAL() error
- func (tx *Tx) Close() error
- func (tx *Tx) Commit() error
- func (tx *Tx) Flush() error
- func (tx *Tx) Page(id PageID) (*Page, error)
- func (tx *Tx) PageSize() int
- func (tx *Tx) Readonly() bool
- func (tx *Tx) Rollback() error
- func (tx *Tx) Root() PageID
- func (tx *Tx) RootPage() (*Page, error)
- func (tx *Tx) SetRoot(id PageID)
- func (tx *Tx) Writable() bool
- type TxOptions
- type TxStats
Constants ¶
const ( PermissionError = vfs.ErrPermission FileExists = vfs.ErrExist FileDoesNotExist = vfs.ErrNotExist FileClosed = vfs.ErrClosed NoDiskSpace = vfs.ErrNoSpace FDLimit = vfs.ErrFDLimit CantResolvePath = vfs.ErrResolvePath IOError = vfs.ErrIO OSOtherError = vfs.ErrOSOther OperationNotSupported = vfs.ErrNotSupported LockFailed = vfs.ErrLockFailed )
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type ErrKind ¶ added in v0.0.2
type ErrKind int
ErrKind defines txfile error kinds(codes). ErrKind is compatible to error, so it can be used with `txerr.Is()`.
const ( NoError ErrKind = iota // no error InternalError // internal error FileCreationFailed // can not create file InitFailed // failed to initialize from file InvalidConfig // configuration error InvalidFileSize // invalid file size InvalidMetaPage // meta page invalid InvalidOp // invalid operation InvalidPageID // page id out of bounds InvalidParam // invalid parameter OutOfMemory // out of memory TxCommitFail // transaction failed during commit TxRollbackFail // transaction failed during rollback TxFailed // transaction failed TxFinished // finished transaction TxReadOnly // readonly transaction )
type Error ¶ added in v0.0.2
type Error struct {
// contains filtered or unexported fields
}
Error is the actual error type returned by all functions/methods within the txfile package. The Error is compatible to error and txerr.Error, but adds a few additional meta-data for applications to report and handle errors. Each single field in Error is optional. Fields can be accessed by methods only. As fields can being optional and Error being used to wrap other errors as well, txerr should be for inspecting errors.
func (*Error) Context ¶ added in v0.0.2
Context returns a formatted string of the related meta-data as key/value pairs.
func (*Error) Error ¶ added in v0.0.2
Error formats the error message. The cause will not be included in the error string. Use fmt with %+v to create a formatted multiline error.
func (*Error) Errors ¶ added in v0.0.2
Errors is similar to `Cause()`, but returns a slice of errors. This way the error value can be consumed and formatted by zap (and propably other loggers).
func (*Error) Format ¶ added in v0.0.2
Format adds support for fmt.Formatter to Error. The format patterns %v and %s print the top-level error message only (similar to `(*Error).Error()`). The format pattern "q" is similar to "%s", but adds double quotes before and after the message. Use %+v to create a multiline string containing the full trace of errors.
func (*Error) Kind ¶ added in v0.0.2
Kind returns the error kind of the error. The kind should be used by applications to check if it is possible to recover from an error condition. Kind return nil if the error value does not define a kind. Better use `txerr.Is` or `txerr.GetKind` to query the error kind.
type File ¶
type File struct {
// contains filtered or unexported fields
}
File provides transactional support to pages of a file. A file is split into pages of type PageSize. Pages within the file are only accessible by page IDs from within active transactions.
func Open ¶
Open opens or creates a new transactional file. Open tries to create the file, if the file does not exist yet. Returns an error if file access fails, file can not be locked or file meta pages are found to be invalid.
func (*File) Begin ¶
Begin creates a new read-write transaction. The transaction returned does hold the Reserved Lock on the file. Use Close, Rollback, or Commit to release the lock.
func (*File) BeginReadonly ¶
BeginReadonly creates a new readonly transaction. The transaction returned does hold the Shared Lock on the file. Use Close() to release the lock.
func (*File) BeginWith ¶
BeginWith creates a new readonly or read-write transaction, with additional transaction settings.
func (*File) Close ¶
Close closes the file, after all transactions have been quit. After closing a file, no more transactions can be started.
type FileStats ¶ added in v0.0.4
type FileStats struct {
Version uint32 // lates file-header version
Size uint64 // actual file size (changes if file did grow dynamically due to allocations)
MaxSize uint64 // max file size as stores in file header
PageSize uint32 // file page size
MetaArea uint // total pages reserved for the meta area
DataAllocated uint // data pages in use
MetaAllocated uint // meta pages in use
}
FileStats reports the current file state like version and allocated/free space.
type Flag ¶ added in v0.0.2
type Flag uint64
Flag configures file opening behavior.
const ( // FlagWaitLock instructs the open function to block until the file lock could // be acquired. By default open will return with an error if the lock file // can not be generated, opened or locked. FlagWaitLock Flag = 1 << iota // FlagUnboundMaxSize configures the file max size to be unbound. This sets // MaxSize to 0. If MaxSize and Prealloc is set, up to MaxSize bytes are // preallocated on disk (truncate). FlagUnboundMaxSize // FlagUpdMaxSize updates the file max size setting. If not set, the max size // setting is read from the file to be opened. // The file will grow if MaxSize is larger then the current max size setting. // If MaxSize is less then the file's max size value, the file is tried to // shrink dynamically whenever pages are freed. Freed pages are returned via // `Truncate`. FlagUpdMaxSize )
type Observer ¶ added in v0.0.4
type Observer interface {
// OnOpen reports initial file stats when successfully open a file.
//
// Memory stats are reported in sizes. Page counts can be derived by dividing
// the sizes by pageSz.
//
// Derived metrics:
// dataAreaSz = maxSz - metaAreaSz // total data area size
// dataAreaActive = dataAreaSz - avail // data area bytes currently in use
OnOpen(stats FileStats)
// OnBegin reports the start of a new transaction.
OnTxBegin(readonly bool)
// OnClose is used to signal the end of a transaction.
//
// If readonly is set, the transaction we a readonly transaction. Only the
// Duration, Total, and Accessed fields will be set.
// Only if `commit` is set will the reported stats be affective in upcoming
// file operations (pages written/freed).
OnTxClose(file FileStats, tx TxStats)
}
Observer defines common callbacks to observe errors, transactions and other state changes in txfile. The callbacks must not block, so to not block any file operations.
type Options ¶
type Options struct {
// Additional flags.
Flags Flag
// Configure file sync behavior
Sync SyncMode
// MaxSize sets the maximum file size in bytes. This should be a multiple of PageSize.
// If it's not a multiple of PageSize, the actual files maximum size is rounded downwards
// to the next multiple of PageSize.
// A value of 0 indicates the file can grow without limits.
MaxSize uint64
// PageSize sets the files page size on file creation. PageSize is ignored if
// the file already exists.
// If PageSize is not configured, the OSes main memory page size is selected.
PageSize uint32
// InitMetaArea configures the minimum amount of page in the meta area.
// The amount of pages is only allocated when the file is generated.
// The meta area grows by double the current meta area. To reduce the
// total amount of pages moved to meta area on grow, it is recommended that
// the value of InitMetaArea is a power of 2.
InitMetaArea uint32
// Prealloc disk space if MaxSize is set.
Prealloc bool
// Open file in readonly mode.
Readonly bool
Observer Observer
}
Options provides common file options used when opening or creating a file.
type Page ¶
type Page struct {
// contains filtered or unexported fields
}
Page provides access to an on disk page. Pages can only be overwritten from within a read-write Transaction. Writes are be buffered until transaction commit, such that other but the current transaction will not be able to see file changes.
func (*Page) Bytes ¶
Bytes returns the page its contents. One can only modify the buffer in write transaction, if Load() or SetBytes() have been called before Bytes(). Otherwise a non-recoverable BUS panic might be triggerd (program will be killed by OS). Bytes returns an error if the page has just been allocated (no backing buffer) or the transaction is already been closed. Use SetBytes() or Load(), to initialize the buffer of a newly allocated page.
func (*Page) Dirty ¶
Dirty reports if the page is marked as dirty and needs to be flushed on commit.
func (*Page) Flush ¶
Flush flushes the page write buffer, if the page is marked as dirty. The page its contents must not be changed after calling Flush, as the flush is executed asynchronously in the background. Dirty pages will be automatically flushed on commit.
func (*Page) Free ¶
Free marks a page as free. Freeing a dirty page will return an error. The page will be returned to the allocator when the transaction commits.
func (*Page) ID ¶
ID returns the pages PageID. The ID can be used to store a reference to this page, for use within another transaction.
func (*Page) Load ¶
Load reads the pages original contents into a cached memory buffer, allowing for in-place modifications to the page. Load returns and error, if used from within a readonly transaction. If the page has been allocated from within the current transaction, a new temporary buffer will be allocated. After load, the write-buffer can be accessed via Bytes(). After modifications to the buffer, one must use MarkDirty(), so the page will be flushed on commit.
func (*Page) MarkDirty ¶
MarkDirty marks a page as dirty. MarkDirty should only be used if in-place modification to the pages buffer have been made, after use of Load().
func (*Page) SetBytes ¶
SetBytes sets the new contents of a page. If the size of contents is less then the files page size, the original contents must be read. If the length of contents matches the page size, a reference to the contents buffer will be held. To enforce a copy, use Load(), Bytes(), copy() and MarkDirty().
type SyncMode ¶ added in v0.0.4
type SyncMode uint8
SyncMode selects the file syncing behavior
const ( // SyncDefault lets the implementation choose the default syncing mode SyncDefault SyncMode = iota // SyncData prefers fdatasync if available. Still uses fsync (or similar) if // implementation wants to enforce fsync. SyncData // SyncFull enforces fsync/or similar. SyncFull // SyncNone disable syncing. Do not use this in production environments, as // this can easily cause file corruption. SyncNone )
type Tx ¶
type Tx struct {
// contains filtered or unexported fields
}
Tx provides access to pages in a File. A transaction MUST always be closed, so to guarantee locks being released as well.
func (*Tx) Active ¶
Active returns true if the transaction can still be used to access pages. A transaction becomes inactive after Close, Commit or Rollback. Errors within a transaction might inactivate the transaction as well. When encountering errors, one should check if the transaction still can be used.
func (*Tx) Alloc ¶
Alloc allocates a new writable page with yet empty contents. Use Load(), Bytes and MarkDirty(), or SetBytes() to fill the page with new contents. Returns an error if the transaction is readonly or no more space is available.
func (*Tx) AllocN ¶
AllocN allocates n potentially non-contious, yet empty pages. Returns an error if the transaction is readonly or no more space is available.
func (*Tx) CheckpointWAL ¶
CheckpointWAL copies all overwrite pages contents into the original pages. Only already committed pages from older transactions will be overwritten. Checkpointing only copies the contents and marks the overwrite pages as freed. The final transaction Commit is required, to propage the WAL mapping changes to all other transactions. Dirty pages are not overwritten. Manual checkpointing should be executed at the end of a transaction, right before committing, so to reduce writes if contents is to be overwritten anyways.
func (*Tx) Close ¶
Close closes the transaction, releasing any locks held by the transaction. It is safe to call Close multiple times. Close on an inactive transaction will be ignored. A non-committed read-write transaction will be rolled back on close. To guaranteed the File and Locking state being valid, even on panic or early return on error, one should also defer the Close operation on new transactions. For example:
tx := f.Begin()
defer tx.Close()
err := some operation
if err != nil {
return err
}
return tx.Commit()
func (*Tx) Commit ¶
Commit commits the current transaction to file. The commit step needs to take the Exclusive Lock, waiting for readonly transactions to be Closed. Returns an error if the transaction has already been closed by Close, Rollback or Commit.
func (*Tx) Page ¶
Page accesses a page by ID. Accessed pages are cached. Retrieving a page that has already been accessed, will return a pointer to the same Page object. Returns an error if the id is known to be invalid or the page has already been freed.
func (*Tx) Readonly ¶
Readonly returns true if no modifications to the page are allowed. Trying to write to a readonly page might result in a non-recoverable panic.
func (*Tx) Rollback ¶
Rollback rolls back and closes the current transaction. Rollback returns an error if the transaction has already been closed by Close, Rollback or Commit.
func (*Tx) Root ¶
Root returns the data root page id. This ID must be set via SetRoot to indicate the start of application data to later transactions. On new files, the default root is 0, as no application data are stored yet.
func (*Tx) RootPage ¶
RootPage returns the application data root page, if the root id has been set in the past. Returns nil, if no root page is set.
type TxOptions ¶
type TxOptions struct {
// Readonly transaction.
Readonly bool
// Allow write transaction to allocate meta pages from overflow area.
// Potentially increasing the file size past the configured max size.
// This setting should only be used to guarantee progress when having a
// transaction only freeing pages.
// Later transactions will try to release pages from the overflow area and
// truncate the file, such that we have a chance to operate within max-size
// limits again.
EnableOverflowArea bool
// MetaAreaGrowPercentage sets the percentage of meta pages in use, until
// the meta-area grows again. The value must be between 0 and 100.
// The default value is 80%.
MetaAreaGrowPercentage int
// Number of pages in wal overwrite log to automatically trigger
// CheckpointWAL on commit.
WALLimit uint
}
TxOptions adds some per transaction options user can set.
type TxStats ¶ added in v0.0.4
type TxStats struct {
Readonly bool // set if transaction is readonly. In this case only Duration, Total and Accessed will be set.
Commit bool // If set reported stats will be affective in future file operations. Otherwise allocation stats will have no effect.
Duration time.Duration // total duration the transaction was live
Total uint // total number of pages accessed(written, read, changed) during the transaction
Accessed uint // number of accessed existing pages (read)
Allocated uint // temporarily allocated pages
Freed uint // total number of freed pages
Written uint // total number of pages being written to
Updated uint // number of pages with changed contents
}
TxStats contains common statistics collected during the life-cycle of a transaction.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
dev-tools
|
|
|
internal
|
|
|
cleanup
Package cleanup provides common helpers for common cleanup patterns on defer Use the helpers with `defer`.
|
Package cleanup provides common helpers for common cleanup patterns on defer Use the helpers with `defer`. |
|
invariant
Package invariant provides helpers for checking and panicing on faulty invariants.
|
Package invariant provides helpers for checking and panicing on faulty invariants. |
|
iter
Package iter provides functions for common array iteration strategies.
|
Package iter provides functions for common array iteration strategies. |
|
strbld
strbld package provides a string Builder that can be used with older go versions as well.
|
strbld package provides a string Builder that can be used with older go versions as well. |
|
Package txerr provides common functions for querying and formatting errors generated by txfile.
|
Package txerr provides common functions for querying and formatting errors generated by txfile. |
|
Package txfiletest provides utilities for testing on top of txfile.
|
Package txfiletest provides utilities for testing on top of txfile. |