Documentation
¶
Overview ¶
Package file deals with low level file storage and transaction management.
StorageFile ¶
StorageFile models a logical storage file which stores fixed size records on disk. Each record has a unique record id. On disk this logical storage file might be split into several smaller files. StorageFiles can be reused after they were closed if the transaction management has been disabled. This is not the case otherwise.
Record ¶
A record is a byte slice of a StorageFile. It is a wrapper data structure for a byte array which provides read and write methods for several data types.
TransactionManager ¶
TransactionManager provides an optional transaction management for StorageFile.
When used each record which is released from use is added to an in memory transaction log. Once the client calls Flush() on the StorageFile the in memory transaction is written to a transaction log on disk. The in-memory log is kept. The in-memory transaction log is written to the actual StorageFile once maxTrans is reached or the StorageFile is closed.
Should the process crash during a transaction, then the transaction log is written to the StorageFile on the next startup using the recover() function.
Index ¶
- Constants
- Variables
- type LogFile
- type Record
- func (r *Record) ClearData()
- func (r *Record) ClearDirty()
- func (r *Record) Data() []byte
- func (r *Record) DecTransCount()
- func (r *Record) Dirty() bool
- func (r *Record) ID() uint64
- func (r *Record) InTransaction() bool
- func (r *Record) IncTransCount()
- func (r *Record) MarshalBinary() (data []byte, err error)
- func (r *Record) PageView() interface{}
- func (r *Record) ReadInt16(pos int) int16
- func (r *Record) ReadInt32(pos int) int32
- func (r *Record) ReadRecord(ior io.Reader) error
- func (r *Record) ReadSingleByte(pos int) byte
- func (r *Record) ReadUInt16(pos int) uint16
- func (r *Record) ReadUInt32(pos int) uint32
- func (r *Record) ReadUInt64(pos int) uint64
- func (r *Record) SetDirty()
- func (r *Record) SetID(id uint64) error
- func (r *Record) SetPageView(view interface{})
- func (r *Record) String() string
- func (r *Record) UnmarshalBinary(data []byte) error
- func (r *Record) WriteInt16(pos int, value int16)
- func (r *Record) WriteInt32(pos int, value int32)
- func (r *Record) WriteRecord(iow io.Writer) error
- func (r *Record) WriteSingleByte(pos int, value byte)
- func (r *Record) WriteUInt16(pos int, value uint16)
- func (r *Record) WriteUInt32(pos int, value uint32)
- func (r *Record) WriteUInt64(pos int, value uint64)
- type StorageFile
- func (s *StorageFile) Close() error
- func (s *StorageFile) Discard(record *Record)
- func (s *StorageFile) Flush() error
- func (s *StorageFile) Get(id uint64) (*Record, error)
- func (s *StorageFile) Name() string
- func (s *StorageFile) RecordSize() uint32
- func (s *StorageFile) ReleaseInUse(record *Record)
- func (s *StorageFile) ReleaseInUseID(id uint64, dirty bool) error
- func (s *StorageFile) Rollback() error
- func (s *StorageFile) String() string
- func (s *StorageFile) Sync()
- type StorageFileError
- type TransactionManager
Constants ¶
const ( SizeByte = 1 SizeUnsignedShort = 2 SizeShort = 2 SizeThreeByteInt = 3 SizeUnsignedInt = 4 SizeInt = 4 SizeSixByteLong = 6 SizeLong = 8 )
Size constants for a record
const DefaultFileSize = 0x2540BE401 // 10000000001 Bytes
DefaultFileSize is the default size of a physical file (10GB)
const DefaultRecordSize = 4096
DefaultRecordSize is the default size of a record in bytes
const DefaultTransInLog = 10
DefaultTransInLog is the default number of transactions which should be kept in memory (affects how often we sync the log from memory)
const DefaultTransSize = 10
DefaultTransSize is the default number of records in a single transaction (affects how many record pointers are allocated at first per transaction)
const LogFileSuffix = "tlg"
LogFileSuffix is the file suffix for transaction log files
Variables ¶
var ( ErrAlreadyInUse = errors.New("Record is already in-use") ErrNotInUse = errors.New("Record was not in-use") ErrInUse = errors.New("Records are still in-use") ErrTransDisabled = errors.New("Transactions are disabled") ErrInTrans = errors.New("Records are still in a transaction") ErrNilData = errors.New("Record has nil data") )
Common storage file related errors.
var (
ErrBadMagic = fmt.Errorf("Bad magic for transaction log")
)
Common TransactionManager related errors
var TransactionLogHeader = []byte{0x66, 0x42}
TransactionLogHeader is the magic number to identify transaction log files
Functions ¶
This section is empty.
Types ¶
type Record ¶
type Record struct {
// contains filtered or unexported fields
}
Record data structure
func ReadRecord ¶
ReadRecord decodes a record by reading from an io.Reader.
func (*Record) ClearData ¶
func (r *Record) ClearData()
ClearData removes all stored data from a Record.
func (*Record) ClearDirty ¶
func (r *Record) ClearDirty()
ClearDirty clears the dirty flag of a Record.
func (*Record) DecTransCount ¶
func (r *Record) DecTransCount()
DecTransCount decrements the transaction count which means the record has been written to disk.
func (*Record) InTransaction ¶
InTransaction returns if the Record is used in a transaction.
func (*Record) IncTransCount ¶
func (r *Record) IncTransCount()
IncTransCount increments the transaction count which means the record is in the log but not yet in the data file.
func (*Record) MarshalBinary ¶
MarshalBinary returns a binary representation of a Record.
func (*Record) PageView ¶
func (r *Record) PageView() interface{}
PageView returns the view on this record. The view determines how the record is being used.
func (*Record) ReadRecord ¶
ReadRecord decodes a record by reading from an io.Reader.
func (*Record) ReadSingleByte ¶
ReadSingleByte reads a byte from a Record.
func (*Record) ReadUInt16 ¶
ReadUInt16 reads a 16-bit unsigned integer from a Record.
func (*Record) ReadUInt32 ¶
ReadUInt32 reads a 32-bit unsigned integer from a Record.
func (*Record) ReadUInt64 ¶
ReadUInt64 reads a 64-bit unsigned integer from a Record.
func (*Record) SetPageView ¶
func (r *Record) SetPageView(view interface{})
SetPageView sets the view on this record.
func (*Record) UnmarshalBinary ¶
UnmarshalBinary decodes a record from a binary blob.
func (*Record) WriteInt16 ¶
WriteInt16 writes a 16-bit signed integer to a Record.
func (*Record) WriteInt32 ¶
WriteInt32 writes a 32-bit signed integer to a Record.
func (*Record) WriteRecord ¶
WriteRecord writes a record to an io.Writer.
func (*Record) WriteSingleByte ¶
WriteSingleByte writes a byte to a Record.
func (*Record) WriteUInt16 ¶
WriteUInt16 writes a 16-bit unsigned integer to a Record.
func (*Record) WriteUInt32 ¶
WriteUInt32 writes a 32-bit unsigned integer to a Record.
func (*Record) WriteUInt64 ¶
WriteUInt64 writes a 64-bit unsigned integer to a Record.
type StorageFile ¶
type StorageFile struct {
// contains filtered or unexported fields
}
StorageFile data structure
func NewDefaultStorageFile ¶
func NewDefaultStorageFile(name string, transDisabled bool) (*StorageFile, error)
NewDefaultStorageFile creates a new storage file with default record size and returns a reference to it.
func NewStorageFile ¶
func NewStorageFile(name string, recordSize uint32, transDisabled bool) (*StorageFile, error)
NewStorageFile creates a new storage file and returns a reference to it.
func (*StorageFile) Close ¶
func (s *StorageFile) Close() error
Close commits all data and closes all physical files.
func (*StorageFile) Flush ¶
func (s *StorageFile) Flush() error
Flush commits the current transaction by flushing all dirty records to the transaction log on disk. If transactions are disabled it simply writes all dirty records to disk.
func (*StorageFile) Get ¶
func (s *StorageFile) Get(id uint64) (*Record, error)
Get returns a record from the file. Other components can write to this record. Any write operation should set the dirty flag on the record. Dirty records will be written back to disk when the file is flushed after which the dirty flag is cleared. Get panics if a record is requested which is still in-use.
func (*StorageFile) Name ¶
func (s *StorageFile) Name() string
Name returns the name of this storage file.
func (*StorageFile) RecordSize ¶
func (s *StorageFile) RecordSize() uint32
RecordSize returns the size of records which can be storerd or retrieved.
func (*StorageFile) ReleaseInUse ¶
func (s *StorageFile) ReleaseInUse(record *Record)
ReleaseInUse releases a record from the in-use map. ReleaseInUse panics if the record was not in use.
func (*StorageFile) ReleaseInUseID ¶
func (s *StorageFile) ReleaseInUseID(id uint64, dirty bool) error
ReleaseInUseID releases a record given by its id from the in-use map. The client code may indicate if the record is not dirty.
func (*StorageFile) Rollback ¶
func (s *StorageFile) Rollback() error
Rollback cancels the current transaction by discarding all dirty records.
func (*StorageFile) String ¶
func (s *StorageFile) String() string
String returns a string representation of a StorageFile.
type StorageFileError ¶ added in v1.2.0
StorageFileError is a storage file related error.
func NewStorageFileError ¶ added in v1.2.0
func NewStorageFileError(sfeType error, sfeDetail string, sfeFilename string) *StorageFileError
NewStorageFileError returns a new StorageFile specific error.
func (*StorageFileError) Error ¶ added in v1.2.0
func (e *StorageFileError) Error() string
Error returns a string representation of the error.
type TransactionManager ¶
type TransactionManager struct {
// contains filtered or unexported fields
}
TransactionManager data structure
func NewTransactionManager ¶
func NewTransactionManager(owner *StorageFile, doRecover bool) (*TransactionManager, error)
NewTransactionManager creates a new transaction manager and returns a reference to it.
func (*TransactionManager) String ¶
func (t *TransactionManager) String() string
String returns a string representation of a TransactionManager.
Source Files