Documentation ¶
Overview ¶
Package ltx reads and writes Liteserver Transaction (LTX) files.
Index ¶
- Constants
- Variables
- func FormatFilename(minTXID, maxTXID TXID) string
- func FormatTimestamp(t time.Time) string
- func IsValidHeaderFlags(flags uint32) bool
- func IsValidPageSize(sz uint32) bool
- func LockPgno(pageSize uint32) uint32
- func NewHasher() hash.Hash64
- func ParseTimestamp(value string) (time.Time, error)
- type Checksum
- type Compactor
- type Decoder
- func (dec *Decoder) Close() error
- func (dec *Decoder) DecodeDatabaseTo(w io.Writer) error
- func (dec *Decoder) DecodeHeader() error
- func (dec *Decoder) DecodePage(hdr *PageHeader, data []byte) error
- func (dec *Decoder) Header() Header
- func (dec *Decoder) N() int64
- func (dec *Decoder) PageN() int
- func (dec *Decoder) PostApplyPos() Pos
- func (dec *Decoder) Trailer() Trailer
- func (dec *Decoder) Verify() error
- type Encoder
- func (enc *Encoder) Close() error
- func (enc *Encoder) EncodeHeader(hdr Header) error
- func (enc *Encoder) EncodePage(hdr PageHeader, data []byte) (err error)
- func (enc *Encoder) Header() Header
- func (enc *Encoder) N() int64
- func (enc *Encoder) PostApplyPos() Pos
- func (enc *Encoder) SetPostApplyChecksum(chksum Checksum)
- func (enc *Encoder) Trailer() Trailer
- type FileSpec
- type Header
- type PageHeader
- type PageSpec
- type Pos
- type PosMismatchError
- type TXID
- type Trailer
Constants ¶
const ( // Magic is the first 4 bytes of every LTX file. Magic = "LTX1" // Version is the current version of the LTX file format. Version = 1 )
const ( HeaderSize = 100 PageHeaderSize = 4 TrailerSize = 16 )
Size constants.
const ( ChecksumSize = 8 TrailerChecksumOffset = TrailerSize - ChecksumSize )
Checksum size & positions.
const ( HeaderFlagMask = uint32(0x00000001) HeaderFlagCompressLZ4 = uint32(0x00000001) )
Header flags.
const MaxPageSize = 65536
MaxPageSize is the maximum allowed page size for SQLite.
const PENDING_BYTE = 0x40000000
const RFC3339Milli = "2006-01-02T15:04:05.000Z07:00"
RFC3339Milli is the standard time format for LTX timestamps. It uses fixed-width millisecond resolution which makes it sortable.
Variables ¶
var ( ErrInvalidFile = errors.New("invalid LTX file") ErrDecoderClosed = errors.New("ltx decoder closed") ErrEncoderClosed = errors.New("ltx encoder closed") ErrNoChecksum = errors.New("no file checksum") ErrInvalidChecksumFormat = errors.New("invalid file checksum format") ErrChecksumMismatch = errors.New("file checksum mismatch") )
Errors
Functions ¶
func FormatFilename ¶
FormatFilename returns an LTX filename representing a range of transactions.
func FormatTimestamp ¶ added in v0.3.12
FormatTimestamp returns t with a fixed-width, millisecond-resolution UTC format.
func IsValidHeaderFlags ¶
IsValidHeaderFlags returns true unless flags outside the valid mask are set.
func IsValidPageSize ¶
IsValidPageSize returns true if sz is between 512 and 64K and a power of two.
Types ¶
type Checksum ¶ added in v0.3.13
type Checksum uint64
Checksum represents an LTX checksum.
const ChecksumFlag Checksum = 1 << 63
ChecksumFlag is a flag on the checksum to ensure it is non-zero.
func ChecksumPage ¶
ChecksumPage returns a CRC64 checksum that combines the page number & page data.
func ChecksumPageWithHasher ¶ added in v0.2.10
ChecksumPageWithHasher returns a CRC64 checksum that combines the page number & page data.
func ChecksumReader ¶ added in v0.2.1
ChecksumReader reads an entire database file from r and computes its rolling checksum.
func ParseChecksum ¶ added in v0.3.13
ParseChecksum parses a 16-character hex string into a checksum.
func (Checksum) MarshalJSON ¶ added in v0.3.13
func (*Checksum) UnmarshalJSON ¶ added in v0.3.13
type Compactor ¶
type Compactor struct { // These flags will be set when encoding the header. HeaderFlags uint32 // If true, the compactor will not validate that input files have contiguous // transaction IDs. This is false by default but can be enabled when // rebuilding snapshots with missing transactions. AllowNonContiguousTXIDs bool // contains filtered or unexported fields }
Compactor represents a compactor of LTX files.
func NewCompactor ¶
NewCompactor returns a new instance of Compactor with default settings.
type Decoder ¶ added in v0.2.2
type Decoder struct {
// contains filtered or unexported fields
}
Decoder represents a decoder of an LTX file.
func NewDecoder ¶ added in v0.2.2
NewDecoder returns a new instance of Decoder.
func (*Decoder) Close ¶ added in v0.2.2
Close verifies the reader is at the end of the file and that the checksum matches.
func (*Decoder) DecodeDatabaseTo ¶ added in v0.3.4
DecodeDatabaseTo decodes the LTX file as a SQLite database to w. The LTX file MUST be a snapshot file.
func (*Decoder) DecodeHeader ¶ added in v0.2.2
DecodeHeader reads the LTX file header frame and stores it internally. Call Header() to retrieve the header after this is successfully called.
func (*Decoder) DecodePage ¶ added in v0.2.2
func (dec *Decoder) DecodePage(hdr *PageHeader, data []byte) error
DecodePage reads the next page header into hdr and associated page data.
func (*Decoder) PostApplyPos ¶ added in v0.3.6
PostApplyPos returns the replication position after underlying the LTX file is applied. Only valid after successful Close().
type Encoder ¶ added in v0.2.2
type Encoder struct {
// contains filtered or unexported fields
}
Encoder implements a encoder for an LTX file.
func NewEncoder ¶ added in v0.2.2
NewEncoder returns a new instance of Encoder.
func (*Encoder) EncodeHeader ¶ added in v0.2.2
EncodeHeader writes hdr to the file's header block.
func (*Encoder) EncodePage ¶ added in v0.2.2
func (enc *Encoder) EncodePage(hdr PageHeader, data []byte) (err error)
EncodePage writes hdr & data to the file's page block.
func (*Encoder) PostApplyPos ¶ added in v0.3.6
PostApplyPos returns the replication position after underlying the LTX file is applied. Only valid after successful Close().
func (*Encoder) SetPostApplyChecksum ¶ added in v0.2.2
SetPostApplyChecksum sets the post-apply checksum of the database. Must call before Close().
type FileSpec ¶
FileSpec is an in-memory representation of an LTX file. Typically used for testing.
type Header ¶
type Header struct { Version int // based on magic Flags uint32 // reserved flags PageSize uint32 // page size, in bytes Commit uint32 // db size after transaction, in pages MinTXID TXID // minimum transaction ID MaxTXID TXID // maximum transaction ID Timestamp int64 // milliseconds since unix epoch PreApplyChecksum Checksum // rolling checksum of database before applying this LTX file WALOffset int64 // file offset from original WAL; zero if journal WALSize int64 // size of original WAL segment; zero if journal WALSalt1 uint32 // header salt-1 from original WAL; zero if journal or compaction WALSalt2 uint32 // header salt-2 from original WAL; zero if journal or compaction NodeID uint64 // node id where the LTX file was created, zero if unset }
Header represents the header frame of an LTX file.
func DecodeHeader ¶ added in v0.3.0
DecodeHeader decodes the header from r. Returns the header & read bytes.
func PeekHeader ¶ added in v0.3.7
PeekHeader reads & unmarshals the header from r. It returns a new io.Reader that prepends the header data back on.
func (*Header) IsSnapshot ¶
IsSnapshot returns true if header represents a complete database snapshot. This is true if the header includes the initial transaction. Snapshots must include all pages in the database.
func (*Header) LockPgno ¶ added in v0.3.11
LockPgno returns the lock page number based on the header's page size.
func (*Header) MarshalBinary ¶
MarshalBinary encodes h to a byte slice.
func (Header) PreApplyPos ¶ added in v0.3.6
PreApplyPos returns the replication position before the LTX file is applies.
func (*Header) UnmarshalBinary ¶
UnmarshalBinary decodes h from a byte slice.
type PageHeader ¶
type PageHeader struct {
Pgno uint32
}
PageHeader represents the header for a single page in an LTX file.
func (*PageHeader) IsZero ¶ added in v0.2.0
func (h *PageHeader) IsZero() bool
IsZero returns true if the header is empty.
func (*PageHeader) MarshalBinary ¶
func (h *PageHeader) MarshalBinary() ([]byte, error)
MarshalBinary encodes h to a byte slice.
func (*PageHeader) UnmarshalBinary ¶
func (h *PageHeader) UnmarshalBinary(b []byte) error
UnmarshalBinary decodes h from a byte slice.
func (*PageHeader) Validate ¶
func (h *PageHeader) Validate() error
Validate returns an error if h is invalid.
type PageSpec ¶ added in v0.2.0
type PageSpec struct { Header PageHeader Data []byte }
PageSpec is an in-memory representation of an LTX page frame. Typically used for testing.
type Pos ¶ added in v0.3.1
Pos represents the transactional position of a database.
type PosMismatchError ¶ added in v0.3.1
type PosMismatchError struct {
Pos Pos `json:"pos"`
}
PosMismatchError is returned when an LTX file is not contiguous with the current position.
func NewPosMismatchError ¶ added in v0.3.1
func NewPosMismatchError(pos Pos) *PosMismatchError
NewPosMismatchError returns a new instance of PosMismatchError.
func (*PosMismatchError) Error ¶ added in v0.3.1
func (e *PosMismatchError) Error() string
Error returns the string representation of the error.
type TXID ¶ added in v0.3.2
type TXID uint64
TXID represents a transaction ID.
func ParseFilename ¶
ParseFilename parses a transaction range from an LTX file.
func (TXID) MarshalJSON ¶ added in v0.3.2
func (*TXID) UnmarshalJSON ¶ added in v0.3.2
type Trailer ¶ added in v0.2.0
type Trailer struct { PostApplyChecksum Checksum // rolling checksum of database after this LTX file is applied FileChecksum Checksum // crc64 checksum of entire file }
Trailer represents the ending frame of an LTX file.
func (*Trailer) MarshalBinary ¶ added in v0.2.0
MarshalBinary encodes h to a byte slice.
func (*Trailer) UnmarshalBinary ¶ added in v0.2.0
UnmarshalBinary decodes h from a byte slice.