wal

package
v0.0.0-...-37009e5 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 20, 2024 License: Apache-2.0 Imports: 10 Imported by: 0

Documentation

Index

Constants

View Source
const (
	B  = 1
	KB = 1024 * B
	MB = 1024 * KB
	GB = 1024 * MB
)
View Source
const (
	DotSEG = ".SEG"
)

Variables

View Source
var (
	ErrClosed     = errors.New("the segment file is closed")
	ErrInvalidCRC = errors.New("invalid crc, the data may be corrupted")
)
View Source
var DefaultOptions = Options{
	DirPath:        os.TempDir(),
	SegmentSize:    GB,
	SegmentFileExt: DotSEG,
	Sync:           false,
	BytesPerSync:   0,
}

Functions

func SegmentFileName

func SegmentFileName(dirPath, extName string, id SegmentID) string

SegmentFileName returns the file name of a segment file.

Types

type ChunkLoc

type ChunkLoc struct {
	SegId       SegmentID
	BlockIndex  uint32
	ChunkOffset int64
	ChunkSize   uint32
}

ChunkLoc represents the location of a chunk in a segment file. Used to read the data from the segment file.

func DecodeChunkLoc

func DecodeChunkLoc(b []byte) *ChunkLoc

DecodeChunkLoc decodes a ChunkLoc from bytes.

func (*ChunkLoc) Encode

func (loc *ChunkLoc) Encode() []byte

Encode encodes a ChunkLoc into bytes. In reverse, decode it by wal.DecodeChunkLoc().

type ChunkType

type ChunkType = byte
const (
	ChunkTypeFull ChunkType = iota
	ChunkTypeFirst
	ChunkTypeMiddle
	ChunkTypeLast
)

type Options

type Options struct {
	// DirPath specifies the directory path where the WAL segment files will
	// be stored.
	DirPath string

	// SegmentSize specifies the maximum size of each segment file in bytes.
	SegmentSize int64

	// SegmentFileExt specifies the file extension of the segment files.
	// The file extension must start with a dot ".", default value is ".SEG".
	// It is used to identify the different types of files(i.e. segment files
	// and hint files) in the directory.
	SegmentFileExt string

	// Sync is whether to synchronize writes through os buffer cache and down
	// onto the actual disk. Sync is required for durability of a single write
	// operation, but also results in slower writes.
	//
	// If false, and the machine crashes, then some recent writes may be lost.
	// Note that if it's just the process crashes, then no write will be lost.
	//
	// In other words, Sync being false has the same semantics as a normal
	// write system call. Sync being true means write followed by fsync.
	Sync bool

	// BytesPerSync specifies the number of bytes to write before calling fsync.
	BytesPerSync uint32
}

Options represents the configuration options for a Write-Ahead Log (WAL).

type Reader

type Reader struct {
	// contains filtered or unexported fields
}

Reader represents a reader for WAL. The readers are *segmentReader for every segment, sorted by segment id. And the currIdx points to current segment reader.

func (*Reader) CurrChunkLoc

func (r *Reader) CurrChunkLoc() *ChunkLoc

CurrChunkLoc returns the location of current chunk.

func (*Reader) CurrSegId

func (r *Reader) CurrSegId() SegmentID

CurrSegId returns the id of current segment.

func (*Reader) Next

func (r *Reader) Next() ([]byte, *ChunkLoc, error)

Next returns the next chunk data with location. If there's no data, io.EOF will be returned.

func (*Reader) Skip

func (r *Reader) Skip()

Skip skips the current segment.

type SegmentID

type SegmentID = uint32

type WAL

type WAL struct {
	// contains filtered or unexported fields
}

WAL (Write-Ahead Log) provides durability and fault-tolerance for incoming writes.

It consists of an activeSeg, which is the current segment file used for new incoming writes, and olderSegs, which is a map of segment files used for read operations.

func Open

func Open(opt Options) (*WAL, error)

func (*WAL) ActiveSegID

func (w *WAL) ActiveSegID() SegmentID

ActiveSegID returns the current active segment id.

func (*WAL) Close

func (w *WAL) Close() error

Close closes the WAL.

func (*WAL) Delete

func (w *WAL) Delete() error

Delete deletes all segment files of the WAL.

func (*WAL) IsEmpty

func (w *WAL) IsEmpty() bool

IsEmpty returns whether the WAL is empty. Only when there is only one active segment, and it's empty.

func (*WAL) NewReader

func (w *WAL) NewReader() *Reader

NewReader returns a new reader for WAL which read all segments.

func (*WAL) NewReaderLE

func (w *WAL) NewReaderLE(segId SegmentID) *Reader

NewReaderLE returns a new reader for WAL which only read data from the segment whose id is less than or equal to the given segId. If segId is 0, meaning read from all segments. You may also call it in the merge process.

func (*WAL) NewReaderWithLoc

func (w *WAL) NewReaderWithLoc(loc *ChunkLoc) (*Reader, error)

NewReaderWithLoc returns a new reader for WAL which only read data from the given chunk location.

func (*WAL) OpenNewActiveSeg

func (w *WAL) OpenNewActiveSeg() error

OpenNewActiveSeg opens a new segment file and sets it as the active segment file regardless of the old one. Calling it in merge process.

func (*WAL) Read

func (w *WAL) Read(loc *ChunkLoc) ([]byte, error)

Read reads the data in the given chunk location from the WAL.

func (*WAL) Sync

func (w *WAL) Sync() error

Sync syncs the active segment file to stable storage like disk.

func (*WAL) Write

func (w *WAL) Write(data []byte) (*ChunkLoc, error)

Write writes the data to the WAL. Actually, it writes the data to the active segment file. Returns the location of the data in the WAL.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL