Documentation
¶
Index ¶
- Constants
- func NewSegmentBufReader(segs ...*Segment) *segmentBufReader
- func NewSegmentsRangeReader(sr ...SegmentRange) (io.ReadCloser, error)
- func NewSegmentsReader(dir string) (io.ReadCloser, error)
- func SegmentName(dir string, i int) string
- func Segments(walDir string) (first, last int, err error)
- type CorruptionErr
- type LogLocation
- type Reader
- type Segment
- type SegmentRange
- type WAL
- func (w *WAL) Close() (err error)
- func (w *WAL) CompressionEnabled() bool
- func (w *WAL) Dir() string
- func (w *WAL) Log(recs ...[]byte) ([]LogLocation, error)
- func (w *WAL) NextSegment() error
- func (w *WAL) Repair(origErr error) error
- func (w *WAL) Size() (int64, error)
- func (w *WAL) Truncate(i int) (err error)
Constants ¶
const (
DefaultSegmentSize = 128 * 1024 * 1024 // 128 MB
)
Variables ¶
This section is empty.
Functions ¶
func NewSegmentBufReader ¶
func NewSegmentBufReader(segs ...*Segment) *segmentBufReader
nolint:golint // TODO: Consider exporting segmentBufReader
func NewSegmentsRangeReader ¶
func NewSegmentsRangeReader(sr ...SegmentRange) (io.ReadCloser, error)
NewSegmentsRangeReader returns a new reader over the given WAL segment ranges. If first or last are -1, the range is open on the respective end.
func NewSegmentsReader ¶
func NewSegmentsReader(dir string) (io.ReadCloser, error)
NewSegmentsReader returns a new reader over all segments in the directory.
func SegmentName ¶
SegmentName builds a segment name for the directory.
Types ¶
type CorruptionErr ¶
CorruptionErr is an error that's returned when corruption is encountered.
func (*CorruptionErr) Error ¶
func (e *CorruptionErr) Error() string
type LogLocation ¶
LogLocation indicates where the log entry is placed inside WAL - by segment number and it's offset in file, in bytes.
type Reader ¶
type Reader struct {
// contains filtered or unexported fields
}
Reader reads WAL records from an io.Reader.
func (*Reader) Err ¶
Err returns the last encountered error wrapped in a corruption error. If the reader does not allow to infer a segment index and offset, a total offset in the reader stream will be provided.
func (*Reader) Next ¶
Next advances the reader to the next records and returns true if it exists. It must not be called again after it returned false.
type Segment ¶
Segment represents a segment file.
func CreateSegment ¶
CreateSegment creates a new segment k in dir.
func OpenReadSegment ¶
OpenReadSegment opens the segment with the given filename.
func OpenWriteSegment ¶
OpenWriteSegment opens segment k in dir. The returned segment is ready for new appends.
type SegmentRange ¶
SegmentRange groups segments by the directory and the first and last index it includes.
type WAL ¶
type WAL struct {
// contains filtered or unexported fields
}
WAL is a write ahead log that stores records in segment files. It must be read from start to end once before logging new data. If an error occurs during read, the repair procedure must be called before it's safe to do further writes.
Segments are written to in pages of 32KB, with records possibly split across page boundaries. Records are never split across segments to allow full segments to be safely truncated. It also ensures that torn writes never corrupt records beyond the most recent segment.
func New ¶
func New(logger zerolog.Logger, reg prometheus.Registerer, dir string, compress bool) (*WAL, error)
New returns a new WAL over the given directory.
func NewSize ¶
func NewSize(logger zerolog.Logger, reg prometheus.Registerer, dir string, segmentSize int, compress bool) (*WAL, error)
NewSize returns a new WAL over the given directory. New segments are created with the specified size.
func (*WAL) CompressionEnabled ¶
CompressionEnabled returns if compression is enabled on this WAL.
func (*WAL) Log ¶
func (w *WAL) Log(recs ...[]byte) ([]LogLocation, error)
Log writes the records into the log. Multiple records can be passed at once to reduce writes and increase throughput.
func (*WAL) NextSegment ¶
NextSegment creates the next segment and closes the previous one.
func (*WAL) Repair ¶
Repair attempts to repair the WAL based on the error. It discards all data after the corruption.
Source Files