Documentation
¶
Index ¶
- Constants
- func NewLiveReaderMetrics(reg prometheus.Registerer) *liveReaderMetrics
- func NewSegmentBufReader(segs ...*Segment) *segmentBufReader
- func NewSegmentsRangeReader(sr ...SegmentRange) (io.ReadCloser, error)
- func NewSegmentsReader(dir string) (io.ReadCloser, error)
- func SegmentName(dir string, i int) string
- type CorruptionErr
- type LiveReader
- type Reader
- type Segment
- type SegmentRange
- type WAL
- func (w *WAL) Close() (err error)
- func (w *WAL) CompressionEnabled() bool
- func (w *WAL) Dir() string
- func (w *WAL) Log(recs ...[]byte) error
- func (w *WAL) NextSegment() error
- func (w *WAL) Repair(origErr error) error
- func (w *WAL) Segments() (first, last int, err error)
- func (w *WAL) Truncate(i int) (err error)
Constants ¶
const (
DefaultSegmentSize = 128 * 1024 * 1024 // 128 MB
)
Variables ¶
This section is empty.
Functions ¶
func NewLiveReaderMetrics ¶ added in v0.9.1
func NewLiveReaderMetrics(reg prometheus.Registerer) *liveReaderMetrics
LiveReaderMetrics instatiates, registers and returns metrics to be injected at LiveReader instantiation.
func NewSegmentBufReader ¶ added in v0.9.0
func NewSegmentBufReader(segs ...*Segment) *segmentBufReader
func NewSegmentsRangeReader ¶
func NewSegmentsRangeReader(sr ...SegmentRange) (io.ReadCloser, error)
NewSegmentsRangeReader returns a new reader over the given WAL segment ranges. If first or last are -1, the range is open on the respective end.
func NewSegmentsReader ¶
func NewSegmentsReader(dir string) (io.ReadCloser, error)
NewSegmentsReader returns a new reader over all segments in the directory.
func SegmentName ¶
SegmentName builds a segment name for the directory.
Types ¶
type CorruptionErr ¶
CorruptionErr is an error that's returned when corruption is encountered.
func (*CorruptionErr) Error ¶
func (e *CorruptionErr) Error() string
type LiveReader ¶ added in v0.4.0
type LiveReader struct {
// contains filtered or unexported fields
}
LiveReader reads WAL records from an io.Reader. It allows reading of WALs that are still in the process of being written, and returns records as soon as they can be read.
func NewLiveReader ¶ added in v0.4.0
func NewLiveReader(logger log.Logger, metrics *liveReaderMetrics, r io.Reader) *LiveReader
NewLiveReader returns a new live reader.
func (*LiveReader) Err ¶ added in v0.4.0
func (r *LiveReader) Err() error
Err returns any errors encountered reading the WAL. io.EOFs are not terminal and Next can be tried again. Non-EOFs are terminal, and the reader should not be used again. It is up to the user to decide when to stop trying should io.EOF be returned.
func (*LiveReader) Next ¶ added in v0.4.0
func (r *LiveReader) Next() bool
Next returns true if Record() will contain a full record. If Next returns false, you should always checked the contents of Error(). Return false guarantees there are no more records if the segment is closed and not corrupt, otherwise if Err() == io.EOF you should try again when more data has been written.
func (*LiveReader) Offset ¶ added in v0.5.0
func (r *LiveReader) Offset() int64
Offset returns the number of bytes consumed from this segment.
func (*LiveReader) Record ¶ added in v0.4.0
func (r *LiveReader) Record() []byte
Record returns the current record. The returned byte slice is only valid until the next call to Next.
type Reader ¶
type Reader struct {
// contains filtered or unexported fields
}
Reader reads WAL records from an io.Reader.
func (*Reader) Err ¶
Err returns the last encountered error wrapped in a corruption error. If the reader does not allow to infer a segment index and offset, a total offset in the reader stream will be provided.
func (*Reader) Next ¶
Next advances the reader to the next records and returns true if it exists. It must not be called again after it returned false.
func (*Reader) Offset ¶ added in v0.3.0
Offset returns the current position of the segment being read.
type Segment ¶
Segment represents a segment file.
func CreateSegment ¶
CreateSegment creates a new segment k in dir.
func OpenReadSegment ¶
OpenReadSegment opens the segment with the given filename.
func OpenWriteSegment ¶
OpenWriteSegment opens segment k in dir. The returned segment is ready for new appends.
type SegmentRange ¶ added in v0.3.0
SegmentRange groups segments by the directory and the first and last index it includes.
type WAL ¶
type WAL struct {
// contains filtered or unexported fields
}
WAL is a write ahead log that stores records in segment files. It must be read from start to end once before logging new data. If an error occurs during read, the repair procedure must be called before it's safe to do further writes.
Segments are written to in pages of 32KB, with records possibly split across page boundaries. Records are never split across segments to allow full segments to be safely truncated. It also ensures that torn writes never corrupt records beyond the most recent segment.
func New ¶
func New(logger log.Logger, reg prometheus.Registerer, dir string, compress bool) (*WAL, error)
New returns a new WAL over the given directory.
func NewSize ¶
func NewSize(logger log.Logger, reg prometheus.Registerer, dir string, segmentSize int, compress bool) (*WAL, error)
NewSize returns a new WAL over the given directory. New segments are created with the specified size.
func Open ¶ added in v0.10.0
func Open(logger log.Logger, reg prometheus.Registerer, dir string) (*WAL, error)
Open an existing WAL.
func (*WAL) CompressionEnabled ¶ added in v0.9.0
CompressionEnabled returns if compression is enabled on this WAL.
func (*WAL) Log ¶
Log writes the records into the log. Multiple records can be passed at once to reduce writes and increase throughput.
func (*WAL) NextSegment ¶ added in v0.9.0
NextSegment creates the next segment and closes the previous one.
func (*WAL) Repair ¶
Repair attempts to repair the WAL based on the error. It discards all data after the corruption.
Source Files