Documentation
¶
Overview ¶
Package replay implements collection and replaying of compaction benchmarking workloads. A workload is a collection of flushed and ingested sstables, along with the corresponding manifests describing the order and grouping with which they were applied. Replaying a workload flushes and ingests the same keys and sstables to reproduce the write workload for the purpose of evaluating compaction heuristics.
Index ¶
- type Metrics
- type PaceByFixedReadAmp
- type PaceByReferenceReadAmp
- type Pacer
- type Plot
- type Runner
- type SampledMetric
- func (m *SampledMetric) Max() int64
- func (m *SampledMetric) Mean() float64
- func (m *SampledMetric) Min() int64
- func (m *SampledMetric) Plot(width, height int, scale float64) string
- func (m *SampledMetric) PlotIncreasingPerSec(width, height int, scale float64) string
- func (m *SampledMetric) Values(n int) []float64
- type Unpaced
- type WorkloadCollector
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Metrics ¶
type Metrics struct {
CompactionCounts struct {
Total int64
Default int64
DeleteOnly int64
ElisionOnly int64
Move int64
Read int64
Rewrite int64
MultiLevel int64
}
EstimatedDebt SampledMetric
Final *pebble.Metrics
Ingest struct {
BytesIntoL0 uint64
// BytesWeightedByLevel is calculated as the number of bytes ingested
// into a level multiplied by the level's distance from the bottommost
// level (L6), summed across all levels. It can be used to guage how
// effective heuristics are at ingesting files into lower levels, saving
// write amplification.
BytesWeightedByLevel uint64
}
// PaceDuration is the time waiting for the pacer to allow the workload to
// continue.
PaceDuration time.Duration
ReadAmp SampledMetric
// QuiesceDuration is the time between completing application of the workload and
// compactions quiescing.
QuiesceDuration time.Duration
TombstoneCount SampledMetric
// TotalSize holds the total size of the database, sampled after each
// workload step.
TotalSize SampledMetric
TotalWriteAmp float64
WorkloadDuration time.Duration
WriteBytes uint64
WriteStalls map[string]int
WriteStallsDuration map[string]time.Duration
WriteThroughput SampledMetric
}
Metrics holds the various statistics on a replay run and its performance.
type PaceByFixedReadAmp ¶
type PaceByFixedReadAmp int
PaceByFixedReadAmp implements Pacer by applying each new write following a fixed read amplification.
type PaceByReferenceReadAmp ¶
type PaceByReferenceReadAmp struct{}
PaceByReferenceReadAmp implements Pacer by applying each new write following the collected workloads read amplification.
type Pacer ¶
type Pacer interface {
// contains filtered or unexported methods
}
A Pacer paces replay of a workload, determining when to apply the next incoming write.
type Runner ¶
type Runner struct {
RunDir string
WorkloadFS vfs.FS
WorkloadPath string
Pacer Pacer
Opts *pebble.Options
MaxWriteBytes uint64
// contains filtered or unexported fields
}
Runner runs a captured workload against a test database, collecting metrics on performance.
func (*Runner) Close ¶
Close closes remaining open resources, including the database. It must be called after Wait.
type SampledMetric ¶
type SampledMetric struct {
// contains filtered or unexported fields
}
SampledMetric holds a metric that is sampled at various points of workload replay. Samples are collected when a new step in the workload is applied to the database, and whenever a compaction completes.
func (*SampledMetric) Max ¶
func (m *SampledMetric) Max() int64
Max calculates the maximum value of the metric.
func (*SampledMetric) Mean ¶
func (m *SampledMetric) Mean() float64
Mean calculates the mean value of the metric.
func (*SampledMetric) Min ¶
func (m *SampledMetric) Min() int64
Min calculates the mininum value of the metric.
func (*SampledMetric) Plot ¶
func (m *SampledMetric) Plot(width, height int, scale float64) string
Plot returns an ASCII graph plot of the metric over time, with the provided width and height determining the size of the graph and the number of representable discrete x and y points. All values are first multiplied by the provided scale parameter before graphing.
func (*SampledMetric) PlotIncreasingPerSec ¶
func (m *SampledMetric) PlotIncreasingPerSec(width, height int, scale float64) string
PlotIncreasingPerSec returns an ASCII graph plot of the increasing delta of a metric over time, per-second. The provided width and height determine the size of the graph and the number of representable discrete x and y points. All deltas are multiplied by the provided scale parameter and scaled to per-second before graphing.
func (*SampledMetric) Values ¶
func (m *SampledMetric) Values(n int) []float64
Values returns the values of the metric, distributed across n discrete buckets that are equally spaced over time. If multiple values fall within a bucket, the latest recorded value is used. If no values fall within a bucket, the next recorded value is used.
type Unpaced ¶
type Unpaced struct{}
Unpaced implements Pacer by applying each new write as soon as possible. It may be useful for examining performance under high read amplification.
type WorkloadCollector ¶
type WorkloadCollector struct {
// contains filtered or unexported fields
}
WorkloadCollector is designed to capture workloads by handling manifest files, flushed SSTs and ingested SSTs. The collector hooks into the pebble.EventListener and pebble.Cleaner in order keep track of file states.
func NewWorkloadCollector ¶
func NewWorkloadCollector(srcDir string) *WorkloadCollector
NewWorkloadCollector is used externally to create a New WorkloadCollector.
func (*WorkloadCollector) Attach ¶
func (w *WorkloadCollector) Attach(opts *pebble.Options)
Attach is used to set up the WorkloadCollector by attaching itself to pebble.Options EventListener and Cleaner.
func (*WorkloadCollector) IsRunning ¶
func (w *WorkloadCollector) IsRunning() bool
IsRunning returns whether the WorkloadCollector is currently running.
func (*WorkloadCollector) Start ¶
func (w *WorkloadCollector) Start(destFS vfs.FS, destPath string)
Start begins collecting a workload. All flushed and ingested sstables, plus corresponding manifests are copied to the provided destination path on the provided FS.
func (*WorkloadCollector) Stop ¶
func (w *WorkloadCollector) Stop()
Stop stops collection of the workload.
func (*WorkloadCollector) WaitAndStop ¶
func (w *WorkloadCollector) WaitAndStop()
WaitAndStop waits for all enqueued sstables to be copied over, and then calls Stop. Gracefully ensures that all sstables referenced in the collected manifest's latest version edit will exist in the copy directory.
Source Files