Documentation ¶
Index ¶
- Variables
- func QuestionMarksStringList(n int) string
- func Run(benchmarkInterface BenchmarkInterface) error
- type AbstractWorkload
- type AutoIncrementGenerator
- func (g *AutoIncrementGenerator) Current() int64
- func (g *AutoIncrementGenerator) Generate(r *Rand) interface{}
- func (g *AutoIncrementGenerator) GenerateTyped(r *Rand) int64
- func (g *AutoIncrementGenerator) Min() int64
- func (g *AutoIncrementGenerator) SampleFromExisting(r *Rand) interface{}
- func (g *AutoIncrementGenerator) SampleFromExistingTyped(r *Rand) int64
- type Benchmark
- type BenchmarkConfig
- type BenchmarkInterface
- type BenchmarkWorker
- type Column
- type Connection
- type DataGenerator
- type DataLogger
- type DataSnapshot
- type DatabaseConfig
- type DatetimeInterval
- type DiscretizedLooper
- type EnumGenerator
- type EventStat
- type ExtendedHdrHistogram
- func (h *ExtendedHdrHistogram) IntervalData(endTime time.Time, histMin, histMax, histSize int64) IntervalData
- func (h *ExtendedHdrHistogram) Merge(other *ExtendedHdrHistogram)
- func (h *ExtendedHdrHistogram) RecordValue(v int64)
- func (h *ExtendedHdrHistogram) ResetDataOnly()
- func (h *ExtendedHdrHistogram) ResetStartTime(startTime time.Time)
- type HistogramCardinalityStringGenerator
- func (g *HistogramCardinalityStringGenerator) Generate(r *Rand) interface{}
- func (g *HistogramCardinalityStringGenerator) GenerateTyped(r *Rand) string
- func (g *HistogramCardinalityStringGenerator) SampleFromExisting(r *Rand) interface{}
- func (g *HistogramCardinalityStringGenerator) SampleFromExistingTyped(r *Rand) string
- type HistogramDistribution
- type HistogramFloatGenerator
- type HistogramIntGenerator
- type HistogramLengthStringGenerator
- func (g *HistogramLengthStringGenerator) Generate(r *Rand) interface{}
- func (g *HistogramLengthStringGenerator) GenerateTyped(r *Rand) string
- func (g *HistogramLengthStringGenerator) SampleFromExisting(r *Rand) interface{}
- func (g *HistogramLengthStringGenerator) SampleFromExistingTyped(r *Rand) string
- type HttpServer
- type IntervalData
- type JSONGenerator
- type LockedDoubleBuffer
- type LooperType
- type NoContextData
- type NormalFloatGenerator
- type NormalIntGenerator
- type NullGenerator
- type OnlineHistogram
- type OuterLoopStat
- type Rand
- func (r *Rand) HistFloat(hist HistogramDistribution) float64
- func (r *Rand) HistInt(hist HistogramDistribution) int64
- func (r *Rand) NormalFloat(mean, stddev float64) float64
- func (r *Rand) NormalInt(mean, stddev int64) int64
- func (r *Rand) UniformFloat(min, max float64) float64
- func (r *Rand) UniformInt(min, max int64) int64
- type RateControlConfig
- type Ring
- type StatusData
- type Table
- func (t Table) CreateTableQuery() string
- func (t Table) DropTableQuery() string
- func (t Table) Generate(r *Rand, column string) interface{}
- func (t Table) InsertQuery(r *Rand, batchSize int, valueOverride map[string]interface{}) (string, []interface{})
- func (t Table) InsertQueryList(r *Rand, valueOverrides []map[string]interface{}) (string, []interface{})
- func (t Table) ReloadData(databaseConfig DatabaseConfig, totalrows int64, batchSize int64, ...)
- func (t Table) SampleFromExisting(r *Rand, column string) interface{}
- type UniformCardinalityStringGenerator
- func (g *UniformCardinalityStringGenerator) Generate(r *Rand) interface{}
- func (g *UniformCardinalityStringGenerator) GenerateTyped(r *Rand) string
- func (g *UniformCardinalityStringGenerator) SampleFromExisting(r *Rand) interface{}
- func (g *UniformCardinalityStringGenerator) SampleFromExistingTyped(r *Rand) string
- type UniformDatetimeGenerator
- type UniformDecimalGenerator
- type UniformFloatGenerator
- type UniformHistogram
- type UniformIntGenerator
- type UniformLengthStringGenerator
- type UniqueStringGenerator
- type UuidGenerator
- type VisualizationConfig
- type WorkerContext
- type Workload
- func (w *Workload[ContextDataT]) Config() WorkloadConfig
- func (w *Workload[ContextDataT]) FinishInitialization(databaseConfig DatabaseConfig, rateControlConfig RateControlConfig)
- func (w *Workload[ContextDataT]) ForEachOnlineHistogram(f func(int, *OnlineHistogram))
- func (w *Workload[ContextDataT]) RateControlConfig() RateControlConfig
- func (w *Workload[ContextDataT]) Run(ctx context.Context, workerInitializationWg *sync.WaitGroup, ...)
- type WorkloadConfig
- type WorkloadDataSnapshot
- type WorkloadInterface
Constants ¶
This section is empty.
Variables ¶
var VersionString = "1.0"
Functions ¶
func QuestionMarksStringList ¶
func Run ¶
func Run(benchmarkInterface BenchmarkInterface) error
Runs a custom defined benchmark that implements the BenchmarkInterface.
Types ¶
type AbstractWorkload ¶
type AbstractWorkload interface { Run(context.Context, *sync.WaitGroup, time.Time) Config() WorkloadConfig // We need to set the RateControlConfig on the Workload object, because the // data logger needs to know the concurrency/event rate of the workload. // See comments in Benchmark.Start for more details. FinishInitialization(DatabaseConfig, RateControlConfig) // The DataLogger need to iterate through all the OnlineHistograms for each // worker so it can perform the double buffer swap. Since the DataLogger only // have access to a map of AbstractWorkload, it doesn't have access to the // underlying BenchmarkWorker array (which are templated). So this // AbstractWorkload needs to provide a way to iterate through all the // OnlineHistograms. The naive way to implement this would be by returning a // slice of OnlineHistograms. This is not a good approach, because it would // allocate memory, which can take unbounded time. Since the code that // iterates through all the online histograms are in a "critical" section that // takes as small amount of a time as possible, this is not acceptable. Thus, // basically create an iterator. Since Golang doesn't have a built-in iterator // pattern, the functional pattern is chosen. // TODO: check that the functional pattern doesn't introduce unnecessary // overhead/memory allocations. ForEachOnlineHistogram(func(int, *OnlineHistogram)) // The DataLogger needs the rate control config to make allocations and record // desired event rates. See comments in Benchmark.Start for more details. RateControlConfig() RateControlConfig }
We want the workload to be templated so the context data can be transparently passed from the workload to the Event() function without going through runtime type selection.
However, we don't want to store a templated workload in the Benchmark object, as it would start to infect every other struct (like DataLogger). Further, since the Workload is stored in the Benchmark in a map, if it is templated, you cannot have different ContextDataT on a per workload basis if you wanted it.
Is this kind of a hack? Maybe. I haven't decided yet.
type AutoIncrementGenerator ¶
type AutoIncrementGenerator struct {
// contains filtered or unexported fields
}
Atomically generate an auto incrementing value from the client-side.
Sample from existing with sample uniformly between the min value to the current value. There is no guarantee that it will land on an existing value if values have been deleted.
TODO: track deletion, but this is problematic too, because golang doesn't offer a concurrent-write map.
func NewAutoIncrementGenerator ¶
func NewAutoIncrementGenerator(min, current int64) *AutoIncrementGenerator
func NewAutoIncrementGeneratorFromDatabase ¶
func NewAutoIncrementGeneratorFromDatabase(databaseConfig DatabaseConfig, table, column string) (*AutoIncrementGenerator, error)
func (*AutoIncrementGenerator) Current ¶
func (g *AutoIncrementGenerator) Current() int64
Get the current value without generating a new value.
func (*AutoIncrementGenerator) Generate ¶
func (g *AutoIncrementGenerator) Generate(r *Rand) interface{}
func (*AutoIncrementGenerator) GenerateTyped ¶
func (g *AutoIncrementGenerator) GenerateTyped(r *Rand) int64
func (*AutoIncrementGenerator) Min ¶
func (g *AutoIncrementGenerator) Min() int64
func (*AutoIncrementGenerator) SampleFromExisting ¶
func (g *AutoIncrementGenerator) SampleFromExisting(r *Rand) interface{}
func (*AutoIncrementGenerator) SampleFromExistingTyped ¶
func (g *AutoIncrementGenerator) SampleFromExistingTyped(r *Rand) int64
type Benchmark ¶
type Benchmark struct { BenchmarkConfig Name string LogInterval time.Duration // TODO: make these two values configurable (move into BenchmarkConfig maybe) LogRingSize int // contains filtered or unexported fields }
func NewBenchmark ¶
func NewBenchmark(benchmarkName string, benchmarkConfig BenchmarkConfig) (*Benchmark, error)
func (*Benchmark) AddWorkload ¶
func (b *Benchmark) AddWorkload(workload AbstractWorkload)
func (*Benchmark) DataSnapshots ¶
func (b *Benchmark) DataSnapshots() []*DataSnapshot
func (*Benchmark) StopAndWait ¶
func (b *Benchmark) StopAndWait()
type BenchmarkConfig ¶
type BenchmarkConfig struct { Bench bool Load bool Duration time.Duration LogFile string LogTable string Note string DatabaseConfig DatabaseConfig RateControlConfig RateControlConfig HttpPort int }
func NewBenchmarkConfig ¶
func NewBenchmarkConfig() *BenchmarkConfig
func (*BenchmarkConfig) Config ¶
func (c *BenchmarkConfig) Config() BenchmarkConfig
func (*BenchmarkConfig) ValidateAndSetDefaults ¶
func (c *BenchmarkConfig) ValidateAndSetDefaults() error
type BenchmarkInterface ¶
type BenchmarkInterface interface { // Returns the name of the benchmark Name() string // Returns a list of constructed workloads (or an error) for this benchmark. Workloads() ([]AbstractWorkload, error) // This function is called if the benchmark ran with the --load flag. This // should load the database when called. RunLoader() error // Returns the benchmark app configuration. The configuration returned must // have already been validated with defaults filled in. Make sure to call // .Validate() on it during the construction of the object that implements // this interface. // // If you implement BenchmarkInterface with BenchmarkConfig embedded in it, // you won't need to define this method as the BenchmarkConfig object already // defines this method for you, and embedding it in your struct will cause // your struct to inherit the method implemented on BenchmarkConfig. Config() BenchmarkConfig }
This is the interface that the benchmark application needs to implement
type BenchmarkWorker ¶
type BenchmarkWorker[ContextDataT any] struct { // contains filtered or unexported fields }
A single goroutine worker that loops and benchmarks MySQL
func NewBenchmarkWorker ¶
func NewBenchmarkWorker[ContextDataT any](workloadIface WorkloadInterface[ContextDataT], databaseConfig DatabaseConfig, rateControlConfig RateControlConfig) (*BenchmarkWorker[ContextDataT], error)
type Column ¶
type Column struct { // Name of the column Name string // SQL definition of the column Definition string // The data generator for the data of this column. Generator DataGenerator }
type Connection ¶
A thin wrapper around https://pkg.go.dev/github.com/go-mysql-org/go-mysql/client#Conn for now. It is possible in the future to extend this to support databases other than MySQL.
This should only be initialized via DatabaseConfig.Connection().
func (*Connection) Close ¶
func (c *Connection) Close() error
func (*Connection) GetRoundRobinConnection ¶
func (c *Connection) GetRoundRobinConnection() *client.Conn
type DataGenerator ¶
An interface for the data generator.
There are two ways to generate data:
1. Generate a new value to be inserted into the database. This is generated via the Generate call. 2. Generate an "existing" value to be used in the WHERE clause of a SQL statement. This is generated via the SampleFromExisting call. Note, most generators cannot guarantee that an existing value is generated, as it would be probihitively expensive to keep track of all the existing data. Consult the documentation of the specific generators for details.
type DataLogger ¶
type DataLogger struct { Interval time.Duration RingSize int OutputFilename string TableName string Note string Benchmark *Benchmark // contains filtered or unexported fields }
func NewDataLogger ¶
func NewDataLogger(dataLogger *DataLogger) (*DataLogger, error)
func (*DataLogger) DataSnapshots ¶
func (d *DataLogger) DataSnapshots() []*DataSnapshot
type DataSnapshot ¶
type DataSnapshot struct { // Time since start of the test. Time float64 // The throughput and latency data for all monitored benchmarks merged // together. AllWorkloadData WorkloadDataSnapshot // The throughput and latency data for individual monitored benchmarks, // indexed by the workload name. PerWorkloadData map[string]WorkloadDataSnapshot }
type DatabaseConfig ¶
type DatabaseConfig struct { // TODO: Unix socket Host string Port int User string Pass string Database string // If this is set, a connection will not be established. This is useful for // non-database-related tests such as selfbench. // TODO: this is kind of a hack... NoConnection bool // The number of underlying connections per Connection object, implemented as a // static pool from which connections are fetched in a round-robin sequence with // each successive request. The sole purpose of this feature is to multiply the // number of open connections to the database to assess any performance impact // specific to the overall number of open database connections. // // Note: this feature does not work on all benchmarks at this moment. It only // works with benchmarks that uses the mybench.Connection.GetRoundRobinConnection // function to get their connections and execute queries. ConnectionMultiplier int // Enable CLIENT_MULTI_STATEMENTS options on the client ClientMultiStatements bool }
The database config object that can be turned into a single connection (without connection pooling).
func (DatabaseConfig) Connection ¶
func (cfg DatabaseConfig) Connection() (*Connection, error)
Returns a connection object based on the database configuration
func (DatabaseConfig) CreateDatabaseIfNeeded ¶
func (cfg DatabaseConfig) CreateDatabaseIfNeeded() error
Creates a new database if it doesn't exist
type DiscretizedLooper ¶
type DiscretizedLooper struct { EventRate float64 OuterLoopRate float64 LooperType LooperType DebugIdentifier string // The context passed is a trace context from runtime/trace package Event func(context.Context) error TraceEvent func(EventStat) TraceOuterLoop func(OuterLoopStat) // contains filtered or unexported fields }
type EnumGenerator ¶
type EnumGenerator[T any] struct { // contains filtered or unexported fields }
Generates values from a discrete set of possible values.
Sample from existing is the exact same as generation, which means it is possible to generate values not in the database but available in the set of values.
func NewEnumGenerator ¶
func NewEnumGenerator[T any](values []T) *EnumGenerator[T]
func (*EnumGenerator[T]) Generate ¶
func (g *EnumGenerator[T]) Generate(r *Rand) interface{}
func (*EnumGenerator[T]) GenerateTyped ¶
func (g *EnumGenerator[T]) GenerateTyped(r *Rand) T
func (*EnumGenerator[T]) SampleFromExisting ¶
func (g *EnumGenerator[T]) SampleFromExisting(r *Rand) interface{}
func (*EnumGenerator[T]) SampleFromExistingTyped ¶
func (g *EnumGenerator[T]) SampleFromExistingTyped(r *Rand) T
type ExtendedHdrHistogram ¶
type ExtendedHdrHistogram struct {
// contains filtered or unexported fields
}
This extends the HDR histogram so it can track: - Start time - Under and overflow counts
func NewExtendedHdrHistogram ¶
func NewExtendedHdrHistogram(startTime time.Time) *ExtendedHdrHistogram
func (*ExtendedHdrHistogram) IntervalData ¶
func (h *ExtendedHdrHistogram) IntervalData(endTime time.Time, histMin, histMax, histSize int64) IntervalData
func (*ExtendedHdrHistogram) Merge ¶
func (h *ExtendedHdrHistogram) Merge(other *ExtendedHdrHistogram)
Should only be called from the data logger, after it is copied away from the double buffer. Maybe some of these "read" methods should be defined on a different type, so it can never be called on an object that could be in the double buffer.
func (*ExtendedHdrHistogram) RecordValue ¶
func (h *ExtendedHdrHistogram) RecordValue(v int64)
func (*ExtendedHdrHistogram) ResetDataOnly ¶
func (h *ExtendedHdrHistogram) ResetDataOnly()
func (*ExtendedHdrHistogram) ResetStartTime ¶
func (h *ExtendedHdrHistogram) ResetStartTime(startTime time.Time)
type HistogramCardinalityStringGenerator ¶
type HistogramCardinalityStringGenerator struct {
// contains filtered or unexported fields
}
Generates a fixed number of unique strings with uniform distribution. For example, if cardinality is 10, then this generator will generate 10 distinct string values. The frequency of the strings are uniform.
Sample from existing is the same as generate, which means it may not sample an existing value.
func NewHistogramCardinalityStringGenerator ¶
func NewHistogramCardinalityStringGenerator(binsEndPoints, frequency []float64, length int) *HistogramCardinalityStringGenerator
See NewHistogramDistribution for documentation the arguments for this function. Note each integer generated by the histogram will be mapped to a string. To specify make sure integers such as 1, 2, 3, 4 are generated, the binsEndPoints must be 0.5, 1.5, 2.5, 3.5, 4.5.
func (*HistogramCardinalityStringGenerator) Generate ¶
func (g *HistogramCardinalityStringGenerator) Generate(r *Rand) interface{}
func (*HistogramCardinalityStringGenerator) GenerateTyped ¶
func (g *HistogramCardinalityStringGenerator) GenerateTyped(r *Rand) string
func (*HistogramCardinalityStringGenerator) SampleFromExisting ¶
func (g *HistogramCardinalityStringGenerator) SampleFromExisting(r *Rand) interface{}
func (*HistogramCardinalityStringGenerator) SampleFromExistingTyped ¶
func (g *HistogramCardinalityStringGenerator) SampleFromExistingTyped(r *Rand) string
type HistogramDistribution ¶
type HistogramDistribution struct {
// contains filtered or unexported fields
}
This generates float64 values based on a discrete probability distribution (represented via a histogram) via the inverse transform sampling algorithm (https://en.wikipedia.org/wiki/Inverse_transform_sampling). Specifically, the steps followed are:
- Normalize the frequency values of the histogram to values of between 0 and 1.
- Compute the cumulative distribution for the normalized histogram such that its output value is also between 0 and 1. So we have the function cdf(bin_value) -> [0, 1].
- Generate a random value, x, between 0 and 1. This value represents a sampled number from the cdf function output. If we compute the inverse function cdf^-1(x) -> bin_value, we will obtain a randomly sampled bin value that will be randomly sampled according to the frequency specified in the histogram.
- The inverse function cdf^-1(x) is calculated via linear interpolation.
Note that the ExistingValue for this distribution is the same as NextValue and thus has no memory of past generated values.
func NewHistogramDistribution ¶
func NewHistogramDistribution(binsEndPoints, frequency []float64) HistogramDistribution
Creates a histogram distribution which is used by Rand to generate random numbers.
The value at frequency[i] correspond to the bin starting at [bins[i], bins[i+1]). Thus, len(bins) == len(frequency) + 1.
Also, the value in bins must be sorted.
type HistogramFloatGenerator ¶
type HistogramFloatGenerator struct {
// contains filtered or unexported fields
}
Generates floating point values according to a histogram distribution.
Sample from existing does not track of values already generated but samples from the same distribution as the Generate. This means it is possible to generate values that doesn't exist in the database.
func NewHistogramFloatGenerator ¶
func NewHistogramFloatGenerator(binsEndPoints, frequency []float64) *HistogramFloatGenerator
See NewHistogramDistribution for documentation the arguments for this function.
func (*HistogramFloatGenerator) Generate ¶
func (g *HistogramFloatGenerator) Generate(r *Rand) interface{}
func (*HistogramFloatGenerator) GenerateTyped ¶
func (g *HistogramFloatGenerator) GenerateTyped(r *Rand) float64
func (*HistogramFloatGenerator) SampleFromExisting ¶
func (g *HistogramFloatGenerator) SampleFromExisting(r *Rand) interface{}
func (*HistogramFloatGenerator) SampleFromExistingTyped ¶
func (g *HistogramFloatGenerator) SampleFromExistingTyped(r *Rand) float64
type HistogramIntGenerator ¶
type HistogramIntGenerator struct {
// contains filtered or unexported fields
}
Generates integers according to a histogram distribution. One possible use case of this is when you want to distribute a foreign key/id with a particular distribution. For example, a `posts` table can have many posts, with 50% of the rows having one `user_id`, and then 25% of the rows with another `user_id`.
Sample from existing does not track of values already generated but samples from the same distribution as the Generate. This means it is possible to generate values that doesn't exist in the database.
func NewHistogramIntGenerator ¶
func NewHistogramIntGenerator(binsEndPoints, frequency []float64) *HistogramIntGenerator
See NewHistogramDistribution for documentation the arguments for this function. Note each integer generated by the histogram will be mapped to a string. To specify make sure integers such as 1, 2, 3, 4 are generated, the binsEndPoints must be 0.5, 1.5, 2.5, 3.5, 4.5.
func (*HistogramIntGenerator) Generate ¶
func (g *HistogramIntGenerator) Generate(r *Rand) interface{}
func (*HistogramIntGenerator) GenerateTyped ¶
func (g *HistogramIntGenerator) GenerateTyped(r *Rand) int64
func (*HistogramIntGenerator) SampleFromExisting ¶
func (g *HistogramIntGenerator) SampleFromExisting(r *Rand) interface{}
func (*HistogramIntGenerator) SampleFromExistingTyped ¶
func (g *HistogramIntGenerator) SampleFromExistingTyped(r *Rand) int64
type HistogramLengthStringGenerator ¶
type HistogramLengthStringGenerator struct {
// contains filtered or unexported fields
}
Generates a random string with length selected by a histogram distribution.
Sample from existing is the same as generate and does not keep track of existing values. Since there are a very large amount of possible strings being generated, there is almost no chance that an existing value will be generated. It is best not to use that method and expect good results.
func NewHistogramLengthStringGenerator ¶
func NewHistogramLengthStringGenerator(binsEndPoints, frequency []float64) *HistogramLengthStringGenerator
See NewHistogramDistribution for documentation the arguments for this function. Note each integer generated by the histogram will be mapped to a string. To specify make sure integers such as 1, 2, 3, 4 are generated, the binsEndPoints must be 0.5, 1.5, 2.5, 3.5, 4.5.
func (*HistogramLengthStringGenerator) Generate ¶
func (g *HistogramLengthStringGenerator) Generate(r *Rand) interface{}
func (*HistogramLengthStringGenerator) GenerateTyped ¶
func (g *HistogramLengthStringGenerator) GenerateTyped(r *Rand) string
func (*HistogramLengthStringGenerator) SampleFromExisting ¶
func (g *HistogramLengthStringGenerator) SampleFromExisting(r *Rand) interface{}
func (*HistogramLengthStringGenerator) SampleFromExistingTyped ¶
func (g *HistogramLengthStringGenerator) SampleFromExistingTyped(r *Rand) string
type HttpServer ¶
type HttpServer struct {
// contains filtered or unexported fields
}
func NewHttpServer ¶
func NewHttpServer(benchmark *Benchmark, note string, port int) *HttpServer
func (*HttpServer) Run ¶
func (h *HttpServer) Run()
type IntervalData ¶
type IntervalData struct { StartTime time.Time EndTime time.Time Count int64 Delta float64 Rate float64 // All data in microseconds Min int64 Mean float64 Max int64 Percentile25 int64 Percentile50 int64 Percentile75 int64 Percentile90 int64 Percentile99 int64 UnderflowCount int64 OverflowCount int64 UniformHist *UniformHistogram }
type JSONGenerator ¶
type JSONGenerator struct {
// contains filtered or unexported fields
}
Generates the same JSON document every time. This is based on map[string]string.
func NewJSONGenerator ¶
func NewJSONGenerator(objLength, valueLength int) *JSONGenerator
func (*JSONGenerator) Generate ¶
func (g *JSONGenerator) Generate(r *Rand) interface{}
func (*JSONGenerator) GenerateTyped ¶
func (g *JSONGenerator) GenerateTyped(r *Rand) string
func (*JSONGenerator) SampleFromExisting ¶
func (g *JSONGenerator) SampleFromExisting(r *Rand) interface{}
func (*JSONGenerator) SampleFromExistingTyped ¶
func (g *JSONGenerator) SampleFromExistingTyped(r *Rand) string
type LockedDoubleBuffer ¶
type LockedDoubleBuffer[T any] struct { // contains filtered or unexported fields }
This is a double buffer implemented using a lock. The target usage is as follows:
- Single consumer single producer.
- The producer goroutine writes data frequently.
- The consumer goroutine reads data infrequently.
- The consumer goroutine will first swap the buffer. It gets the non-active data during the swap. After the swap, it will read the data and then reset the data to 0, so it can be swapped again.
- We can never swap before the non-active data is reset.
- While the producer goroutine is writing to the data, the swap is not allowed to occur.
func NewLockedDoubleBuffer ¶
func NewLockedDoubleBuffer[T any](newT func() T) *LockedDoubleBuffer[T]
func (*LockedDoubleBuffer[T]) SafeActiveWrite ¶
func (b *LockedDoubleBuffer[T]) SafeActiveWrite(f func(T))
Since we need to prevent swapping from happening while writing, we are using a lock. Lock-less could work as well but will be more complex and may not be necessary.
func (*LockedDoubleBuffer[T]) Swap ¶
func (b *LockedDoubleBuffer[T]) Swap(preSwapCallback func(nonActiveData T)) T
Swap the active and non-active data with a lock. Returns the non-active data.
type LooperType ¶
type LooperType int
const ( LooperTypeUniform LooperType = iota LooperTypePoisson )
type NoContextData ¶
type NoContextData struct{}
This is a convenience type defined to indicate that there's no context data. If you don't need a custom context data for your workload, use this.
Further, the workload should embed this struct so the NewContextData function does not need to be defined
func (NoContextData) NewContextData ¶
func (NoContextData) NewContextData(*Connection) (NoContextData, error)
Any struct that embeds NoContextData will inherit this method. If this struct is also trying to implement WorkloadInterface, then it does not have to define this method manually.
type NormalFloatGenerator ¶
type NormalFloatGenerator struct {
// contains filtered or unexported fields
}
Generates a floating point number with a given normal distribution.
Sample from existing is the same as generating a number, which means it is not guaranteed to land on an existing value.
func NewNormalFloatGenerator ¶
func NewNormalFloatGenerator(mean, stddev float64) *NormalFloatGenerator
func (*NormalFloatGenerator) Generate ¶
func (g *NormalFloatGenerator) Generate(r *Rand) interface{}
func (*NormalFloatGenerator) GenerateTyped ¶
func (g *NormalFloatGenerator) GenerateTyped(r *Rand) float64
func (*NormalFloatGenerator) SampleFromExisting ¶
func (g *NormalFloatGenerator) SampleFromExisting(r *Rand) interface{}
func (*NormalFloatGenerator) SampleFromExistingTyped ¶
func (g *NormalFloatGenerator) SampleFromExistingTyped(r *Rand) float64
type NormalIntGenerator ¶
type NormalIntGenerator struct {
// contains filtered or unexported fields
}
Generates a random integer value according to a normal distribution.
Sample from existing is the same as generation.
func NewNormalIntGenerator ¶
func NewNormalIntGenerator(mean, stddev int64) *NormalIntGenerator
func (*NormalIntGenerator) Generate ¶
func (g *NormalIntGenerator) Generate(r *Rand) interface{}
func (*NormalIntGenerator) GenerateTyped ¶
func (g *NormalIntGenerator) GenerateTyped(r *Rand) int64
func (*NormalIntGenerator) SampleFromExisting ¶
func (g *NormalIntGenerator) SampleFromExisting(r *Rand) interface{}
func (*NormalIntGenerator) SampleFromExistingTyped ¶
func (g *NormalIntGenerator) SampleFromExistingTyped(r *Rand) int64
type NullGenerator ¶
type NullGenerator struct{}
A boring generator that only generates only null values.
func NewNullGenerator ¶
func NewNullGenerator() NullGenerator
func (NullGenerator) Generate ¶
func (NullGenerator) Generate(*Rand) interface{}
func (NullGenerator) SampleFromExisting ¶
func (NullGenerator) SampleFromExisting(*Rand) interface{}
type OnlineHistogram ¶
type OnlineHistogram struct { *LockedDoubleBuffer[*ExtendedHdrHistogram] }
func NewOnlineHistogram ¶
func NewOnlineHistogram(startTime time.Time) *OnlineHistogram
func (*OnlineHistogram) RecordValue ¶
func (h *OnlineHistogram) RecordValue(v int64)
func (*OnlineHistogram) Swap ¶
func (h *OnlineHistogram) Swap(preSwapCallback func(nonActiveData *ExtendedHdrHistogram)) *ExtendedHdrHistogram
type OuterLoopStat ¶
type OuterLoopStat struct { // The desired wake up time. DesiredWakeupTime time.Time // The start time of the outer loop iteration and the actual wake up time. ActualWakeupTime time.Time // Number of Event() calls to make in this outer loop iteration. EventBatchSize int64 // The end time of the outer loop iteration, but doesn't actually include the // time taken to calculate the next wakeup time, as that code is assumed to be // very fast and negligible. EventsEnd time.Time // The total time taken to process all Event() calls, including the TraceEvent calls. EventsLatency time.Duration // The next desired wakeup time NextDesiredWakeupTime time.Time // The next expected event's activation time. If this timestamp is really // behind the actual wakeup time, then the system is very backlogged. NextExpectedEventTime time.Time // The cumulative number of events executed. CumulativeNumberOfEvents int64 }
type Rand ¶
func (*Rand) HistFloat ¶
func (r *Rand) HistFloat(hist HistogramDistribution) float64
func (*Rand) HistInt ¶
func (r *Rand) HistInt(hist HistogramDistribution) int64
func (*Rand) NormalFloat ¶
func (*Rand) UniformFloat ¶
func (*Rand) UniformInt ¶
type RateControlConfig ¶
type RateControlConfig struct { // The total rate at which Event is called in hz, across all goroutines. EventRate float64 // Number of goroutines to drive the EventRate. Each goroutine will get the // same portion of the EventRate. This needs to be increased if a single // goroutine cannot drive the EventRate. If not specified, it will be // calculated from EventRate and MaxEventRatePerWorker. Concurrency int // The maximum event rate per goroutine, used to calculate Concurrency. // If not specified, it will default to 100. MaxEventRatePerWorker float64 // The desired rate of the outer loop that batches events. Default: 50. OuterLoopRate float64 // The type of looper used. Default to Uniform looper. LooperType LooperType }
type Ring ¶
type Ring[T any] struct { // contains filtered or unexported fields }
A terrible implementation of a ring, based on the Golang ring which is not thread-safe nor offers a nice API.
I can't believe there are no simple ring buffer data structure in Golang, with generics.
func (*Ring[T]) ReadAllOrdered ¶
func (r *Ring[T]) ReadAllOrdered() []T
type StatusData ¶
type StatusData struct { CurrentTime float64 Note string Workloads []string DataSnapshots []*DataSnapshot }
type Table ¶
type Table struct { // The name of the table Name string // The list of columns. The order of the columns in the table follows the // order of this slice. Columns []*Column // The columns for the primary key for the table. This is a slice as it // supports a composite primary key. PrimaryKey []string // A list of columns for indices. Indices [][]string // A list of columns for unique indices. UniqueKeys [][]string // Additional table options appended to the end of the CREATE TABLE // statements, such as compression settings, auto increment settings, and so // on. TableOptions string // contains filtered or unexported fields }
This struct provides helpers for creating and seeding a table.
func InitializeTable ¶
func (Table) CreateTableQuery ¶
func (Table) DropTableQuery ¶
func (Table) InsertQuery ¶
func (Table) InsertQueryList ¶
func (Table) ReloadData ¶
func (t Table) ReloadData(databaseConfig DatabaseConfig, totalrows int64, batchSize int64, concurrency int)
Drop and recreate the table with data seeded via the data generators.
totalrows specifies the total number of rows to insert into the new table. batchSize controls how many rows to insert in one INSERT statement (200 is usually a good starting point), concurrency is the number of goroutines used to insert the data.
If concurrency is 0, it is set by default to 16. This allows the loader to reuse the -concurrency flag (which is default 0).
func (Table) SampleFromExisting ¶
type UniformCardinalityStringGenerator ¶
type UniformCardinalityStringGenerator struct {
// contains filtered or unexported fields
}
Generates a fixed number of unique strings with uniform distribution. For example, if cardinality is 10, then this generator will generate 10 distinct string values. The frequency of the strings are uniform.
Sample from existing is the same as generate, which means it may not sample an existing value.
func NewUniformCardinalityStringGenerator ¶
func NewUniformCardinalityStringGenerator(cardinality, length int) *UniformCardinalityStringGenerator
func (*UniformCardinalityStringGenerator) Generate ¶
func (g *UniformCardinalityStringGenerator) Generate(r *Rand) interface{}
func (*UniformCardinalityStringGenerator) GenerateTyped ¶
func (g *UniformCardinalityStringGenerator) GenerateTyped(r *Rand) string
func (*UniformCardinalityStringGenerator) SampleFromExisting ¶
func (g *UniformCardinalityStringGenerator) SampleFromExisting(r *Rand) interface{}
func (*UniformCardinalityStringGenerator) SampleFromExistingTyped ¶
func (g *UniformCardinalityStringGenerator) SampleFromExistingTyped(r *Rand) string
type UniformDatetimeGenerator ¶
type UniformDatetimeGenerator struct {
// contains filtered or unexported fields
}
Generates a date time value in two modes:
- GenerateNow == true will cause Generate to return time.Now.
- GenerateNow == false will cause Generate to generate a random time between the intervals specified in Intervals with uniform probability distribution.
SampleFromExisting always will sample from the Intervals. However, if GenerateNow == true, then it will also sample between an extra interval between when Generate() is first called and the moment the SampleFromExisting call is made.
Generate and SampleFromExisting will return a string of the time formatted with YYYY-MM-DD hh:mm:ss, which is what SQL expects. GenerateTyped and SampleFromExistingTyped will return time.Time.
func NewNowGenerator ¶
func NewNowGenerator() *UniformDatetimeGenerator
func NewUniformDatetimeGenerator ¶
func NewUniformDatetimeGenerator(intervals []DatetimeInterval, generateNow bool) *UniformDatetimeGenerator
func (*UniformDatetimeGenerator) Generate ¶
func (g *UniformDatetimeGenerator) Generate(r *Rand) interface{}
func (*UniformDatetimeGenerator) GenerateTyped ¶
func (g *UniformDatetimeGenerator) GenerateTyped(r *Rand) time.Time
func (*UniformDatetimeGenerator) SampleFromExisting ¶
func (g *UniformDatetimeGenerator) SampleFromExisting(r *Rand) interface{}
func (*UniformDatetimeGenerator) SampleFromExistingTyped ¶
func (g *UniformDatetimeGenerator) SampleFromExistingTyped(r *Rand) time.Time
type UniformDecimalGenerator ¶
type UniformDecimalGenerator struct {
// contains filtered or unexported fields
}
TODO: can this be folded into the UniformFloatGenerator? Generates an random decimal value
Sampling from existing is the same as the generation, which mean it is not guaranteed to generate an existing value if the number of rows in the database is small or the decimal has a large precision
func NewUniformDecimalGenerator ¶
func NewUniformDecimalGenerator(precision, scale int) *UniformDecimalGenerator
func (*UniformDecimalGenerator) Generate ¶
func (g *UniformDecimalGenerator) Generate(r *Rand) interface{}
func (*UniformDecimalGenerator) GenerateTyped ¶
func (g *UniformDecimalGenerator) GenerateTyped(r *Rand) string
func (*UniformDecimalGenerator) SampleFromExisting ¶
func (g *UniformDecimalGenerator) SampleFromExisting(r *Rand) interface{}
func (*UniformDecimalGenerator) SampleFromExistingTyped ¶
func (g *UniformDecimalGenerator) SampleFromExistingTyped(r *Rand) string
type UniformFloatGenerator ¶
type UniformFloatGenerator struct {
// contains filtered or unexported fields
}
Generates a random floating point value according to an uniform distribution between min (inclusive) and max (exclusive).
Sampling from existing is the same as the generation, since there are a large number of floating point values, it is unlikely to generate an exact value that has been used before. However, the generated value may still be useful in WHERE clauses that uses the greater than or less than operators.
func NewUniformFloatGenerator ¶
func NewUniformFloatGenerator(min, max float64) *UniformFloatGenerator
func (*UniformFloatGenerator) Generate ¶
func (g *UniformFloatGenerator) Generate(r *Rand) interface{}
func (*UniformFloatGenerator) GenerateTyped ¶
func (g *UniformFloatGenerator) GenerateTyped(r *Rand) float64
func (*UniformFloatGenerator) SampleFromExisting ¶
func (g *UniformFloatGenerator) SampleFromExisting(r *Rand) interface{}
func (*UniformFloatGenerator) SampleFromExistingTyped ¶
func (g *UniformFloatGenerator) SampleFromExistingTyped(r *Rand) float64
type UniformHistogram ¶
type UniformHistogram struct { Buckets []hdrhistogram.Bar // contains filtered or unexported fields }
func NewUniformHistogram ¶
func NewUniformHistogram(histMin, histMax, histSize int64) *UniformHistogram
func (*UniformHistogram) RecordValues ¶
func (h *UniformHistogram) RecordValues(v int64, count int64)
type UniformIntGenerator ¶
type UniformIntGenerator struct {
// contains filtered or unexported fields
}
Generates an integer value in the inclusive range between min (inclusive) and max (exclusive) with an uniform distribution.
Sampling from existing is the same as the generation, which mean it is not guaranteed to generate an existing value if the number of rows in the database is small.
func NewUniformIntGenerator ¶
func NewUniformIntGenerator(min, max int64) *UniformIntGenerator
func (*UniformIntGenerator) Generate ¶
func (g *UniformIntGenerator) Generate(r *Rand) interface{}
func (*UniformIntGenerator) GenerateTyped ¶
func (g *UniformIntGenerator) GenerateTyped(r *Rand) int64
func (*UniformIntGenerator) SampleFromExisting ¶
func (g *UniformIntGenerator) SampleFromExisting(r *Rand) interface{}
func (*UniformIntGenerator) SampleFromExistingTyped ¶
func (g *UniformIntGenerator) SampleFromExistingTyped(r *Rand) int64
type UniformLengthStringGenerator ¶
type UniformLengthStringGenerator struct {
// contains filtered or unexported fields
}
Generates a random string with length selected between the min and max specified with uniform probability.
Sample from existing is the same as generate and does not keep track of existing values. Since there are a very large amount of possible strings being generated, there is almost no chance that an existing value will be generated. It is best not to use that method and expect good results.
func NewUniformLengthStringGenerator ¶
func NewUniformLengthStringGenerator(minLength, maxLength int) *UniformLengthStringGenerator
func (*UniformLengthStringGenerator) Generate ¶
func (g *UniformLengthStringGenerator) Generate(r *Rand) interface{}
func (*UniformLengthStringGenerator) GenerateTyped ¶
func (g *UniformLengthStringGenerator) GenerateTyped(r *Rand) string
func (*UniformLengthStringGenerator) SampleFromExisting ¶
func (g *UniformLengthStringGenerator) SampleFromExisting(r *Rand) interface{}
func (*UniformLengthStringGenerator) SampleFromExistingTyped ¶
func (g *UniformLengthStringGenerator) SampleFromExistingTyped(r *Rand) string
type UniqueStringGenerator ¶
type UniqueStringGenerator struct {
// contains filtered or unexported fields
}
Generates an unique string with a fixed length every time Generate is called. The internal generation is based on an atomic, incrementing integer. Each integer is converted into a string (via a hash function).
Sample from existing will generate a value that has previously been generated. However, the value may have been deleted in the database so it's not guaranteed that the value generated will exist on the database.
func NewUniqueStringGenerator ¶
func NewUniqueStringGenerator(length int, min, current int64) *UniqueStringGenerator
length is the length of the string to be generated min and current are the integer values used to generate the strings. For loading data (when there are nothing in the database), min and current both should be 0. When there are already data in the database, min and current should be set to the min and max integer values used to generate strings that already exist in the database.
func NewUniqueStringGeneratorFromDatabase ¶
func NewUniqueStringGeneratorFromDatabase(databaseConfig DatabaseConfig, table, column string) (*UniqueStringGenerator, error)
func (*UniqueStringGenerator) Generate ¶
func (g *UniqueStringGenerator) Generate(r *Rand) interface{}
func (*UniqueStringGenerator) GenerateTyped ¶
func (g *UniqueStringGenerator) GenerateTyped(r *Rand) string
func (*UniqueStringGenerator) SampleFromExisting ¶
func (g *UniqueStringGenerator) SampleFromExisting(r *Rand) interface{}
func (*UniqueStringGenerator) SampleFromExistingTyped ¶
func (g *UniqueStringGenerator) SampleFromExistingTyped(r *Rand) string
type UuidGenerator ¶
type UuidGenerator struct {
Version int
}
Generates UUIDs SampleFromExisting is basically broken as this should only very rarely generate a duplicate UUID. Version 1 uuid's have the timestamp at which they were generated embedded in them Version 4 uuid's are random
func NewUuidGenerator ¶
func NewUuidGenerator(version int) *UuidGenerator
NewUuidGenerator Only version 1 (timebased) and version 4 (random) supported
func (*UuidGenerator) Generate ¶
func (g *UuidGenerator) Generate(r *Rand) interface{}
func (*UuidGenerator) GenerateTyped ¶
func (g *UuidGenerator) GenerateTyped(r *Rand) string
func (*UuidGenerator) SampleFromExisting ¶
func (g *UuidGenerator) SampleFromExisting(r *Rand) interface{}
func (*UuidGenerator) SampleFromExistingTyped ¶
func (g *UuidGenerator) SampleFromExistingTyped(r *Rand) string
type VisualizationConfig ¶
type VisualizationConfig struct { // The min, max, and size of the visualization histogram that tracks the // per-event latency. This is not used to actually track the latency as the code // internally uses HDR histogram to track a much wider range. Instead, this is // used for the final data logging and web UI display. // The min and max has units of microseconds and defaults to 0 and 100000. LatencyHistMin int64 LatencyHistMax int64 LatencyHistSize int64 }
type WorkerContext ¶
type WorkerContext[T any] struct { // A worker-specific connection object to the database. Conn *Connection // A worker-specific Rand object. This is needed because the global rand.* // functions in Go uses a global mutex underneath the hood and can cause // severe performance problems within mybench due to lock contention. Rand *Rand // User-defined custom data. If no custom data is needed, set T to be // mybench.NoContextData. Data T // A context.Context object used for tracing only. The tracing used is // Golang's runtime/trace package. Each Event executed is already in a task // called OuterLoopIteration{WorkloadName}. By the time the Event function is // called, it is done so inside a trace region called `Event`. Nested regions // can be created by the user. TraceCtx context.Context }
This is the object type that holds the thread-local context data for each benchmark worker. Each benchmark worker has its own copy of this. This object is passed by the benchmark worker to the workload Event() function through the Event() function argument.
The WorkerContext also holds arbitrary data with user-defined types under the Data attribute. This allows the user to store custom thread-local data on each benchmark for their workloads. A common use case is to store a statement object in an user-defined struct.
type Workload ¶
type Workload[ContextDataT any] struct { WorkloadConfig // contains filtered or unexported fields }
The actual benchmark struct for a single workload.
func NewWorkload ¶
func NewWorkload[ContextDataT any](workloadIface WorkloadInterface[ContextDataT]) *Workload[ContextDataT]
func (*Workload[ContextDataT]) Config ¶
func (w *Workload[ContextDataT]) Config() WorkloadConfig
func (*Workload[ContextDataT]) FinishInitialization ¶
func (w *Workload[ContextDataT]) FinishInitialization(databaseConfig DatabaseConfig, rateControlConfig RateControlConfig)
func (*Workload[ContextDataT]) ForEachOnlineHistogram ¶
func (w *Workload[ContextDataT]) ForEachOnlineHistogram(f func(int, *OnlineHistogram))
func (*Workload[ContextDataT]) RateControlConfig ¶
func (w *Workload[ContextDataT]) RateControlConfig() RateControlConfig
type WorkloadConfig ¶
type WorkloadConfig struct { // The name of the workload, for identification purposes only. Name string // scales the workload by the given percentage // this currently scales various RateControl parameters WorkloadScale float64 // Configures the visualization for this workload. // Some workload may know the latency bounds, and may wish to choose a better scale for the histograms Visualization VisualizationConfig }
Config used to create the Workload
func (WorkloadConfig) Config ¶
func (w WorkloadConfig) Config() WorkloadConfig
type WorkloadDataSnapshot ¶
type WorkloadDataSnapshot struct { // Throughput and latency data IntervalData // The desired throughput DesiredRate float64 }
Merges the IntervalData with other data. Represents all the stats collected for a single workload.
type WorkloadInterface ¶
type WorkloadInterface[ContextDataT any] interface { // Returns the config for the workload. Should always return the same // value. // // If the struct that implements this interface embeds WorkloadConfig, it // does not need to manually define this method as WorkloadConfig already // defines this method and structs that embeds WorkloadConfig will inherit // that method. Config() WorkloadConfig // Run the code for a single event. Run your queries here. This function is // called repeatedly at the desired rate. // // This function can be called concurrently from many goroutines. Each // goroutine will pass a different connection object to this function, so the // connection should be safe to use normally. However, if you rely on any // variables set in the Prepare function, make sure it is thread safe but also // performant. Event(WorkerContext[ContextDataT]) error // Since the Event() function can be called from multiple goroutines, there is // no convenient place to store goroutine-specific data. Storing it on the // WorkloadInterface won't work because that object is shared // between all BenchmarkWorkers (and hence goroutines and threads) . // // Using the WorkerContext.Data field, each BenchmarkWorker can store its own // goroutine-local data. This is passed to the Event() function call on that // goroutine only, which means use of the data is thread-safe, as it is not // accessed from any other threads. // // This method creates a new context data object and it is called once per // goroutine/BenchmarkWorker before the main event loop. // // If a workload does not need context data, the struct that implements // workload interface should simply embed NoContextData, which defines this // method. NewContextData(*Connection) (ContextDataT, error) }
An interface for implementing the workload. Although only one Workload struct exists, the functions defined in this can be called concurrently from many (thousands) goroutines. Care must be taken to ensure no data races (use atomic or mutexes, although the latter could be slow as the number of goroutines increase)
A example of a workload could be selecting data from a single table. Another workload could be inserting into the same table. This is similar in meaning as the "class" Section 6.2 in the paper: "OLTP-Bench: An Extensible Testbed for Benchmarking Relational Databases" https://www.cs.cmu.edu/~pavlo/papers/oltpbench-vldb.pdf.
Each benchmark supports multiple workloads that can be scaled and reported on seperately allowing for evolving workload mixtures.