performance

package module
v0.0.9 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 26, 2019 License: Apache-2.0 Imports: 9 Imported by: 0

README

Babelfish performance testing

Build

go generate ./...
cd cmd/bblfsh-performance
go build

Currently supports only 2 commands

parse-and-store
./bblfsh-performance parse-and-store --help
parse file(s) with golang benchmark output and store it into a given storage

Usage:
  bblfsh-performance parse-and-store [--language=<language>] [--commit=<commit-id>] [--storage=<storage>] <file ...> [flags]

Aliases:
  parse-and-store, pas, parse-and-dump

Examples:
WARNING! To access storage corresponding environment variables should be set.
Full examples of usage scripts are following:

# for prometheus pushgateway
export PROM_ADDRESS="localhost:9091"
export PROM_JOB=pushgateway
bblfsh-performance parse-and-store --language=go --commit=3d9682b --storage="prom" /var/log/bench0 /var/log/bench1

# for influx db
export INFLUX_ADDRESS="http://localhost:8086"
export INFLUX_USERNAME=""
export INFLUX_PASSWORD=""
export INFLUX_DB=mydb
export INFLUX_MEASUREMENT=benchmark
bblfsh-performance parse-and-store --language=go --commit=3d9682b --storage="influxdb" /var/log/bench0 /var/log/bench1

Flags:
  -c, --commit string     commit id that's being tested and will be used as a tag in performance report
  -h, --help              help for parse-and-store
  -l, --language string   name of the language to be tested
  -s, --storage string    storage kind to store the results(prom, influxdb, file) (default "prom")
Command usage

Either locally or in CI:

  1. pull driver repo
  2. perform benchmarks over the fixtures
  3. save output test benchmark output to the file
  4. run bblfsh-performance parse-and-store and pass the filepath(s) as an argument
driver-native
./bblfsh-performance driver-native --help
run language driver container and perform benchmark tests over the native driver, store results into a given storage

Usage:
  bblfsh-performance driver-native [--language=<language>] [--commit=<commit-id>] [--storage=<storage>] [--filter-prefix=<filter-prefix>] [--native=<path-to-native>] <directory> [flags]

Aliases:
  driver-native, dn, native

Examples:
WARNING! Requires native-driver-performance binary to be build
WARNING! To access storage corresponding environment variables should be set.
Full examples of usage scripts are following:

# for prometheus pushgateway
export PROM_ADDRESS="localhost:9091"
export PROM_JOB=pushgateway
./bblfsh-performance driver-native \
--language go \
--commit 096361d09049c27e829fd5a6658f1914fd3b62ac \
--native /home/lwsanty/goproj/lwsanty/performance/cmd/native-driver-performance/native-driver-performance \
/var/testdata/fixtures

# for influx db
export INFLUX_ADDRESS="http://localhost:8086"
export INFLUX_USERNAME=""
export INFLUX_PASSWORD=""
export INFLUX_DB=mydb
export INFLUX_MEASUREMENT=benchmark
./bblfsh-performance driver-native \
--language go \
--commit 096361d09049c27e829fd5a6658f1914fd3b62ac \
--native /home/lwsanty/goproj/lwsanty/performance/cmd/native-driver-performance/native-driver-performance \
--storage=influxdb \
/var/testdata/fixtures


Flags:
  -c, --commit string              commit id that's being tested and will be used as a tag in performance report
      --exclude-suffixes strings   file suffixes to be excluded (default [.legacy,.native,.uast])
      --filter-prefix string       file prefix to be filtered (default "bench_")
  -h, --help                       help for driver-native
  -l, --language string            name of the language to be tested
  -n, --native string              path to native driver performance util (default "/root/utils/native-driver-test")
  -s, --storage string             storage kind to store the results(prom, influxdb, file) (default "prom")
driver
./bblfsh-performance driver --help
run language driver container and perform benchmark tests over the driver, store results into a given storage

Usage:
  bblfsh-performance driver [--language=<language>] [--commit=<commit-id>] [--storage=<storage>] [--filter-prefix=<filter-prefix>] <directory> [flags]

Aliases:
  driver, d

Examples:
WARNING! To access storage corresponding environment variables should be set.
Full examples of usage scripts are following:

# for prometheus pushgateway
export PROM_ADDRESS="localhost:9091"
export PROM_JOB=pushgateway
./bblfsh-performance driver \
--language go \
--commit 096361d09049c27e829fd5a6658f1914fd3b62ac \
/var/testdata/fixtures

# for influx db
export INFLUX_ADDRESS="http://localhost:8086"
export INFLUX_USERNAME=""
export INFLUX_PASSWORD=""
export INFLUX_DB=mydb
export INFLUX_MEASUREMENT=benchmark
./bblfsh-performance driver \
--language go \
--commit 096361d09049c27e829fd5a6658f1914fd3b62ac \
--storage=influxdb \
/var/testdata/fixtures


Flags:
  -c, --commit string              commit id that's being tested and will be used as a tag in performance report
      --exclude-suffixes strings   file suffixes to be excluded (default [.legacy,.native,.uast])
      --filter-prefix string       file prefix to be filtered (default "bench_")
  -h, --help                       help for driver
  -l, --language string            name of the language to be tested
  -s, --storage string             storage kind to store the results(prom, influxdb, file) (default "prom")
end-2-end
./bblfsh-performance end-to-end --help
run bblfshd container and perform benchmark tests, store results into a given storage

Usage:
  bblfsh-performance end-to-end [--language=<language>] [--commit=<commit-id>] [--extension=<files-extension>] [--docker-tag=<docker-tag>] [--storage=<storage>] <directory ...> [flags]

Aliases:
  end-to-end, e2e

Examples:
To use external bblfshd set BBLFSHD_LOCAL=${bblfshd_address}

WARNING! To access storage corresponding environment variables should be set.
Full examples of usage scripts are following:

# for prometheus pushgateway
export PROM_ADDRESS="localhost:9091"
export PROM_JOB=pushgateway
./bblfsh-performance end-to-end --language=go --commit=3d9682b --filter-prefix="bench_" --exclude-suffixes=".legacy",".native",".uast" --storage="prom" /var/testdata/benchmarks

# for influx db
export INFLUX_ADDRESS="http://localhost:8086"
export INFLUX_USERNAME=""
export INFLUX_PASSWORD=""
export INFLUX_DB=mydb
export INFLUX_MEASUREMENT=benchmark
bblfsh-performance end-to-end --language=go --commit=3d9682b --filter-prefix="bench_" --exclude-suffixes=".legacy",".native",".uast" --storage="influxdb" /var/testdata/benchmarks

Flags:
  -c, --commit string              commit id that's being tested and will be used as a tag in performance report
      --custom-driver              if this flag is set to true CLI pulls corresponding language driver repo's commit, builds docker image and installs it onto the bblfsh container
  -t, --docker-tag string          bblfshd docker image tag to be tested (default "latest-drivers")
      --exclude-suffixes strings   file suffixes to be excluded (default [.legacy,.native,.uast])
      --filter-prefix string       file prefix to be filtered (default "bench_")
  -h, --help                       help for end-to-end
  -l, --language string            name of the language to be tested
  -s, --storage string             storage kind to store the results(prom, influxdb, file) (default "prom")

Documentation

Index

Constants

View Source 
const (
	// BblfshdLevel is a metrics tag that represents benchmarks being run over bblfshd container
	BblfshdLevel = "bblfshd"
	// DriverLevel is a metrics tag that represents benchmarks being run over language driver container
	DriverLevel = "driver"
	// DriverNativeLevel is a metrics tag that represents benchmarks being run over native language driver
	DriverNativeLevel = "driver-native"
	// TransformsLevel is a metrics tag that represents benchmarks being run over transformations layer
	TransformsLevel = "transforms"

	// FileFilterPrefix is a fileFilterPrefix of file that would be filtered from the list of files in a directory.
	// Currently we use benchmark fixtures, file name pattern in this case is bench_*.${extension}
	FileFilterPrefix = "bench_"
)

Variables

View Source 
var (
	// ErrCannotInstallCustomDriver is used when driver installation process has failed or test conditions
	// do not allow to install it
	ErrCannotInstallCustomDriver = errors.NewKind("custom driver cannot be installed: %v")
)

Functions

func Bench added in v0.0.6 

func Bench(f func()) func(b *testing.B)

Bench wraps given function into function that performs benchmark over it

func ExecCmd 

func ExecCmd(command string) error

ExecCmd executes the specified Bash script. If execution fails, the error contains the combined output from stdout and stderr of the script. Do not use this for scripts that produce a large volume of output.

func GetFiles 

func GetFiles(pref string, exclusionSuffixes []string, dirs ...string) ([]string, error)

GetFiles is a simple "get files by pattern" function Purpose: filter required fixtures

func SplitStringMap 

func SplitStringMap(m map[string]string) ([]string, []string)

SplitStringMap splits map[string]string to arrays of keys and arrays of values

func WrapErr 

func WrapErr(err error, kinds ...*errors.Kind) error

WrapErr wraps given error with a given amount of error kinds. Works in inside-to-outside direction

Types

type Benchmark 

type Benchmark struct {
	Benchmark parse.Benchmark
}

Benchmark is a wrapper around parse.Benchmark and serves for formatting and arranging data before storing

func BenchmarkResultToBenchmark 

func BenchmarkResultToBenchmark(name string, b *testing.BenchmarkResult, trimPrefixes ...string) Benchmark

BenchmarkResultToBenchmark converts b *testing.BenchmarkResult *parse.Benchmark for further storing

func NewBenchmark 

func NewBenchmark(pb *parse.Benchmark, trimPrefixes ...string) Benchmark

NewBenchmark is a constructor for Benchmark

type RunE 

type RunE func(cmd *cobra.Command, args []string) error

RunE is a type that represents a standard Run function for cobra commands

func RunESilenced 

func RunESilenced(f RunE) RunE

TODO(lwsanty): https://github.com/spf13/cobra/issues/340 RunESilenced is a wrapper over standard cobra's RunE function Purpose: hide the command usage output in the case of internal error inside the command

Source Files

View all Source files

Directories

Path Synopsis
cmd
bblfsh-performance
bblfsh-performance/driver
bblfsh-performance/drivernative
bblfsh-performance/endtoend
bblfsh-performance/parseandstore
native-driver-performance
file
influxdb
pushgateway

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.