golden

package
v1.6.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 9, 2024 License: Apache-2.0 Imports: 18 Imported by: 0

Documentation

Overview

Package golden holds tools for testing documentation code.

Index

Constants

View Source
const (
	// ArgInputReplacement is the placeholder for the input file path in the
	// command arguments.
	ArgInputReplacement = "GOLDEN_INPUT"
	// ArgOutputReplacement is the placeholder for the output file path in the
	// command arguments.
	ArgOutputReplacement = "GOLDEN_OUTPUT"
)
View Source
const (
	// StableTime is a stable time to be used in golden file tests.
	StableTime string = "2023-01-01T00:00:00Z"
	// StableDuration is a stable duration to be used in golden file tests.
	StableDuration string = "123ms"
	// StableText is a stable text to be used in golden file tests.
	StableText string = "text"
	// StableFloat is a stable float to be used in golden file tests.
	StableFloat float64 = 0.123
	// StableInt is a stable int to be used in golden file tests.
	StableInt int = 123
	// StableBool is a stable bool to be used in golden file tests.
	StableBool bool = true
	// StableVersion is a stable version to be used in golden file tests.
	StableVersion string = "VERSION"
)

Variables

This section is empty.

Functions

func BashTest

func BashTest(
	t *testing.T,
	goldenDir string,
	bashConfig BashConfig,
)

BashTest executes a golden file test for a bash command. It walks over the goldenDir to gather all .sh scripts present in the dir. It then executes each of the scripts and compares expected vs. actual outputs. If displayStdout or displayStderr are true, the output of each script will be composed of the resulting stderr + stdout.

func BashTestFile added in v1.6.0

func BashTestFile(
	t *testing.T,
	script string,
	bashConfig BashConfig,
)

BashTestFile executes a golden file test for a single bash script. The script is executed and the expected output is compared with the actual output.

func CopyFile

func CopyFile(original, newFile string)

CopyFile copies a file from the original path to the new path.

func DagTest added in v1.6.0

func DagTest(t *testing.T, cases []DagTestCase)

DagTest runs a set of test cases in topological order. Each test case is a BashTest, and the test cases are connected by their dependencies. If a test case has dependencies, it will only be run after all of its dependencies have been run.

Sample usage:

cases := []golden.DagTestCase{
  {
    name:   "app-create",
    needs:  []string{},
    config: BashConfig{ /**/ },
    path:   "app-create",
  },
  {
    name:   "app-push",
    needs:  []string{"app-create"},
    config: BashConfig{ /**/ },
    path:   "app-push",
  },
}
golden.DagTest(t, cases)

func FileTest

func FileTest(t *testing.T, inputPath string, config Config)

FileTest performs a golden file test using input as the input file for the program found at the path given by main. The output is compared against the golden file (.golden). If the update flag is set, the .golden file is updated to the observed output.

func FileTests

func FileTests(t *testing.T, location string, config Config)

FileTests performs golden file tests for all the <.json/csv> files contained in the given location. A golden file test uses an input file to execute a program. The output of the program is compared against an expected output which is a .golden file. If the location is a <.json/csv> file, then a single test is carried out for that file. If the location is a directory, all <.json/csv> files are gathered and a golden file test is executed for each file. You can read more about GoldenFileTesting in this blog.

func GenerateTemplate

func GenerateTemplate(templateName, destination string)

GenerateTemplate generates a template with the given name. If destination is not empty, it will move the template to the destination directory.

func Reset

func Reset(keep []string)

Reset cleans up golden file testing. Must be called after test execution. Keep is a map of files that will not be deleted.

func Setup

func Setup()

Setup sets up golden file testing. Must be called before test execution.

func SetupTemplateTest

func SetupTemplateTest(templateName string)

SetupTemplateTest sets up golden file testing with a template. Must be called before test execution.

func SetupWithFileGen

func SetupWithFileGen(templateName string)

SetupWithFileGen gets the main.go from the template before setting up golden file testing. Must be called before test execution.

func Teardown

func Teardown(dirs ...string)

Teardown cleans up golden file testing. Must be called after test execution. Passing multiple directories is optional and it will clean up all directories passed to it.

func TemplateTeardown

func TemplateTeardown(templateName string)

TemplateTeardown cleans up the template directory. Must be called after test execution.

func ValidateAgainstSchema

func ValidateAgainstSchema(name string, docBytes, schemaBytes []byte) error

ValidateAgainstSchema validates the given JSON document (docBytes) against a JSON schema (schemaBytes). 'name' can be used for identifying the failing document type in the error.

Types

type BashConfig

type BashConfig struct {
	// DisplayStdout indicates whether to display or suppress stdout.
	DisplayStdout bool
	// DisplayStderr indicates whether to display or suppress stderr.
	DisplayStderr bool
	// OutputProcessConfig defines how to process the output before comparison.
	OutputProcessConfig OutputProcessConfig
	// Envs specifies the environment variables to set for execution.
	Envs [][2]string
	// PostProcessFunctions defines a list of functions to be executed after the bash
	// script has been run. This can be used to make use of the output of the bash script
	// and perform additional operations on it. The functions are executed in the order
	// they are defined and are not used for comparison.
	PostProcessFunctions []func(goldenFile string) error
	// WorkingDir is the directory where the bash script(s) will be
	// executed.
	WorkingDir string
	// WaitBefore adds a delay before running the bash script. This is useful
	// when throttling is needed, e.g., when dealing with rate limiting.
	WaitBefore time.Duration
}

BashConfig defines the configuration for a golden bash test.

type CompareConfig

type CompareConfig struct {
	// Pure string comparison. If true, the output is compared as a string. If
	// false, the output is parsed as JSON and compared as a JSON object.
	TxtParse         bool
	TxtCompareLength int
}

CompareConfig configures how to compare actual and expected.

type Config

type Config struct {
	// VerifyFunc is used to validate output against input, if provided.
	VerifyFunc func(input, output []byte) error
	// InputSchema definition to validate input JSON against
	InputSchema []byte
	// OutputSchema definition to validate output JSON against
	OutputSchema []byte
	// Args specifies the arguments to supply to the solver.
	Args []string
	// Envs specifies the environment variables to set for execution.
	Envs [][2]string
	// CompareConfig defines how to compare output against expectation.
	CompareConfig CompareConfig
	// OutputProcessConfig defines how to process the output before comparison.
	OutputProcessConfig OutputProcessConfig
	// SkipGoldenComparison skips the comparison against the golden file.
	SkipGoldenComparison bool
	// ExitCode defines the expected exit code of the command.
	ExitCode int
	// UseStdIn indicates whether to feed the file via stdin.
	UseStdIn bool
	// UseStdOut indicates whether to write output to stdout instead of a file.
	UseStdOut bool
	// IgnoreStdOut indicates whether to ignore the output of the command. This
	// is useful when the command writes to a file and we are not interested in
	// the output.
	IgnoreStdOut bool
	// TransientFields are keys that hold values which are transient (dynamic)
	// in nature, such as the elapsed time, version, start time, etc. Transient
	// fields have a special parsing in the .golden file and they are
	// stabilized in the comparison.
	TransientFields []TransientField
	// Tresholds by data type to be used when comparing actual and expected.
	// This configuration is optional, and if not provided then the comparison
	// between values is hard equality.
	Thresholds Tresholds
	// When DedicatedComparison is defined, then the golden file test will only
	// compare the keys that are defined in the slice. The keys are defined as
	// a [JSONPath]-like key. In general, use a dot (.) to recursively enter
	// nested objects and brackets ([]) to access array elements.
	//
	// [JSONPath]: https://goessner.net/articles/JsonPath/
	DedicatedComparison []string
	// ExecutionConfig defines the configuration for a Python golden file test. If
	// it is absent, then the golden file test is not a Python test.
	ExecutionConfig *ExecutionConfig
}

Config lets a user configure the golden file tests.

type CustomThresholds

type CustomThresholds struct {
	// Float defines specific thresholds for specific keys.
	Float map[string]float64
	// Int defines specific thresholds for specific keys.
	Int map[string]int
	// Time defines specific thresholds for specific keys.
	Time map[string]time.Duration
	// Duration defines specific thresholds for specific keys.
	Duration map[string]time.Duration
}

CustomThresholds defines threshold for specific keys that override the generic thresholds.

type DagTestCase added in v1.6.0

type DagTestCase struct {
	Name   string
	Needs  []string
	Config *BashConfig
	Path   string
}

DagTestCase represents a test case in a directed acyclic graph (DAG) test.

type ExecutionConfig

type ExecutionConfig struct {
	// Command is the command of the entrypoint of the app to be executed. E.g.,
	// "python3".
	Command string
	// Args are the arguments to be passed to the entrypoint of the app to be
	// executed. E.g., ["main.py"]. If InputFlag and OutputFlag are not
	// specified, then GOLDEN_INPUT and GOLDEN_OUTPUT are replaced by the input
	// and output file paths, respectively.
	Args []string
	// WorkDir is the working directory where the command will be executed. When
	// specified, the input file path will be adapted.
	WorkDir string
	// InputFlag is the argument to be used to pass the input file to the app to
	// be executed. E.g., "-input".
	InputFlag string
	// OutputFlag is the argument to be used to pass the output file to the app
	// to be executed. E.g., "-output".
	OutputFlag string
}

ExecutionConfig defines the configuration for non-SDK golden file tests.

type OutputProcessConfig

type OutputProcessConfig struct {
	// AlwaysUpdate makes the comparison always update the golden file.
	AlwaysUpdate bool
	// KeepVolatileData indicates whether to keep or replace frequently
	// changing data.
	KeepVolatileData bool
	// VolatileRegexReplacements defines regex replacements to be applied to the
	// golden file before comparison.
	VolatileRegexReplacements []VolatileRegexReplacement
	// VolatileDataFiles are files that contain volatile data and should get
	// post-processed to be more stable. This is only supported in directory
	// mode ([BashTest]) of golden bash testing, i.e., this will be ignored in
	// single file mode ([BashTestFile]).
	VolatileDataFiles []string
	// RelativeDestination is the relative path to the directory where the
	// output file will be stored. If not provided, then the output file is
	// stored in the current directory.
	RelativeDestination string
}

OutputProcessConfig defines how to process the output before comparison.

type TransientField

type TransientField struct {
	// Key is a representation of a [JSONPath]-like key. Here are some examples
	// of transient fields and how to override them in the comparison:
	//  - "version" key in the root: ".version"
	//  - "elapsed" key in the stats object in the root: ".stats.elapsed"
	//  - "start" key in the stats object in the root: ".stats.start"
	//  - "time" key in the first element of the solutions array in the root: ".solutions[0].time"
	// In general, use a dot (.) to recursively enter nested objects and
	// brackets ([]) to access array elements.
	//
	// [JSONPath]: https://goessner.net/articles/JsonPath/
	Key string

	// Replacement is optional, and it is the value that is used to stabilize
	// the transient field. If a replacement is not provided for the key, the
	// stabilization happens according to the data type. For example, a
	// [time.Time] is replaced using [StableTime], a [time.Duration] is
	// replaced using [StableDuration], etc. You can use the constants provided
	// by this package to stabilize the transient fields.
	Replacement any
}

TransientField represents a field that is transient, this is, dynamic in nature. Examples of such fields include durations, times, versions, etc. Transient fields are replaced in golden file tests to always obtain the same result regardless of the moment it is executed. If a dynamic field always has a static value, then the golden file tests can run successfully. The transient field is represented by a key and a replacement. Please see the documentation of the fields for more information.

type Tresholds

type Tresholds struct {
	// Float is the threshold to be used when comparing floats.
	Float float64
	// Int is the threshold to be used when comparing ints.
	Int int
	// Time is the threshold to be used when comparing times. Two times are
	// considered the same if the absolute difference between them, which is a
	// duration, is less than or equal to the given threshold.
	Time time.Duration
	// Duration is the threshold to be used when comparing durations.
	Duration time.Duration
	// CustomThresholds defines threshold for specific keys that override the
	// generic thresholds.
	CustomThresholds CustomThresholds
}

Tresholds by data type to be used when comparing actual and expected. If the absolute difference between the two values is less than or equal to the given threshold, then we consider the two values to be equal.

type VolatileRegexReplacement

type VolatileRegexReplacement struct {
	// Regex to match.
	Regex string
	// Replacement to apply.
	Replacement string
}

VolatileRegexReplacement defines a regex replacement to be applied to the output before comparison.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL