Documentation ¶
Overview ¶
Package golden holds tools for testing documentation code.
Index ¶
- Constants
- func BashTest(t *testing.T, goldenDir string, bashConfig BashConfig)
- func BashTestFile(t *testing.T, script string, bashConfig BashConfig)
- func CopyFile(original, newFile string)
- func DagTest(t *testing.T, cases []DagTestCase)
- func FileTest(t *testing.T, inputPath string, config Config)
- func FileTests(t *testing.T, location string, config Config)
- func GenerateTemplate(templateName, destination string)
- func Reset(keep []string)
- func Setup()
- func SetupTemplateTest(templateName string)
- func SetupWithFileGen(templateName string)
- func Teardown(dirs ...string)
- func TemplateTeardown(templateName string)
- func ValidateAgainstSchema(name string, docBytes, schemaBytes []byte) error
- type BashConfig
- type CompareConfig
- type Config
- type CustomThresholds
- type DagTestCase
- type ExecutionConfig
- type OutputProcessConfig
- type TransientField
- type Tresholds
- type VolatileRegexReplacement
Constants ¶
const ( // ArgInputReplacement is the placeholder for the input file path in the // command arguments. ArgInputReplacement = "GOLDEN_INPUT" // ArgOutputReplacement is the placeholder for the output file path in the // command arguments. ArgOutputReplacement = "GOLDEN_OUTPUT" )
const ( // StableTime is a stable time to be used in golden file tests. StableTime string = "2023-01-01T00:00:00Z" // StableDuration is a stable duration to be used in golden file tests. StableDuration string = "123ms" // StableText is a stable text to be used in golden file tests. StableText string = "text" // StableFloat is a stable float to be used in golden file tests. StableFloat float64 = 0.123 // StableInt is a stable int to be used in golden file tests. StableInt int = 123 // StableBool is a stable bool to be used in golden file tests. StableBool bool = true // StableVersion is a stable version to be used in golden file tests. StableVersion string = "VERSION" )
Variables ¶
This section is empty.
Functions ¶
func BashTest ¶
func BashTest( t *testing.T, goldenDir string, bashConfig BashConfig, )
BashTest executes a golden file test for a bash command. It walks over the goldenDir to gather all .sh scripts present in the dir. It then executes each of the scripts and compares expected vs. actual outputs. If displayStdout or displayStderr are true, the output of each script will be composed of the resulting stderr + stdout.
func BashTestFile ¶ added in v1.6.0
func BashTestFile( t *testing.T, script string, bashConfig BashConfig, )
BashTestFile executes a golden file test for a single bash script. The script is executed and the expected output is compared with the actual output.
func CopyFile ¶
func CopyFile(original, newFile string)
CopyFile copies a file from the original path to the new path.
func DagTest ¶ added in v1.6.0
func DagTest(t *testing.T, cases []DagTestCase)
DagTest runs a set of test cases in topological order. Each test case is a BashTest, and the test cases are connected by their dependencies. If a test case has dependencies, it will only be run after all of its dependencies have been run.
Sample usage:
cases := []golden.DagTestCase{ { name: "app-create", needs: []string{}, config: BashConfig{ /**/ }, path: "app-create", }, { name: "app-push", needs: []string{"app-create"}, config: BashConfig{ /**/ }, path: "app-push", }, } golden.DagTest(t, cases)
func FileTest ¶
FileTest performs a golden file test using input as the input file for the program found at the path given by main. The output is compared against the golden file (.golden). If the update flag is set, the .golden file is updated to the observed output.
func FileTests ¶
FileTests performs golden file tests for all the <.json/csv> files contained in the given location. A golden file test uses an input file to execute a program. The output of the program is compared against an expected output which is a .golden file. If the location is a <.json/csv> file, then a single test is carried out for that file. If the location is a directory, all <.json/csv> files are gathered and a golden file test is executed for each file. You can read more about GoldenFileTesting in this blog.
func GenerateTemplate ¶
func GenerateTemplate(templateName, destination string)
GenerateTemplate generates a template with the given name. If destination is not empty, it will move the template to the destination directory.
func Reset ¶
func Reset(keep []string)
Reset cleans up golden file testing. Must be called after test execution. Keep is a map of files that will not be deleted.
func SetupTemplateTest ¶
func SetupTemplateTest(templateName string)
SetupTemplateTest sets up golden file testing with a template. Must be called before test execution.
func SetupWithFileGen ¶
func SetupWithFileGen(templateName string)
SetupWithFileGen gets the main.go from the template before setting up golden file testing. Must be called before test execution.
func Teardown ¶
func Teardown(dirs ...string)
Teardown cleans up golden file testing. Must be called after test execution. Passing multiple directories is optional and it will clean up all directories passed to it.
func TemplateTeardown ¶
func TemplateTeardown(templateName string)
TemplateTeardown cleans up the template directory. Must be called after test execution.
func ValidateAgainstSchema ¶
ValidateAgainstSchema validates the given JSON document (docBytes) against a JSON schema (schemaBytes). 'name' can be used for identifying the failing document type in the error.
Types ¶
type BashConfig ¶
type BashConfig struct { // DisplayStdout indicates whether to display or suppress stdout. DisplayStdout bool // DisplayStderr indicates whether to display or suppress stderr. DisplayStderr bool // OutputProcessConfig defines how to process the output before comparison. OutputProcessConfig OutputProcessConfig // Envs specifies the environment variables to set for execution. Envs [][2]string // PostProcessFunctions defines a list of functions to be executed after the bash // script has been run. This can be used to make use of the output of the bash script // and perform additional operations on it. The functions are executed in the order // they are defined and are not used for comparison. PostProcessFunctions []func(goldenFile string) error // WorkingDir is the directory where the bash script(s) will be // executed. WorkingDir string // WaitBefore adds a delay before running the bash script. This is useful // when throttling is needed, e.g., when dealing with rate limiting. WaitBefore time.Duration }
BashConfig defines the configuration for a golden bash test.
type CompareConfig ¶
type CompareConfig struct { // Pure string comparison. If true, the output is compared as a string. If // false, the output is parsed as JSON and compared as a JSON object. TxtParse bool TxtCompareLength int }
CompareConfig configures how to compare actual and expected.
type Config ¶
type Config struct { // VerifyFunc is used to validate output against input, if provided. VerifyFunc func(input, output []byte) error // InputSchema definition to validate input JSON against InputSchema []byte // OutputSchema definition to validate output JSON against OutputSchema []byte // Args specifies the arguments to supply to the solver. Args []string // Envs specifies the environment variables to set for execution. Envs [][2]string // CompareConfig defines how to compare output against expectation. CompareConfig CompareConfig // OutputProcessConfig defines how to process the output before comparison. OutputProcessConfig OutputProcessConfig // SkipGoldenComparison skips the comparison against the golden file. SkipGoldenComparison bool // ExitCode defines the expected exit code of the command. ExitCode int // UseStdIn indicates whether to feed the file via stdin. UseStdIn bool // UseStdOut indicates whether to write output to stdout instead of a file. UseStdOut bool // IgnoreStdOut indicates whether to ignore the output of the command. This // is useful when the command writes to a file and we are not interested in // the output. IgnoreStdOut bool // TransientFields are keys that hold values which are transient (dynamic) // in nature, such as the elapsed time, version, start time, etc. Transient // fields have a special parsing in the .golden file and they are // stabilized in the comparison. TransientFields []TransientField // Tresholds by data type to be used when comparing actual and expected. // This configuration is optional, and if not provided then the comparison // between values is hard equality. Thresholds Tresholds // When DedicatedComparison is defined, then the golden file test will only // compare the keys that are defined in the slice. The keys are defined as // a [JSONPath]-like key. In general, use a dot (.) to recursively enter // nested objects and brackets ([]) to access array elements. // // [JSONPath]: https://goessner.net/articles/JsonPath/ DedicatedComparison []string // ExecutionConfig defines the configuration for a Python golden file test. If // it is absent, then the golden file test is not a Python test. ExecutionConfig *ExecutionConfig }
Config lets a user configure the golden file tests.
type CustomThresholds ¶
type CustomThresholds struct { // Float defines specific thresholds for specific keys. Float map[string]float64 // Int defines specific thresholds for specific keys. Int map[string]int // Time defines specific thresholds for specific keys. Time map[string]time.Duration // Duration defines specific thresholds for specific keys. Duration map[string]time.Duration }
CustomThresholds defines threshold for specific keys that override the generic thresholds.
type DagTestCase ¶ added in v1.6.0
type DagTestCase struct { Name string Needs []string Config *BashConfig Path string }
DagTestCase represents a test case in a directed acyclic graph (DAG) test.
type ExecutionConfig ¶
type ExecutionConfig struct { // Command is the command of the entrypoint of the app to be executed. E.g., // "python3". Command string // Args are the arguments to be passed to the entrypoint of the app to be // executed. E.g., ["main.py"]. If InputFlag and OutputFlag are not // specified, then GOLDEN_INPUT and GOLDEN_OUTPUT are replaced by the input // and output file paths, respectively. Args []string // WorkDir is the working directory where the command will be executed. When // specified, the input file path will be adapted. WorkDir string // InputFlag is the argument to be used to pass the input file to the app to // be executed. E.g., "-input". InputFlag string // OutputFlag is the argument to be used to pass the output file to the app // to be executed. E.g., "-output". OutputFlag string }
ExecutionConfig defines the configuration for non-SDK golden file tests.
type OutputProcessConfig ¶
type OutputProcessConfig struct { // AlwaysUpdate makes the comparison always update the golden file. AlwaysUpdate bool // KeepVolatileData indicates whether to keep or replace frequently // changing data. KeepVolatileData bool // VolatileRegexReplacements defines regex replacements to be applied to the // golden file before comparison. VolatileRegexReplacements []VolatileRegexReplacement // VolatileDataFiles are files that contain volatile data and should get // post-processed to be more stable. This is only supported in directory // mode ([BashTest]) of golden bash testing, i.e., this will be ignored in // single file mode ([BashTestFile]). VolatileDataFiles []string // RelativeDestination is the relative path to the directory where the // output file will be stored. If not provided, then the output file is // stored in the current directory. RelativeDestination string }
OutputProcessConfig defines how to process the output before comparison.
type TransientField ¶
type TransientField struct { // Key is a representation of a [JSONPath]-like key. Here are some examples // of transient fields and how to override them in the comparison: // - "version" key in the root: ".version" // - "elapsed" key in the stats object in the root: ".stats.elapsed" // - "start" key in the stats object in the root: ".stats.start" // - "time" key in the first element of the solutions array in the root: ".solutions[0].time" // In general, use a dot (.) to recursively enter nested objects and // brackets ([]) to access array elements. // // [JSONPath]: https://goessner.net/articles/JsonPath/ Key string // Replacement is optional, and it is the value that is used to stabilize // the transient field. If a replacement is not provided for the key, the // stabilization happens according to the data type. For example, a // [time.Time] is replaced using [StableTime], a [time.Duration] is // replaced using [StableDuration], etc. You can use the constants provided // by this package to stabilize the transient fields. Replacement any }
TransientField represents a field that is transient, this is, dynamic in nature. Examples of such fields include durations, times, versions, etc. Transient fields are replaced in golden file tests to always obtain the same result regardless of the moment it is executed. If a dynamic field always has a static value, then the golden file tests can run successfully. The transient field is represented by a key and a replacement. Please see the documentation of the fields for more information.
type Tresholds ¶
type Tresholds struct { // Float is the threshold to be used when comparing floats. Float float64 // Int is the threshold to be used when comparing ints. Int int // Time is the threshold to be used when comparing times. Two times are // considered the same if the absolute difference between them, which is a // duration, is less than or equal to the given threshold. Time time.Duration // Duration is the threshold to be used when comparing durations. Duration time.Duration // CustomThresholds defines threshold for specific keys that override the // generic thresholds. CustomThresholds CustomThresholds }
Tresholds by data type to be used when comparing actual and expected. If the absolute difference between the two values is less than or equal to the given threshold, then we consider the two values to be equal.
type VolatileRegexReplacement ¶
type VolatileRegexReplacement struct { // Regex to match. Regex string // Replacement to apply. Replacement string }
VolatileRegexReplacement defines a regex replacement to be applied to the output before comparison.