ztest

Overview ¶

Package ztest runs formulaic tests ("ztests") that can be (1) run in-process with the compiled-ini zq code base, (2) run as a sub-process using the zq executable build artifact, or (3) run as a bash script running a sequence of arbitrary shell commands invoking any of the build artifacts. The first two cases comprise the "Zed test style" and the last case comprises the "script test style". Case (1) is easier to debug by simply running "go test" compared replicating the test using "go run". Script-style tests don't have this convenience.

In the Zed style, ztest runs a Zed program on an input and checks for an expected output.

A Zed-style test is defined in a YAML file.

zed: count()

input: |
  #0:record[i:int64]
  0:[1;]
  0:[2;]

output: |
  #0:record[count:uint64]
  0:[2;]

Input format is detected automatically and can be anything recognized by "zq -i auto" (including optional gzip compression). Output format defaults to tzng but can be set to anything accepted by "zq -f".

zed: count()

input: |
  #0:record[i:int64]
  0:[1;]
  0:[2;]

output-flags: -f table

output: |
  count
  2

Alternatively, tests can be configured to run as shell scripts. In this style of test, arbitrary bash scripts can run chaining together any of the tools in cmd/ in addition to zq. Scripts are executed by "bash -e -o pipefail", and a nonzero shell exit code causes a test failure, so any failed command generally results in a test failure. Here, the yaml sets up a collection of input files and stdin, the script runs, and the test driver compares expected output files, stdout, and stderr with data in the yaml spec. In this case, instead of specifying, "zed", "input", "output", you specify the yaml arrays "inputs" and "outputs" --- where each array element defines a file, stdin, stdout, or stderr --- and a "script" that specifies a multi-line yaml string defining the script, e.g.,

inputs:

• name: in1.tzng data: | #0:record[i:int64] 0:[1;]
• name: stdin data: | #0:record[i:int64] 0:[2;]

script: |

zq -o out.tzng in1.tzng -
zq -o count.tzng "count()" out.tzng

outputs:

• name: out.tzng data: | #0:record[i:int64] 0:[1;] 0:[2;]
• name: count.tzng data: | #0:record[count:uint64] 0:[2;]

Each input and output has a name. For inputs, a file (source) or inline data (data) may be specified. If no data is specified, then a file of the same name as the name field is looked for in the same directory as the yaml file. The source spec is a file path relative to the directory of the yaml file. For outputs, expected output is defined in the same fashion as the inputs though you can also specify a "regexp" string instead of expected data. If an output is named "stdout" or "stderr" then the actual output is taken from the stdout or stderr of the the shell script.

Ztest YAML files for a package should reside in a subdirectory named testdata/ztest.

pkg/
  pkg.go
  pkg_test.go
  testdata/
    ztest/
      test-1.yaml
      test-2.yaml
      ...

Name YAML files descriptively since each ztest runs as a subtest named for the file that defines it.

pkg_test.go should contain a Go test named TestZTest that calls Run.

func TestZTest(t *testing.T) { ztest.Run(t, "testdata/ztest") }

If the ZTEST_PATH environment variable is unset or empty and the test is not a script test, Run runs ztests in the current process and skips the script tests. Otherwise, Run runs each ztest in a separate process using the zq executable in the directories specified by ZTEST_PATH.

Tests of either style can be skipped by setting the skip field to a non-empty string. A message containing the string will be written to the test log.