README
¶
Simple Conformance Tests
The simple test suite checks end-to-end functionality of the entire parse-(check)-eval pipeline. Test inputs and output are fully specified. The parse and check phases are expected to complete without error and their outputs are not individually validated - we only check the final results of evaluation.
The simple suite is executed by the driver binary in simple_test.go.
This binary is invoked with one or more of the following flags:
--serverPath to the default ConformanceService server binary, to be used when no phase-specific server is specified.--parse_serverPath to the ConformanceService server binary to use for parsing.--check_serverPath to the ConformanceService server binary to use for checking.--eval_serverPath to the ConformanceService server binary to use for evaluation.
The remaining arguments are paths to data files containing the test inputs and expected outputs.
The driver binary should be run from this directory, looking in
the testdata/ subdirectory for its data files.
Implementations which use the bazel build system
can invoke the simple test suite as their own unit tests with the following
entry in a BUILD.bazel file, substituting their own conformance server
binary target for //server/main:cel_server:
sh_test(
name = "conformance_simple",
srcs = ["@com_google_cel_spec//tests:conftest.sh"],
args = [
"$(location @com_google_cel_spec//tests/simple:simple_test)",
"--server=$(location //server/main:cel_server)",
"$(location @com_google_cel_spec//tests/simple:testdata/plumbing.textproto)",
"$(location @com_google_cel_spec//tests/simple:testdata/basic.textproto)",
"$(location @com_google_cel_spec//tests/simple:testdata/integer_math.textproto)",
],
data = [
"@com_google_cel_spec//tests/simple:simple_test",
"//server/main:cel_server",
"@com_google_cel_spec//tests/simple:testdata/plumbing.textproto",
"@com_google_cel_spec//tests/simple:testdata/basic.textproto",
"@com_google_cel_spec//tests/simple:testdata/integer_math.textproto",
],
)
Implementations should add more test data files as they increase their conformance. See testdata for a description of the individual test files.
Documentation
¶
Overview ¶
Package simple runs end-to-end CEL conformance tests against ConformanceService servers. The "simple" tests run the Parse / Check (optional) / Eval pipeline and compare the result against an expected value, error, or unknown from the Eval phase. To validate the intermediate results from the Parse or Check phases, use a different test driver.
Each phase can be sent to a different ConformanceService server. Thus a partial implementation can be tested by using other implementations for the missing phases. This also validates the interoperativity.
Example test data:
name: "basic"
description: "Basic tests that all implementations should pass."
section {
name: "self_eval"
description: "Simple self-evaluating forms."
test {
name: "self_eval_zero"
expr: "0"
value: { int64_value: 0 }
}
}
section {
name: "arithmetic"
description: "Numeric arithmetic checks."
test {
name: "one plus one"
description: "Uses implicit match against 'true'."
expr: "1 + 1 == 2"
}
}
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Match ¶
func Match(t *spb.SimpleTest, actual *exprpb.ExprValue) error
Match checks the expectation in the result matcher against the actual result of evaluation. Returns nil if the expectation matches the actual, otherwise returns an error describing the difference. Calling this function implies that the interpretation succeeded in the parse and check phases. See MatchValue() for the normalization applied to values for matching.
func MatchValue ¶
MatchValue returns whether the actual value is equal to the expected value, modulo the following normalization:
- All floating-point NaN values are equal.
- Map comparisons ignore order.
func ValidateTest ¶
func ValidateTest(t *spb.SimpleTest) error
ValidateTest checks whether a simple test has the required fields.
Types ¶
This section is empty.
Source Files