The CEL conformance test suites provide a complementary specification
of the CEL language in the form of executable software. All CEL
implementations which pass the conformance tests should give valid CEL
expressions the same meaning.
A language-independent gRPC protocol allows the tests to run against CEL
components in any language which implement a server for the protocol.
As much as possible, the tests are driven by data files. Implementations
which do not provide a ConformanceService server are welcome to write
their own drivers which read the same testdata
files.
Test Suites
We currently have the following test suites:
- simple The simple tests check the end-to-end
parse-(check)-eval pipeline for fully-specified expressions,
inputs, and output without parse or check failures.
- envcheck The envcheck suite confirms that the checker and
runtime support a set of functions and overloads (though we don't check
the behavior of these functions in this suite).
- Test suites more suitable for other kinds of validation may be introduced
later.
Integrating with the Tests
Implementations should write a gRPC server for the
ConformanceService
API
See the cel-go server
as an example.
The server should be available as an executable file. When invoked without
without arguments, the server should listen on an arbitrary available TCP
port on the local loopback address (127.0.0.1
or ::1
), then write its
address on stdout
in the format
Listening on
address:
port
See the celrpc
library for an
example of implementing this protocol.
An implementation with only a subset of the execution phases (particularly
runtime-only implementations without a parser or checker) is free to
support only the corresponding API methods. The test suite driver can be
instructed to use a different implementation for the unsupported methods.
Each conformance test suite will contain specific integration instructions.
Implementations which use the bazel build system
should copy the relevant entries out of the WORKSPACE
file at the root
of this repository.
Each CEL implementation should also have its own unit tests and benchmarks
for testing subcomponents, its API, and other implementation-dependent
features.