README
¶
This directory contains end to end tests of LUCI CV.
These tests are designed to run locally with fakes instead of Gerrit, Datastore, Cloud Task Queue.
How to run?
The usual Go way, e.g. go test ./... or go test go.chromium.org/luci/cv/tests/e2e/...
How to debug?
Select failing test, e.g. TestXYZ. Run just this test in verbose mode,
which emits copious amounts of logging, as if you are looking at production CV
app, except log lines not attributed nor grouped to individual task queue
executions.
go test -v -run TestXYZ
To make sense of logs, it may be easiest to save them to a file:
go test -v -run TestXYZ 2>&1 >out.log
If test seems to hang, save your time:
go test -v -run TestXYZ -test.timeout=1s 2>&1 >out.log
If test is flaky, try:
go test -v -run TestXYZ -test.count=1000 -test.failfast 2>&1 >out.log
Guidance on writing new tests
- Actual tests should be added into
..._test.gofiles, as usual. - Prefer re-using existing project config setups, instead of generating new ones.
- Mark each
TestXYZfunc witht.Parallel().- Corollary: in rare cases where your test really must not be run in
parallel, create a dedicated sub-package for just your test and change
global vars as necessary.
e2epackage is intentionally exportable.
- Corollary: in rare cases where your test really must not be run in
parallel, create a dedicated sub-package for just your test and change
global vars as necessary.
- Helper functions should be added to
e2e.go. They should be public (see above). - Use exactly 1 Convey block inside each
TestXYZfunction. - Avoid deeply nested Convey blocks. Ideally there is 1 top level only.
- Rationale: easier to debug, since each section may produce substantial amount of logs, which are harder to attribute to individual sub-blocks.
- Corollary: moderate copy-pasta is fine.
- Tests should be fast, finishing with 1s even on under-powered laptops.
Advanced: run with flaky Datastore:
Use special -cv.dsflakiness flag, e.g. go test ./... -cv.dsflakiness=0.06.
See flakifyDS function for explanation of what 0.06 value actually means and
the shortcomings of the simulation.
Documentation
¶
Overview ¶
Package e2e contains all CV end-to-end tests.
Index ¶
- func MakeCfgCombinable(cgName, gHost, gRepo, gRef string, builders ...*cfgpb.Verifiers_Tryjob_Builder) *cfgpb.Config
- func MakeCfgSingular(cgName, gHost, gRepo, gRef string, builders ...*cfgpb.Verifiers_Tryjob_Builder) *cfgpb.Config
- type Test
- func (t *Test) AddCommitter(email string)
- func (t *Test) AddDryRunner(email string)
- func (t *Test) AddNewPatchsetRunner(email string)
- func (t *Test) EarliestCreatedRunOf(ctx context.Context, lProject string) *run.Run
- func (t *Test) ExportedBQAttemptsCount() int
- func (t *Test) LastMessage(gHost string, gChange int64) *gerritpb.ChangeMessageInfo
- func (t *Test) LatestRunWithGerritCL(ctx context.Context, gHost string, gChange int64) *run.Run
- func (t *Test) LoadCL(ctx context.Context, id common.CLID) *changelist.CL
- func (t *Test) LoadGerritCL(ctx context.Context, gHost string, gChange int64) *changelist.CL
- func (t *Test) LoadGerritRuns(ctx context.Context, gHost string, gChange int64) []*run.Run
- func (t *Test) LoadProject(ctx context.Context, lProject string) *prjmanager.Project
- func (t *Test) LoadRun(ctx context.Context, id common.RunID) *run.Run
- func (t *Test) LoadRunsOf(ctx context.Context, lProject string) []*run.Run
- func (t *Test) LogPhase(ctx context.Context, format string, args ...any)
- func (t *Test) MaxCQVote(ctx context.Context, gHost string, gChange int64) int32
- func (t *Test) MaxVote(ctx context.Context, gHost string, gChange int64, gLabel string) int32
- func (t *Test) Now() time.Time
- func (t *Test) RunEndedPubSubTasks() tqtesting.TaskList
- func (t *Test) RunUntil(ctx context.Context, stopIf func() bool)
- func (t *Test) RunUntilT(ctx context.Context, targetTasksCount int, stopIf func() bool)
- func (t *Test) SetUp(testingT *testing.T) (context.Context, func())
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func MakeCfgCombinable ¶
func MakeCfgCombinable(cgName, gHost, gRepo, gRef string, builders ...*cfgpb.Verifiers_Tryjob_Builder) *cfgpb.Config
MakeCfgCombinable return project config with a combinable ConfigGroup.
func MakeCfgSingular ¶
func MakeCfgSingular(cgName, gHost, gRepo, gRef string, builders ...*cfgpb.Verifiers_Tryjob_Builder) *cfgpb.Config
MakeCfgSingular return project config with a single ConfigGroup.
Types ¶
type Test ¶
type Test struct {
*cvtesting.Test // auto-initialized if nil
PMNotifier *prjmanager.Notifier
RunNotifier *run.Notifier
AdminServer adminpb.AdminServer
// contains filtered or unexported fields
}
Test encapsulates e2e setup for a CV test.
Embeds cvtesting.Test, which sets CV's dependencies and some simple CV components (e.g. TreeClient), while this Test focuses on setup of CV's own components.
Typical use:
ct := Test{CVDev: true}
ctx, cancel := ct.SetUp(t)
defer cancel()
...
ct.RunUntil(ctx, func() bool { return len(ct.LoadRunsOf("project")) > 0 })
func (*Test) AddCommitter ¶
AddCommitter adds a given member into the committer group.
func (*Test) AddDryRunner ¶
AddDryRunner adds a given member into the dry-runner group.
func (*Test) AddNewPatchsetRunner ¶
AddNewPatchsetRunner adds a given member into the new-patchset-runner group.
func (*Test) EarliestCreatedRunOf ¶
EarliestCreatedRunOf returns the earliest created Run in a project.
If there are several such runs, may return any one of them.
Returns nil if there are no Runs.
func (*Test) ExportedBQAttemptsCount ¶
ExportedBQAttemptsCount returns number of exported CQ Attempts.
func (*Test) LastMessage ¶
func (t *Test) LastMessage(gHost string, gChange int64) *gerritpb.ChangeMessageInfo
LastMessage returns the last message posted on a Gerrit CL from Gerrit fake.
Returns nil if there are no messages. Panics if the CL doesn't exist.
func (*Test) LatestRunWithGerritCL ¶
LatestRunWithGerritCL returns the latest created Run containing given CL.
If there are several, returns the one with latest .StartTime. Returns nil if there is such Runs, including if Gerrit CL isn't yet in DS.
func (*Test) LoadGerritCL ¶
LoadGerritCL returns CL entity or nil if not exists.
func (*Test) LoadGerritRuns ¶
LoadGerritRuns loads all Runs from Datastore which include a Gerrit CL.
func (*Test) LoadProject ¶
LoadProject returns Project entity or nil if not exists.
func (*Test) LoadRunsOf ¶
LoadRunsOf loads all Runs of a project from Datastore.
func (*Test) LogPhase ¶
LogPhase emits easy to recognize log like =========================== PHASE: .... ===========================
func (*Test) MaxCQVote ¶
MaxCQVote returns max CQ vote of a Gerrit CL loaded from Gerrit fake.
Returns 0 if there are no votes. Panics if CL doesn't exist.
func (*Test) MaxVote ¶
MaxVote returns max vote of a Gerrit CL loaded from Gerrit fake.
Returns 0 if there are no votes. Panics if CL doesn't exist.
func (*Test) RunEndedPubSubTasks ¶
RunEndedPubSubTasks returns all the succeeded TQ tasks with RunEnded pubsub events.
func (*Test) RunUntil ¶
RunUntil runs TQ tasks, while stopIf returns false.
If `dsFlakinessFlag` is set, uses flaky datastore for running TQ tasks. If `tqParallelFlag` is set, runs TQ tasks concurrently.
Not goroutine safe.
Source Files