test-infra

module
v0.0.0-...-fe50fa5 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 8, 2018 License: Apache-2.0

README

Kubernetes Test Infrastructure

Build Status Go Report Card GoDoc

The test-infra repository contains a collection of tools for testing Kubernetes and displaying Kubernetes tests results. See also CONTRIBUTING.md.

See the architecture diagram for an overview of how the different services interact.

Viewing test results

Automated testing

Test anything with the following pattern:

git clone https://github.com/kubernetes/test-infra
test-infra/jenkins/bootstrap.py --job=J --repo=R --service-account=S.json --upload=gs://B

The --job=J flag specifies what test job to run. The --repo=R (or --bare) flag controls what we check out from git.

Anyone can reconfigure our CI system with a test-infra PR that updates the appropriate files. Detailed instructions follow:

E2E Testing

Our e2e testing uses kubetest to build/deploy/test kubernetes clusters on various providers. Please see those documents for additional details about this tool as well as e2e testing generally.

Create a new job

Create a PR in this repo to add/update/remove a job or suite. Specifically you'll need to do the following:

  • Create an entry in jobs/config.json for the job
    • If this is a kubetest job create the corresponding jobs/env/FOO.env file
    • It will pick a free project from boskos pool by default, or
    • You can also set --gcp-project=foo in jobs/config.json for a dedicated project, make sure the project has the right IAM grants
  • Add the job name to the test_groups list in testgrid/config.yaml
    • Also the group to at least one dashboard_tab
  • Add the job to the appropriate section in prow/config.yaml
    • Presubmit jobs run on unmerged code in PRs
    • Postsubmit jobs run after merging code
    • Periodic job run on a timed basis

The configs need to be sorted and kubernetes must be in sync with the security repo, or else presubmit will fail. You can run the script below to keep them valid:

hack/update-config.sh

NOTE: kubernetes/kubernetes and kubernetes-security/kubernetes must have matching presubmits.

Please test the job on your local workstation before creating a PR:

mkdir /tmp/whatever && cd /tmp/whatever
$GOPATH/src/k8s.io/test-infra/jenkins/bootstrap.py \
  --job=J \  # aka your new job
  --repo=R1 --repo=R2 \  # what repos to check out
  --service-account ~/S.json  # the service account to use to launch GCE/GKE clusters
# Note: create a service account at the cloud console for the project J uses
Release branch jobs & Image validation jobs

Release branch jobs and image validation jobs are defined in test_config.yaml. We test different master/node image versions against multiple k8s branches on different features.

Those jobs are using channel based versions, current supported testing map is:

  • k8s-dev : master
  • k8s-beta : master (inactive)
  • k8s-stable1 : release-1.10
  • k8s-stable2 : release-1.9
  • k8s-stable3 : release-1.8

Our build job will generate a ci/(channel-name) file pointer in gcs.

After you update test_config.yaml, please run

bazel run //experiment:generate_tests -- --yaml-config-path=experiment/test_config.yaml --json-config-path=jobs/config.json  --prow-config-path=prow/config.yaml && bazel run //jobs:config_sort

to regenerate the job configs.

We are moving towards making more jobs to fit into the generated config.

Presubmit will tell you if you forget to do any of this correctly.

Merge your PR and @k8s-ci-robot will deploy your change automatically.

Update an existing job

Largely similar to creating a new job, except you can just modify the existing entries rather than adding new ones.

Update what a job does by editing its definition in jobs/config.json. For the kubetest jobs this typically means editing the jobs/FOO.env files it uses.

Update when a job runs by changing its definition in prow/config.yaml. The test-infra oncall must push prow changes (make -C prow update-config).

Update where the job appears on testgrid by changing testgrid/config.yaml.

Delete a job

The reverse of creating a new job: delete the appropriate entries in jobs/config.json, prow/config.yaml and testgrid/config.yaml.

Merge your PR and @k8s-ci-robot will deploy your change automatically.

Building and testing the test-infra

We use Bazel to build and test the code in this repo. The commands bazel build //... and bazel test //... should be all you need for most cases. If you modify Go code, run ./hack/update-bazel.sh to keep BUILD.bazel files up-to-date.

Contributing Test Results

The Kubernetes project encourages organizations to contribute execution of e2e test jobs for a variety of platforms (e.g., Azure, rktnetes). For information about how to contribute test results, see Contributing Test Results.

Other Docs

Directories

Path Synopsis
def
configmap
configmap will write a configmap to --output from --data=name=/path/to/source
configmap will write a configmap to --output from --data=name=/path/to/source
dind
experiment
bootstrap
bootstrap implements a drop-in-replacement for jenkins/bootstrap.py
bootstrap implements a drop-in-replacement for jenkins/bootstrap.py
config-updater
config-updater watches for merged PRs which update a set of files and update the corresponding files in a given deployment
config-updater watches for merged PRs which update a set of files and update the corresponding files in a given deployment
manual-trigger
manual-trigger triggers jenkins jobs based a specified github pull request
manual-trigger triggers jenkins jobs based a specified github pull request
refresh
Refresh retries Github status updates for stale PR statuses.
Refresh retries Github status updates for stale PR statuses.
gcsweb
Package ghclient provides a github client that wraps go-github with retry logic, rate limiting, and depagination where necessary.
Package ghclient provides a github client that wraps go-github with retry logic, rate limiting, and depagination where necessary.
greenhouse implements a bazel remote cache service [1] supporting arbitrarily many workspaces stored within the same top level directory.
greenhouse implements a bazel remote cache service [1] supporting arbitrarily many workspaces stored within the same top level directory.
diskcache
Package diskcache implements disk backed cache storage for use in greenhouse
Package diskcache implements disk backed cache storage for use in greenhouse
diskutil
Package diskutil implements disk related utilities for greenhouse
Package diskutil implements disk related utilities for greenhouse
images
Package main / gke.go provides the Google Container Engine (GKE) kubetest deployer via newGKE().
Package main / gke.go provides the Google Container Engine (GKE) kubetest deployer via newGKE().
conformance
Package conformance implements conformance test kubetest code.
Package conformance implements conformance test kubetest code.
dind
Package dind implements dind-specific kubetest code.
Package dind implements dind-specific kubetest code.
This is a label_sync tool, details in README.md
This is a label_sync tool, details in README.md
logexporter
cmd
maintenance
fixconfig
fixconfig automatically fixes the prow config to have automatically generated security repo presubmits transformed from the kubernetes presubmits
fixconfig automatically fixes the prow config to have automatically generated security repo presubmits transformed from the kubernetes presubmits
mungers
Package mungers deals with munging of github pull requests
Package mungers deals with munging of github pull requests
prow
clonerefs
Package clonerefs is a library for cloning references
Package clonerefs is a library for cloning references
cmd/config
Package main knows how to validate config files.
Package main knows how to validate config files.
cmd/gcsupload
gcsupload uploads the files and folders specified to GCS using the Prow-defined job configuration
gcsupload uploads the files and folders specified to GCS using the Prow-defined job configuration
cmd/initupload
initupload parses the logs from the clonerefs container and determines if that container was successful or not.
initupload parses the logs from the clonerefs container and determines if that container was successful or not.
cmd/tot
Tot vends (rations) incrementing numbers for use in builds.
Tot vends (rations) incrementing numbers for use in builds.
cmd/tot/fallbackcheck
fallbackcheck reports whether jobs in the provided prow deployment have fallback build numbers in GCS.
fallbackcheck reports whether jobs in the provided prow deployment have fallback build numbers in GCS.
commentpruner
Package commentpruner facilitates efficiently deleting bot comments as a reaction to webhook events.
Package commentpruner facilitates efficiently deleting bot comments as a reaction to webhook events.
config
Package config knows how to read and parse config.yaml.
Package config knows how to read and parse config.yaml.
cron
Package cron provides a wrapper of robfig/cron, which manages schedule cron jobs for horologium
Package cron provides a wrapper of robfig/cron, which manages schedule cron jobs for horologium
entrypoint
Package entrypoint is a library that knows how to wrap a process and write it's output and exit code to disk
Package entrypoint is a library that knows how to wrap a process and write it's output and exit code to disk
errorutil
errorutil provides utilities for errors
errorutil provides utilities for errors
external-plugins/cherrypicker
Cherrypicker automates cherry-picking of merged PRs in master to release branches.
Cherrypicker automates cherry-picking of merged PRs in master to release branches.
gcsupload
Package gcsupload uploads artifacts to a GCS path resolved from job configuration
Package gcsupload uploads artifacts to a GCS path resolved from job configuration
genfiles
Package genfiles understands the .generated_files config file.
Package genfiles understands the .generated_files config file.
gerrit
Package gerrit implements a gerrit-fetcher using https://github.com/andygrunwald/go-gerrit
Package gerrit implements a gerrit-fetcher using https://github.com/andygrunwald/go-gerrit
git
Package git provides a client to plugins that can do git operations.
Package git provides a client to plugins that can do git operations.
git/localgit
Package localgit creates a local git repo that can be used for testing code that uses a git.Client.
Package localgit creates a local git repo that can be used for testing code that uses a git.Client.
initupload
Package initupload determines the output status of clone operations and posts that status along with artifacts and logs to cloud storage
Package initupload determines the output status of clone operations and posts that status along with artifacts and logs to cloud storage
jenkins
Package jenkins includes a client and the operational logic for managing Jenkins masters in prow.
Package jenkins includes a client and the operational logic for managing Jenkins masters in prow.
logrusutil
Package logrusutil implements some helpers for using logrus
Package logrusutil implements some helpers for using logrus
metrics
Package metrics contains utilities for working with metrics in prow.
Package metrics contains utilities for working with metrics in prow.
pjutil
Package pjutil contains helpers for working with ProwJobs.
Package pjutil contains helpers for working with ProwJobs.
pluginhelp
Package pluginhelp defines structures that represent plugin help information.
Package pluginhelp defines structures that represent plugin help information.
pluginhelp/externalplugins
Package externalplugins provides the plugin help components to be compiled into external plugin binaries.
Package externalplugins provides the plugin help components to be compiled into external plugin binaries.
pluginhelp/hook
Package hook provides the plugin help components to be compiled into the hook binary.
Package hook provides the plugin help components to be compiled into the hook binary.
plugins/blockade
Package blockade defines a plugin that adds the 'do-not-merge/blocked-paths' label to PRs that modify protected file paths.
Package blockade defines a plugin that adds the 'do-not-merge/blocked-paths' label to PRs that modify protected file paths.
plugins/buildifier
buildifier defines a Prow plugin that runs buildifier over modified BUILD, WORKSPACE, and skylark (.bzl) files in pull requests.
buildifier defines a Prow plugin that runs buildifier over modified BUILD, WORKSPACE, and skylark (.bzl) files in pull requests.
plugins/cat
Package cat adds cat images to issues in response to a /meow comment
Package cat adds cat images to issues in response to a /meow comment
plugins/docs-no-retest
Package docsnoretest contains a Prow plugin which manages a label indicating whether a given pull requests only changes documentation.
Package docsnoretest contains a Prow plugin which manages a label indicating whether a given pull requests only changes documentation.
plugins/dog
Package dog adds dog images to issues in response to a /woof comment
Package dog adds dog images to issues in response to a /woof comment
plugins/hold
Package hold contains a plugin which will allow users to label their own pull requests as not ready or ready for merge.
Package hold contains a plugin which will allow users to label their own pull requests as not ready or ready for merge.
plugins/milestone
Package setmilestone implements the `/milestone` command which allows members of the milestone maintainers team to specify a milestone to be applied to an Issue or PR.
Package setmilestone implements the `/milestone` command which allows members of the milestone maintainers team to specify a milestone to be applied to an Issue or PR.
plugins/milestonestatus
Package milestonestatus implements the `/status` command which allows members of the milestone maintainers team to specify a `status/*` label to be applied to an Issue or PR.
Package milestonestatus implements the `/status` command which allows members of the milestone maintainers team to specify a `status/*` label to be applied to an Issue or PR.
plugins/sigmention
Package sigmention recognize SIG '@' mentions and adds 'sig/*' and 'kind/*' labels as appropriate.
Package sigmention recognize SIG '@' mentions and adds 'sig/*' and 'kind/*' labels as appropriate.
plugins/size
Package size contains a Prow plugin which counts the number of lines changed in a pull request, buckets this number into a few size classes (S, L, XL, etc), and finally labels the pull request with this size.
Package size contains a Prow plugin which counts the number of lines changed in a pull request, buckets this number into a few size classes (S, L, XL, etc), and finally labels the pull request with this size.
plugins/skip
Package skip implements the `/skip` command which allows users to clean up commit statuses of non-blocking presubmits on PRs.
Package skip implements the `/skip` command which allows users to clean up commit statuses of non-blocking presubmits on PRs.
plugins/wip
Package wip will label a PR a work-in-progress if the author provides a prefix to their pull request title to the same effect.
Package wip will label a PR a work-in-progress if the author provides a prefix to their pull request title to the same effect.
pod-utils/decorate
decorate is a library for adding to a user-provided PodSpec in order to create a full Pod that will fulfill a test job
decorate is a library for adding to a user-provided PodSpec in order to create a full Pod that will fulfill a test job
pod-utils/downwardapi
downwardapi declares the types used to expose job configuration to the jobs themselves
downwardapi declares the types used to expose job configuration to the jobs themselves
pod-utils/gcs
gcs handles uploading files and raw data to GCS and determines where in the GCS bucket data should go given a specific job specification
gcs handles uploading files and raw data to GCS and determines where in the GCS bucket data should go given a specific job specification
pod-utils/options
Package options abstracts the options loading flow for pod utilities
Package options abstracts the options loading flow for pod utilities
pod-utils/wrapper
wrapper contains utilities for the processes that wrap the test execution in a ProwJob test container
wrapper contains utilities for the processes that wrap the test execution in a ProwJob test container
report
Package report contains helpers for writing comments and updating statuses in Github.
Package report contains helpers for writing comments and updating statuses in Github.
sidecar
Package sidecar is a library that knows how to report on the output of a process that writes its output and exit code to disk
Package sidecar is a library that knows how to report on the output of a process that writes its output and exit code to disk
tide
Package tide contains a controller for managing a tide pool of PRs.
Package tide contains a controller for managing a tide pool of PRs.
robots
commenter
Commenter provides a way to --query for issues and append a --comment to matches.
Commenter provides a way to --query for issues and append a --comment to matches.
testgrid
velodrome
sql

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL