my-test-infra

module
v0.0.0-...-97238d6 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 29, 2019 License: Apache-2.0

README

Kubernetes Test Infrastructure

Go Report Card GoDoc

The test-infra repository contains a collection of tools for testing Kubernetes and displaying Kubernetes tests results. See also CONTRIBUTING.md.

See the architecture diagram for an overview of how the different services interact.

Viewing test results

E2E Testing

Our e2e testing uses kubetest to build/deploy/test kubernetes clusters on various providers. Please see those documents for additional details about this tool as well as e2e testing generally.

Anyone can reconfigure our CI system with a test-infra PR that updates the appropriate files. Detailed instructions follow:

CI Job management

Create a new job

Bootstrap is deprecated, Please utilize the podutils to create new prowjobs.

Create a PR in this repo to add/update/remove a job or suite. Specifically you'll need to do the following:

  • Add the job to the appropriate section in config/jobs

    • Directory Structure:
      • In general for jobs for github.com/org/repo use config/jobs/org/repo/filename.yaml
      • For Kubernetes repos we also allow config/jobs/kubernetes/sig-foo/filename.yaml
      • We use basename of the config name as a key in the prow configmap, so the name of your config file need to be unique across the config subdir
    • Type of jobs:
      • Presubmit jobs run on unmerged code in PRs
      • Postsubmit jobs run after merging code
      • Periodic job run on a timed basis
      • You can find more prowjob definitions at how-to-add-new-jobs
    • A simple sample job uses podutil looks like:
      - name: foo-repo-presubmit-test
        decorate: true
        spec:
          containers:
          - image: gcr.io/k8s-testimages/kubekins-e2e:latest-master
            command:
            - /path/to/cmd
            args:
            - positional
            - --and
            - flags
      
  • Add the job name to the test_groups list in testgrid/config.yaml

    • Also the group to at least one dashboard_tab

The configs need to be sorted and kubernetes must be in sync with the security repo, or else presubmit will fail. You can run the script below to keep them valid:

hack/update-config.sh
Local testing

docker run your image locally, and mount in the repos you depend on.

Release branch jobs & Image validation jobs

Release branch jobs and image validation jobs are defined in test_config.yaml. We test different master/node image versions against multiple k8s branches on different features.

Those jobs are using channel based versions, current supported testing map is:

  • k8s-dev : master
  • k8s-beta : release-1.13
  • k8s-stable1 : release-1.12
  • k8s-stable2 : release-1.11
  • k8s-stable3 : release-1.10

Our build job will generate a ci/(channel-name) file pointer in gcs.

After you update test_config.yaml, please run

bazel run //experiment:generate_tests -- --yaml-config-path=experiment/test_config.yaml

to regenerate the job configs.

We are moving towards making more jobs to fit into the generated config.

Presubmit will tell you if you forget to do any of this correctly.

Merge your PR and @k8s-ci-robot will deploy your change automatically.

Update an existing job

Largely similar to creating a new job, except you can just modify the existing entries rather than adding new ones.

Update what a job does by editing its definition in config/jobs.

Update where the job appears on testgrid by changing testgrid/config.yaml.

Delete a job

The reverse of creating a new job: delete the appropriate entries in config/jobs and testgrid/config.yaml.

Merge your PR and @k8s-ci-robot will deploy your change automatically.

Building and testing the test-infra

We use Bazel to build and test the code in this repo. The commands bazel build //... and bazel test //... should be all you need for most cases. If you modify Go code, run ./hack/update-bazel.sh to keep BUILD.bazel files up-to-date.

Contributing Test Results

The Kubernetes project encourages organizations to contribute execution of e2e test jobs for a variety of platforms (e.g., Azure, rktnetes). For information about how to contribute test results, see Contributing Test Results.

Other Docs

Directories

Path Synopsis
config
jobs/kubernetes-security
genjobs automatically generates the security repo presubmits from the kubernetes presubmits
genjobs automatically generates the security repo presubmits from the kubernetes presubmits
def
configmap
configmap will write a configmap to --output from --data=name=/path/to/source
configmap will write a configmap to --output from --data=name=/path/to/source
experiment
bootstrap
bootstrap implements a drop-in-replacement for jenkins/bootstrap.py
bootstrap implements a drop-in-replacement for jenkins/bootstrap.py
ci-janitor
ci-janitor cleans up dedicated projects in k8s prowjob configs
ci-janitor cleans up dedicated projects in k8s prowjob configs
manual-trigger
manual-trigger triggers jenkins jobs based a specified github pull request
manual-trigger triggers jenkins jobs based a specified github pull request
gcsweb
ghcache
Package ghcache implements an HTTP cache optimized for caching responses from the GitHub API (http://192.168.1.167).
Package ghcache implements an HTTP cache optimized for caching responses from the GitHub API (http://192.168.1.167).
pkg/cov/junit/calculation
Package calculation calculates coverage through summarizing and filtering.
Package calculation calculates coverage through summarizing and filtering.
greenhouse implements a bazel remote cache service [1] supporting arbitrarily many workspaces stored within the same top level directory.
greenhouse implements a bazel remote cache service [1] supporting arbitrarily many workspaces stored within the same top level directory.
diskcache
Package diskcache implements disk backed cache storage for use in greenhouse
Package diskcache implements disk backed cache storage for use in greenhouse
diskutil
Package diskutil implements disk related utilities for greenhouse
Package diskutil implements disk related utilities for greenhouse
images
Package main / eks.go implements kubetest deployer interface for EKS.
Package main / eks.go implements kubetest deployer interface for EKS.
conformance
Package conformance implements conformance test kubetest code.
Package conformance implements conformance test kubetest code.
e2e
kubeadmdind
Package kubeadmdind implements a kubetest deployer based on the scripts in the github.com/kubernetes-sigs/kubeadm-dind-cluster repo.
Package kubeadmdind implements a kubetest deployer based on the scripts in the github.com/kubernetes-sigs/kubeadm-dind-cluster repo.
This is a label_sync tool, details in README.md
This is a label_sync tool, details in README.md
logexporter
cmd
maintenance
pkg
flagutil
Package flagutil contains utilities and interfaces shared between several test-infra commands.
Package flagutil contains utilities and interfaces shared between several test-infra commands.
ghclient
Package ghclient provides a github client that wraps go-github with retry logic, rate limiting, and depagination where necessary.
Package ghclient provides a github client that wraps go-github with retry logic, rate limiting, and depagination where necessary.
prow
apis/prowjobs/v1
Package v1 is the v1 version of the API.
Package v1 is the v1 version of the API.
client/clientset/versioned
This package has the automatically generated clientset.
This package has the automatically generated clientset.
client/clientset/versioned/fake
This package has the automatically generated fake clientset.
This package has the automatically generated fake clientset.
client/clientset/versioned/scheme
This package contains the scheme of the automatically generated clientset.
This package contains the scheme of the automatically generated clientset.
client/clientset/versioned/typed/prowjobs/v1
This package has the automatically generated typed clients.
This package has the automatically generated typed clients.
client/clientset/versioned/typed/prowjobs/v1/fake
Package fake has the automatically generated clients.
Package fake has the automatically generated clients.
clonerefs
Package clonerefs is a library for cloning references
Package clonerefs is a library for cloning references
cmd/checkconfig
checkconfig loads configuration for Prow to validate it
checkconfig loads configuration for Prow to validate it
cmd/gcsupload
gcsupload uploads the files and folders specified to GCS using the Prow-defined job configuration
gcsupload uploads the files and folders specified to GCS using the Prow-defined job configuration
cmd/initupload
initupload parses the logs from the clonerefs container and determines if that container was successful or not.
initupload parses the logs from the clonerefs container and determines if that container was successful or not.
cmd/tot
Tot vends (rations) incrementing numbers for use in builds.
Tot vends (rations) incrementing numbers for use in builds.
cmd/tot/fallbackcheck
fallbackcheck reports whether jobs in the provided prow deployment have fallback build numbers in GCS.
fallbackcheck reports whether jobs in the provided prow deployment have fallback build numbers in GCS.
commentpruner
Package commentpruner facilitates efficiently deleting bot comments as a reaction to webhook events.
Package commentpruner facilitates efficiently deleting bot comments as a reaction to webhook events.
config
Package config knows how to read and parse config.yaml.
Package config knows how to read and parse config.yaml.
config/secret
Package secret implements an agent to read and reload the secrets.
Package secret implements an agent to read and reload the secrets.
crier
Package crier reports finished prowjob status to git providers.
Package crier reports finished prowjob status to git providers.
cron
Package cron provides a wrapper of robfig/cron, which manages schedule cron jobs for horologium
Package cron provides a wrapper of robfig/cron, which manages schedule cron jobs for horologium
deck/jobs
Package jobs implements methods on job information used by Prow component deck
Package jobs implements methods on job information used by Prow component deck
entrypoint
Package entrypoint is a library that knows how to wrap a process and write it's output and exit code to disk
Package entrypoint is a library that knows how to wrap a process and write it's output and exit code to disk
errorutil
Package errorutil provides utilities for errors
Package errorutil provides utilities for errors
external-plugins/refresh
Refresh retries Github status updates for stale PR statuses.
Refresh retries Github status updates for stale PR statuses.
flagutil
Package flagutil contains utilities and interfaces shared between several Prow commands.
Package flagutil contains utilities and interfaces shared between several Prow commands.
gcsupload
Package gcsupload uploads artifacts to a GCS path resolved from job configuration
Package gcsupload uploads artifacts to a GCS path resolved from job configuration
genfiles
Package genfiles understands the .generated_files config file.
Package genfiles understands the .generated_files config file.
gerrit/adapter
Package adapter implements a controller that interacts with gerrit instances
Package adapter implements a controller that interacts with gerrit instances
gerrit/client
Package client implements a client that can handle multiple gerrit instances derived from http://github.com/andygrunwald/go-gerrit
Package client implements a client that can handle multiple gerrit instances derived from http://github.com/andygrunwald/go-gerrit
gerrit/reporter
Package reporter implements a reporter interface for gerrit
Package reporter implements a reporter interface for gerrit
git
Package git provides a client to plugins that can do git operations.
Package git provides a client to plugins that can do git operations.
git/localgit
Package localgit creates a local git repo that can be used for testing code that uses a git.Client.
Package localgit creates a local git repo that can be used for testing code that uses a git.Client.
github/reporter
Package reporter implements a reporter interface for github TODO(krzyzacy): move logic from report.go here
Package reporter implements a reporter interface for github TODO(krzyzacy): move logic from report.go here
initupload
Package initupload determines the output status of clone operations and posts that status along with artifacts and logs to cloud storage.
Package initupload determines the output status of clone operations and posts that status along with artifacts and logs to cloud storage.
jenkins
Package jenkins includes a client and the operational logic for managing Jenkins masters in prow.
Package jenkins includes a client and the operational logic for managing Jenkins masters in prow.
logrusutil
Package logrusutil implements some helpers for using logrus
Package logrusutil implements some helpers for using logrus
metrics
Package metrics contains utilities for working with metrics in prow.
Package metrics contains utilities for working with metrics in prow.
pjutil
Package pjutil contains helpers for working with ProwJobs.
Package pjutil contains helpers for working with ProwJobs.
pluginhelp
Package pluginhelp defines structures that represent plugin help information.
Package pluginhelp defines structures that represent plugin help information.
pluginhelp/externalplugins
Package externalplugins provides the plugin help components to be compiled into external plugin binaries.
Package externalplugins provides the plugin help components to be compiled into external plugin binaries.
pluginhelp/hook
Package hook provides the plugin help components to be compiled into the hook binary.
Package hook provides the plugin help components to be compiled into the hook binary.
plugins/blockade
Package blockade defines a plugin that adds the 'do-not-merge/blocked-paths' label to PRs that modify protected file paths.
Package blockade defines a plugin that adds the 'do-not-merge/blocked-paths' label to PRs that modify protected file paths.
plugins/buildifier
Package buildifier defines a Prow plugin that runs buildifier over modified BUILD, WORKSPACE, and skylark (.bzl) files in pull requests.
Package buildifier defines a Prow plugin that runs buildifier over modified BUILD, WORKSPACE, and skylark (.bzl) files in pull requests.
plugins/cat
Package cat adds cat images to an issue or PR in response to a /meow comment
Package cat adds cat images to an issue or PR in response to a /meow comment
plugins/cherrypickunapproved
Package cherrypickunapproved adds the `do-not-merge/cherry-pick-not-approved` label to PRs against a release branch which do not have the `cherry-pick-approved` label.
Package cherrypickunapproved adds the `do-not-merge/cherry-pick-not-approved` label to PRs against a release branch which do not have the `cherry-pick-approved` label.
plugins/dco
Package dco implements a DCO (http://developercertificate.org/) checker plugin
Package dco implements a DCO (http://developercertificate.org/) checker plugin
plugins/docs-no-retest
Package docsnoretest contains a Prow plugin which manages a label indicating whether a given pull requests only changes documentation.
Package docsnoretest contains a Prow plugin which manages a label indicating whether a given pull requests only changes documentation.
plugins/dog
Package dog adds dog images to the issue or PR in response to a /woof comment
Package dog adds dog images to the issue or PR in response to a /woof comment
plugins/hold
Package hold contains a plugin which will allow users to label their own pull requests as not ready or ready for merge.
Package hold contains a plugin which will allow users to label their own pull requests as not ready or ready for merge.
plugins/lgtm
Package lgtm implements the lgtm plugin
Package lgtm implements the lgtm plugin
plugins/milestone
Package milestone implements the `/milestone` command which allows members of the milestone maintainers team to specify a milestone to be applied to an Issue or PR.
Package milestone implements the `/milestone` command which allows members of the milestone maintainers team to specify a milestone to be applied to an Issue or PR.
plugins/milestonestatus
Package milestonestatus implements the `/status` command which allows members of the milestone maintainers team to specify a `status/*` label to be applied to an Issue or PR.
Package milestonestatus implements the `/status` command which allows members of the milestone maintainers team to specify a `status/*` label to be applied to an Issue or PR.
plugins/override
Package override supports the /override context command.
Package override supports the /override context command.
plugins/pony
Package pony adds pony images to the issue or PR in response to a /pony comment
Package pony adds pony images to the issue or PR in response to a /pony comment
plugins/require-matching-label
Package requirematchinglabel implements the `require-matching-label` plugin.
Package requirematchinglabel implements the `require-matching-label` plugin.
plugins/sigmention
Package sigmention recognize SIG '@' mentions and adds 'sig/*' and 'kind/*' labels as appropriate.
Package sigmention recognize SIG '@' mentions and adds 'sig/*' and 'kind/*' labels as appropriate.
plugins/size
Package size contains a Prow plugin which counts the number of lines changed in a pull request, buckets this number into a few size classes (S, L, XL, etc), and finally labels the pull request with this size.
Package size contains a Prow plugin which counts the number of lines changed in a pull request, buckets this number into a few size classes (S, L, XL, etc), and finally labels the pull request with this size.
plugins/skip
Package skip implements the `/skip` command which allows users to clean up commit statuses of non-blocking presubmits on PRs.
Package skip implements the `/skip` command which allows users to clean up commit statuses of non-blocking presubmits on PRs.
plugins/stage
Package stage defines a Prow plugin that defines the stage of the issue in the features process.
Package stage defines a Prow plugin that defines the stage of the issue in the features process.
plugins/welcome
Package welcome implements a prow plugin to welcome new contributors
Package welcome implements a prow plugin to welcome new contributors
plugins/wip
Package wip will label a PR a work-in-progress if the author provides a prefix to their pull request title to the same effect.
Package wip will label a PR a work-in-progress if the author provides a prefix to their pull request title to the same effect.
pod-utils/decorate
Package decorate is a library for adding to a user-provided PodSpec in order to create a full Pod that will fulfill a test job
Package decorate is a library for adding to a user-provided PodSpec in order to create a full Pod that will fulfill a test job
pod-utils/downwardapi
Package downwardapi declares the types used to expose job configuration to the jobs themselves
Package downwardapi declares the types used to expose job configuration to the jobs themselves
pod-utils/gcs
Package gcs handles uploading files and raw data to GCS and determines where in the GCS bucket data should go given a specific job specification
Package gcs handles uploading files and raw data to GCS and determines where in the GCS bucket data should go given a specific job specification
pod-utils/options
Package options abstracts the options loading flow for pod utilities
Package options abstracts the options loading flow for pod utilities
pod-utils/wrapper
Package wrapper contains utilities for the processes that wrap the test execution in a ProwJob test container
Package wrapper contains utilities for the processes that wrap the test execution in a ProwJob test container
pubsub/reporter
Package reporter contains helpers for publishing statues to Pub statuses in Github.
Package reporter contains helpers for publishing statues to Pub statuses in Github.
report
Package report contains helpers for writing comments and updating statuses in Github.
Package report contains helpers for writing comments and updating statuses in Github.
sidecar
Package sidecar is a library that knows how to report on the output of a process that writes its output and exit code to disk
Package sidecar is a library that knows how to report on the output of a process that writes its output and exit code to disk
spyglass
Package spyglass creates views for Prow job artifacts.
Package spyglass creates views for Prow job artifacts.
spyglass/lenses
Package lenses provides interfaces and methods necessary for implementing custom artifact viewers
Package lenses provides interfaces and methods necessary for implementing custom artifact viewers
spyglass/lenses/buildlog
Package buildlog provides a build log viewer for Spyglass
Package buildlog provides a build log viewer for Spyglass
spyglass/lenses/junit
Package junit provides a junit viewer for Spyglass
Package junit provides a junit viewer for Spyglass
spyglass/lenses/metadata
Package metadata provides a metadata viewer for Spyglass
Package metadata provides a metadata viewer for Spyglass
statusreconciler
Package statusreconciler ensures that changes to required presubmits do not cause PRs in flight to get stuck in the merge queue
Package statusreconciler ensures that changes to required presubmits do not cause PRs in flight to get stuck in the merge queue
tide
Package tide contains a controller for managing a tide pool of PRs.
Package tide contains a controller for managing a tide pool of PRs.
tide/history
Package history provides an append only, size limited log of recent actions that Tide has taken for each subpool.
Package history provides an append only, size limited log of recent actions that Tide has taken for each subpool.
robots
commenter
Commenter provides a way to --query for issues and append a --comment to matches.
Commenter provides a way to --query for issues and append a --comment to matches.
coverage/diff
Package diff calculates the difference of two coverage lists and produces a collection of individual coverage difference.
Package diff calculates the difference of two coverage lists and produces a collection of individual coverage difference.
coverage/downloader
Package downloader finds and downloads the coverage profile file from the latest healthy build stored in given gcs directory
Package downloader finds and downloads the coverage profile file from the latest healthy build stored in given gcs directory
pr-labeler
PR labeler provides a way to add a missing ok-to-test label on trusted PRs.
PR labeler provides a way to add a missing ok-to-test label on trusted PRs.
testgrid
velodrome
sql

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL