shared

package
v0.0.0-...-2504d47 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 26, 2024 License: BSD-3-Clause Imports: 46 Imported by: 2

Documentation

Index

Constants

View Source
const (
	// TestStatusUnknown is an uninitialized TestStatus and should
	// not be used.
	TestStatusUnknown TestStatus = 0

	// TestStatusPass indicates that all tests completed successfully and passed.
	TestStatusPass TestStatus = 1

	// TestStatusOK indicates that all tests completed successfully.
	TestStatusOK TestStatus = 2

	// TestStatusError indicates that some tests did not complete
	// successfully.
	TestStatusError TestStatus = 3

	// TestStatusTimeout indicates that some tests timed out.
	TestStatusTimeout TestStatus = 4

	// TestStatusNotRun indicates that a test was not run.
	TestStatusNotRun TestStatus = 5

	// TestStatusFail indicates that a test failed.
	TestStatusFail TestStatus = 6

	// TestStatusCrash indicates that the WPT test runner crashed attempting to run the test.
	TestStatusCrash TestStatus = 7

	// TestStatusSkip indicates that the test was disabled for this test run.
	TestStatusSkip TestStatus = 8

	// TestStatusAssert indicates that a non-fatal assertion failed. This test
	// status is supported by, at least, Mozilla.
	TestStatusAssert TestStatus = 9

	// TestStatusNameUnknown is the string representation for an uninitialized
	// TestStatus and should not be used.
	TestStatusNameUnknown string = "UNKNOWN"

	// TestStatusNamePass is the string representation of a test result where the
	// test passed.
	TestStatusNamePass string = "PASS"

	// TestStatusNameOK is the string represnetation of a test result where the
	// test ran completely but may not have passed (and/or not all of its subtests
	// passed).
	TestStatusNameOK string = "OK"

	// TestStatusNameError is the string representation for a test result where
	// a test harness error was encountered at test runtime.
	TestStatusNameError string = "ERROR"

	// TestStatusNameTimeout is the string representation for a test result where
	// the test timed out.
	TestStatusNameTimeout string = "TIMEOUT"

	// TestStatusNameNotRun is  the string representation for a test result where
	// the test exists but was not run.
	TestStatusNameNotRun string = "NOTRUN"

	// TestStatusNameFail is the string representation of a test result where the
	// test failed.
	TestStatusNameFail string = "FAIL"

	// TestStatusNameCrash is the string representation of a test result where the
	// test runner crashed.
	TestStatusNameCrash string = "CRASH"

	// TestStatusNameSkip is the string representation of a test result where the
	// test was disabled for this test run.
	TestStatusNameSkip string = "SKIP"

	// TestStatusNameAssert is the string representation of a test result where
	// a non-fatal assertion failed. This test status is supported by, at least,
	// Mozilla.
	TestStatusNameAssert string = "ASSERT"

	// TestStatusDefault is the default value used when a status string cannot be
	// interpreted.
	TestStatusDefault TestStatus = TestStatusUnknown

	// TestStatusNameDefault is the default string used when a status value cannot
	// be interpreted.
	TestStatusNameDefault string = TestStatusNameUnknown
)
View Source
const BetaLabel = "beta"

BetaLabel is the implicit label present for runs marked 'beta'.

View Source
const ExperimentalLabel = "experimental"

ExperimentalLabel is the implicit label present for runs marked 'experimental'.

View Source
const LatestSHA = "latest"

LatestSHA is a helper for the 'latest' keyword/special case.

View Source
const MasterLabel = "master"

MasterLabel is the implicit label present for runs marked 'master', i.e. run from the master branch.

View Source
const MaxCountMaxValue = 500

MaxCountMaxValue is the maximum allowed value for the max-count param.

View Source
const MaxCountMinValue = 1

MaxCountMinValue is the minimum allowed value for the max-count param.

View Source
const MaxKeysPerLookup = 1000

MaxKeysPerLookup is the max number of keys allowed per lookup (e.g. GetMulti). https://cloud.google.com/datastore/docs/concepts/limits

View Source
const MetadataFileName = "META.yml"

MetadataFileName is the name of Metadata files in the wpt-metadata repo.

View Source
const PRBaseLabel = "pr_base"

PRBaseLabel is the implicit label for running just the affected tests on a PR but without the changes (i.e. against the base branch).

View Source
const PRHeadLabel = "pr_head"

PRHeadLabel is the implicit label for running just the affected tests on the head of a PR (with the changes).

View Source
const PendingMetadataCacheKey = "WPT-PENDING-METADATA"

PendingMetadataCacheKey is the key for the set that stores a list of pending metadata PRs in Redis.

View Source
const PendingMetadataCachePrefix = "PENDING-PR-"

PendingMetadataCachePrefix is the key prefix for pending metadata stored in Redis.

View Source
const ShowMetadataParam = "metadataInfo"

ShowMetadataParam determines whether Metadata Information returns along with a test result query request.

View Source
const SourceOwner string = "web-platform-tests"

SourceOwner is the owner name of the wpt-metadata repo.

View Source
const SourceRepo string = "wpt-metadata"

SourceRepo is the wpt-metadata repo.

View Source
const StableLabel = "stable"

StableLabel is the implicit label present for runs marked 'stable'.

View Source
const UserLabelPrefix = "user:"

UserLabelPrefix is a prefix used to denote a label for a user's GitHub handle, prefixed because usernames are essentially user input.

View Source
const WPTRepoName = "wpt"

WPTRepoName is the repo name for the GitHub wpt repo.

View Source
const WPTRepoOwner = "web-platform-tests"

WPTRepoOwner is the owner (username) for the GitHub wpt repo.

View Source
const WebFeatureManifestRepo = "wpt"

WebFeatureManifestRepo is where the web feature manifest is published.

Variables

View Source
var (
	// ErrEntityAlreadyExists is returned by Datastore.Insert when the entity already exists.
	ErrEntityAlreadyExists = errors.New("datastore: entity already exists")
	// ErrNoSuchEntity is returned by Datastore.Get when the key doesn't exist.
	ErrNoSuchEntity = errors.New("datastore: entity not found")
)
View Source
var Clients clientsImpl

Clients is a singleton containing heavyweight (e.g. with connection pools) clients that should be bound to the runtime instead of each request in order to be reused. They are initialized and authenticated at startup using the background context; each request should use its own context.

View Source
var ErrInvalidManifest = errors.New("invalid manifest")

ErrInvalidManifest is the error returned when the manifest is a valid JSON but without the correct structure.

View Source
var ErrRunNotInSearchCache = errors.New("Run is still being loaded into the searchcache")

ErrRunNotInSearchCache is an error for 422 responses from the searchcache.

View Source
var SHARegex = regexp.MustCompile(`^[0-9a-fA-F]{7,40}$`)

SHARegex is the pattern for a valid SHA1 hash that's at least 7 characters long.

Functions

func AddRoute

func AddRoute(route, name string, h http.HandlerFunc) *mux.Route

AddRoute is a helper for registering a handler for an http path (route). Note that it adds an HSTS header to the response.

func AlwaysCachable

func AlwaysCachable(r *http.Request) bool

AlwaysCachable is a helper for returning true for all requests.

func CacheStatusOK

func CacheStatusOK(ctx context.Context, statusCode int, payload []byte) bool

CacheStatusOK is a hlper for indicating that a request is cacheable iff the status code is http.StatusOK.

func CropString

func CropString(s string, i int) string

CropString conditionally crops a string to the given length, if it is longer. Returns the original string otherwise.

func DeleteCache

func DeleteCache(key string) error

DeleteCache deletes the object stored at key in Redis. A key is ignored if it does not exist.

func ExplodePossibleFilenames

func ExplodePossibleFilenames(filePath string) []string

ExplodePossibleFilenames explodes the given single filename into the test names that could be created for it at runtime.

func ExplodePossibleRenames

func ExplodePossibleRenames(before, after string) map[string]string

ExplodePossibleRenames returns a map of equivalent renames for the given file rename.

func FetchJSON

func FetchJSON(url string, value interface{}) error

FetchJSON fetches the given URL, which is expected to be JSON, and unmarshals it into the given value pointer, fatally logging any errors.

func FlushCache

func FlushCache() error

FlushCache purges everything from Memorystore.

func GetDefaultBrowserNames

func GetDefaultBrowserNames() []string

GetDefaultBrowserNames returns an alphabetically-ordered array of the names of the browsers which are to be included by default.

func GetMetadataByteMap

func GetMetadataByteMap(log Logger, fetcher MetadataFetcher) (sha *string, metadata map[string]Metadata, err error)

GetMetadataByteMap collects and parses all META.yml files from the wpt-metadata repository.

func GetMetadataFilePath

func GetMetadataFilePath(folderName string) string

GetMetadataFilePath appends MetadataFileName to a Metadata folder path.

func GetResultsDiff

func GetResultsDiff(
	before ResultsSummary,
	after ResultsSummary,
	filter DiffFilterParam,
	paths mapset.Set,
	renames map[string]string) map[string]TestDiff

GetResultsDiff returns a map of test name to an array of [newly-passing, newly-failing, total-delta], for tests which had different results counts in their map (which is test name to array of [count-passed, total]).

func GetResultsURL

func GetResultsURL(run TestRun, testFile string) (resultsURL string)

GetResultsURL constructs the URL to the result of a single test file in the given run.

func GetSecret

func GetSecret(ds Datastore, tokenName string) (string, error)

GetSecret is a helper wrapper for loading a token's secret from the datastore by name.

func GetSharedPath

func GetSharedPath(paths ...string) string

GetSharedPath gets the longest path shared between the given paths.

func GetUserLabel

func GetUserLabel(username string) string

GetUserLabel prefixes the given username with the prefix for using as a label.

func GetWPTMetadataArchive

func GetWPTMetadataArchive(client *http.Client, ref *string) (res map[string][]byte, err error)

GetWPTMetadataArchive iterates through wpt-metadata repository and returns a map that maps a test path to its META.yml file content, using a given ref.

func GetWPTMetadataMasterSHA

func GetWPTMetadataMasterSHA(ctx context.Context, gitHubClient *github.Client) (*string, error)

GetWPTMetadataMasterSHA returns the SHA of the master branch of the wpt-metadata repo.

func GetWPTTestPath

func GetWPTTestPath(folderPath string, testname string) string

GetWPTTestPath concatenates a folder path and a test name into a WPT test path.

func HandleWithLogging

func HandleWithLogging(h http.HandlerFunc) http.HandlerFunc

HandleWithLogging handles the request with the given handler, setting the logger on the request's context to be either a logrus logger (when running locally) or a Google Cloud logger (when running on GCP).

func IsBrowserName

func IsBrowserName(name string) bool

IsBrowserName determines whether the given name string is a valid browser name. Used for validating user-input params for browsers.

func IsFeatureEnabled

func IsFeatureEnabled(ds Datastore, flagName string) bool

IsFeatureEnabled returns true if a feature with the given flag name exists, and Enabled is set to true.

func IsLatest

func IsLatest(sha string) bool

IsLatest returns whether a SHA[0:10] is empty or "latest", both of which are treated as looking up the latest run for each browser.

func IsStableBrowserName

func IsStableBrowserName(name string) bool

IsStableBrowserName determines whether the given name string is a valid browser name of a stable browser (i.e. not using the -experimental suffix).

func MapStringKeys

func MapStringKeys(m interface{}) ([]string, error)

MapStringKeys returns the keys in the given string-keyed map.

func NewCachingHandler

func NewCachingHandler(ctx context.Context, delegate http.Handler, cache ReadWritable, isCacheable func(*http.Request) bool, getCacheKey func(*http.Request) interface{}, shouldCache func(context.Context, int, []byte) bool) http.Handler

NewCachingHandler produces a caching handler with an underlying delegate handler, cache, cacheability decision function, and cache key producer.

func NewGitHubClientFromToken

func NewGitHubClientFromToken(ctx context.Context, token string) *github.Client

NewGitHubClientFromToken returns a new GitHub client from an access token.

func NewMultiError

func NewMultiError(errors []error, when string) error

NewMultiError creates a MultiError from a slice of errors. The "when" parameter will be included in the error string in a "when" clause. If the slice is empty, nil will be returned.

func NewMultiErrorFromChan

func NewMultiErrorFromChan(errors chan error, when string) error

NewMultiErrorFromChan creates a MultiError by reading from an error channel. The "when" parameter will be included in the error string in a "when" clause. If there is no error in the channel, nil will be returned.

Note that it uses `range` over the channel, so users need to close the channel before calling this function or running it in a goroutine.

func NewSecureCookie

func NewSecureCookie(store Datastore) (*securecookie.SecureCookie, error)

NewSecureCookie returns a SecureCookie instance for wpt.fyi. This instance can be used to encode and decode cookies set by wpt.fyi.

func NewSetFromStringSlice

func NewSetFromStringSlice(items []string) mapset.Set

NewSetFromStringSlice is a helper for the inability to cast []string to []interface{}

func ParseAlignedParam

func ParseAlignedParam(v url.Values) (aligned *bool, err error)

ParseAlignedParam parses the "aligned" param. See ParseBooleanParam.

func ParseBooleanParam

func ParseBooleanParam(v url.Values, name string) (result *bool, err error)

ParseBooleanParam parses the given param name as a bool. Return nil if the param is missing, true if if it's present with no value, otherwise the parsed boolean value of the param's value.

func ParseBrowsersParam

func ParseBrowsersParam(v url.Values) (browsers []string, err error)

ParseBrowsersParam returns a list of browser params for the request. It parses the 'browsers' parameter, split on commas, and also checks for the (repeatable) 'browser' params.

func ParseDateTimeParam

func ParseDateTimeParam(v url.Values, name string) (*time.Time, error)

ParseDateTimeParam flexibly parses a date/time param with the given name as a time.Time.

func ParseIntParam

func ParseIntParam(v url.Values, param string) (*int, error)

ParseIntParam parses the result of ParseParam as int64.

func ParseLabelsParam

func ParseLabelsParam(v url.Values) []string

ParseLabelsParam returns a set list of test-run labels to include, or nil if no labels are provided.

func ParseMaxCountParam

func ParseMaxCountParam(v url.Values) (*int, error)

ParseMaxCountParam parses the 'max-count' parameter as an integer

func ParseMaxCountParamWithDefault

func ParseMaxCountParamWithDefault(v url.Values, defaultValue int) (count int, err error)

ParseMaxCountParamWithDefault parses the 'max-count' parameter as an integer, or returns the default when no param is present, or on error.

func ParseOffsetParam

func ParseOffsetParam(v url.Values) (*int, error)

ParseOffsetParam parses the 'offset' parameter as an integer

func ParsePRParam

func ParsePRParam(v url.Values) (*int, error)

ParsePRParam parses the "pr" parameter. If it's not a valid int64, an error will be returned.

func ParsePathsParam

func ParsePathsParam(v url.Values) []string

ParsePathsParam returns a set list of test paths to include, or nil if no filter is provided (and all tests should be included). It parses the 'paths' parameter, split on commas, and also checks for the (repeatable) 'path' params

func ParseQueryParamInt

func ParseQueryParamInt(v url.Values, key string) (*int, error)

ParseQueryParamInt parses the URL query parameter at key. If the parameter is empty or missing, nil is returned.

func ParseRepeatedInt64Param

func ParseRepeatedInt64Param(v url.Values, singular, plural string) (params []int64, err error)

ParseRepeatedInt64Param parses the result of ParseRepeatedParam as int64.

func ParseRepeatedParam

func ParseRepeatedParam(v url.Values, singular string, plural string) (params []string)

ParseRepeatedParam parses a param that may be a plural name, with all values comma-separated, or a repeated singular param. e.g. ?label=foo&label=bar vs ?labels=foo,bar

func ParseSHA

func ParseSHA(shaParam string) (sha string, err error)

ParseSHA parses and validates the given 'sha'. It returns "latest" by default (and in error cases).

func ParseTestURL

func ParseTestURL(testURL string) (filePath, query string)

ParseTestURL parses a WPT test URL and returns its file path and query components. If the test is a multi-global (auto-generated) test, the function returns the underlying file name of the test. e.g. testURL="foo/bar/test.any.worker.html?variant"

filepath="foo/bar/test.any.js"
query="?variant"

func ParseViewParam

func ParseViewParam(v url.Values) (*string, error)

ParseViewParam parses the 'view' parameter and ensures it is a valid value.

func PrepareLinkFilter

func PrepareLinkFilter(metadata MetadataResults) map[string][]string

PrepareLinkFilter maps a MetadataResult test name to its URLs.

func PrepareTestLabelFilter

func PrepareTestLabelFilter(metadata MetadataResults) map[string][]string

PrepareTestLabelFilter maps a MetadataResult test name to its labels.

func ProductChannelToLabel

func ProductChannelToLabel(channel string) string

ProductChannelToLabel maps known product-specific channel names to the wpt.fyi model's equivalent.

func Router

func Router() *mux.Router

Router returns the global mux.Router used for handling all requests.

func SetFeature

func SetFeature(ds Datastore, flag Flag) error

SetFeature puts a feature with the given flag name and enabled state.

func SplitWPTTestPath

func SplitWPTTestPath(githubPath string) (string, string)

SplitWPTTestPath splits a WPT test path into a folder path and a test name.

func StringSliceContains

func StringSliceContains(ss []string, s string) bool

StringSliceContains returns true if the given slice contains the given string.

func ToStringSlice

func ToStringSlice(set mapset.Set) []string

ToStringSlice converts a set to a typed string slice.

func URLAsCacheKey

func URLAsCacheKey(r *http.Request) interface{}

URLAsCacheKey is a helper for returning the request's full URL as a cache key. If this string is too long to be a redis key then writes to redis will fail, but that is not a big concern; it simply means that requests for cacheable long URLs will not be cached.

func WrapApplicationJSON

func WrapApplicationJSON(h http.HandlerFunc) http.HandlerFunc

WrapApplicationJSON wraps the given handler func in one that sets a Content-Type header of "text/json" on the response.

func WrapHSTS

func WrapHSTS(h http.HandlerFunc) http.HandlerFunc

WrapHSTS wraps the given handler func in one that sets the Strict-Transport-Security header on the response.

func WrapPermissiveCORS

func WrapPermissiveCORS(h http.HandlerFunc, methods ...string) http.HandlerFunc

WrapPermissiveCORS wraps the given handler func in one that sets an all-permissive CORS header on the response.

func WrapTrustedCORS

func WrapTrustedCORS(h http.HandlerFunc, origins []string, methods []string) http.HandlerFunc

WrapTrustedCORS wraps the given handler func in one that sets an Allow-Credentials CORS header with specified origins and methods on the response.

Types

type AppEngineAPI

type AppEngineAPI interface {
	Context() context.Context

	// GitHub OAuth client using the bot account (wptfyibot), which has
	// repo and read:org permissions.
	GetGitHubClient() (*github.Client, error)

	// http.Client
	GetHTTPClient() *http.Client
	GetHTTPClientWithTimeout(time.Duration) *http.Client

	// GetVersion returns the version name for the current environment.
	GetVersion() string
	// GetHostname returns the canonical hostname for the current AppEngine
	// project, i.e. staging.wpt.fyi or wpt.fyi.
	GetHostname() string
	// GetVersionedHostname returns the AppEngine hostname for the current
	// version of the default service, i.e.,
	//   VERSION-dot-wptdashboard{,-staging}.REGION.r.appspot.com.
	// Note: if the default service does not have the current version,
	// AppEngine routing will find a version according to traffic split.
	// https://cloud.google.com/appengine/docs/standard/go/how-requests-are-routed#soft_routing
	GetVersionedHostname() string
	// GetServiceHostname returns the AppEngine hostname for the current
	// version of the given service, i.e.,
	//   VERSION-dot-SERVICE-dot-wptdashboard{,-staging}.REGION.r.appspot.com.
	// Note: if the specified service does not have the current version,
	// AppEngine routing will find a version according to traffic split;
	// if the service does not exist at all, AppEngine will fall back to
	// the default service.
	GetServiceHostname(service string) string

	// GetResultsURL returns a URL to {staging.,}wpt.fyi/results with the
	// given filter.
	GetResultsURL(filter TestRunFilter) *url.URL
	// GetRunsURL returns a URL to {staging.,}wpt.fyi/runs with the given
	// filter.
	GetRunsURL(filter TestRunFilter) *url.URL
	// GetResultsUploadURL returns a URL for uploading results.
	GetResultsUploadURL() *url.URL

	// Simple wrappers that delegate to Datastore
	IsFeatureEnabled(featureName string) bool
	GetUploader(uploader string) (Uploader, error)

	// ScheduleTask schedules an AppEngine POST task on Cloud Tasks.
	// taskName can be empty, in which case one will be generated by Cloud
	// Tasks. Returns the final taskName and error.
	ScheduleTask(queueName, taskName, target string, params url.Values) (string, error)
}

AppEngineAPI is an abstraction of some appengine context helper methods.

func NewAppEngineAPI

func NewAppEngineAPI(ctx context.Context) AppEngineAPI

NewAppEngineAPI returns an AppEngineAPI for the given context.

type BSFData

type BSFData struct {
	// The latest WPT Revision updated in this BSF data.
	LastUpdateRevision string `json:"lastUpdateRevision"`
	// Fields correspond to the fields (columns) in this BSF data table.
	Fields []string `json:"fields"`
	// BSF data table, defined by the fields.
	Data [][]string `json:"data"`
}

BSFData stores BSF data of the latest WPT revision.

func FilterandExtractBSFData

func FilterandExtractBSFData(rawBSFdata [][]string, from *time.Time, to *time.Time) BSFData

FilterandExtractBSFData filters rawBSFdata based on query filters `from` and `to`, and generates BSFData for bsf_handler. rawBSFdata is [][]string with the 0th index as fields and the rest as the BSF data table in chronological order; e.g. [[sha,date,chrome-version,chrome,firefox-version,firefox,safari-version,safari], [eea0b54014e970a2f94f1c35ec6e18ece76beb76,2018-08-07,70.0.3510.0 dev,602.0505256721168,63.0a1,1617.1788882804883,12.1,2900.3438625831423], [203c34855f6871d6e55eaf7b55b50dad563f781f,2018-08-18,70.0.3521.2 dev,605.3869030161061,63.0a1,1521.908686731921,12.1,2966.686195133767], ...]

type Browser

type Browser struct {
	InitiallyLoaded bool   `json:"initially_loaded"`
	CurrentlyRun    bool   `json:"currently_run"`
	BrowserName     string `json:"browser_name"`
	BrowserVersion  string `json:"browser_version"`
	OSName          string `json:"os_name"`
	OSVersion       string `json:"os_version"`
	Sauce           bool   `json:"sauce"`
}

Browser holds objects that appear in browsers.json

type ByBrowserName

type ByBrowserName []Product

ByBrowserName is a []Product sortable by BrowserName values.

func (ByBrowserName) Len

func (e ByBrowserName) Len() int

func (ByBrowserName) Less

func (e ByBrowserName) Less(i, j int) bool

func (ByBrowserName) Swap

func (e ByBrowserName) Swap(i, j int)

type CachedStore

type CachedStore interface {
	Get(cacheID, storeID, value interface{}) error
}

CachedStore is a read-only interface that attempts to read from a cache, and when entities are not found, read from a store and write the result to the cache.

func NewByteCachedStore

func NewByteCachedStore(ctx context.Context, cache ReadWritable, store Readable) CachedStore

NewByteCachedStore produces a CachedStore that composes a ReadWritable cache and a Readable store, operating over the input context.Context.

func NewObjectCachedStore

func NewObjectCachedStore(ctx context.Context, cache ObjectCache, store ObjectStore) CachedStore

NewObjectCachedStore constructs a new CachedStore backed by an ObjectCache and ObjectStore.

type CachingResponseWriter

type CachingResponseWriter interface {
	http.ResponseWriter

	WriteTo(io.Writer) (int64, error)
	StatusCode() int
	Bytes() []byte
}

CachingResponseWriter is an http.ResponseWriter that can produce a new io.Reader instances that can replay the response.

func NewCachingResponseWriter

func NewCachingResponseWriter(delegate http.ResponseWriter) CachingResponseWriter

NewCachingResponseWriter wraps the input http.ResponseWriter with a caching implementation.

type CheckSuite

type CheckSuite struct {
	// SHA of the revision that requested a check suite.
	SHA string `json:"sha"`
	// The GitHub app ID for the custom wpt.fyi check.
	AppID int64 `json:"app_id"`
	// The GitHub app installation ID for custom wpt.fyi check
	InstallationID int64  `json:"installation"`
	Owner          string `json:"owner"` // Owner username
	Repo           string `json:"repo"`
	PRNumbers      []int  `json:"pr_numbers"`
}

CheckSuite entities represent a GitHub check request that has been noted by wpt.fyi, and will cause creation of a completed check_run when results arrive for the PR.

type CloudSecretManager

type CloudSecretManager struct {
	// contains filtered or unexported fields
}

CloudSecretManager is the implementation of the SecretManager for GCP. https://cloud.google.com/secret-manager

func NewAppEngineSecretManager

func NewAppEngineSecretManager(ctx context.Context, projectID string) CloudSecretManager

NewAppEngineSecretManager instantiates a new secret manager for a given context.

func (CloudSecretManager) GetSecret

func (m CloudSecretManager) GetSecret(name string) ([]byte, error)

GetSecret attempts to get the latest version of the provided secret from Google Cloud Secret Manager.

type Datastore

type Datastore interface {
	Context() context.Context
	Done() interface{}
	NewQuery(typeName string) Query
	NewIDKey(typeName string, id int64) Key
	NewIncompleteKey(typeName string) Key
	NewNameKey(typeName string, name string) Key
	ReserveID(typeName string) (Key, error)
	Get(key Key, dst interface{}) error
	GetAll(q Query, dst interface{}) ([]Key, error)
	GetMulti(keys []Key, dst interface{}) error
	Put(key Key, src interface{}) (Key, error)
	PutMulti(keys []Key, src interface{}) ([]Key, error)

	// Atomically insert a new entity.
	Insert(key Key, src interface{}) error
	// Atomically update or create an entity: the entity is first fetched
	// by key into dst, which must be a struct pointer; if the key cannot
	// be found, no error is returned and dst is not modified. Then
	// mutator(dst) is called; the transaction will be aborted if non-nil
	// error is returned. Finally, write dst back by key.
	Update(key Key, dst interface{}, mutator func(obj interface{}) error) error

	TestRunQuery() TestRunQuery
}

Datastore abstracts a datastore, hiding the distinctions between cloud and appengine's datastores.

func NewAppEngineDatastore

func NewAppEngineDatastore(ctx context.Context, cached bool) Datastore

NewAppEngineDatastore creates a Datastore implementation, or a Datastore implementation with Redis in front to cache all TestRun reads if cached is true.

Both variants (cached or not) are backed by Cloud Datastore SDK, using Clients initialized at startup in webapp.

func NewCloudDatastore

func NewCloudDatastore(ctx context.Context, client *datastore.Client) Datastore

NewCloudDatastore creates a Datastore implementation that is backed by a given Cloud Datastore client.

type DiffAPI

type DiffAPI interface {
	GetRunsDiff(before, after TestRun, filter DiffFilterParam, paths mapset.Set) (RunDiff, error)
	GetDiffURL(before, after TestRun, diffFilter *DiffFilterParam) *url.URL
	GetMasterDiffURL(testRun TestRun, diffFilter *DiffFilterParam) *url.URL
}

DiffAPI is an abstraction for computing run differences.

func NewDiffAPI

func NewDiffAPI(ctx context.Context) DiffAPI

NewDiffAPI return and implementation of the DiffAPI interface.

type DiffFilterParam

type DiffFilterParam struct {
	// Added tests are present in the 'after' state of the diff, but not present
	// in the 'before' state of the diff.
	Added bool

	// Deleted tests are present in the 'before' state of the diff, but not present
	// in the 'after' state of the diff.
	Deleted bool

	// Changed tests are present in both the 'before' and 'after' states of the diff,
	// but the number of passes, failures, or total tests has changed.
	Changed bool

	// Unchanged tests are present in both the 'before' and 'after' states of the diff,
	// and the number of passes, failures, or total tests is unchanged.
	Unchanged bool
}

DiffFilterParam represents the types of changed test paths to include.

func ParseDiffFilterParams

func ParseDiffFilterParams(v url.Values) (param DiffFilterParam, paths mapset.Set, err error)

ParseDiffFilterParams collects the diff filtering params for the given request. It splits the filter param into the differences to include. The filter param is inspired by Git's --diff-filter flag. It also adds the set of test paths to include; see ParsePathsParam below.

func (DiffFilterParam) String

func (d DiffFilterParam) String() string

type FetchBSF

type FetchBSF interface {
	Fetch(isExperimental bool) ([][]string, error)
}

FetchBSF encapsulates the Fetch(isExperimental bool) method for testing.

func NewFetchBSF

func NewFetchBSF() FetchBSF

NewFetchBSF returns an instance of FetchBSF for apiBSFHandler.

type Flag

type Flag struct {
	Name    string `datastore:"-"` // Name is the key in datastore.
	Enabled bool
}

Flag represents an enviroment feature flag's default state.

func GetFeatureFlags

func GetFeatureFlags(ds Datastore) (flags []Flag, err error)

GetFeatureFlags returns all feature flag defaults set in the datastore.

type GitHubAccessControl

type GitHubAccessControl interface {
	// IsValid* functions also verify the access token with GitHub.
	IsValidWPTMember() (bool, error)
	IsValidAdmin() (bool, error)
}

GitHubAccessControl encapsulates implementation details of access control for the wpt-metadata repository.

func NewGitHubAccessControl

func NewGitHubAccessControl(ctx context.Context, ds Datastore, botClient *github.Client, user *User, token string) (GitHubAccessControl, error)

NewGitHubAccessControl returns a GitHubAccessControl for checking the permission of a logged-in GitHub user.

func NewGitHubAccessControlFromRequest

func NewGitHubAccessControlFromRequest(aeAPI AppEngineAPI, ds Datastore, r *http.Request) (GitHubAccessControl, error)

NewGitHubAccessControlFromRequest returns a GitHubAccessControl for checking the permission of a logged-in GitHub user from a request. (nil, nil) will be returned if the user is not logged in.

type GitHubOAuth

type GitHubOAuth interface {
	Context() context.Context
	Datastore() Datastore
	GetAccessToken() string
	GetAuthCodeURL(state string, opts ...oauth2.AuthCodeOption) string
	GetUser(client *github.Client) (*github.User, error)
	NewClient(oauthCode string) (*github.Client, error)
	SetRedirectURL(url string)
}

GitHubOAuth encapsulates implementation details of GitHub OAuth flow.

func NewGitHubOAuth

func NewGitHubOAuth(ctx context.Context) (GitHubOAuth, error)

NewGitHubOAuth returns an instance of GitHubOAuth for loginHandler and oauthHandler.

type GitHubWebFeaturesClient

type GitHubWebFeaturesClient struct {
	// contains filtered or unexported fields
}

GitHubWebFeaturesClient is the entrypoint to retrieving web features from GitHub.

func NewGitHubWebFeaturesClient

func NewGitHubWebFeaturesClient(ghClient *github.Client) *GitHubWebFeaturesClient

NewGitHubWebFeaturesClient constructs an instance of GitHubWebFeaturesClient with default values.

func (GitHubWebFeaturesClient) Get

Get returns the latest web features data from GitHub.

type GitHubWebFeaturesClientOption

type GitHubWebFeaturesClientOption func(*gitHubWebFeaturesClientOptions)

A GitHubWebFeaturesClientOption configures GitHubWebFeaturesClient.

func SetHTTPClientForGitHubWebFeatures

func SetHTTPClientForGitHubWebFeatures(netClient *http.Client) GitHubWebFeaturesClientOption

SetHTTPClientForGitHubWebFeatures overrides the http client used to download the found asset from the release.

type Iterator

type Iterator interface {
	Next(dst interface{}) (Key, error)
}

Iterator abstracts a datastore.Iterator

type Key

type Key interface {
	IntID() int64
	StringID() string
	Kind() string // Type name, e.g. TestRun
}

Key abstracts an int64 based datastore.Key

type KeysByProduct

type KeysByProduct []ProductTestRunKeys

KeysByProduct is an array of tuples of {product, matching keys}, returned when a TestRun key query is executed.

func (KeysByProduct) AllKeys

func (t KeysByProduct) AllKeys() []Key

AllKeys returns an array of all the loaded keys.

type LegacySearchRunResult

type LegacySearchRunResult struct {
	// Passes is the number of test results in a PASS/OK state.
	Passes int `json:"passes"`
	// Total is the total number of test results for this run/file pair.
	Total int `json:"total"`
	// Status represents either the test status or harness status.
	// This will be an empty string for old summaries.
	Status string `json:"status"`
	// NewAggProcess represents whether the summary was created with the old
	// or new aggregation process.
	NewAggProcess bool `json:"newAggProcess"`
}

LegacySearchRunResult is the results data from legacy test summarys. These summaries contain a "pass count" and a "total count", where the test itself counts as 1, and each subtest counts as 1. The "pass count" contains any status values that are "PASS" or "OK".

type Logger

type Logger interface {
	Debugf(format string, args ...interface{})
	Errorf(format string, args ...interface{})
	Infof(format string, args ...interface{})
	Warningf(format string, args ...interface{})
}

Logger is an abstract logging interface that contains an intersection of logrus and GAE logging functionality.

func GetLogger

func GetLogger(ctx context.Context) Logger

GetLogger retrieves a non-nil Logger that is appropriate for use in ctx. If ctx does not provide a logger, then a nil-logger is returned.

func NewNilLogger

func NewNilLogger() Logger

NewNilLogger returns a new logger that silently ignores all Logger calls.

type LoggerCtxKey

type LoggerCtxKey struct{}

LoggerCtxKey is a key for attaching a Logger to a context.Context.

func DefaultLoggerCtxKey

func DefaultLoggerCtxKey() LoggerCtxKey

DefaultLoggerCtxKey returns the default key where a logger instance should be stored in a context.Context object.

type Manifest

type Manifest struct {
	Items   map[string]rawManifestTrie `json:"items,omitempty"`
	Version int                        `json:"version,omitempty"`
	URLBase string                     `json:"url_base,omitempty"`
	// contains filtered or unexported fields
}

Manifest represents a JSON blob of all the WPT tests.

func (*Manifest) ContainsFile

func (m *Manifest) ContainsFile(path string) (bool, error)

ContainsFile checks whether m contains a file path (including directories).

func (*Manifest) ContainsTest

func (m *Manifest) ContainsTest(testURL string) (bool, error)

ContainsTest checks whether m contains a full test URL.

func (Manifest) FilterByPath

func (m Manifest) FilterByPath(paths ...string) (*Manifest, error)

FilterByPath filters all the manifest items by path prefixes.

type Metadata

type Metadata struct {
	Links MetadataLinks `yaml:"links"`
}

Metadata represents a wpt-metadata META.yml file.

type MetadataFetcher

type MetadataFetcher interface {
	Fetch() (sha *string, res map[string][]byte, err error)
}

MetadataFetcher is an abstract interface that encapsulates the Fetch() method. Fetch() fetches metadata for webapp and searchcache.

type MetadataLink struct {
	Product ProductSpec          `yaml:"product,omitempty" json:"product,omitempty"`
	URL     string               `yaml:"url,omitempty"     json:"url"`
	Label   string               `yaml:"label,omitempty"  json:"label,omitempty"`
	Results []MetadataTestResult `yaml:"results" json:"results,omitempty"`
}

MetadataLink is an item in the `links` node of a wpt-metadata META.yml file, which lists an external reference, optionally filtered by product and a specific test.

type MetadataLinks []MetadataLink

MetadataLinks is a helper type for a MetadataLink slice.

type MetadataResults

type MetadataResults map[string]MetadataLinks

MetadataResults is a map from test paths to all of the links under that test path. It represents a flattened copy of the wpt-metadata repository, which has metadata sharded across as large number of files in a directory structure.

func GetMetadataResponse

func GetMetadataResponse(testRuns []TestRun, includeTestLevel bool, log Logger, fetcher MetadataFetcher) (MetadataResults, error)

GetMetadataResponse retrieves the response to a WPT Metadata query. Metadata is included for any product that matches a passed TestRun. Test-level metadata (i.e. that is not associated with any product) may be fetched by passing true for includeTestLevel.

func GetMetadataResponseOnProducts

func GetMetadataResponseOnProducts(productSpecs ProductSpecs, includeTestLevel bool, log Logger, fetcher MetadataFetcher) (MetadataResults, error)

GetMetadataResponseOnProducts constructs the response to a WPT Metadata query, given ProductSpecs. Metdata is included for any product that matches a passed ProductSpec. Test-level metadata (i.e. that is not associated with any product) may be fetched by passing true for includeTestLevel.

type MetadataTestResult

type MetadataTestResult struct {
	TestPath    string      `yaml:"test"    json:"test,omitempty"`
	SubtestName *string     `yaml:"subtest,omitempty" json:"subtest,omitempty"`
	Status      *TestStatus `yaml:"status,omitempty"  json:"status,omitempty"`
}

MetadataTestResult is a filter for test results to which the Metadata link should apply.

type MultiError

type MultiError struct {
	// contains filtered or unexported fields
}

MultiError is a convenient wrapper of multiple errors and is itself an implementation of the error interface.

func (MultiError) Count

func (e MultiError) Count() int

Count returns the number of errors in this MultiError.

func (MultiError) Error

func (e MultiError) Error() string

func (MultiError) Errors

func (e MultiError) Errors() []error

Errors returns the inner error slice of a MultiError.

type ObjectCache

type ObjectCache interface {
	ObjectStore
	Put(id, value interface{}) error
}

ObjectCache is an ObjectStore that also supports Put() for arbitrary id/value pairs.

func NewJSONObjectCache

func NewJSONObjectCache(ctx context.Context, delegate ReadWritable) ObjectCache

NewJSONObjectCache constructs a new JSON object cache, bound to the input context.Context and delgating to the input ReadWritable.

type ObjectStore

type ObjectStore interface {
	Get(id, value interface{}) error
}

ObjectStore is a store that populates an arbitrary output object on Get().

type PendingTestRun

type PendingTestRun struct {
	ID int64 `json:"id" datastore:"-"`
	ProductAtRevision
	CheckRunID int64               `json:"check_run_id" datastore:",omitempty"`
	Uploader   string              `json:"uploader"`
	Error      string              `json:"error" datastore:",omitempty"`
	Stage      PendingTestRunStage `json:"stage"`

	Created time.Time `json:"created"`
	Updated time.Time `json:"updated"`
}

PendingTestRun represents a TestRun that has started, but is not yet completed.

func (*PendingTestRun) Load

func (s *PendingTestRun) Load(ps []datastore.Property) error

Load is part of the datastore.PropertyLoadSaver interface. We use it to reset all time to UTC and trim their monotonic clock.

func (*PendingTestRun) Save

func (s *PendingTestRun) Save() ([]datastore.Property, error)

Save is part of the datastore.PropertyLoadSaver interface. Delegate to the default behaviour.

func (*PendingTestRun) Transition

func (s *PendingTestRun) Transition(next PendingTestRunStage) error

Transition sets Stage to next if the transition is allowed; otherwise an error is returned.

type PendingTestRunByUpdated

type PendingTestRunByUpdated []PendingTestRun

PendingTestRunByUpdated sorts the pending test runs by updated (asc)

func (PendingTestRunByUpdated) Len

func (a PendingTestRunByUpdated) Len() int

func (PendingTestRunByUpdated) Less

func (a PendingTestRunByUpdated) Less(i, j int) bool

func (PendingTestRunByUpdated) Swap

func (a PendingTestRunByUpdated) Swap(i, j int)

type PendingTestRunStage

type PendingTestRunStage int

PendingTestRunStage represents the stage of a test run in its life cycle.

const (
	StageGitHubQueued     PendingTestRunStage = 100
	StageGitHubInProgress PendingTestRunStage = 200
	StageCIRunning        PendingTestRunStage = 300
	StageCIFinished       PendingTestRunStage = 400
	StageGitHubSuccess    PendingTestRunStage = 500
	StageGitHubFailure    PendingTestRunStage = 550
	StageWptFyiReceived   PendingTestRunStage = 600
	StageWptFyiProcessing PendingTestRunStage = 700
	StageValid            PendingTestRunStage = 800
	StageInvalid          PendingTestRunStage = 850
	StageEmpty            PendingTestRunStage = 851
	StageDuplicate        PendingTestRunStage = 852
)

Constant enums for PendingTestRunStage

func (PendingTestRunStage) MarshalJSON

func (s PendingTestRunStage) MarshalJSON() ([]byte, error)

MarshalJSON is the custom JSON marshaler for PendingTestRunStage.

func (PendingTestRunStage) String

func (s PendingTestRunStage) String() string

func (*PendingTestRunStage) UnmarshalJSON

func (s *PendingTestRunStage) UnmarshalJSON(b []byte) error

UnmarshalJSON is the custom JSON unmarshaler for PendingTestRunStage.

type Product

type Product struct {
	BrowserName    string `json:"browser_name"`
	BrowserVersion string `json:"browser_version"`
	OSName         string `json:"os_name"`
	OSVersion      string `json:"os_version"`
}

Product uniquely defines a browser version, running on an OS version.

func ParseBrowserParam

func ParseBrowserParam(v url.Values) (product *Product, err error)

ParseBrowserParam parses and validates the 'browser' param for the request. It returns "" by default (and in error cases).

func ParseProduct

func ParseProduct(product string) (result Product, err error)

ParseProduct parses the `browser-version-os-version` input as a Product struct.

func (Product) String

func (p Product) String() string

type ProductAtRevision

type ProductAtRevision struct {
	Product

	// The first 10 characters of the SHA1 of the tested WPT revision.
	//
	// Deprecated: The authoritative git revision indicator is FullRevisionHash.
	Revision string `json:"revision"`

	// The complete SHA1 hash of the tested WPT revision.
	FullRevisionHash string `json:"full_revision_hash"`
}

ProductAtRevision defines a WPT run for a specific product, at a specific hash of the WPT repo.

func (ProductAtRevision) String

func (p ProductAtRevision) String() string

type ProductSpec

type ProductSpec struct {
	ProductAtRevision

	Labels mapset.Set
}

ProductSpec is a struct representing a parsed product spec string.

func ParseProductParam

func ParseProductParam(v url.Values) (product *ProductSpec, err error)

ParseProductParam parses and validates the 'product' param for the request.

func ParseProductSpec

func ParseProductSpec(spec string) (productSpec ProductSpec, err error)

ParseProductSpec parses a test-run spec into a ProductAtRevision struct.

func ParseProductSpecUnsafe

func ParseProductSpecUnsafe(s string) ProductSpec

ParseProductSpecUnsafe ignores any potential error parsing the given product spec.

func (ProductSpec) DisplayName

func (p ProductSpec) DisplayName() string

DisplayName returns a capitalized version of the product's name.

func (ProductSpec) IsExperimental

func (p ProductSpec) IsExperimental() bool

IsExperimental returns true if the product spec is restricted to experimental runs (i.e. has the label "experimental").

func (ProductSpec) MarshalJSON

func (p ProductSpec) MarshalJSON() ([]byte, error)

MarshalJSON treats the set as an array so it can be marshalled.

func (ProductSpec) MarshalYAML

func (p ProductSpec) MarshalYAML() (interface{}, error)

MarshalYAML serializes a ProductSpec into a YAML string.

func (ProductSpec) Matches

func (p ProductSpec) Matches(run TestRun) bool

Matches returns whether the ProductSpec matches the given run.

func (ProductSpec) MatchesLabels

func (p ProductSpec) MatchesLabels(labels mapset.Set) bool

MatchesLabels returns whether the ProductSpec's labels matches the given labels.

func (ProductSpec) MatchesProductAtRevision

func (p ProductSpec) MatchesProductAtRevision(productAtRevision ProductAtRevision) bool

MatchesProductAtRevision returns whether the spec matches the given ProductAtRevision.

func (ProductSpec) MatchesProductSpec

func (p ProductSpec) MatchesProductSpec(productSpec ProductSpec) bool

MatchesProductSpec returns whether the ProductSpec matches the given ProductSpec.

func (ProductSpec) String

func (p ProductSpec) String() string

func (*ProductSpec) UnmarshalJSON

func (p *ProductSpec) UnmarshalJSON(data []byte) (err error)

UnmarshalJSON parses an array so that ProductSpec can be unmarshalled.

func (*ProductSpec) UnmarshalYAML

func (p *ProductSpec) UnmarshalYAML(unmarshal func(interface{}) error) (err error)

UnmarshalYAML parses an array so that ProductSpec can be unmarshalled.

type ProductSpecs

type ProductSpecs []ProductSpec

ProductSpecs is a helper type for a slice of ProductSpec structs.

func GetDefaultProducts

func GetDefaultProducts() ProductSpecs

GetDefaultProducts returns the default set of products to show on wpt.fyi

func ParseBeforeAndAfterParams

func ParseBeforeAndAfterParams(v url.Values) (ProductSpecs, error)

ParseBeforeAndAfterParams parses the before and after params used when intending to diff two test runs. Either both or neither of the params must be present.

func ParseProductOrBrowserParams

func ParseProductOrBrowserParams(v url.Values) (products ProductSpecs, err error)

ParseProductOrBrowserParams parses the product (or, browser) params present in the given request.

func ParseProductSpecs

func ParseProductSpecs(specs ...string) (products ProductSpecs, err error)

ParseProductSpecs parses multiple product specs

func ParseProductsParam

func ParseProductsParam(v url.Values) (ProductSpecs, error)

ParseProductsParam returns a list of product params for the request. It parses the 'products' parameter, split on commas, and also checks for the (repeatable) 'product' params.

func (ProductSpecs) Len

func (p ProductSpecs) Len() int

func (ProductSpecs) Less

func (p ProductSpecs) Less(i, j int) bool

func (ProductSpecs) OrDefault

func (p ProductSpecs) OrDefault() ProductSpecs

OrDefault returns the current product specs, or the default if the set is empty.

func (ProductSpecs) Products

func (p ProductSpecs) Products() []Product

Products gets the slice of products specified in the ProductSpecs slice.

func (ProductSpecs) Strings

func (p ProductSpecs) Strings() []string

Strings returns the array of the ProductSpec items as their string representations.

func (ProductSpecs) Swap

func (p ProductSpecs) Swap(i, j int)

type ProductTestRunKeys

type ProductTestRunKeys struct {
	Product ProductSpec
	Keys    []Key
}

ProductTestRunKeys is a tuple of a product and test run keys loaded for it.

type ProductTestRuns

type ProductTestRuns struct {
	Product  ProductSpec
	TestRuns TestRuns
}

ProductTestRuns is a tuple of a product and test runs loaded for it.

type Query

type Query interface {
	Filter(filterStr string, value interface{}) Query
	Project(fields ...string) Query
	Limit(limit int) Query
	Offset(offset int) Query
	Order(order string) Query
	KeysOnly() Query
	Distinct() Query
	Run(Datastore) Iterator
}

Query abstracts a datastore.Query

func VersionPrefix

func VersionPrefix(query Query, fieldName, versionPrefix string, desc bool) Query

VersionPrefix returns the given query with a prefix filter on the given field name, using the >= and < filters.

type QueryFilter

type QueryFilter struct {
	RunIDs []int64
	Q      string
}

QueryFilter represents the ways search results can be filtered in the webapp search API.

func ParseQueryFilterParams

func ParseQueryFilterParams(v url.Values) (filter QueryFilter, err error)

ParseQueryFilterParams parses shared params for the search APIs.

type ReadWritable

type ReadWritable interface {
	Readable
	// NewWriteCloser provides an io.WriteCloser for the entity keyed by its input
	// argument.
	NewWriteCloser(interface{}) (io.WriteCloser, error)
}

ReadWritable is a provider interface for io.ReadCloser and io.WriteCloser.

func NewGZReadWritable

func NewGZReadWritable(delegate ReadWritable) ReadWritable

NewGZReadWritable produces a ReadWritable that ungzips on read and gzips on write, and delegates the input argument.

func NewRedisReadWritable

func NewRedisReadWritable(ctx context.Context, expiry time.Duration) ReadWritable

NewRedisReadWritable produces a ReadWritable that performs read/write operations via the App Engine redis API through the input context.Context.

type Readable

type Readable interface {
	// NewReadCloser provides an io.ReadCloser for the entity keyed by its input
	// argument.
	NewReadCloser(interface{}) (io.ReadCloser, error)
}

Readable is a provider interface for an io.ReadCloser.

func NewHTTPReadable

func NewHTTPReadable(ctx context.Context) Readable

NewHTTPReadable produces a Readable bound to the input context.Context.

type RedisSet

type RedisSet interface {
	// Add inserts value to the set stored at key; ignored if value is
	// already a member of the set.
	Add(key string, value string) error
	// Remove removes value from the set stored at key; ignored if value is
	// not a member of the set.
	Remove(key string, value string) error
	// GetAll returns all the members of the set stored at key; returns an
	// empty string[] if the key is not present.
	GetAll(key string) ([]string, error)
}

RedisSet is an interface for an redisSetReadWritable, which performs Add/Remove/GetAll operations via the App Engine Redis API.

func NewRedisSet

func NewRedisSet() RedisSet

NewRedisSet returns a new redisSetReadWritable.

type ResultsDiff

type ResultsDiff map[string]TestDiff

ResultsDiff is a collection of test diffs, keyed by the test path.

func (ResultsDiff) Add

func (r ResultsDiff) Add(k string, diff TestDiff)

Add adds the given diff to the TestDiff for the given key, or puts it in the map if it's not yet present.

func (ResultsDiff) Regressions

func (r ResultsDiff) Regressions() mapset.Set

Regressions returns the set of test paths for tests that have a regression value in their diff. A change is considered a regression when tests that existed both before and after have an increase in the number of failing tests has increased, which will of course include newly-added tests that are failing. Additionally, we flag a decrease in the total number of tests as a regression, since that can often indicate a failure in a test's setup.

type ResultsSummary

type ResultsSummary map[string]TestSummary

ResultsSummary is a collection of [pass, total] summary pairs, keyed by test.

func FetchRunResultsJSON

func FetchRunResultsJSON(ctx context.Context, run TestRun) (results ResultsSummary, err error)

FetchRunResultsJSON fetches the results JSON summary for the given test run, but does not include subtests (since a full run can span 20k files).

func FetchRunResultsJSONForParam

func FetchRunResultsJSONForParam(
	ctx context.Context, r *http.Request, param string) (results ResultsSummary, err error)

FetchRunResultsJSONForParam fetches the results JSON blob for the given [product]@[SHA] param.

func FetchRunResultsJSONForSpec

func FetchRunResultsJSONForSpec(
	ctx context.Context, r *http.Request, spec ProductSpec) (results ResultsSummary, err error)

FetchRunResultsJSONForSpec fetches the result JSON blob for the given spec.

func (ResultsSummary) Add

func (s ResultsSummary) Add(k string, other TestSummary)

Add adds the given summary to the summary for the given path, adding it to the map if it's not present already.

type RunDiff

type RunDiff struct {
	// Differences is a map from test-path to an array of
	// [newly-passing, newly-failing, total-delta],
	// where newly-pa
	Before        TestRun           `json:"-"`
	BeforeSummary ResultsSummary    `json:"-"`
	After         TestRun           `json:"-"`
	AfterSummary  ResultsSummary    `json:"-"`
	Differences   ResultsDiff       `json:"diff"`
	Renames       map[string]string `json:"renames"`
}

RunDiff represents a summary of the differences between 2 runs.

func RunDiffFromSearchResponse

func RunDiffFromSearchResponse(aeAPI AppEngineAPI, before, after TestRun, scDiff SearchResponse) (RunDiff, error)

RunDiffFromSearchResponse builds a RunDiff from a searchcache response.

type SHAs

type SHAs []string

SHAs is a helper type for a slice of commit/revision SHAs.

func ParseSHAParam

func ParseSHAParam(v url.Values) (SHAs, error)

ParseSHAParam parses and validates any 'sha' param(s) for the request.

func (SHAs) EmptyOrLatest

func (s SHAs) EmptyOrLatest() bool

EmptyOrLatest returns whether the shas slice is empty, or only contains one item, which is the latest keyword.

func (SHAs) FirstOrLatest

func (s SHAs) FirstOrLatest() string

FirstOrLatest returns the first sha in the slice, or the latest keyword.

func (SHAs) ShortSHAs

func (s SHAs) ShortSHAs() []string

ShortSHAs returns an array of the given SHAs' first 7-chars.

type SearchResponse

type SearchResponse struct {
	// Runs is the specific runs for which results were retrieved. Each run, in
	// order, corresponds to a Status entry in each SearchResult in Results.
	Runs []TestRun `json:"runs"`
	// IgnoredRuns is any runs that the client requested to be included in the
	// query, but were not included. This optional field may be non-nil if, for
	// example, results are being served from an incompelte cache of runs and some
	// runs described in the query request are not resident in the cache.
	IgnoredRuns []TestRun `json:"ignored_runs,omitempty"`
	// Results is the collection of test results, grouped by test file name.
	Results []SearchResult `json:"results"`
	// MetadataResponse is a response to a wpt-metadata query.
	MetadataResponse MetadataResults `json:"metadata,omitempty"`
}

SearchResponse contains a response to search API calls, including specific runs whose results were searched and the search results themselves.

type SearchResult

type SearchResult struct {
	// Test is the name of a test; this often corresponds to a test file path in
	// the WPT source reposiory.
	Test string `json:"test"`
	// LegacyStatus is the results data from legacy test summaries. These
	// summaries contain a "pass count" and a "total count", where the test itself
	// counts as 1, and each subtest counts as 1. The "pass count" contains any
	// status values that are "PASS" or "OK".
	LegacyStatus []LegacySearchRunResult `json:"legacy_status,omitempty"`

	// Interoperability scores. For N browsers, we have an array of
	// N+1 items, where the index X is the number of items passing in exactly
	// X of the N browsers. e.g. for 4 browsers, [0/4, 1/4, 2/4, 3/4, 4/4].
	Interop []int `json:"interop,omitempty"`

	// Subtests (names) which are included in the LegacyStatus summary.
	Subtests []string `json:"subtests,omitempty"`

	// Diff count of subtests which are included in the LegacyStatus summary.
	Diff TestDiff `json:"diff,omitempty"`
}

SearchResult contains data regarding a particular test file over a collection of runs. The runs are identified externally in a parallel slice (see SearchResponse).

type SecretManager

type SecretManager interface {
	GetSecret(name string) ([]byte, error)
}

SecretManager is a simple interface for getting secrets.

type TestDiff

type TestDiff []int

TestDiff is an array of differences between 2 tests.

func NewTestDiff

func NewTestDiff(before, after []int, filter DiffFilterParam) TestDiff

NewTestDiff computes the differences between two test-run pass-count summaries, namely an array of [passing, total] counts.

func (TestDiff) Add

func (d TestDiff) Add(other TestDiff)

Add adds the given other TestDiff to this TestDiff's value. Used for summing.

func (TestDiff) Append

func (d TestDiff) Append(before, after TestStatus, filter *DiffFilterParam)

Append the difference between the two given statuses, if any.

func (TestDiff) IsEmpty

func (d TestDiff) IsEmpty() bool

IsEmpty returns true if the diff is empty (all zeroes)

func (TestDiff) NewlyPassing

func (d TestDiff) NewlyPassing() int

NewlyPassing is the delta/increase in the number of passing tests when comparing before/after.

func (TestDiff) Regressions

func (d TestDiff) Regressions() int

Regressions is the delta/increase in the number of failing tests when comparing before/after.

func (TestDiff) TotalDelta

func (d TestDiff) TotalDelta() int

TotalDelta is the delta in the number of total subtests when comparing before/after.

type TestHistoryEntry

type TestHistoryEntry struct {
	BrowserName string
	RunID       string
	Date        string
	TestName    string
	SubtestName string
	Status      string
}

TestHistoryEntry formats Test History data for the datastore.

type TestRun

type TestRun struct {
	ID int64 `json:"id" datastore:"-"`

	ProductAtRevision

	// URL for summary of results, which is derived from raw results.
	ResultsURL string `json:"results_url"`

	// Time when the test run metadata was first created.
	CreatedAt time.Time `json:"created_at"`

	// Time when the test run started.
	TimeStart time.Time `json:"time_start"`

	// Time when the test run ended.
	TimeEnd time.Time `json:"time_end"`

	// URL for raw results JSON object. Resembles the JSON output of the
	// wpt report tool.
	RawResultsURL string `json:"raw_results_url"`

	// Labels for the test run.
	Labels []string `json:"labels"`
}

TestRun stores metadata for a test run (produced by run/run.py)

func FetchRunForSpec

func FetchRunForSpec(ctx context.Context, spec ProductSpec) (*TestRun, error)

FetchRunForSpec loads the wpt.fyi TestRun metadata for the given spec.

func (TestRun) Channel

func (r TestRun) Channel() string

Channel return the channel label, if any, for the given run.

func (TestRun) IsExperimental

func (r TestRun) IsExperimental() bool

IsExperimental returns true if the run is labelled experimental.

func (TestRun) IsPRBase

func (r TestRun) IsPRBase() bool

IsPRBase returns true if the run is labelled experimental.

func (TestRun) LabelsSet

func (r TestRun) LabelsSet() mapset.Set

LabelsSet creates a set from the run's labels.

func (*TestRun) Load

func (r *TestRun) Load(ps []datastore.Property) error

Load is part of the datastore.PropertyLoadSaver interface. We use it to reset all time to UTC and trim their monotonic clock.

func (*TestRun) Save

func (r *TestRun) Save() ([]datastore.Property, error)

Save is part of the datastore.PropertyLoadSaver interface. Delegate to the default behaviour.

type TestRunFilter

type TestRunFilter struct {
	SHAs     SHAs         `json:"shas,omitempty"`
	Labels   mapset.Set   `json:"labels,omitempty"`
	Aligned  *bool        `json:"aligned,omitempty"`
	From     *time.Time   `json:"from,omitempty"`
	To       *time.Time   `json:"to,omitempty"`
	MaxCount *int         `json:"maxcount,omitempty"`
	Offset   *int         `json:"offset,omitempty"` // Used for paginating with MaxCount.
	Products ProductSpecs `json:"products,omitempty"`
	View     *string      `json:"view,omitempty"`
}

TestRunFilter represents the ways TestRun entities can be filtered in the webapp and api.

func ParsePageToken

func ParsePageToken(v url.Values) (*TestRunFilter, error)

ParsePageToken decodes a base64 encoding of a TestRunFilter struct.

func ParseTestRunFilterParams

func ParseTestRunFilterParams(v url.Values) (filter TestRunFilter, err error)

ParseTestRunFilterParams parses all of the filter params for a TestRun query.

func (TestRunFilter) GetProductsOrDefault

func (filter TestRunFilter) GetProductsOrDefault() (products ProductSpecs)

GetProductsOrDefault parses the 'products' (and legacy 'browsers') params, returning the ordered list of products to include, or a default list.

func (TestRunFilter) IsDefaultProducts

func (filter TestRunFilter) IsDefaultProducts() bool

IsDefaultProducts returns whether the params products are empty, or the equivalent of the default product set.

func (TestRunFilter) IsDefaultQuery

func (filter TestRunFilter) IsDefaultQuery() bool

IsDefaultQuery returns whether the params are just an empty query (or, the equivalent defaults of an empty query).

func (TestRunFilter) MarshalJSON

func (filter TestRunFilter) MarshalJSON() ([]byte, error)

MarshalJSON treats the set as an array so it can be marshalled.

func (TestRunFilter) MasterOnly

func (filter TestRunFilter) MasterOnly() TestRunFilter

MasterOnly returns the current filter, ensuring it has with the master-only restriction (a label of "master").

func (TestRunFilter) NextPage

func (filter TestRunFilter) NextPage(loadedRuns TestRunsByProduct) *TestRunFilter

NextPage returns a filter for the next page of results that would match the current filter, based on the given results that were loaded.

func (TestRunFilter) OrAlignedStableRuns

func (filter TestRunFilter) OrAlignedStableRuns() TestRunFilter

OrAlignedStableRuns returns the current filter, or, if it is a default query, returns a query for stable runs, with an aligned SHA.

func (TestRunFilter) OrDefault

func (filter TestRunFilter) OrDefault() TestRunFilter

OrDefault returns the current filter, or, if it is a default query, returns the query used by default in wpt.fyi.

func (TestRunFilter) OrExperimentalRuns

func (filter TestRunFilter) OrExperimentalRuns() TestRunFilter

OrExperimentalRuns returns the current filter, or, if it is a default query, returns a query for the latest experimental runs.

func (TestRunFilter) ToQuery

func (filter TestRunFilter) ToQuery() (q url.Values)

ToQuery converts the filter set to a url.Values (set of query params).

func (TestRunFilter) Token

func (filter TestRunFilter) Token() (string, error)

Token returns a base64 encoded copy of the filter.

func (*TestRunFilter) UnmarshalJSON

func (filter *TestRunFilter) UnmarshalJSON(data []byte) error

UnmarshalJSON parses an array so that TestRunFilter can be unmarshalled.

type TestRunIDs

type TestRunIDs []int64

TestRunIDs is a helper for an array of TestRun IDs.

func ExtractRunIDsBodyParam

func ExtractRunIDsBodyParam(r *http.Request, replay bool) (TestRunIDs, error)

ExtractRunIDsBodyParam extracts {"run_ids": <run ids>} from a request JSON body. Optionally replace r.Body so that it can be replayed by subsequent request handling code can process it.

func GetTestRunIDs

func GetTestRunIDs(keys []Key) TestRunIDs

GetTestRunIDs extracts the TestRunIDs from loaded datastore keys.

func ParseRunIDsParam

func ParseRunIDsParam(v url.Values) (ids TestRunIDs, err error)

ParseRunIDsParam parses the "run_ids" parameter. If the ID is not a valid int64, an error will be returned.

func (TestRunIDs) GetKeys

func (ids TestRunIDs) GetKeys(store Datastore) []Key

GetKeys returns a slice of keys for the TestRunIDs in the given datastore.

func (TestRunIDs) LoadTestRuns

func (ids TestRunIDs) LoadTestRuns(store Datastore) (testRuns TestRuns, err error)

LoadTestRuns is a helper for fetching the TestRuns from the datastore, for the gives TestRunIDs.

type TestRunQuery

type TestRunQuery interface {
	// LoadTestRuns loads the test runs for the TestRun entities for the given parameters.
	// It is encapsulated because we cannot run single queries with multiple inequality
	// filters, so must load the keys and merge the results.
	LoadTestRuns(
		products []ProductSpec,
		labels mapset.Set,
		revisions []string,
		from *time.Time,
		to *time.Time,
		limit,
		offset *int) (result TestRunsByProduct, err error)

	// LoadTestRunKeys loads the keys for the TestRun entities for the given parameters.
	// It is encapsulated because we cannot run single queries with multiple inequality
	// filters, so must load the keys and merge the results.
	LoadTestRunKeys(
		products []ProductSpec,
		labels mapset.Set,
		revisions []string,
		from *time.Time,
		to *time.Time,
		limit *int,
		offset *int) (result KeysByProduct, err error)

	// LoadTestRunsByKeys loads test runs by keys and sets their IDs.
	LoadTestRunsByKeys(KeysByProduct) (result TestRunsByProduct, err error)

	// GetAlignedRunSHAs returns an array of the SHA[0:10] for runs that
	// exists for all the given products, ordered by most-recent, as well as a map
	// of those SHAs to a KeysByProduct map of products to the TestRun keys, for the
	// runs in the aligned run.
	GetAlignedRunSHAs(
		products ProductSpecs,
		labels mapset.Set,
		from,
		to *time.Time,
		limit *int,
		offset *int) (shas []string, keys map[string]KeysByProduct, err error)
}

TestRunQuery abstracts complex queries of TestRun entities.

func NewTestRunQuery

func NewTestRunQuery(store Datastore) TestRunQuery

NewTestRunQuery creates a concrete TestRunQuery backed by a Datastore interface.

type TestRuns

type TestRuns []TestRun

TestRuns is a helper type for an array of TestRun entities.

func FetchLatestRuns

func FetchLatestRuns(wptdHost string) (TestRuns, error)

FetchLatestRuns fetches the TestRun metadata for the latest runs, using the API on the given host.

func FetchRuns

func FetchRuns(wptdHost string, filter TestRunFilter) (TestRuns, error)

FetchRuns fetches the TestRun metadata for the given sha / labels, using the API on the given host.

func (TestRuns) GetTestRunIDs

func (t TestRuns) GetTestRunIDs() TestRunIDs

GetTestRunIDs gets an array of the IDs for the TestRun entities in the array.

func (TestRuns) Len

func (t TestRuns) Len() int

func (TestRuns) Less

func (t TestRuns) Less(i, j int) bool

func (TestRuns) OldestRunTimeStart

func (t TestRuns) OldestRunTimeStart() time.Time

OldestRunTimeStart returns the TimeStart of the oldest run in the set.

func (TestRuns) SetTestRunIDs

func (t TestRuns) SetTestRunIDs(ids TestRunIDs)

SetTestRunIDs sets the ID field for each run, from the given ids.

func (TestRuns) Swap

func (t TestRuns) Swap(i, j int)

type TestRunsByProduct

type TestRunsByProduct []ProductTestRuns

TestRunsByProduct is an array of tuples of {product, matching runs}, returned when a TestRun query is executed.

func (TestRunsByProduct) AllRuns

func (t TestRunsByProduct) AllRuns() TestRuns

AllRuns returns an array of all the loaded runs.

func (TestRunsByProduct) First

func (t TestRunsByProduct) First() *TestRun

First returns the first TestRun

type TestStatus

type TestStatus int64

TestStatus represents the possible test result statuses.

func TestStatusValueFromString

func TestStatusValueFromString(str string) TestStatus

TestStatusValueFromString returns the enum value associated with str (if any), or else TestStatusDefault.

func (TestStatus) IsHarnessStatus

func (s TestStatus) IsHarnessStatus() bool

IsHarnessStatus is true if the value is TestStatusPass or TestStatusError, statuses which are used for the harness-level result.

func (TestStatus) IsPass

func (s TestStatus) IsPass() bool

IsPass is true if the value is TestStatusPass.

func (TestStatus) IsPassOrOK

func (s TestStatus) IsPassOrOK() bool

IsPassOrOK is true if the value is TestStatusPass or TestStatusOK

func (*TestStatus) MarshalYAML

func (s *TestStatus) MarshalYAML() (interface{}, error)

MarshalYAML serializes TestStatus into a YAML string.

func (TestStatus) String

func (s TestStatus) String() string

String returns the string associated with s (if any), or else TestStatusStringDefault.

func (*TestStatus) UnmarshalJSON

func (s *TestStatus) UnmarshalJSON(b []byte) error

UnmarshalJSON is the custom JSON unmarshaler for TestStatus.

func (*TestStatus) UnmarshalYAML

func (s *TestStatus) UnmarshalYAML(unmarshal func(interface{}) error) (err error)

UnmarshalYAML unmarshals a TestStatus as either a name string or a number.

type TestSummary

type TestSummary []int

TestSummary is a pair of [passing, total] counts for a test file.

func (TestSummary) Add

func (s TestSummary) Add(other TestSummary)

Add adds the other summary counts to this one. Used for summing folders.

type Token

type Token struct {
	Secret string `json:"secret"`
}

Token is used for test result uploads.

type TriageMetadata

type TriageMetadata interface {
	Triage(metadata MetadataResults) (string, error)
}

TriageMetadata encapsulates the Triage() method for testing.

func NewTriageMetadata

func NewTriageMetadata(ctx context.Context, githubClient *github.Client, authorName string, authorEmail string, fetcher MetadataFetcher) TriageMetadata

NewTriageMetadata returns an instance of the triageMetadata struct to run Triage() method.

type Uploader

type Uploader struct {
	Username string
	Password string
}

Uploader is a username/password combo accepted by the results receiver.

func GetUploader

func GetUploader(m SecretManager, uploader string) (Uploader, error)

GetUploader gets the Uploader by the given name.

type User

type User struct {
	GitHubHandle string `json:"github_handle,omitempty"`
	GitHubEmail  string `json:"github_email,omitempty"`
}

User represents an authenticated GitHub user.

func GetUserFromCookie

func GetUserFromCookie(ctx context.Context, ds Datastore, r *http.Request) (*User, string)

GetUserFromCookie extracts the User and GitHub OAuth token from a request's session cookie, if it exists. If the cookie does not exist or cannot be decoded, (nil, "") will be returned.

type Version

type Version struct {
	Major    int
	Minor    *int
	Build    *int
	Revision *int
	Channel  string
}

Version is a struct for a parsed version string.

func ParseVersion

func ParseVersion(version string) (result *Version, err error)

ParseVersion parses the given version as a semantically versioned string.

func (Version) String

func (v Version) String() string

type WebFeaturesData

type WebFeaturesData map[string]map[string]interface{}

WebFeaturesData is the public data type that represents the data parsed from a given manifest file.

func (WebFeaturesData) TestMatchesWithWebFeature

func (d WebFeaturesData) TestMatchesWithWebFeature(test, webFeature string) bool

TestMatchesWithWebFeature performs two checks. If the given test path is present in the data. If not, return false If it is present, check if the given web feature applies to that test.

Directories

Path Synopsis
Package sharedtest is a generated GoMock package.
Package sharedtest is a generated GoMock package.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL