updater

package
v0.0.173 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 18, 2024 License: Apache-2.0 Imports: 32 Imported by: 0

Documentation

Overview

Package updater reads the latest test results and saves updated state.

Index

Constants

View Source 
const ElapsedKey = "test-duration-minutes"

ElapsedKey is the key for the target duration metric.

View Source 
const EmailListKey = "EmailAddresses"

EmailListKey is the expected metadata key for email addresses.

View Source 
const TestMethodsElapsedKey = "test-methods-duration-minutes"

TestMethodsElapsedKey is the key for the test results duration metric.

Variables

This section is empty.

Functions

func AddManualSubscription added in v0.0.91 

func AddManualSubscription(projID, subID, prefix string)

AddManualSubscription allows injecting additional subscriptions that are not specified by the test group itself.

Likely to be removed (or migrated into the config.proto) in a future version.

func AppendColumn added in v0.0.133 

func AppendColumn(grid *statepb.Grid, rows map[string]*statepb.Row, inflated InflatedColumn)

AppendColumn adds the build column to the grid.

This handles details like: * rows appearing/disappearing in the middle of the run. * adding auto metadata like duration, commit as well as any user-added metadata * extracting build metadata into the appropriate column header * Ensuring row names are unique and formatted with metadata

func ConstructGrid added in v0.0.72 

func ConstructGrid(log logrus.FieldLogger, cols []InflatedColumn, issues map[string][]string, failuresToAlert, passesToDisableAlert int, useCommitAsBuildID bool, userProperty string, brokenThreshold float32, columnHeader []*configpb.TestGroup_ColumnHeader) *statepb.Grid

ConstructGrid will append all the inflatedColumns into the returned Grid.

The returned Grid has correctly compressed row values.

func CustomStatus added in v0.0.121 

func CustomStatus(rules []*evalpb.Rule, testResult TestResult) *tspb.TestStatus

CustomStatus evaluates the result according to the rules.

Returns nil if no rule matches, otherwise returns the overridden status.

func CustomTargetStatus added in v0.0.158 

func CustomTargetStatus(rules []*evalpb.Rule, targetResult TargetResult) *tspb.TestStatus

CustomTargetStatus evaluates the result according to the rules.

Returns nil if no rule matches, otherwise returns the overridden status.

func FormatStrftime added in v0.0.164 

func FormatStrftime(in string) string

FormatStrftime replaces python codes with what go expects.

aka %Y-%m-%d becomes 2006-01-02

func InflateDropAppend added in v0.0.56 

func InflateDropAppend(ctx context.Context, alog logrus.FieldLogger, client gcs.Client, tg *configpb.TestGroup, gridPath gcs.Path, write bool, readCols ColumnReader, reprocess time.Duration) (bool, error)

InflateDropAppend updates groups by downloading the existing grid, dropping old rows and appending new ones.

func Means added in v0.0.56 

func Means(properties map[string][]string) map[string]float64

Means returns means for each given property's values.

func SortStarted added in v0.0.64 

func SortStarted(cols []InflatedColumn)

SortStarted sorts InflatedColumns by column start time.

func SplitCells added in v0.0.56 

func SplitCells(originalName string, cells ...Cell) map[string]Cell

SplitCells appends a unique suffix to each cell.

When an excessive number of cells contain the same name the list gets truncated, replaced with a synthetic "... [overflow]" cell.

func TestGroupPath added in v0.0.118 

func TestGroupPath(g gcs.Path, gridPrefix, groupName string) (*gcs.Path, error)

TestGroupPath returns the path to a test_group proto given this proto

func Update 

func Update(parent context.Context, client gcs.ConditionalClient, mets *Metrics, updateGroup GroupUpdater, opts *UpdateOptions, fixers ...Fixer) error

Update test groups with the specified freq.

Retries errors at double and unfinished groups as soon as possible.

Filters down to a single group when set. Returns after all groups updated once if freq is zero.

Types

type Cell added in v0.0.56 

type Cell struct {
	// Result determines the color of the cell, defaulting to NO_RESULT (clear)
	Result statuspb.TestStatus

	// The name of the row before user-customized formatting
	ID string

	// CellID specifies the an identifier to the build, which allows
	// clicking different cells in a column to go to different locations.
	CellID string

	// Properties maps key:value pairs for cell IDs.
	Properties map[string]string

	// Icon is a short string that appears on the cell
	Icon string
	// Message is a longer string that appears on mouse-over
	Message string

	// Metrics holds numerical data, such as how long it ran, coverage, etc.
	Metrics map[string]float64

	// UserProperty holds the value of a user-defined property, which allows
	// runtime flexibility in generating links to click on.
	UserProperty string

	// Issues relevant to this cell
	// TODO(fejta): persist cell association, currently gets written out as a row-association.
	// TODO(fejta): support issue association when parsing prow job results.
	Issues []string
}

Cell holds a row's values for a given column

func MergeCells added in v0.0.56 

func MergeCells(flaky bool, cells ...Cell) Cell

MergeCells will combine the cells into a single result.

The flaky argument determines whether returned result is flaky (true) or failing when merging cells with both passing and failing results.

Merging multiple results will set the icon to n/N passes

Includes the message from the "most relevant" cell that includes a message. Where relevance is determined by result.GTE.

type ColumnReader added in v0.0.56 

type ColumnReader func(ctx context.Context, log logrus.FieldLogger, tg *configpb.TestGroup, oldCols []InflatedColumn, stop time.Time, receivers chan<- InflatedColumn) error

ColumnReader finds, processes and new columns to send to the receivers.

* Columns with the same Name and Build will get merged together. * Readers must be reentrant.

  • Processing must expect every sent column to be the final column this cycle. AKA calling this method once and reading two columns should be equivalent to calling the method once, reading one column and then calling it a second time and reading a second column.

type Fixer added in v0.0.91 

type Fixer func(context.Context, logrus.FieldLogger, *config.TestGroupQueue, []*configpb.TestGroup) error

Fixer will fix the TestGroupQueue's next time for TestGroups.

Fixer should: * work continually and not return until the context expires. * expect to be called multiple times with different contexts and test groups.

For example, it might use the last updated time of the test group to specify the next update time. Or it might watch the data backing these groups and request an immediate update whenever the data changes.

func FixGCS added in v0.0.91 

func FixGCS(subscriber pubsub.Subscriber) Fixer

FixGCS listens for changes to GCS files and schedules another update of those groups ~immediately.

Limited to test groups with a gcs_config result_source that includes pubsub info. Returns when the context is canceled or a processing error occurs.

func FixPersistent added in v0.0.120 

func FixPersistent(log logrus.FieldLogger, client queue.PersistClient, path gcs.Path, tick <-chan time.Time) Fixer

FixPersistent persists the updater queue using queue.FixPersistent.

type GroupUpdater added in v0.0.29 

type GroupUpdater func(parent context.Context, log logrus.FieldLogger, client gcs.Client, tg *configpb.TestGroup, gridPath gcs.Path) (bool, error)

GroupUpdater will compile the grid state proto for the specified group and upload it.

This typically involves downloading the existing state, dropping old columns, compiling any new columns and inserting them into the front and then uploading the proto to GCS.

Disable pooled downloads with a nil poolCtx, otherwise at most concurrency builds will be downloaded at the same time.

Return true if there are more results to process.

func GCS added in v0.0.29 

func GCS(poolCtx context.Context, colClient gcs.Client, groupTimeout, buildTimeout time.Duration, concurrency int, write bool, enableIgnoreSkip bool) GroupUpdater

GCS returns a GCS-based GroupUpdater, which knows how to process result data stored in GCS.

type InflatedColumn added in v0.0.56 

type InflatedColumn struct {
	// Column holds the header data.
	Column *statepb.Column
	// Cells holds each row's uncompressed data for this column.
	Cells map[string]Cell // TODO(fejta): *Cell
}

InflatedColumn holds all the entries for a given column.

This includes both: * Column state metadata and * Cell values for every row in this column

func InflateGrid added in v0.0.70 

func InflateGrid(ctx context.Context, grid *statepb.Grid, earliest, latest time.Time) ([]InflatedColumn, map[string][]string, error)

InflateGrid inflates the grid's rows into an InflatedColumn channel.

Drops columns before earliest or more recent than latest. Also returns a map of issues associated with each row name.

type Metrics added in v0.0.72 

type Metrics struct {
	UpdateState  metrics.Cyclic
	DelaySeconds metrics.Duration
}

Metrics holds metrics relevant to the Updater.

func CreateMetrics added in v0.0.111 

func CreateMetrics(factory metrics.Factory) *Metrics

CreateMetrics creates metrics for this controller

type TargetResult added in v0.0.158 

type TargetResult interface {
	TargetStatus() tspb.TestStatus
}

TargetResult defines the interface for accessing data about the target/suite result.

type TestResult added in v0.0.121 

type TestResult interface {
	// Properties for the test result.
	Properties() map[string][]string
	// Name of the test case
	Name() string
	// The number of errors in the test case.
	Errors() int
	// The number of failures in the test case.
	Failures() int
	// The sequence of exception/error messages in the test case.
	Exceptions() []string
}

TestResult defines the interface for accessing data about the result.

type UpdateOptions added in v0.0.156 

type UpdateOptions struct {
	ConfigPath       gcs.Path
	GridPrefix       string
	GroupConcurrency int
	GroupNames       []string
	Write            bool
	Freq             time.Duration
}

UpdateOptions aggregates the Update function parameter into a single structure.

Source Files

View all Source files

Directories

Path Synopsis
Package resultstore fetches and process results from ResultStore.
 Package resultstore fetches and process results from ResultStore.
query

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.