uidetection

package
v0.0.0-...-9474aff Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 28, 2022 License: BSD-3-Clause Imports: 26 Imported by: 0

Documentation

Overview

Package uidetection provides image-based UI detections/interactions.

Index

Constants

View Source
const (
	// KeyType represents the var name of the key type used for UI detection API.
	KeyType = "uidetection.key_type"
	// Key represents the var name of the key used for UI detection API.
	Key = "uidetection.key"
	// Server represents the var name of the server address used for UI detection API.
	Server = "uidetection.server"
)
View Source
const (
	// ErrNotFound is the error when there is no matching elements found.
	ErrNotFound = "no matching elements found"
	// ErrMultipleMatch is the error when there are multiple matching elements found.
	ErrMultipleMatch = "there are multiple matches"
	// ErrNthNotFound is the error when the Nth element doesn't exist.
	ErrNthNotFound = "Nth element not found"
	// ErrEmptyBoundingBox is the error when the relative matchers create a
	// screenshot of zero size.
	ErrEmptyBoundingBox = "The element you're trying to screenshot has zero size"
)

Variables

View Source
var UIDetectionVars = []string{
	KeyType,
	Key,
	Server,
}

UIDetectionVars contains a list of all variables used by the UI detection library.

Functions

This section is empty.

Types

type Context

type Context struct {
	// contains filtered or unexported fields
}

Context provides functionalities for image-based UI automation.

func New

func New(t *chrome.TestConn, keyType, key, server string) *Context

New returns a new UI Detection automation instance.

func NewDefault

func NewDefault(t *chrome.TestConn) *Context

NewDefault returns a new UI Detection automation instance with default params.

func (*Context) Exists

func (uda *Context) Exists(s *Finder) uiauto.Action

Exists returns an action that returns nil if the specified element exists.

func (*Context) Gone

func (uda *Context) Gone(s *Finder) uiauto.Action

Gone returns an action that returns nil if the specified element doesn't exist.

func (*Context) LeftClick

func (uda *Context) LeftClick(s *Finder) uiauto.Action

LeftClick returns an action that left-clicks a finder.

func (*Context) Location

func (uda *Context) Location(ctx context.Context, s *Finder) (*Location, error)

Location finds the location of a finder in the screen.

func (*Context) RightClick

func (uda *Context) RightClick(s *Finder) uiauto.Action

RightClick returns an action that right-clicks a finder.

func (*Context) Tap

func (uda *Context) Tap(s *Finder) uiauto.Action

Tap performs a single touchscreen tap.

func (*Context) WaitUntilExists

func (uda *Context) WaitUntilExists(s *Finder) uiauto.Action

WaitUntilExists returns an action that waits until the specified element exists.

func (*Context) WaitUntilGone

func (uda *Context) WaitUntilGone(s *Finder) uiauto.Action

WaitUntilGone returns an action that waits until the specified element doesnt exist.

func (*Context) WithInterval

func (uda *Context) WithInterval(interval time.Duration) *Context

WithInterval returns a new Context with the specified polling interval.

func (*Context) WithOptions

func (uda *Context) WithOptions(optionList ...Option) *Context

WithOptions returns a new Context with the specified detection options.

func (*Context) WithPollOpts

func (uda *Context) WithPollOpts(pollOpts testing.PollOptions) *Context

WithPollOpts returns a new Context with the specified polling options.

func (*Context) WithScreenshotStrategy

func (uda *Context) WithScreenshotStrategy(s ScreenshotStrategy) *Context

WithScreenshotStrategy returns a new Context with the specified screenshot strategy.

func (*Context) WithTimeout

func (uda *Context) WithTimeout(timeout time.Duration) *Context

WithTimeout returns a new Context with the specified timeout.

type CustomIconParam

type CustomIconParam = func(*CustomIconParams)

CustomIconParam is a modifier to apply to CustomIconParams.

func MatchCount

func MatchCount(matchCount int32) CustomIconParam

MatchCount controls the MatchCount param.

func MinConfidence

func MinConfidence(minConfidence float64) CustomIconParam

MinConfidence controls the min confidence threshold.

type CustomIconParams

type CustomIconParams struct {
	// MatchCount is the limit of number of matches.
	// Set to -1 to not limit the number of matches.
	MatchCount int32
	// MinConfidence is the threshold in the range [0.0, 1.0] below which
	// the matches will be considered as non-existent.
	MinConfidence float64
}

CustomIconParams provides all of the ways which you can configure a custom icon detection.

NOTE: The default values are the recommended values. The users are not expected to modify them if the test is passing with the defaults. If the defaults are not working, the users will need to find a proper value with the trial and error method.

See https://chromium.googlesource.com/chromiumos/platform/tast/+/HEAD/docs/using_uidetection.md for more details.

func DefaultCustomIconParams

func DefaultCustomIconParams() *CustomIconParams

DefaultCustomIconParams return params with default values.

type Finder

type Finder struct {
	// contains filtered or unexported fields
}

Finder represents a data structure that consists of arguments to find a UI element.

func CustomIcon

func CustomIcon(iconFile string, paramList ...CustomIconParam) *Finder

CustomIcon returns a finder for a given icon.

func TextBlock

func TextBlock(words []string, paramList ...TextParam) *Finder

TextBlock returns a finder for a given textblock. Use TextBlock if you want to find a group of nearby text, generally representing an entire text element. E.g. use uidetection.TextBlock([]string{"Save", "As"}) to find the "Save As" menu item.

func Word

func Word(word string, paramList ...TextParam) *Finder

Word returns a finder for a given word.

func (*Finder) Above

func (f *Finder) Above(other *Finder) *Finder

Above ensures that the element returned must be above the element returned by the finder on the screen.

func (*Finder) AboveA11yNode

func (f *Finder) AboveA11yNode(other *nodewith.Finder) *Finder

AboveA11yNode ensures that the element returned must be above the element returned by the a11y node finder on the screen.

func (*Finder) AboveDp

func (f *Finder) AboveDp(dp int) *Finder

AboveDp ensures that the element returned must be above dp display pixels on the screen.

func (*Finder) AbovePx

func (f *Finder) AbovePx(px int) *Finder

AbovePx ensures that the element returned must be above px pixels on the screen.

func (*Finder) Below

func (f *Finder) Below(other *Finder) *Finder

Below ensures that the element returned must be below the element returned by the finder on the screen.

func (*Finder) BelowA11yNode

func (f *Finder) BelowA11yNode(other *nodewith.Finder) *Finder

BelowA11yNode ensures that the element returned must be below the element returned by the a11y node finder on the screen.

func (*Finder) BelowDp

func (f *Finder) BelowDp(dp int) *Finder

BelowDp ensures that the element returned must be below dp display pixels on the screen.

func (*Finder) BelowPx

func (f *Finder) BelowPx(px int) *Finder

BelowPx ensures that the element returned must be below px pixels on the screen.

func (*Finder) ExactMatch

func (f *Finder) ExactMatch() *Finder

ExactMatch turns off the approximate match for the word finder. The results will be filtered by exact string matching. USE WITH CAUTION. Due to the performance of the OCR (optical character recognition) model, approximate match is the default behavior for error-tolerance. An example use case is when the matching word is short with two or three letters. TODO(b/211937254): Allow exact matches with max_edit_distance in new proto.

func (*Finder) First

func (f *Finder) First() *Finder

First enables the finder to choose the first match of a UI element if there are multiple matches.

func (*Finder) LeftOf

func (f *Finder) LeftOf(other *Finder) *Finder

LeftOf ensures that the element returned must be left of the element returned by the finder on the screen.

func (*Finder) LeftOfA11yNode

func (f *Finder) LeftOfA11yNode(other *nodewith.Finder) *Finder

LeftOfA11yNode ensures that the element returned must be left of the element returned by the a11y node finder on the screen.

func (*Finder) LeftOfDp

func (f *Finder) LeftOfDp(dp int) *Finder

LeftOfDp ensures that the element returned must be left of dp display pixels on the screen.

func (*Finder) LeftOfPx

func (f *Finder) LeftOfPx(px int) *Finder

LeftOfPx ensures that the element returned must be left of px pixels on the screen.

func (*Finder) Nth

func (f *Finder) Nth(nth int) *Finder

Nth enables the finder to choose the n-th match of a UI element. if there are multiple matches.

func (*Finder) RightOf

func (f *Finder) RightOf(other *Finder) *Finder

RightOf ensures that the element returned must be right of the element returned by the finder on the screen.

func (*Finder) RightOfA11yNode

func (f *Finder) RightOfA11yNode(other *nodewith.Finder) *Finder

RightOfA11yNode ensures that the element returned must be right of the element returned by the a11y node finder on the screen.

func (*Finder) RightOfDp

func (f *Finder) RightOfDp(dp int) *Finder

RightOfDp ensures that the element returned must be right of dp display pixels on the screen.

func (*Finder) RightOfPx

func (f *Finder) RightOfPx(px int) *Finder

RightOfPx ensures that the element returned must be right of px pixels on the screen.

func (*Finder) Within

func (f *Finder) Within(other *Finder) *Finder

Within ensures that the element returned must be within the element returned by the finder on the screen.

func (*Finder) WithinA11yNode

func (f *Finder) WithinA11yNode(other *nodewith.Finder) *Finder

WithinA11yNode ensures that the element returned must be above the element returned by the a11y node finder on the screen.

func (*Finder) WithinDp

func (f *Finder) WithinDp(r coords.Rect) *Finder

WithinDp ensures that the element returned must be within the rectangle on the screen, measuren in dp.

func (*Finder) WithinPx

func (f *Finder) WithinPx(r coords.Rect) *Finder

WithinPx ensures that the element returned must be within the rectangle on the screen, measured in pixels.

type Location

type Location struct {
	// Rectangle of the location.
	coords.Rect
	// Text associated with the element, if any.
	Text string
}

Location represents the location of a matching UI element.

type Option

type Option = func(*Options)

Option is a modifier to apply to Options.

func Retries

func Retries(retries int) Option

Retries controls the number of retries.

func RetryInterval

func RetryInterval(retryInterval time.Duration) Option

RetryInterval controls the interval between retries.

type Options

type Options struct {
	// The number of times of retries.
	Retries int
	// The interval between retries.
	RetryInterval time.Duration
}

Options provides all of the ways which you can configure the UI detection API.

func DefaultOptions

func DefaultOptions() *Options

DefaultOptions return options with default values.

type ScreenshotStrategy

type ScreenshotStrategy int

ScreenshotStrategy holds the different screenshot strategies that can be used for image-based UI detection.

const (
	StableScreenshot ScreenshotStrategy = iota
	ImmediateScreenshot
)

Holds all the screenshot types that can be used.

type TextParam

type TextParam = func(*TextParams)

TextParam is a modifier to apply to TextParams.

func DisableApproxMatch

func DisableApproxMatch(disableApproxMatch bool) TextParam

DisableApproxMatch controls the DisableApproxMatch param.

func MaxEditDistance

func MaxEditDistance(maxEditDistance int32) TextParam

MaxEditDistance controls the MaxEditDistance param.

func RegexMode

func RegexMode(regexMode bool) TextParam

RegexMode controls the RegexMode param.

type TextParams

type TextParams struct {
	// RegexMode indicates whether the query string is a regex or not.
	RegexMode bool
	// DisableApproxMatch disables the approximate match.
	// approxmiate match is enabled by default to tolerate recognition errors,
	// so that some similar characters (i.e., "5" and "s") are treated the same.
	// Normally, you don't want to turn the approxmiate match off.
	DisableApproxMatch bool
	// MaxEditDistance is the Levenshtein distance threshold.
	// For example "string" and "sting" is the same match if MaxEditDistance=1.
	// NOTE: this param is applicable only if RegexMode is False.
	MaxEditDistance int32
}

TextParams provides all of the ways which you can configure a word detection or a textblock detection.

NOTE: The default values are the recommended values. The users are not expected to modify them if the test is passing with the defaults. If the defaults are not working, the users will need to find a proper value with the trial and error method.

See https://chromium.googlesource.com/chromiumos/platform/tast/+/HEAD/docs/using_uidetection.md for more details.

func DefaultTextParams

func DefaultTextParams() *TextParams

DefaultTextParams return params with default values.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL