luci: go.chromium.org/luci/resultdb/internal/services/bqexporter Index | Files

package bqexporter

import "go.chromium.org/luci/resultdb/internal/services/bqexporter"

Index

Package Files

bqexporter.go quota_iterator.go test_result_row.go

func InitServer Uses

func InitServer(srv *server.Server, opts Options)

InitServer initializes a bqexporter server.

type Invocation Uses

type Invocation struct {
    // ID is the ID of the invocation.
    ID  string `bigquery:"id"`

    // Tags represents Invocation-level string key-value pairs.
    // A key can be repeated.
    Tags []StringPair `bigquery:"tags"`

    // The LUCI Realm the invocation exists under.
    Realm string `bigquery:"realm"`
}

Invocation is a subset of pb.Invocation for the invocation fields that need to be saved in a BQ table.

type Options Uses

type Options struct {
    // How often to query for tasks.
    TaskQueryInterval time.Duration

    // How long to lease a task for.
    TaskLeaseDuration time.Duration

    // Whether to use InsertIDs in BigQuery Streaming Inserts.
    UseInsertIDs bool

    // Maximum number of rows in a batch.
    MaxBatchRowCount int

    // Maximum size of a batch in bytes, approximate.
    MaxBatchSizeApprox int

    // Maximum size of all batches held in memory, approximate.
    MaxBatchTotalSizeApprox int

    // Maximum rate for BigQuery Streaming Inserts.
    RateLimit rate.Limit

    // Number of invocations to export concurrently.
    // This number should be small (e.g. 10) if this ResultDB instance mostly
    // exports huge invocations (10k-100k results per invocation), and it should
    // be large (e.g. 100) if exports small invocations (1000 results per
    // invocation).
    TaskWorkers int
}

Options is bqexpoerter configuration.

func DefaultOptions Uses

func DefaultOptions() Options

DefaultOptions returns Options with default values.

type StringPair Uses

type StringPair struct {
    Key   string `bigquery:"key"`
    Value string `bigquery:"value"`
}

StringPair is a copy of pb.StringPair, suitable for representing a key:value pair in a BQ table. Inferred to be a field of type RECORD with Key and Value string fields.

type TestLocation Uses

type TestLocation struct {
    FileName string `bigquery:"file_name"`
    Line     int    `bigquery:"line"`
}

TestLocation is a location of a test definition, e.g. the file name. For field description, see the comments in the TestLocation protobuf message.

type TestResultRow Uses

type TestResultRow struct {
    // ExportedInvocation contains info of the exported invocation.
    //
    // Note: it's possible that this invocation is not the result's
    // immediate parent invocation, but the including invocation.
    // For example if the BigQuery table is for all test results of CI builds,
    // then the exported invocation is for a CI build, which includes multiple
    // invocations for swarming tasks within that build.
    ExportedInvocation Invocation `bigquery:"exported"`

    // ParentInvocation contains info of the result's immediate parent
    // invocation.
    ParentInvocation Invocation `bigquery:"parent"`

    // TestID is a unique identifier of the test in a LUCI project.
    // Refer to pb.TestResult.TestId for details.
    TestID string `bigquery:"test_id"`

    // ResultID identifies a test result in a given invocation and test id.
    ResultID string `bigquery:"result_id"`

    // Variant describes one specific way of running the test,
    // e.g. a specific bucket, builder and a test suite.
    Variant []StringPair `bigquery:"variant"`

    // A hex-encoded sha256 of concatenated "<key>:<value>\n" variant pairs.
    VariantHash string `bigquery:"variant_hash"`

    // Expected is a flag indicating whether the result of test case execution is expected.
    // Refer to pb.TestResult.Expected for details.
    Expected bool `bigquery:"expected"`

    // Status of the test result.
    // See pb.TestStatus for possible values.
    Status string `bigquery:"status"`

    // SummaryHTML is a human-readable explanation of the result, in HTML.
    SummaryHTML string `bigquery:"summary_html"`

    // StartTime is the point in time when the test case started to execute.
    StartTime bigquery.NullTimestamp `bigquery:"start_time"`

    // Duration of the test case execution in seconds.
    Duration bigquery.NullFloat64 `bigquery:"duration"`

    // Tags contains metadata for this test result.
    // It might describe this particular execution or the test case.
    Tags []StringPair `bigquery:"tags"`

    // If the failures of the test variant are exonerated.
    // Note: the exoneration is at the test variant level, not result level.
    Exonerated bool `bigquery:"exonerated"`

    // PartitionTime is used to partition the table.
    // It is the time when exported invocation was created in Spanner.
    // https://cloud.google.com/bigquery/docs/creating-column-partitions#limitations
    // mentions "The partitioning column must be a top-level field."
    // So we keep this column here instead of adding the CreateTime to Invocation.
    PartitionTime time.Time `bigquery:"partition_time"`

    // TestLocation is the location of the test definition.
    TestLocation *TestLocation `bigquery:"test_location"`
}

TestResultRow represents a row in a BigQuery table for result of a functional test case.

func (*TestResultRow) Name Uses

func (tr *TestResultRow) Name() string

Name returns test result name.

Package bqexporter imports 30 packages (graph) and is imported by 1 packages. Updated 2020-09-30. Refresh now. Tools for package owners.