bq

package
v0.0.0-...-a6a2865 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Feb 15, 2024 License: Apache-2.0 Imports: 9 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source
var (
	// ErrTableNotFound is returned when trying to load to non-existent tmp tables.
	ErrTableNotFound = errors.New("table not found")
	// ErrDatatypeNotSupported is returned by Query for unsupported datatypes.
	ErrDatatypeNotSupported = errors.New("datatype not supported")

	// ErrNilBigQueryClient is returned when an invalid client is provided.
	ErrNilBigQueryClient = errors.New("nil bigquery client")
	// ErrNilQuery is returned when the query is empty or nil.
	ErrNilQuery = errors.New("bigquery query is empty or nil")
)

Functions

This section is empty.

Types

type TableOps

type TableOps struct {
	LoadSource string // The bucket/path to load from.
	Project    string
	Date       string // Name of the partition field
	Job        tracker.Job
	// map key is the single field name, value is fully qualified name
	PartitionKeys map[string]string
	OrderKeys     string
	// contains filtered or unexported fields
}

TableOps is used to construct and execute table partition operations.

func NewTableOps

func NewTableOps(ctx context.Context, job tracker.Job, project string, loadSource string) (*TableOps, error)

NewTableOps creates a suitable QueryParams for a Job. The context is used to create a bigquery client, and should be kept alive while the querier is in use.

func NewTableOpsWithClient

func NewTableOpsWithClient(client bqiface.Client, job tracker.Job, project string, loadSource string) (*TableOps, error)

NewTableOpsWithClient creates a suitable QueryParams for a Job.

func (TableOps) CopyToRaw

func (to TableOps) CopyToRaw(ctx context.Context, dryRun bool) (bqiface.Job, error)

CopyToRaw copies the job "tmp" table partition to the "raw" table partition.

func (TableOps) Dedup

func (to TableOps) Dedup(ctx context.Context, dryRun bool) (bqiface.Job, error)

Dedup initiates a deduplication query, and returns the bqiface.Job.

func (TableOps) DeleteTmp

func (to TableOps) DeleteTmp(ctx context.Context) error

DeleteTmp deletes the tmp table partition.

func (TableOps) Join

func (to TableOps) Join(ctx context.Context, dryRun bool) (bqiface.Job, error)

Join joins the raw tables into annotated tables.

func (TableOps) LoadToTmp

func (to TableOps) LoadToTmp(ctx context.Context, dryRun bool) (bqiface.Job, error)

LoadToTmp loads the "tmp" experiment table from files previously written to GCS by the parsers (or other source).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL