cockroach: github.com/cockroachdb/cockroach/pkg/ccl/workloadccl Index | Files | Directories

package workloadccl

import "github.com/cockroachdb/cockroach/pkg/ccl/workloadccl"

Index

Package Files

fixture.go

func FixtureURL Uses

func FixtureURL(config FixtureConfig, gen workload.Generator) string

FixtureURL returns the URL for pre-computed Generator data stored on GCS.

func ImportFixture Uses

func ImportFixture(
    ctx context.Context,
    sqlDB *gosql.DB,
    gen workload.Generator,
    dbName string,
    filesPerNode int,
    injectStats bool,
    csvServer string,
) (int64, error)

ImportFixture works like MakeFixture, but instead of stopping halfway or writing a backup to cloud storage, it finishes ingesting the data. It also includes the option to inject pre-calculated table statistics if injectStats is true.

func ListFixtures Uses

func ListFixtures(
    ctx context.Context, gcs *storage.Client, config FixtureConfig,
) ([]string, error)

ListFixtures returns the object paths to all fixtures stored in a FixtureConfig.

func RestoreFixture Uses

func RestoreFixture(
    ctx context.Context, sqlDB *gosql.DB, fixture Fixture, database string, injectStats bool,
) (int64, error)

RestoreFixture loads a fixture into a CockroachDB cluster. An enterprise license is required to have been set in the cluster.

type Fixture Uses

type Fixture struct {
    Config    FixtureConfig
    Generator workload.Generator
    Tables    []FixtureTable
}

Fixture describes pre-computed data for a Generator, allowing quick initialization of large clusters.

func GetFixture Uses

func GetFixture(
    ctx context.Context, gcs *storage.Client, config FixtureConfig, gen workload.Generator,
) (Fixture, error)

GetFixture returns a handle for pre-computed Generator data stored on GCS. It is expected that the generator will have had Configure called on it.

func MakeFixture Uses

func MakeFixture(
    ctx context.Context,
    sqlDB *gosql.DB,
    gcs *storage.Client,
    config FixtureConfig,
    gen workload.Generator,
    filesPerNode int,
) (Fixture, error)

MakeFixture regenerates a fixture, storing it to GCS. It is expected that the generator will have had Configure called on it.

There's some ideal world in which we can generate backups (and thus fixtures) directly from a Generator, but for now, we use `IMPORT ... CSV DATA`. First a CSV file with the table data is written to GCS. `IMPORT ... CSV DATA` works by turning a set of CSV files for a single table into a backup file, then restoring that file into a cluster. The `transform` option gives us only the first half (which is all we want for fixture generation).

type FixtureConfig Uses

type FixtureConfig struct {
    // GCSBucket is a Google Cloud Storage bucket.
    GCSBucket string

    // GCSPrefix is a prefix to prepend to each Google Cloud Storage object
    // path.
    GCSPrefix string

    // CSVServerURL is a url to a `./workload csv-server` to use as a source of
    // CSV data. The url is anything accepted by our backup/restore. Notably, if
    // you run a csv-server next to each CockroachDB node,
    // `http://localhost:<port>` will work.
    CSVServerURL string

    // BillingProject if non-empty, is the Google Cloud project to bill for all
    // storage requests. This is required to be set if using a "requestor pays"
    // bucket.
    BillingProject string
}

FixtureConfig describes a storage place for fixtures.

type FixtureTable Uses

type FixtureTable struct {
    TableName string
    BackupURI string
}

FixtureTable describes pre-computed data for a single table in a Generator, allowing quick initializaiton of large clusters.

type ImportDataLoader Uses

type ImportDataLoader struct {
    FilesPerNode int
    InjectStats  bool
    CSVServer    string
}

ImportDataLoader is an InitialDataLoader implementation that loads data with IMPORT. The zero-value gets some sane defaults for the tunable settings.

func (ImportDataLoader) InitialDataLoad Uses

func (l ImportDataLoader) InitialDataLoad(
    ctx context.Context, db *gosql.DB, gen workload.Generator,
) (int64, error)

InitialDataLoad implements the InitialDataLoader interface.

Directories

PathSynopsis
allccl
cliccl
format
roachmartccl

Package workloadccl imports 21 packages (graph) and is imported by 3 packages. Updated 2019-11-12. Refresh now. Tools for package owners.