go-vitess.v1: gopkg.in/src-d/go-vitess.v1/vt/vttest Index | Files

package vttest

import "gopkg.in/src-d/go-vitess.v1/vt/vttest"

Package vttest contains helpers to set up Vitess for testing.


Package Files

environment.go local_cluster.go mysqlctl.go randomdata.go shard_name.go vtprocess.go


const DefaultCharset = "utf8"

DefaultCharset is the default charset used by MySQL instances

const DefaultMySQLFlavor = "MySQL56"

DefaultMySQLFlavor is the MySQL flavor used by vttest when MYSQL_FLAVOR is not set in the environment


var NewDefaultEnv = defaultEnvFactory

NewDefaultEnv is an user-configurable callback that returns a new Environment instance with the default settings. This callback is only used in cases where the user hasn't explicitly set the Env variable when initializing a LocalCluster

var QueryServerArgs = []string{
    "-queryserver-config-pool-size", "4",
    "-queryserver-config-query-timeout", "300",
    "-queryserver-config-schema-reload-time", "60",
    "-queryserver-config-stream-pool-size", "4",
    "-queryserver-config-transaction-cap", "4",
    "-queryserver-config-transaction-timeout", "300",
    "-queryserver-config-txpool-timeout", "300",

QueryServerArgs are the default arguments passed to all Vitess query servers

func GetMySQLOptions Uses

func GetMySQLOptions(flavor string) (string, []string, error)

GetMySQLOptions returns the default option set for the given MySQL flavor. If flavor is not set, the value from the `MYSQL_FLAVOR` env variable is used, and if this is not set, DefaultMySQLFlavor will be used. Returns the name of the MySQL flavor being used, the set of MySQL CNF files specific to this flavor, and any errors.

func GetShardName Uses

func GetShardName(shard, total int) string

GetShardName returns an appropriate shard name, as a string. A single shard name is simply 0; otherwise it will attempt to split up 0x100 into multiple shards. For example, in a two sharded keyspace, shard 0 is -80, shard 1 is 80-. This function currently only applies to sharding setups where the shard count is 256 or less, and all shards are equal width.

func GetShardNames Uses

func GetShardNames(total int) (names []string)

GetShardNames creates a slice of shard names for N shards

func LoadSQLFile Uses

func LoadSQLFile(filename, sourceroot string) ([]string, error)

LoadSQLFile loads a parses a .sql file from disk, removing all the different comments that mysql/mysqldump inserts in these, and returning each individual SQL statement as its own string. If sourceroot is set, that directory will be used when resolving `source ` statements in the SQL file.

type Config Uses

type Config struct {
    // Topology defines the fake cluster's topology. This field is mandatory.
    // See: vt/proto/vttest.VTTestTopology
    Topology *vttestpb.VTTestTopology

    // Seed can be set with a SeedConfig struct to enable
    // auto-initialization of the database cluster with random data.
    // If nil, no random initialization will be performed.
    // See: SeedConfig
    Seed *SeedConfig

    // SchemaDir is the directory for schema files. Within this dir,
    // there should be a subdir for each keyspace. Within each keyspace
    // dir, each file is executed as SQL after the database is created on
    // each shard.
    // If the directory contains a `vschema.json`` file, it will be used
    // as the VSchema for the V3 API
    SchemaDir string

    // DefaultSchemaDir is the default directory for initial schema files.
    // If no schema is found in SchemaDir, default to this location.
    DefaultSchemaDir string

    // DataDir is the directory where the data files will be placed.
    // If no directory is specified a random directory will be used
    // under VTDATAROOT.
    DataDir string

    // Charset is the default charset used by MySQL
    Charset string

    // WebDir is the location of the vtcld web server files
    WebDir string

    // WebDir2 is the location of the vtcld2 web server files
    WebDir2 string

    // ExtraMyCnf are the extra .CNF files to be added to the MySQL config
    ExtraMyCnf []string

    // OnlyMySQL can be set so only MySQL is initialized as part of the
    // local cluster configuration. The rest of the Vitess components will
    // not be started.
    OnlyMySQL bool

    // MySQL protocol bind address.
    // vtcombo will bind to this address when exposing the mysql protocol socket
    MySQLBindHost string

    // SnapshotFile is the path to the MySQL Snapshot that will be used to
    // initialize the mysqld instance in the cluster. Note that some environments
    // do not suppport initialization through snapshot files.
    SnapshotFile string

    // TransactionMode is SINGLE, MULTI or TWOPC
    TransactionMode string

    TransactionTimeout float64

Config are the settings used to configure the self-contained Vitess cluster. The LocalCluster struct embeds Config so it's possible to either initialize a LocalCluster with the given settings, or set the settings directly after initialization. All settings must be set before LocalCluster.Setup() is called.

func (*Config) DbName Uses

func (cfg *Config) DbName() string

DbName returns the default name for a database in this cluster. If OnlyMySQL is set, this will be the name of the single database created in MySQL. Otherwise, this will be the database that stores the first keyspace in the topology.

func (*Config) InitSchemas Uses

func (cfg *Config) InitSchemas(keyspace, schema string, vschema *vschemapb.Keyspace) error

InitSchemas is a shortcut for tests that just want to setup a single keyspace with a single SQL file, and/or a vschema. It creates a temporary directory, and puts the schema/vschema in there. It then sets the right value for cfg.SchemaDir. At the end of the test, the caller should os.RemoveAll(cfg.SchemaDir).

type Environment Uses

type Environment interface {
    // BinaryPath returns the full path to the given executable
    BinaryPath(bin string) string

    // MySQLManager is the constructor for the MySQL manager that will
    // be used by the cluster. The manager must take care of initializing
    // and destructing the MySQL instance(s) that will be used by the cluster.
    // See: vttest.MySQLManager for the interface the manager must implement
    MySQLManager(mycnf []string, snapshot string) (MySQLManager, error)

    // Directory is the path where the local cluster will store all its
    // data and metadata. For local testing, this should probably be an
    // unique temporary directory.
    Directory() string

    // LogDirectory is the directory where logs for all services in the
    // cluster will be stored.
    LogDirectory() string

    // VtcomoboArguments are the extra commandline arguments that will be
    // passed to `vtcombo`
    VtcomboArguments() []string

    // ProcessHealthCheck returns a HealthChecker for the given service.
    // The HealthChecker takes an address and attempts to check whether
    // the service is up and healthy.
    // If a given service does not require any custom health checks,
    // nil can be returned.
    ProcessHealthCheck(name string) HealthChecker

    // DefaultProtocol is the protocol used to communicate with the
    // Vitess cluster. This is usually "grpc".
    DefaultProtocol() string

    // PortForProtocol returns the listening port for a given service
    // on the given protocol. If protocol is empty, the default protocol
    // for each service is assumed.
    PortForProtocol(name, protocol string) int

    // EnvVars returns the environment variables that will be passed
    // to all Vitess processes spawned by the local cluster. These variables
    // always take precedence over the variables inherited from the current
    // process.
    EnvVars() []string

    // TearDown is called during LocalCluster.TearDown() to cleanup
    // any temporary data in the environment. Environments that can
    // last through several test runs do not need to implement it.
    TearDown() error

Environment is the interface that customizes the global settings for the test cluster. Usually the same environment settings are shared by all the LocalCluster instances in a given test suite, with each instance receiving a different Config for specific tests. For Environments that create temporary data on-disk and clean it up on termination, a brand new instance of Environment should be passed to each LocalCluster.

type FieldGenerator Uses

type FieldGenerator func(name, t string, rng *rand.Rand) (string, error)

FieldGenerator is a callback that generates the value of a random field in when seeding the database with random data. `name` is the name of the column where the field belongs, `t` is its SQL tyoe, and `rng` is the RNG currently being used, as to ensure reproducible generation between runs. A FieldGenerator must return the raw SQL data for the field, ready to be placed into a SQL statement. The returned value will _NOT_ be escaped.

type HealthChecker Uses

type HealthChecker func(addr string) bool

HealthChecker is a callback that impements a service-specific health check It must return true if the service at the given `addr` is reachable, false otherwerise.

type LocalCluster Uses

type LocalCluster struct {

    // Env is the Environment which will be used for unning this local cluster.
    // It can be set by the user before calling Setup(). If not set, Setup() will
    // use the NewDefaultEnv callback to instantiate an environment with the system
    // default settings
    Env Environment
    // contains filtered or unexported fields

LocalCluster controls a local Vitess setup for testing, containing a MySQL instance and one or more vtgate-equivalent access points. To use, simply create a new LocalCluster instance and either pass in the desired Config, or manually set each field on the struct itself. Once the struct is configured, call LocalCluster.Setup() to instantiate the cluster. See: Config for configuration settings on the cluster

func (*LocalCluster) Execute Uses

func (db *LocalCluster) Execute(sql []string, dbname string) error

Execute runs a series of SQL statements on the MySQL instance backing this local cluster. This is provided for debug/introspection purposes; normal cluster access should be performed through the Vitess GRPC interface.

func (*LocalCluster) JSONConfig Uses

func (db *LocalCluster) JSONConfig() interface{}

JSONConfig returns a key/value object with the configuration settings for the local cluster. It should be serialized with `json.Marshal`

func (*LocalCluster) MySQLAppDebugConnParams Uses

func (db *LocalCluster) MySQLAppDebugConnParams() mysql.ConnParams

MySQLAppDebugConnParams returns a mysql.ConnParams struct that can be used to connect directly to the mysqld service in the self-contained cluster, using the appdebug user. It's valid only if you used MySQLOnly option.

func (*LocalCluster) MySQLConnParams Uses

func (db *LocalCluster) MySQLConnParams() mysql.ConnParams

MySQLConnParams returns a mysql.ConnParams struct that can be used to connect directly to the mysqld service in the self-contained cluster This connection should be used for debug/introspection purposes; normal cluster access should be performed through the vtgate port.

func (*LocalCluster) Query Uses

func (db *LocalCluster) Query(sql, dbname string, limit int) (*sqltypes.Result, error)

Query runs a SQL query on the MySQL instance backing this local cluster and returns its result. This is provided for debug/introspection purposes; normal cluster access should be performed through the Vitess GRPC interface.

func (*LocalCluster) Setup Uses

func (db *LocalCluster) Setup() error

Setup brings up the self-contained Vitess cluster by spinning up MySQL and Vitess instances. The spawned processes will be running until the TearDown() method is called. Please ensure to `defer db.TearDown()` after calling this method

func (*LocalCluster) TearDown Uses

func (db *LocalCluster) TearDown() error

TearDown shuts down all the processes in the local cluster and cleans up any temporary on-disk data. If an error is returned, some of the running processes may not have been shut down cleanly and may need manual cleanup.

type LocalTestEnv Uses

type LocalTestEnv struct {
    BasePort     int
    TmpPath      string
    DefaultMyCnf []string
    Env          []string

LocalTestEnv is an Environment implementation for local testing See: NewLocalTestEnv()

func NewLocalTestEnv Uses

func NewLocalTestEnv(flavor string, basePort int) (*LocalTestEnv, error)

NewLocalTestEnv returns an instance of the default test environment used for local testing Vitess. The defaults are as follows: - Directory() is a random temporary directory in VTDATAROOT, which is cleaned up when closing the Environment. - LogDirectory() is the `logs` subdir inside Directory() - The MySQL flavor is set to `flavor`. If the argument is not set, it will default to the value of MYSQL_FLAVOR, and if this variable is not set, to DefaultMySQLFlavor - PortForProtocol() will return ports based off the given basePort. If basePort is zero, a random port between 10000 and 20000 will be chosen. - DefaultProtocol() is always "grpc" - ProcessHealthCheck() performs no service-specific health checks - BinaryPath() will look up the default Vitess binaries in VTROOT - MySQLManager() will return a vttest.Mysqlctl instance, configured with the given MySQL flavor. This will use the `mysqlctl` command to initialize and teardown a single mysqld instance.

func NewLocalTestEnvWithDirectory Uses

func NewLocalTestEnvWithDirectory(flavor string, basePort int, directory string) (*LocalTestEnv, error)

NewLocalTestEnvWithDirectory returns a new instance of the default test environment with a directory explicitly specified.

func (*LocalTestEnv) BinaryPath Uses

func (env *LocalTestEnv) BinaryPath(binary string) string

BinaryPath implements BinaryPath for LocalTestEnv

func (*LocalTestEnv) DefaultProtocol Uses

func (env *LocalTestEnv) DefaultProtocol() string

DefaultProtocol implements DefaultProtocol for LocalTestEnv. It is always GRPC.

func (*LocalTestEnv) Directory Uses

func (env *LocalTestEnv) Directory() string

Directory implements Directory for LocalTestEnv.

func (*LocalTestEnv) EnvVars Uses

func (env *LocalTestEnv) EnvVars() []string

EnvVars implements EnvVars for LocalTestEnv

func (*LocalTestEnv) LogDirectory Uses

func (env *LocalTestEnv) LogDirectory() string

LogDirectory implements LogDirectory for LocalTestEnv.

func (*LocalTestEnv) MySQLManager Uses

func (env *LocalTestEnv) MySQLManager(mycnf []string, snapshot string) (MySQLManager, error)

MySQLManager implements MySQLManager for LocalTestEnv

func (*LocalTestEnv) PortForProtocol Uses

func (env *LocalTestEnv) PortForProtocol(name, proto string) int

PortForProtocol implements PortForProtocol for LocalTestEnv.

func (*LocalTestEnv) ProcessHealthCheck Uses

func (env *LocalTestEnv) ProcessHealthCheck(name string) HealthChecker

ProcessHealthCheck implements ProcessHealthCheck for LocalTestEnv. By default, it performs no service-specific health checks

func (*LocalTestEnv) TearDown Uses

func (env *LocalTestEnv) TearDown() error

TearDown implements TearDown for LocalTestEnv

func (*LocalTestEnv) VtcomboArguments Uses

func (env *LocalTestEnv) VtcomboArguments() []string

VtcomboArguments implements VtcomboArguments for LocalTestEnv.

type MySQLManager Uses

type MySQLManager interface {
    Setup() error
    TearDown() error
    Auth() (string, string)
    Address() (string, int)
    UnixSocket() string
    Params(dbname string) mysql.ConnParams

MySQLManager is an interface to a mysqld process manager, capable of starting/shutting down mysqld services and initializing them.

type Mysqlctl Uses

type Mysqlctl struct {
    Binary    string
    InitFile  string
    Directory string
    Port      int
    MyCnf     []string
    Env       []string

Mysqlctl implements MySQLManager through Vitess' mysqlctld tool

func (*Mysqlctl) Address Uses

func (ctl *Mysqlctl) Address() (string, int)

Address returns the hostname/tcp port pair required to connect to mysqld

func (*Mysqlctl) Auth Uses

func (ctl *Mysqlctl) Auth() (string, string)

Auth returns the username/password tuple required to log in to mysqld

func (*Mysqlctl) Params Uses

func (ctl *Mysqlctl) Params(dbname string) mysql.ConnParams

Params returns the mysql.ConnParams required to connect directly to mysqld using Vitess' mysql client.

func (*Mysqlctl) Setup Uses

func (ctl *Mysqlctl) Setup() error

Setup spawns a new mysqld service and initializes it with the defaults. The service is kept running in the background until TearDown() is called.

func (*Mysqlctl) TearDown Uses

func (ctl *Mysqlctl) TearDown() error

TearDown shutdowns the running mysqld service

func (*Mysqlctl) UnixSocket Uses

func (ctl *Mysqlctl) UnixSocket() string

UnixSocket returns the path to the local Unix socket required to connect to mysqld

type SeedConfig Uses

type SeedConfig struct {
    // RngSeed is the seed uset to initialize the random number
    // generator that will be used to fill the database with
    // random data. Multiple runs with the same seed will result
    // in the same initial data
    RngSeed int

    // MinSize is the minimum number of initial rows in each tale shard
    MinSize int

    // MaxSize is the maximum number of initial rows in each table shard
    MaxSize int

    // NullProbability is the chance to initialize a field a NULL value.
    // Only applies to fields that can contain NULL values
    NullProbability float64

    // RandomField is a callback to generate the value of a random field
    RandomField FieldGenerator

SeedConfig are the settings to enable the initialization of the local cluster with random data. This struct must be set in Config before Setup() is called.

func SeedConfigDefaults Uses

func SeedConfigDefaults() *SeedConfig

SeedConfigDefaults returns the default values for SeedConfig

type VtProcess Uses

type VtProcess struct {
    Name         string
    Directory    string
    LogDirectory string
    Binary       string
    ExtraArgs    []string
    Env          []string
    Port         int
    PortGrpc     int
    HealthCheck  HealthChecker
    // contains filtered or unexported fields

VtProcess is a generic handle for a running Vitess process. It can be spawned manually or through one of the available helper methods.

func VtcomboProcess Uses

func VtcomboProcess(env Environment, args *Config, mysql MySQLManager) *VtProcess

VtcomboProcess returns a VtProcess handle for a local `vtcombo` service, configured with the given Config. The process must be manually started by calling WaitStart()

func (*VtProcess) Address Uses

func (vtp *VtProcess) Address() string

Address returns the main address for this Vitess process. This is usually the main HTTP endpoint for the service.

func (*VtProcess) IsHealthy Uses

func (vtp *VtProcess) IsHealthy() bool

IsHealthy returns whether the monitored Vitess process has started successfully.

func (*VtProcess) WaitStart Uses

func (vtp *VtProcess) WaitStart() (err error)

WaitStart spawns this Vitess process and waits for it to be up and running. The process is considered "up" when it starts serving its debug HTTP endpoint -- this means the process was successfully started. If the process is not healthy after 60s, this method will timeout and return an error.

func (*VtProcess) WaitTerminate Uses

func (vtp *VtProcess) WaitTerminate() error

WaitTerminate attemps to gracefully shutdown the Vitess process by sending a SIGTERM, then wait for up to 10s for it to exit. If the process hasn't exited cleanly after 10s, a SIGKILL is forced and the corresponding exit error is returned to the user

Package vttest imports 23 packages (graph) and is imported by 1 packages. Updated 2019-06-13. Refresh now. Tools for package owners.