go-vitess.v1: gopkg.in/src-d/go-vitess.v1/vt/wrangler Index | Files | Directories

package wrangler

import "gopkg.in/src-d/go-vitess.v1/vt/wrangler"

Package wrangler contains the Wrangler object to manage complex topology actions.

Index

Package Files

cleaner.go hook.go keyspace.go permissions.go rebuild.go reparent.go schema.go shard.go split.go tablet.go validator.go version.go wrangler.go

Constants

const (
    //
    // ChangeSlaveTypeAction CleanerFunction
    //
    // ChangeSlaveTypeActionName is the name of the action to change a slave type
    // (can be used to find such an action by name)
    ChangeSlaveTypeActionName = "ChangeSlaveTypeAction"
    //
    // TabletTagAction CleanerFunction
    //
    // TabletTagActionName is the name of the Tag action
    TabletTagActionName = "TabletTagAction"
    //
    // StartSlaveAction CleanerAction
    //
    // StartSlaveActionName is the name of the slave start action
    StartSlaveActionName = "StartSlaveAction"
    //
    // VReplication CleanerAction
    //
    // VReplicationActionName is the name of the action to execute VReplication commands
    VReplicationActionName = "VReplicationAction"
)
const (
    // DefaultFilteredReplicationWaitTime is the default value for argument filteredReplicationWaitTime.
    DefaultFilteredReplicationWaitTime = 30 * time.Second
)
const (
    // DefaultWaitForFilteredReplicationMaxDelay is the default maximum delay value used in WaitForFilteredReplication.
    DefaultWaitForFilteredReplicationMaxDelay = 30 * time.Second
)
const (
    // DefaultWaitSlaveTimeout is the default value for waitSlaveTimeout, which is used when calling method CopySchemaShardFromShard.
    DefaultWaitSlaveTimeout = 10 * time.Second
)

Variables

var (
    // DefaultActionTimeout is a good default for interactive
    // remote actions. We usually take a lock then do an action,
    // lock actions use RemoteOperationTimeout,
    // so basing this to be greater than RemoteOperationTimeout is good.
    // Use this as the default value for Context that need a deadline.
    DefaultActionTimeout = *topo.RemoteOperationTimeout * 4
)

func RecordChangeSlaveTypeAction Uses

func RecordChangeSlaveTypeAction(cleaner *Cleaner, tabletAlias *topodatapb.TabletAlias, from topodatapb.TabletType, to topodatapb.TabletType)

RecordChangeSlaveTypeAction records a new ChangeSlaveTypeAction into the specified Cleaner

func RecordStartSlaveAction Uses

func RecordStartSlaveAction(cleaner *Cleaner, tablet *topodatapb.Tablet)

RecordStartSlaveAction records a new action to restart binlog replication on a server into the specified Cleaner

func RecordTabletTagAction Uses

func RecordTabletTagAction(cleaner *Cleaner, tabletAlias *topodatapb.TabletAlias, name, value string)

RecordTabletTagAction records a new action to set / remove a tag into the specified Cleaner

func RecordVReplicationAction Uses

func RecordVReplicationAction(cleaner *Cleaner, tablet *topodatapb.Tablet, query string)

RecordVReplicationAction records an action to restart binlog replication on a server into the specified Cleaner

func ResetDebugVarsGetVersion Uses

func ResetDebugVarsGetVersion()

ResetDebugVarsGetVersion is used by tests to reset the getVersionFromTablet variable to the default one. That way we can run the unit tests in testlib/ even when another implementation of getVersionFromTablet is used.

type Cleaner Uses

type Cleaner struct {
    // contains filtered or unexported fields
}

Cleaner remembers a list of cleanup steps to perform. Just record action cleanup steps, and execute them at the end in reverse order, with various guarantees.

func (*Cleaner) CleanUp Uses

func (cleaner *Cleaner) CleanUp(wr *Wrangler) error

CleanUp will run the recorded actions. If an action on a target fails, it will not run the next action on the same target. We return the aggregate errors for all cleanups. CleanUp uses its own context, with a timeout of 5 minutes, so that clean up action will run even if the original context times out. TODO(alainjobart) Actions should run concurrently on a per target basis. They are then serialized on each target.

func (*Cleaner) Record Uses

func (cleaner *Cleaner) Record(name, target string, action CleanerFunction)

Record will add a cleaning action to the list

type CleanerFunction Uses

type CleanerFunction func(context.Context, *Wrangler) error

CleanerFunction is the interface that clean-up actions need to implement

type Wrangler Uses

type Wrangler struct {
    // contains filtered or unexported fields
}

Wrangler manages complex actions on the topology, like reparents, backups, resharding, ...

Multiple go routines can use the same Wrangler at the same time, provided they want to share the same logger / topo server / lock timeout.

func New Uses

func New(logger logutil.Logger, ts *topo.Server, tmc tmclient.TabletManagerClient) *Wrangler

New creates a new Wrangler object.

func (*Wrangler) CancelResharding Uses

func (wr *Wrangler) CancelResharding(ctx context.Context, keyspace, shard string) (err error)

CancelResharding cancels any resharding in progress on the specified keyspace/shard. This works for horizontal as well as vertical resharding.

func (*Wrangler) ChangeSlaveType Uses

func (wr *Wrangler) ChangeSlaveType(ctx context.Context, tabletAlias *topodatapb.TabletAlias, tabletType topodatapb.TabletType) error

ChangeSlaveType changes the type of tablet and recomputes all necessary derived paths in the serving graph, if necessary.

Note we don't update the master record in the Shard here, as we can't ChangeType from and out of master anyway.

func (*Wrangler) CopySchemaShard Uses

func (wr *Wrangler) CopySchemaShard(ctx context.Context, sourceTabletAlias *topodatapb.TabletAlias, tables, excludeTables []string, includeViews bool, destKeyspace, destShard string, waitSlaveTimeout time.Duration) error

CopySchemaShard copies the schema from a source tablet to the specified shard. The schema is applied directly on the master of the destination shard, and is propogated to the replicas through binlogs.

func (*Wrangler) CopySchemaShardFromShard Uses

func (wr *Wrangler) CopySchemaShardFromShard(ctx context.Context, tables, excludeTables []string, includeViews bool, sourceKeyspace, sourceShard, destKeyspace, destShard string, waitSlaveTimeout time.Duration) error

CopySchemaShardFromShard copies the schema from a source shard to the specified destination shard. For both source and destination it picks the master tablet. See also CopySchemaShard.

func (*Wrangler) DeleteKeyspace Uses

func (wr *Wrangler) DeleteKeyspace(ctx context.Context, keyspace string, recursive bool) error

DeleteKeyspace will do all the necessary changes in the topology server to entirely remove a keyspace.

func (*Wrangler) DeleteShard Uses

func (wr *Wrangler) DeleteShard(ctx context.Context, keyspace, shard string, recursive, evenIfServing bool) error

DeleteShard will do all the necessary changes in the topology server to entirely remove a shard.

func (*Wrangler) DeleteTablet Uses

func (wr *Wrangler) DeleteTablet(ctx context.Context, tabletAlias *topodatapb.TabletAlias, allowMaster bool) (err error)

DeleteTablet removes a tablet from a shard. - if allowMaster is set, we can Delete a master tablet (and clear its record from the Shard record if it was the master).

func (*Wrangler) EmergencyReparentShard Uses

func (wr *Wrangler) EmergencyReparentShard(ctx context.Context, keyspace, shard string, masterElectTabletAlias *topodatapb.TabletAlias, waitSlaveTimeout time.Duration) (err error)

EmergencyReparentShard will make the provided tablet the master for the shard, when the old master is completely unreachable.

func (*Wrangler) ExecuteFetchAsApp Uses

func (wr *Wrangler) ExecuteFetchAsApp(ctx context.Context, tabletAlias *topodatapb.TabletAlias, usePool bool, query string, maxRows int) (*querypb.QueryResult, error)

ExecuteFetchAsApp executes a query remotely using the App pool

func (*Wrangler) ExecuteFetchAsDba Uses

func (wr *Wrangler) ExecuteFetchAsDba(ctx context.Context, tabletAlias *topodatapb.TabletAlias, query string, maxRows int, disableBinlogs bool, reloadSchema bool) (*querypb.QueryResult, error)

ExecuteFetchAsDba executes a query remotely using the DBA pool

func (*Wrangler) ExecuteHook Uses

func (wr *Wrangler) ExecuteHook(ctx context.Context, tabletAlias *topodatapb.TabletAlias, hook *hk.Hook) (hookResult *hk.HookResult, err error)

ExecuteHook will run the hook on the tablet

func (*Wrangler) ExecuteTabletHook Uses

func (wr *Wrangler) ExecuteTabletHook(ctx context.Context, tablet *topodatapb.Tablet, hook *hk.Hook) (hookResult *hk.HookResult, err error)

ExecuteTabletHook will run the hook on the provided tablet.

func (*Wrangler) GetPermissions Uses

func (wr *Wrangler) GetPermissions(ctx context.Context, tabletAlias *topodatapb.TabletAlias) (*tabletmanagerdatapb.Permissions, error)

GetPermissions returns the permissions set on a remote tablet

func (*Wrangler) GetSchema Uses

func (wr *Wrangler) GetSchema(ctx context.Context, tabletAlias *topodatapb.TabletAlias, tables, excludeTables []string, includeViews bool) (*tabletmanagerdatapb.SchemaDefinition, error)

GetSchema uses an RPC to get the schema from a remote tablet

func (*Wrangler) GetVersion Uses

func (wr *Wrangler) GetVersion(ctx context.Context, tabletAlias *topodatapb.TabletAlias) (string, error)

GetVersion returns the version string from a tablet

func (*Wrangler) InitShardMaster Uses

func (wr *Wrangler) InitShardMaster(ctx context.Context, keyspace, shard string, masterElectTabletAlias *topodatapb.TabletAlias, force bool, waitSlaveTimeout time.Duration) (err error)

InitShardMaster will make the provided tablet the master for the shard.

func (*Wrangler) InitTablet Uses

func (wr *Wrangler) InitTablet(ctx context.Context, tablet *topodatapb.Tablet, allowMasterOverride, createShardAndKeyspace, allowUpdate bool) error

InitTablet creates or updates a tablet. If no parent is specified in the tablet, and the tablet has a slave type, we will find the appropriate parent. If createShardAndKeyspace is true and the parent keyspace or shard don't exist, they will be created. If allowUpdate is true, and a tablet with the same ID exists, just update it. If a tablet is created as master, and there is already a different master in the shard, allowMasterOverride must be set.

func (*Wrangler) Logger Uses

func (wr *Wrangler) Logger() logutil.Logger

Logger returns the logger associated with this wrangler.

func (*Wrangler) MigrateServedFrom Uses

func (wr *Wrangler) MigrateServedFrom(ctx context.Context, keyspace, shard string, servedType topodatapb.TabletType, cells []string, reverse bool, filteredReplicationWaitTime time.Duration) (err error)

MigrateServedFrom is used during vertical splits to migrate a served type from a keyspace to another.

func (*Wrangler) MigrateServedTypes Uses

func (wr *Wrangler) MigrateServedTypes(ctx context.Context, keyspace, shard string, cells []string, servedType topodatapb.TabletType, reverse, skipReFreshState bool, filteredReplicationWaitTime time.Duration, reverseReplication bool) (err error)

MigrateServedTypes is used during horizontal splits to migrate a served type from a list of shards to another.

func (*Wrangler) PlannedReparentShard Uses

func (wr *Wrangler) PlannedReparentShard(ctx context.Context, keyspace, shard string, masterElectTabletAlias, avoidMasterAlias *topodatapb.TabletAlias, waitSlaveTimeout time.Duration) (err error)

PlannedReparentShard will make the provided tablet the master for the shard, when both the current and new master are reachable and in good shape.

func (*Wrangler) PreflightSchema Uses

func (wr *Wrangler) PreflightSchema(ctx context.Context, tabletAlias *topodatapb.TabletAlias, changes []string) ([]*tabletmanagerdatapb.SchemaChangeResult, error)

PreflightSchema will try a schema change on the remote tablet.

func (*Wrangler) RebuildKeyspaceGraph Uses

func (wr *Wrangler) RebuildKeyspaceGraph(ctx context.Context, keyspace string, cells []string) error

RebuildKeyspaceGraph rebuilds the serving graph data while locking out other changes.

func (*Wrangler) RefreshTabletState Uses

func (wr *Wrangler) RefreshTabletState(ctx context.Context, tabletAlias *topodatapb.TabletAlias) error

RefreshTabletState refreshes tablet state

func (*Wrangler) RefreshTabletsByShard Uses

func (wr *Wrangler) RefreshTabletsByShard(ctx context.Context, si *topo.ShardInfo, tabletTypes []topodatapb.TabletType, cells []string) error

RefreshTabletsByShard calls RefreshState on all the tables of a given type in a shard. It would work for the master, but the discovery wouldn't be very efficient.

func (*Wrangler) ReloadSchema Uses

func (wr *Wrangler) ReloadSchema(ctx context.Context, tabletAlias *topodatapb.TabletAlias) error

ReloadSchema forces the remote tablet to reload its schema.

func (*Wrangler) ReloadSchemaKeyspace Uses

func (wr *Wrangler) ReloadSchemaKeyspace(ctx context.Context, keyspace string, concurrency *sync2.Semaphore, includeMaster bool) error

ReloadSchemaKeyspace reloads the schema in all shards in a keyspace. The concurrency is shared across all shards (only that many tablets will be reloaded at once).

func (*Wrangler) ReloadSchemaShard Uses

func (wr *Wrangler) ReloadSchemaShard(ctx context.Context, keyspace, shard, replicationPos string, concurrency *sync2.Semaphore, includeMaster bool)

ReloadSchemaShard reloads the schema for all slave tablets in a shard, after they reach a given replication position (empty pos means immediate). In general, we don't always expect all slaves to be ready to reload, and the periodic schema reload makes them self-healing anyway. So we do this on a best-effort basis, and log warnings for any tablets that fail to reload within the context deadline.

func (*Wrangler) RemoveKeyspaceCell Uses

func (wr *Wrangler) RemoveKeyspaceCell(ctx context.Context, keyspace, cell string, force, recursive bool) error

RemoveKeyspaceCell will remove a cell from the Cells list in all shards of a keyspace (by calling RemoveShardCell on every shard). It will also remove the SrvKeyspace for that keyspace/cell.

func (*Wrangler) RemoveShardCell Uses

func (wr *Wrangler) RemoveShardCell(ctx context.Context, keyspace, shard, cell string, force, recursive bool) error

RemoveShardCell will remove a cell from the Cells list in a shard.

It will first check the shard has no tablets there. If 'force' is specified, it will remove the cell even when the tablet map cannot be retrieved. This is intended to be used when a cell is completely down and its topology server cannot even be reached.

If 'recursive' is specified, it will delete any tablets in the cell/shard, with the assumption that the tablet processes have already been terminated.

func (*Wrangler) ReparentTablet Uses

func (wr *Wrangler) ReparentTablet(ctx context.Context, tabletAlias *topodatapb.TabletAlias) error

ReparentTablet tells a tablet to reparent this tablet to the current master, based on the current replication position. If there is no match, it will fail.

func (*Wrangler) SetKeyspaceServedFrom Uses

func (wr *Wrangler) SetKeyspaceServedFrom(ctx context.Context, keyspace string, servedType topodatapb.TabletType, cells []string, sourceKeyspace string, remove bool) (err error)

SetKeyspaceServedFrom locks a keyspace and changes its ServerFromMap

func (*Wrangler) SetKeyspaceShardingInfo Uses

func (wr *Wrangler) SetKeyspaceShardingInfo(ctx context.Context, keyspace, shardingColumnName string, shardingColumnType topodatapb.KeyspaceIdType, force bool) (err error)

SetKeyspaceShardingInfo locks a keyspace and sets its ShardingColumnName and ShardingColumnType

func (*Wrangler) SetLogger Uses

func (wr *Wrangler) SetLogger(logger logutil.Logger)

SetLogger can be used to change the current logger. Not synchronized, no calls to this wrangler should be in progress.

func (*Wrangler) SetShardIsMasterServing Uses

func (wr *Wrangler) SetShardIsMasterServing(ctx context.Context, keyspace, shard string, isMasterServing bool) (err error)

SetShardIsMasterServing changes the IsMasterServing parameter of a shard. It does not rebuild any serving graph or do any consistency check. This is an emergency manual operation.

func (*Wrangler) SetShardTabletControl Uses

func (wr *Wrangler) SetShardTabletControl(ctx context.Context, keyspace, shard string, tabletType topodatapb.TabletType, cells []string, remove bool, blacklistedTables []string) (err error)

SetShardTabletControl changes the TabletControl records for a shard. It does not rebuild any serving graph or do cross-shard consistency check. - sets black listed tables in tablet control record

This takes the keyspace lock as to not interfere with resharding operations.

func (*Wrangler) SetSourceShards Uses

func (wr *Wrangler) SetSourceShards(ctx context.Context, keyspace, shard string, sources []*topodatapb.TabletAlias, tables []string) error

SetSourceShards is a utility function to override the SourceShards fields on a Shard.

func (*Wrangler) ShardReplicationStatuses Uses

func (wr *Wrangler) ShardReplicationStatuses(ctx context.Context, keyspace, shard string) ([]*topo.TabletInfo, []*replicationdatapb.Status, error)

ShardReplicationStatuses returns the ReplicationStatus for each tablet in a shard.

func (*Wrangler) ShowResharding Uses

func (wr *Wrangler) ShowResharding(ctx context.Context, keyspace, shard string) (err error)

ShowResharding shows all resharding related metadata for the keyspace/shard.

func (*Wrangler) SourceShardAdd Uses

func (wr *Wrangler) SourceShardAdd(ctx context.Context, keyspace, shard string, uid uint32, skeyspace, sshard string, keyRange *topodatapb.KeyRange, tables []string) (err error)

SourceShardAdd will add a new SourceShard inside a shard.

func (*Wrangler) SourceShardDelete Uses

func (wr *Wrangler) SourceShardDelete(ctx context.Context, keyspace, shard string, uid uint32) (err error)

SourceShardDelete will delete a SourceShard inside a shard, by index.

This takes the keyspace lock as not to interfere with resharding operations.

func (*Wrangler) SplitClone Uses

func (wr *Wrangler) SplitClone(ctx context.Context, keyspace string, from, to []string) error

SplitClone initiates a SplitClone workflow.

func (*Wrangler) TabletManagerClient Uses

func (wr *Wrangler) TabletManagerClient() tmclient.TabletManagerClient

TabletManagerClient returns the tmclient.TabletManagerClient this wrangler is using.

func (*Wrangler) TopoServer Uses

func (wr *Wrangler) TopoServer() *topo.Server

TopoServer returns the topo.Server this wrangler is using.

func (*Wrangler) UpdateDisableQueryService Uses

func (wr *Wrangler) UpdateDisableQueryService(ctx context.Context, keyspace, shard string, tabletType topodatapb.TabletType, cells []string, disableQueryService bool) (err error)

UpdateDisableQueryService changes the TabletControl records for a shard. It updates serving graph

This takes the keyspace lock as to not interfere with resharding operations.

func (*Wrangler) UpdateSrvKeyspacePartitions Uses

func (wr *Wrangler) UpdateSrvKeyspacePartitions(ctx context.Context, keyspace, shard string, tabletType topodatapb.TabletType, cells []string, remove bool) (err error)

UpdateSrvKeyspacePartitions changes the SrvKeyspaceGraph for a shard. It updates serving graph

This takes the keyspace lock as to not interfere with resharding operations.

func (*Wrangler) VReplicationExec Uses

func (wr *Wrangler) VReplicationExec(ctx context.Context, tabletAlias *topodatapb.TabletAlias, query string) (*querypb.QueryResult, error)

VReplicationExec executes a query remotely using the DBA pool

func (*Wrangler) Validate Uses

func (wr *Wrangler) Validate(ctx context.Context, pingTablets bool) error

Validate a whole TopologyServer tree

func (*Wrangler) ValidateKeyspace Uses

func (wr *Wrangler) ValidateKeyspace(ctx context.Context, keyspace string, pingTablets bool) error

ValidateKeyspace will validate a bunch of information in a keyspace is correct.

func (*Wrangler) ValidatePermissionsKeyspace Uses

func (wr *Wrangler) ValidatePermissionsKeyspace(ctx context.Context, keyspace string) error

ValidatePermissionsKeyspace validates all the permissions are the same in a keyspace

func (*Wrangler) ValidatePermissionsShard Uses

func (wr *Wrangler) ValidatePermissionsShard(ctx context.Context, keyspace, shard string) error

ValidatePermissionsShard validates all the permissions are the same in a shard

func (*Wrangler) ValidateSchemaKeyspace Uses

func (wr *Wrangler) ValidateSchemaKeyspace(ctx context.Context, keyspace string, excludeTables []string, includeViews bool) error

ValidateSchemaKeyspace will diff the schema from all the tablets in the keyspace.

func (*Wrangler) ValidateSchemaShard Uses

func (wr *Wrangler) ValidateSchemaShard(ctx context.Context, keyspace, shard string, excludeTables []string, includeViews bool) error

ValidateSchemaShard will diff the schema from all the tablets in the shard.

func (*Wrangler) ValidateShard Uses

func (wr *Wrangler) ValidateShard(ctx context.Context, keyspace, shard string, pingTablets bool) error

ValidateShard will validate a bunch of information in a shard is correct.

func (*Wrangler) ValidateVersionKeyspace Uses

func (wr *Wrangler) ValidateVersionKeyspace(ctx context.Context, keyspace string) error

ValidateVersionKeyspace validates all versions are the same in all tablets in a keyspace

func (*Wrangler) ValidateVersionShard Uses

func (wr *Wrangler) ValidateVersionShard(ctx context.Context, keyspace, shard string) error

ValidateVersionShard validates all versions are the same in all tablets in a shard

func (*Wrangler) VerticalSplitClone Uses

func (wr *Wrangler) VerticalSplitClone(ctx context.Context, fromKeyspace, toKeyspace string, tables []string) error

VerticalSplitClone initiates a VerticalSplitClone workflow.

func (*Wrangler) WaitForDrain Uses

func (wr *Wrangler) WaitForDrain(ctx context.Context, cells []string, keyspace, shard string, servedType topodatapb.TabletType,
    retryDelay, healthCheckTopologyRefresh, healthcheckRetryDelay, healthCheckTimeout, initialWait time.Duration) error

WaitForDrain blocks until the selected tablets (cells/keyspace/shard/tablet_type) have reported a QPS rate of 0.0. NOTE: This is just an observation of one point in time and no guarantee that the tablet was actually drained. At later times, a QPS rate > 0.0 could still be observed.

func (*Wrangler) WaitForFilteredReplication Uses

func (wr *Wrangler) WaitForFilteredReplication(ctx context.Context, keyspace, shard string, maxDelay time.Duration) error

WaitForFilteredReplication will wait until the Filtered Replication process has finished.

Directories

PathSynopsis
testlibPackage testlib contains utility methods to include in unit tests to deal with topology common tasks, like fake tablets and action loops.

Package wrangler imports 43 packages (graph) and is imported by 20 packages. Updated 2019-06-13. Refresh now. Tools for package owners.