go-vitess.v1: gopkg.in/src-d/go-vitess.v1/vt/automation Index | Files

package automation

import "gopkg.in/src-d/go-vitess.v1/vt/automation"

Package automation contains code to execute high-level cluster operations (e.g. resharding) as a series of low-level operations (e.g. vtctl, shell commands, ...).

Index

Package Files

cluster_operation_instance.go copy_schema_shard_task.go horizontal_resharding_task.go id_generator.go migrate_served_from_task.go migrate_served_types_task.go rebuild_keyspace_graph_task.go scheduler.go split_clone_task.go split_diff_task.go task.go task_containers.go tasks.go vertical_split_clone_task.go vertical_split_diff_task.go vertical_split_task.go vtctlclient_wrapper.go vtworkerclient_wrapper.go wait_for_filtered_replication_task.go

func AddMissingTaskID Uses

func AddMissingTaskID(tc []*automationpb.TaskContainer, taskIDGenerator *IDGenerator)

AddMissingTaskID assigns a task id to each task in "tc".

func AddTask Uses

func AddTask(t *automationpb.TaskContainer, taskName string, parameters map[string]string)

AddTask adds a new task to an existing task container.

func ExecuteVtctl Uses

func ExecuteVtctl(ctx context.Context, server string, args []string) (string, error)

ExecuteVtctl runs vtctl using vtctlclient. The stream of Event messages is concatenated into one output string. Additionally, the start and the end of the command will be logged to make it easier to debug which command was executed and how long it took.

func ExecuteVtworker Uses

func ExecuteVtworker(ctx context.Context, server string, args []string) (string, error)

ExecuteVtworker executes the vtworker command in "args" via an RPC to "server". The output of the RPC, a stream of LoggerEvent messages, is concatenated into one output string. If a retryable error is encountered (e.g. the vtworker process is already executing another command), this function will keep retrying infinitely until "ctx" is cancelled.

func MarkTaskFailed Uses

func MarkTaskFailed(t *automationpb.Task, output string, err error)

MarkTaskFailed marks the task as failed.

func MarkTaskSucceeded Uses

func MarkTaskSucceeded(t *automationpb.Task, output string)

MarkTaskSucceeded marks the task as done.

func NewTask Uses

func NewTask(taskName string, parameters map[string]string) *automationpb.Task

NewTask creates a new task protobuf message for "taskName" with "parameters".

func NewTaskContainer Uses

func NewTaskContainer() *automationpb.TaskContainer

NewTaskContainer creates an empty task container. Use AddTask() to add tasks to it.

func NewTaskContainerWithSingleTask Uses

func NewTaskContainerWithSingleTask(taskName string, parameters map[string]string) *automationpb.TaskContainer

NewTaskContainerWithSingleTask creates a new task container with exactly one task.

type ClusterOperationInstance Uses

type ClusterOperationInstance struct {
    automationpb.ClusterOperation
    // contains filtered or unexported fields
}

ClusterOperationInstance is a runtime type which enhances the protobuf message "ClusterOperation" with runtime specific data. Unlike the protobuf message, the additional runtime data will not be part of a checkpoint. Methods of this struct are not thread-safe.

func NewClusterOperationInstance Uses

func NewClusterOperationInstance(clusterOpID string, initialTask *automationpb.TaskContainer, taskIDGenerator *IDGenerator) ClusterOperationInstance

NewClusterOperationInstance creates a new cluster operation instance with one initial task.

func (ClusterOperationInstance) Clone Uses

func (c ClusterOperationInstance) Clone() ClusterOperationInstance

Clone creates a deep copy of the inner protobuf. Other elements e.g. taskIDGenerator are not deep-copied.

func (*ClusterOperationInstance) InsertTaskContainers Uses

func (c *ClusterOperationInstance) InsertTaskContainers(newTaskContainers []*automationpb.TaskContainer, pos int)

InsertTaskContainers inserts "newTaskContainers" at pos in the current list of task containers. Existing task containers will be moved after the new task containers.

type CopySchemaShardTask Uses

type CopySchemaShardTask struct {
}

CopySchemaShardTask runs vtctl CopySchemaShard to copy the schema from one shard to another.

func (*CopySchemaShardTask) OptionalParameters Uses

func (t *CopySchemaShardTask) OptionalParameters() []string

OptionalParameters is part of the Task interface.

func (*CopySchemaShardTask) RequiredParameters Uses

func (t *CopySchemaShardTask) RequiredParameters() []string

RequiredParameters is part of the Task interface.

func (*CopySchemaShardTask) Run Uses

func (t *CopySchemaShardTask) Run(parameters map[string]string) ([]*automationpb.TaskContainer, string, error)

Run is part of the Task interface.

type HorizontalReshardingTask Uses

type HorizontalReshardingTask struct {
}

HorizontalReshardingTask is a cluster operation which allows to increase the number of shards.

func (*HorizontalReshardingTask) OptionalParameters Uses

func (t *HorizontalReshardingTask) OptionalParameters() []string

OptionalParameters is part of the Task interface.

func (*HorizontalReshardingTask) RequiredParameters Uses

func (t *HorizontalReshardingTask) RequiredParameters() []string

RequiredParameters is part of the Task interface.

func (*HorizontalReshardingTask) Run Uses

func (t *HorizontalReshardingTask) Run(parameters map[string]string) ([]*automationpb.TaskContainer, string, error)

Run is part of the Task interface.

type IDGenerator Uses

type IDGenerator struct {
    // contains filtered or unexported fields
}

IDGenerator generates unique task and cluster operation IDs.

func (*IDGenerator) GetNextID Uses

func (ig *IDGenerator) GetNextID() string

GetNextID returns an ID which wasn't returned before.

type MigrateServedFromTask Uses

type MigrateServedFromTask struct {
}

MigrateServedFromTask runs vtctl MigrateServedFrom to let vertically split out tables get served from the new destination keyspace.

func (*MigrateServedFromTask) OptionalParameters Uses

func (t *MigrateServedFromTask) OptionalParameters() []string

OptionalParameters is part of the Task interface.

func (*MigrateServedFromTask) RequiredParameters Uses

func (t *MigrateServedFromTask) RequiredParameters() []string

RequiredParameters is part of the Task interface.

func (*MigrateServedFromTask) Run Uses

func (t *MigrateServedFromTask) Run(parameters map[string]string) ([]*automationpb.TaskContainer, string, error)

Run is part of the Task interface.

type MigrateServedTypesTask Uses

type MigrateServedTypesTask struct {
}

MigrateServedTypesTask runs vtctl MigrateServedTypes to migrate a serving type from the source shard to the shards that it replicates to.

func (*MigrateServedTypesTask) OptionalParameters Uses

func (t *MigrateServedTypesTask) OptionalParameters() []string

OptionalParameters is part of the Task interface.

func (*MigrateServedTypesTask) RequiredParameters Uses

func (t *MigrateServedTypesTask) RequiredParameters() []string

RequiredParameters is part of the Task interface.

func (*MigrateServedTypesTask) Run Uses

func (t *MigrateServedTypesTask) Run(parameters map[string]string) ([]*automationpb.TaskContainer, string, error)

Run is part of the Task interface.

type RebuildKeyspaceGraphTask Uses

type RebuildKeyspaceGraphTask struct {
}

RebuildKeyspaceGraphTask runs vtctl RebuildKeyspaceGraph to migrate a serving type from the source shard to the shards that it replicates to.

func (*RebuildKeyspaceGraphTask) OptionalParameters Uses

func (t *RebuildKeyspaceGraphTask) OptionalParameters() []string

OptionalParameters is part of the Task interface.

func (*RebuildKeyspaceGraphTask) RequiredParameters Uses

func (t *RebuildKeyspaceGraphTask) RequiredParameters() []string

RequiredParameters is part of the Task interface.

func (*RebuildKeyspaceGraphTask) Run Uses

func (t *RebuildKeyspaceGraphTask) Run(parameters map[string]string) ([]*automationpb.TaskContainer, string, error)

Run is part of the Task interface.

type Scheduler Uses

type Scheduler struct {
    // contains filtered or unexported fields
}

Scheduler executes automation tasks and maintains the execution state.

func NewScheduler Uses

func NewScheduler() (*Scheduler, error)

NewScheduler creates a new instance.

func (*Scheduler) Checkpoint Uses

func (s *Scheduler) Checkpoint(clusterOp ClusterOperationInstance)

Checkpoint should be called every time the state of the cluster op changes. It is used to update the copy of the state in activeClusterOperations.

func (*Scheduler) EnqueueClusterOperation Uses

func (s *Scheduler) EnqueueClusterOperation(ctx context.Context, req *automationpb.EnqueueClusterOperationRequest) (*automationpb.EnqueueClusterOperationResponse, error)

EnqueueClusterOperation can be used to start a new cluster operation.

func (*Scheduler) GetClusterOperationDetails Uses

func (s *Scheduler) GetClusterOperationDetails(ctx context.Context, req *automationpb.GetClusterOperationDetailsRequest) (*automationpb.GetClusterOperationDetailsResponse, error)

GetClusterOperationDetails can be used to query the full details of active or finished operations.

func (*Scheduler) Run Uses

func (s *Scheduler) Run()

Run processes queued cluster operations.

func (*Scheduler) ShutdownAndWait Uses

func (s *Scheduler) ShutdownAndWait()

ShutdownAndWait shuts down the scheduler and waits infinitely until all pending cluster operations have finished.

type SplitCloneTask Uses

type SplitCloneTask struct {
}

SplitCloneTask runs SplitClone on a remote vtworker to split an existing shard.

func (*SplitCloneTask) OptionalParameters Uses

func (t *SplitCloneTask) OptionalParameters() []string

OptionalParameters is part of the Task interface.

func (*SplitCloneTask) RequiredParameters Uses

func (t *SplitCloneTask) RequiredParameters() []string

RequiredParameters is part of the Task interface.

func (*SplitCloneTask) Run Uses

func (t *SplitCloneTask) Run(parameters map[string]string) ([]*automationpb.TaskContainer, string, error)

Run is part of the Task interface.

type SplitDiffTask Uses

type SplitDiffTask struct {
}

SplitDiffTask runs SplitDiff on a remote vtworker to compare the old shard against its new split shards.

func (*SplitDiffTask) OptionalParameters Uses

func (t *SplitDiffTask) OptionalParameters() []string

OptionalParameters is part of the Task interface.

func (*SplitDiffTask) RequiredParameters Uses

func (t *SplitDiffTask) RequiredParameters() []string

RequiredParameters is part of the Task interface.

func (*SplitDiffTask) Run Uses

func (t *SplitDiffTask) Run(parameters map[string]string) ([]*automationpb.TaskContainer, string, error)

Run is part of the Task interface.

type Task Uses

type Task interface {
    // Run executes the task using the key/values from parameters.
    // "newTaskContainers" contains new tasks which the task can emit. They'll be inserted in the cluster operation directly after this task. It may be "nil".
    // "output" may be empty. It contains any text which maybe must e.g. to debug the task or show it in the UI.
    Run(parameters map[string]string) (newTaskContainers []*automationpb.TaskContainer, output string, err error)

    // RequiredParameters() returns a list of parameter keys which must be provided as input for Run().
    RequiredParameters() []string

    // OptionalParameters() returns a list of parameter keys which are optional input for Run().
    OptionalParameters() []string
}

Task implementations can be executed by the scheduler.

type VerticalSplitCloneTask Uses

type VerticalSplitCloneTask struct {
}

VerticalSplitCloneTask runs VerticalSplitClone on a remote vtworker to split out tables from an existing keyspace to a different keyspace.

func (*VerticalSplitCloneTask) OptionalParameters Uses

func (t *VerticalSplitCloneTask) OptionalParameters() []string

OptionalParameters is part of the Task interface.

func (*VerticalSplitCloneTask) RequiredParameters Uses

func (t *VerticalSplitCloneTask) RequiredParameters() []string

RequiredParameters is part of the Task interface.

func (*VerticalSplitCloneTask) Run Uses

func (t *VerticalSplitCloneTask) Run(parameters map[string]string) ([]*automationpb.TaskContainer, string, error)

Run is part of the Task interface.

type VerticalSplitDiffTask Uses

type VerticalSplitDiffTask struct {
}

VerticalSplitDiffTask runs VerticalSplitDiff on a remote vtworker to compare the split out tables against the source keyspace.

func (*VerticalSplitDiffTask) OptionalParameters Uses

func (t *VerticalSplitDiffTask) OptionalParameters() []string

OptionalParameters is part of the Task interface.

func (*VerticalSplitDiffTask) RequiredParameters Uses

func (t *VerticalSplitDiffTask) RequiredParameters() []string

RequiredParameters is part of the Task interface.

func (*VerticalSplitDiffTask) Run Uses

func (t *VerticalSplitDiffTask) Run(parameters map[string]string) ([]*automationpb.TaskContainer, string, error)

Run is part of the Task interface.

type VerticalSplitTask Uses

type VerticalSplitTask struct {
}

VerticalSplitTask is a cluster operation to split out specific tables of one keyspace to a different keyspace.

func (*VerticalSplitTask) OptionalParameters Uses

func (t *VerticalSplitTask) OptionalParameters() []string

OptionalParameters is part of the Task interface.

func (*VerticalSplitTask) RequiredParameters Uses

func (t *VerticalSplitTask) RequiredParameters() []string

RequiredParameters is part of the Task interface.

func (*VerticalSplitTask) Run Uses

func (t *VerticalSplitTask) Run(parameters map[string]string) ([]*automationpb.TaskContainer, string, error)

Run is part of the Task interface.

type WaitForFilteredReplicationTask Uses

type WaitForFilteredReplicationTask struct {
}

WaitForFilteredReplicationTask runs vtctl WaitForFilteredReplication to block until the destination master (i.e. the receiving side of the filtered replication) has caught up to max_delay with the source shard.

func (*WaitForFilteredReplicationTask) OptionalParameters Uses

func (t *WaitForFilteredReplicationTask) OptionalParameters() []string

OptionalParameters is part of the Task interface.

func (*WaitForFilteredReplicationTask) RequiredParameters Uses

func (t *WaitForFilteredReplicationTask) RequiredParameters() []string

RequiredParameters is part of the Task interface.

func (*WaitForFilteredReplicationTask) Run Uses

func (t *WaitForFilteredReplicationTask) Run(parameters map[string]string) ([]*automationpb.TaskContainer, string, error)

Run is part of the Task interface.

Package automation imports 17 packages (graph) and is imported by 2 packages. Updated 2019-06-13. Refresh now. Tools for package owners.