worker

package
v0.0.0-...-9a5932e Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 10, 2014 License: BSD-3-Clause Imports: 18 Imported by: 0

Documentation

Overview

'worker' package contains the framework, utility methods and core functions for long running actions. 'vtworker' binary will use these.

Index

Constants

View Source
const (
	// all the states for the worker
	SQLDiffNotSarted              sqlDiffWorkerState = "not started"
	SQLDiffDone                   sqlDiffWorkerState = "done"
	SQLDiffError                  sqlDiffWorkerState = "error"
	SQLDiffFindTargets            sqlDiffWorkerState = "finding target instances"
	SQLDiffSynchronizeReplication sqlDiffWorkerState = "synchronizing replication"
	SQLDiffRunning                sqlDiffWorkerState = "running the diff"
	SQLDiffCleanUp                sqlDiffWorkerState = "cleaning up"
)

Variables

This section is empty.

Functions

func CompareRows

func CompareRows(fields []mproto.Field, compareCount int, left, right []sqltypes.Value) (int, error)

CompareRows returns: -1 if left is smaller than right 0 if left and right are equal +1 if left is bigger than right

func RowsEqual

func RowsEqual(left, right []sqltypes.Value) int

RowsEqual returns the index of the first different fields, or -1 if both rows are the same

func SignalInterrupt

func SignalInterrupt()

Types

type DiffReport

type DiffReport struct {
	// contains filtered or unexported fields
}

DiffReport has the stats for a diff job

func (*DiffReport) ComputeQPS

func (dr *DiffReport) ComputeQPS()

ComputeQPS fills in processingQPS

func (*DiffReport) HasDifferences

func (dr *DiffReport) HasDifferences() bool

HasDifferences returns true if the diff job recorded any difference

func (*DiffReport) String

func (dr *DiffReport) String() string

type QueryResultReader

type QueryResultReader struct {
	Output chan *mproto.QueryResult
	Fields []mproto.Field
	// contains filtered or unexported fields
}

QueryResultReader will stream rows towards the output channel.

func NewQueryResultReaderForTablet

func NewQueryResultReaderForTablet(ts topo.Server, tabletAlias topo.TabletAlias, sql string) (*QueryResultReader, error)

NewQueryResultReaderForTablet creates a new QueryResultReader for the provided tablet / sql query

func TableScan

func TableScan(ts topo.Server, tabletAlias topo.TabletAlias, tableDefinition *myproto.TableDefinition) (*QueryResultReader, error)

TableScan returns a QueryResultReader that gets all the rows from a table, ordered by Primary Key. The returned columns are ordered with the Primary Key columns in front.

func TableScanByKeyRange

func TableScanByKeyRange(ts topo.Server, tabletAlias topo.TabletAlias, tableDefinition *myproto.TableDefinition, keyRange key.KeyRange, keyspaceIdType key.KeyspaceIdType) (*QueryResultReader, error)

TableScanByKeyRange returns a QueryResultReader that gets all the rows from a table that match the supplied KeyRange, ordered by Primary Key. The returned columns are ordered with the Primary Key columns in front.

func (*QueryResultReader) Close

func (qrr *QueryResultReader) Close() error

func (*QueryResultReader) Error

func (qrr *QueryResultReader) Error() error

type RowDiffer

type RowDiffer struct {
	// contains filtered or unexported fields
}

RowDiffer will consume rows on both sides, and compare them. It assumes left and right are sorted by ascending primary key. it will record errors if extra rows exist on either side.

func NewRowDiffer

func NewRowDiffer(left, right *QueryResultReader, tableDefinition *myproto.TableDefinition) (*RowDiffer, error)

NewRowDiffer returns a new RowDiffer

func (*RowDiffer) Go

func (rd *RowDiffer) Go() (dr DiffReport, err error)

Go runs the diff. If there is no error, it will drain both sides. If an error occurs, it will just return it and stop.

type RowReader

type RowReader struct {
	// contains filtered or unexported fields
}

RowReader returns individual rows from a QueryResultReader

func NewRowReader

func NewRowReader(queryResultReader *QueryResultReader) *RowReader

NewRowReader returns a RowReader based on the QueryResultReader

func (*RowReader) Drain

func (rr *RowReader) Drain() (int, error)

Drain will empty the RowReader and return how many rows we got

func (*RowReader) Fields

func (rr *RowReader) Fields() []mproto.Field

Fields returns the types for the rows

func (*RowReader) Next

func (rr *RowReader) Next() ([]sqltypes.Value, error)

Next will return: (row, nil) for the next row (nil, nil) for EOF (nil, error) if an error occured

type RowSubsetDiffer

type RowSubsetDiffer struct {
	// contains filtered or unexported fields
}

RowSubsetDiffer will consume rows on both sides, and compare them. It assumes superset and subset are sorted by ascending primary key. It will record errors in DiffReport.extraRowsRight if extra rows exist on the subset side, and DiffReport.extraRowsLeft will always be zero.

func NewRowSubsetDiffer

func NewRowSubsetDiffer(superset, subset *QueryResultReader, pkFieldCount int) (*RowSubsetDiffer, error)

NewRowSubsetDiffer returns a new RowSubsetDiffer

func (*RowSubsetDiffer) Go

func (rd *RowSubsetDiffer) Go() (dr DiffReport, err error)

Go runs the diff. If there is no error, it will drain both sides. If an error occurs, it will just return it and stop.

type SQLDiffWorker

type SQLDiffWorker struct {
	// contains filtered or unexported fields
}

SQLDiffWorker runs a sanity check in in a system with a lookup database: any row in the subset spec needs to have a conuterpart in the superset spec.

func (*SQLDiffWorker) CheckInterrupted

func (worker *SQLDiffWorker) CheckInterrupted() bool

func (*SQLDiffWorker) Run

func (worker *SQLDiffWorker) Run()

Run is mostly a wrapper to run the cleanup at the end.

func (*SQLDiffWorker) StatusAsHTML

func (worker *SQLDiffWorker) StatusAsHTML() string

func (*SQLDiffWorker) StatusAsText

func (worker *SQLDiffWorker) StatusAsText() string

type SourceSpec

type SourceSpec struct {
	Keyspace string
	Shard    string
	SQL      string
	// contains filtered or unexported fields
}

SourceSpec specifies a SQL query in some keyspace and shard.

type SplitDiffWorker

type SplitDiffWorker struct {
	// contains filtered or unexported fields
}

SplitDiffWorker executes a diff between a destination shard and its source shards in a shard split case.

func (*SplitDiffWorker) CheckInterrupted

func (sdw *SplitDiffWorker) CheckInterrupted() bool

func (*SplitDiffWorker) Run

func (sdw *SplitDiffWorker) Run()

Run is mostly a wrapper to run the cleanup at the end.

func (*SplitDiffWorker) StatusAsHTML

func (sdw *SplitDiffWorker) StatusAsHTML() string

func (*SplitDiffWorker) StatusAsText

func (sdw *SplitDiffWorker) StatusAsText() string

type VerticalSplitDiffWorker

type VerticalSplitDiffWorker struct {
	// contains filtered or unexported fields
}

VerticalSplitDiffWorker executes a diff between a destination shard and its source shards in a shard split case.

func (*VerticalSplitDiffWorker) CheckInterrupted

func (vsdw *VerticalSplitDiffWorker) CheckInterrupted() bool

func (*VerticalSplitDiffWorker) Run

func (vsdw *VerticalSplitDiffWorker) Run()

Run is mostly a wrapper to run the cleanup at the end.

func (*VerticalSplitDiffWorker) StatusAsHTML

func (vsdw *VerticalSplitDiffWorker) StatusAsHTML() string

func (*VerticalSplitDiffWorker) StatusAsText

func (vsdw *VerticalSplitDiffWorker) StatusAsText() string

type Worker

type Worker interface {
	// StatusAsHTML returns the current worker status in HTML
	StatusAsHTML() string

	// StatusAsText returns the current worker status in plain text
	StatusAsText() string

	// Run is the main entry point for the worker. It will be called
	// in a go routine.
	// When the SignalInterrupt() is called, Run should exit as soon as
	// possible.
	Run()
}

Worker is the base interface for all long running workers.

func NewSQLDiffWorker

func NewSQLDiffWorker(wr *wrangler.Wrangler, cell string, superset, subset SourceSpec) Worker

NewSQLDiffWorker returns a new SQLDiffWorker object.

func NewSplitDiffWorker

func NewSplitDiffWorker(wr *wrangler.Wrangler, cell, keyspace, shard string) Worker

NewSplitDiff returns a new SplitDiffWorker object.

func NewVerticalSplitDiffWorker

func NewVerticalSplitDiffWorker(wr *wrangler.Wrangler, cell, keyspace, shard string) Worker

NewVerticalSplitDiff returns a new VerticalSplitDiffWorker object.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL