tabletmanager

package
v0.0.0-...-bc668d9 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 3, 2015 License: BSD-3-Clause Imports: 51 Imported by: 0

Documentation

Overview

Package tabletmanager exports the ActionAgent object. It keeps the local tablet state, starts / stops all associated services (query service, update stream, binlog players, ...), and handles tabletmanager RPCs to update the state.

The agent is responsible for maintaining the tablet record in the topology server. Only 'ScrapTablet -force' and 'DeleteTablet' should be run by other processes, everything else should ask the tablet server to make the change.

Most RPC calls lock the actionMutex, except the easy read-only ones. RPC calls that change the tablet record will also call updateState.

See rpc_server.go for all cases, and which actions take the actionMutex, and which run changeCallback.

Index

Constants

View Source 
const (
	BPM_STATE_RUNNING int64 = iota
	BPM_STATE_STOPPED
)

Variables

View Source 
var RegisterQueryServices []RegisterQueryService

RegisterQueryServices is a list of functions to call when the delayed registration is triggered.

Functions

func HttpHandleSnapshots 

func HttpHandleSnapshots(mycnf *mysqlctl.Mycnf, uid uint32)

HttpHandleSnapshots handles the serving of files from the local tablet

func RegisterBinlogPlayerMap 

func RegisterBinlogPlayerMap(blm *BinlogPlayerMap)

RegisterBinlogPlayerMap registers the varz for the players

func SetReparentFlags 

func SetReparentFlags(fast bool, timeout time.Duration)

SetReparentFlags changes flag values. It should only be used in tests.

Types

type ActionAgent 

type ActionAgent struct {
	// The following fields are set during creation
	TopoServer      topo.Server
	TabletAlias     topo.TabletAlias
	Mysqld          *mysqlctl.Mysqld
	MysqlDaemon     mysqlctl.MysqlDaemon
	DBConfigs       *dbconfigs.DBConfigs
	SchemaOverrides []tabletserver.SchemaOverride
	BinlogPlayerMap *BinlogPlayerMap
	LockTimeout     time.Duration

	// This is the History of the health checks, public so status
	// pages can display it
	History *history.History
	// contains filtered or unexported fields
}

ActionAgent is the main class for the agent.

func NewActionAgent 

func NewActionAgent(
	batchCtx context.Context,
	tabletAlias topo.TabletAlias,
	dbcfgs *dbconfigs.DBConfigs,
	mycnf *mysqlctl.Mycnf,
	port, securePort int,
	overridesFile string,
	lockTimeout time.Duration,
) (agent *ActionAgent, err error)

NewActionAgent creates a new ActionAgent and registers all the associated services.

batchCtx is the context that the agent will use for any background tasks it spawns.

func NewTestActionAgent 

func NewTestActionAgent(batchCtx context.Context, ts topo.Server, tabletAlias topo.TabletAlias, port int, mysqlDaemon mysqlctl.MysqlDaemon) (agent *ActionAgent)

NewTestActionAgent creates an agent for test purposes. Only a subset of features are supported now, but we'll add more over time.

func (*ActionAgent) ApplySchema 

func (agent *ActionAgent) ApplySchema(ctx context.Context, change *myproto.SchemaChange) (*myproto.SchemaChangeResult, error)

ApplySchema will apply a schema change Should be called under RPCWrapLockAction.

func (*ActionAgent) BlacklistedTables 

func (agent *ActionAgent) BlacklistedTables() []string

BlacklistedTables reads the list of blacklisted tables from the TabletControl record (if any) stored in the agent, protected by mutex.

func (*ActionAgent) BreakSlaves 

func (agent *ActionAgent) BreakSlaves(ctx context.Context) error

BreakSlaves will tinker with the replication stream in a way that will stop all the slaves. Should be called under RPCWrapLockAction.

func (*ActionAgent) BroadcastHealthStreamReply 

func (agent *ActionAgent) BroadcastHealthStreamReply(hsr *actionnode.HealthStreamReply)

BroadcastHealthStreamReply will send the HealthStreamReply to all listening clients.

func (*ActionAgent) ChangeType 

func (agent *ActionAgent) ChangeType(ctx context.Context, tabletType topo.TabletType) error

ChangeType changes the tablet type Should be called under RPCWrapLockAction.

func (*ActionAgent) DemoteMaster 

func (agent *ActionAgent) DemoteMaster(ctx context.Context) error

DemoteMaster demotes the current master, and marks it read-only in the topo. Should be called under RPCWrapLockAction.

func (*ActionAgent) DisableQueryService 

func (agent *ActionAgent) DisableQueryService() bool

DisableQueryService reads the DisableQueryService field from the TabletControl record (if any) stored in the agent, protected by mutex.

func (*ActionAgent) ExecuteFetch 

func (agent *ActionAgent) ExecuteFetch(ctx context.Context, query string, maxrows int, wantFields, disableBinlogs bool) (*proto.QueryResult, error)

ExecuteFetch will execute the given query, possibly disabling binlogs. Should be called under RPCWrap.

func (*ActionAgent) ExecuteHook 

func (agent *ActionAgent) ExecuteHook(ctx context.Context, hk *hook.Hook) *hook.HookResult

ExecuteHook executes the provided hook locally, and returns the result. Should be called under RPCWrapLockAction.

func (*ActionAgent) GetPermissions 

func (agent *ActionAgent) GetPermissions(ctx context.Context) (*myproto.Permissions, error)

GetPermissions returns the db permissions. Should be called under RPCWrap.

func (*ActionAgent) GetSchema 

func (agent *ActionAgent) GetSchema(ctx context.Context, tables, excludeTables []string, includeViews bool) (*myproto.SchemaDefinition, error)

GetSchema returns the schema. Should be called under RPCWrap.

func (*ActionAgent) GetSlaves 

func (agent *ActionAgent) GetSlaves(ctx context.Context) ([]string, error)

GetSlaves returns the address of all the slaves Should be called under RPCWrap.

func (*ActionAgent) HealthStreamMapSize 

func (agent *ActionAgent) HealthStreamMapSize() int

HealthStreamMapSize returns the size of the healthStreamMap (used for tests).

func (*ActionAgent) Healthy 

func (agent *ActionAgent) Healthy() (time.Duration, error)

Healthy reads the result of the latest healthcheck, protected by mutex.

func (*ActionAgent) InitTablet 

func (agent *ActionAgent) InitTablet(port, securePort int) error

InitTablet initializes the tablet record if necessary.

func (*ActionAgent) IsRunningHealthCheck 

func (agent *ActionAgent) IsRunningHealthCheck() bool

IsRunningHealthCheck indicates if the agent is configured to run healthchecks.

func (*ActionAgent) MasterPosition 

func (agent *ActionAgent) MasterPosition(ctx context.Context) (myproto.ReplicationPosition, error)

MasterPosition returns the master position Should be called under RPCWrap.

func (*ActionAgent) Ping 

func (agent *ActionAgent) Ping(ctx context.Context, args string) string

Ping makes sure RPCs work, and refreshes the tablet record. Should be called under RPCWrap.

func (*ActionAgent) PreflightSchema 

func (agent *ActionAgent) PreflightSchema(ctx context.Context, change string) (*myproto.SchemaChangeResult, error)

PreflightSchema will try out the schema change Should be called under RPCWrapLockAction.

func (*ActionAgent) PromoteSlave 

func (agent *ActionAgent) PromoteSlave(ctx context.Context) (*actionnode.RestartSlaveData, error)

PromoteSlave transforms the current tablet from a slave to a master. It returns the data needed for other tablets to become a slave. Should be called under RPCWrapLockAction.

func (*ActionAgent) RPCWrap 

func (agent *ActionAgent) RPCWrap(ctx context.Context, name string, args, reply interface{}, f func() error) error

RPCWrap is for read-only actions that can be executed concurrently. verbose is forced to false.

func (*ActionAgent) RPCWrapLock 

func (agent *ActionAgent) RPCWrapLock(ctx context.Context, name string, args, reply interface{}, verbose bool, f func() error) error

RPCWrapLock is for actions that should not run concurrently with each other.

func (*ActionAgent) RPCWrapLockAction 

func (agent *ActionAgent) RPCWrapLockAction(ctx context.Context, name string, args, reply interface{}, verbose bool, f func() error) error

RPCWrapLockAction is the same as RPCWrapLock, plus it will call refreshTablet after the action returns.

func (*ActionAgent) RefreshState 

func (agent *ActionAgent) RefreshState(ctx context.Context)

RefreshState reload the tablet record from the topo server. Should be called under RPCWrapLockAction, so it actually works.

func (*ActionAgent) RegisterHealthStream 

func (agent *ActionAgent) RegisterHealthStream(c chan<- *actionnode.HealthStreamReply) (int, error)

RegisterHealthStream adds a health stream channel to our list

func (*ActionAgent) ReloadSchema 

func (agent *ActionAgent) ReloadSchema(ctx context.Context)

ReloadSchema will reload the schema Should be called under RPCWrapLockAction.

func (*ActionAgent) ReparentPosition 

func (agent *ActionAgent) ReparentPosition(ctx context.Context, rp *myproto.ReplicationPosition) (*actionnode.RestartSlaveData, error)

ReparentPosition returns the RestartSlaveData for the provided ReplicationPosition. Should be called under RPCWrap.

func (*ActionAgent) ReserveForRestore 

func (agent *ActionAgent) ReserveForRestore(ctx context.Context, args *actionnode.ReserveForRestoreArgs) error

ReserveForRestore reserves the current tablet for an upcoming restore operation. Should be called under RPCWrapLockAction.

func (*ActionAgent) RestartSlave 

func (agent *ActionAgent) RestartSlave(ctx context.Context, rsd *actionnode.RestartSlaveData) error

RestartSlave tells the tablet it has a new master Should be called under RPCWrapLockAction.

func (*ActionAgent) Restore 

func (agent *ActionAgent) Restore(ctx context.Context, args *actionnode.RestoreArgs, logger logutil.Logger) error

Restore stops the tablet's mysqld, replaces its data folder with a snapshot, and then restarts it.

Check that the SnapshotManifest is valid and the master has not changed. Shutdown mysqld. Load the snapshot from source tablet. Restart mysqld and replication. Put tablet into the replication graph as a spare. Should be called under RPCWrapLockAction.

func (*ActionAgent) RunBlpUntil 

func (agent *ActionAgent) RunBlpUntil(ctx context.Context, bpl *blproto.BlpPositionList, waitTime time.Duration) (*myproto.ReplicationPosition, error)

RunBlpUntil runs the binlog player server until the position is reached, and returns the current mysql master replication position.

func (*ActionAgent) RunHealthCheck 

func (agent *ActionAgent) RunHealthCheck(ctx context.Context, targetTabletType topo.TabletType)

RunHealthCheck will manually run the health check on the tablet Should be called under RPCWrap.

func (*ActionAgent) Scrap 

func (agent *ActionAgent) Scrap(ctx context.Context) error

Scrap scraps the live running tablet Should be called under RPCWrapLockAction.

func (*ActionAgent) SetReadOnly 

func (agent *ActionAgent) SetReadOnly(ctx context.Context, rdonly bool) error

SetReadOnly makes the mysql instance read-only or read-write Should be called under RPCWrapLockAction.

func (*ActionAgent) SlaveStatus 

func (agent *ActionAgent) SlaveStatus(ctx context.Context) (*myproto.ReplicationStatus, error)

SlaveStatus returns the replication status Should be called under RPCWrap.

func (*ActionAgent) SlaveWasPromoted 

func (agent *ActionAgent) SlaveWasPromoted(ctx context.Context) error

SlaveWasPromoted promotes a slave to master, no questions asked. Should be called under RPCWrapLockAction.

func (*ActionAgent) SlaveWasRestarted 

func (agent *ActionAgent) SlaveWasRestarted(ctx context.Context, swrd *actionnode.SlaveWasRestartedArgs) error

SlaveWasRestarted updates the parent record for a tablet. Should be called under RPCWrapLockAction.

func (*ActionAgent) Sleep 

func (agent *ActionAgent) Sleep(ctx context.Context, duration time.Duration)

Sleep sleeps for the duration Should be called under RPCWrapLockAction.

func (*ActionAgent) Snapshot 

func (agent *ActionAgent) Snapshot(ctx context.Context, args *actionnode.SnapshotArgs, logger logutil.Logger) (*actionnode.SnapshotReply, error)

Snapshot takes a db snapshot Should be called under RPCWrapLockAction.

func (*ActionAgent) SnapshotSourceEnd 

func (agent *ActionAgent) SnapshotSourceEnd(ctx context.Context, args *actionnode.SnapshotSourceEndArgs) error

SnapshotSourceEnd restores the state of the server after a Snapshot(server_mode =true) Should be called under RPCWrapLockAction.

func (*ActionAgent) Start 

func (agent *ActionAgent) Start(mysqlPort, vtPort, vtsPort int) error

Start validates and updates the topology records for the tablet, and performs the initial state change callback to start tablet services.

func (*ActionAgent) StartBlp 

func (agent *ActionAgent) StartBlp(ctx context.Context) error

StartBlp starts the binlog players Should be called under RPCWrapLockAction.

func (*ActionAgent) StartSlave 

func (agent *ActionAgent) StartSlave(ctx context.Context) error

StartSlave will start the replication Should be called under RPCWrapLock.

func (*ActionAgent) Stop 

func (agent *ActionAgent) Stop()

Stop shutdowns this agent.

func (*ActionAgent) StopBlp 

func (agent *ActionAgent) StopBlp(ctx context.Context) (*blproto.BlpPositionList, error)

StopBlp stops the binlog players, and return their positions. Should be called under RPCWrapLockAction.

func (*ActionAgent) StopSlave 

func (agent *ActionAgent) StopSlave(ctx context.Context) error

StopSlave will stop the replication Should be called under RPCWrapLock.

func (*ActionAgent) StopSlaveMinimum 

func (agent *ActionAgent) StopSlaveMinimum(ctx context.Context, position myproto.ReplicationPosition, waitTime time.Duration) (*myproto.ReplicationStatus, error)

StopSlaveMinimum will stop the slave after it reaches at least the provided position.

func (*ActionAgent) Tablet 

func (agent *ActionAgent) Tablet() *topo.TabletInfo

Tablet reads the stored TabletInfo from the agent, protected by mutex.

func (*ActionAgent) TabletExternallyReparented 

func (agent *ActionAgent) TabletExternallyReparented(ctx context.Context, externalID string) error

TabletExternallyReparented updates all topo records so the current tablet is the new master for this shard. Should be called under RPCWrapLock.

func (*ActionAgent) UnregisterHealthStream 

func (agent *ActionAgent) UnregisterHealthStream(id int) error

UnregisterHealthStream removes a health stream channel from our list

func (*ActionAgent) WaitBlpPosition 

func (agent *ActionAgent) WaitBlpPosition(ctx context.Context, blpPosition *blproto.BlpPosition, waitTime time.Duration) error

WaitBlpPosition waits until a specific filtered replication position is reached. Should be called under RPCWrapLock.

func (*ActionAgent) WaitSlavePosition 

func (agent *ActionAgent) WaitSlavePosition(ctx context.Context, position myproto.ReplicationPosition, waitTimeout time.Duration) (*myproto.ReplicationStatus, error)

WaitSlavePosition waits until we reach the provided position, and returns the current position Should be called under RPCWrapLock.

type BinlogPlayerController 

type BinlogPlayerController struct {
	// contains filtered or unexported fields
}

BinlogPlayerController controls one player

func (*BinlogPlayerController) BlpPosition 

func (bpc *BinlogPlayerController) BlpPosition(vtClient *binlogplayer.DBClient) (*blproto.BlpPosition, string, error)

BlpPosition returns the current position for a controller, as read from the database.

func (*BinlogPlayerController) Iteration 

func (bpc *BinlogPlayerController) Iteration() (err error)

Iteration is a single iteration for the player: get the current status, try to play, and plays until interrupted, or until an error occurs.

func (*BinlogPlayerController) Loop 

func (bpc *BinlogPlayerController) Loop()

Loop runs the main player loop: try to play, and in case of error, sleep for 5 seconds and try again.

func (*BinlogPlayerController) Start 

func (bpc *BinlogPlayerController) Start()

Start will start the player in the background and run forever

func (*BinlogPlayerController) StartUntil 

func (bpc *BinlogPlayerController) StartUntil(stopPos myproto.ReplicationPosition) error

StartUntil will start the Player until we reach the given position.

func (*BinlogPlayerController) Stop 

func (bpc *BinlogPlayerController) Stop()

Stop will ask the controller to stop playing, and wait until it is stopped.

func (*BinlogPlayerController) String 

func (bpc *BinlogPlayerController) String() string

func (*BinlogPlayerController) WaitForStop 

func (bpc *BinlogPlayerController) WaitForStop(waitTimeout time.Duration) error

WaitForStop will wait until the player is stopped. Use this after StartUntil.

type BinlogPlayerControllerStatus 

type BinlogPlayerControllerStatus struct {
	// configuration values
	Index        uint32
	SourceShard  topo.SourceShard
	StopPosition myproto.ReplicationPosition

	// stats and current values
	LastPosition        myproto.ReplicationPosition
	SecondsBehindMaster int64
	Counts              map[string]int64
	Rates               map[string][]float64
	State               string
	SourceTablet        topo.TabletAlias
	LastError           string
}

BinlogPlayerControllerStatus is the status of an individual controller

type BinlogPlayerControllerStatusList 

type BinlogPlayerControllerStatusList []*BinlogPlayerControllerStatus

BinlogPlayerControllerStatusList is the list of statuses

func (BinlogPlayerControllerStatusList) Len 

func (bpcsl BinlogPlayerControllerStatusList) Len() int

Len is part of sort.Interface

func (BinlogPlayerControllerStatusList) Less 

func (bpcsl BinlogPlayerControllerStatusList) Less(i, j int) bool

Less is part of sort.Interface

func (BinlogPlayerControllerStatusList) Swap 

func (bpcsl BinlogPlayerControllerStatusList) Swap(i, j int)

Swap is part of sort.Interface

type BinlogPlayerMap 

type BinlogPlayerMap struct {
	// contains filtered or unexported fields
}

BinlogPlayerMap controls all the players. It can be stopped and restarted.

func NewBinlogPlayerMap 

func NewBinlogPlayerMap(ts topo.Server, dbConfig *mysql.ConnectionParams, mysqld *mysqlctl.Mysqld) *BinlogPlayerMap

NewBinlogPlayerMap creates a new map of players

func (*BinlogPlayerMap) BlpPositionList 

func (blm *BinlogPlayerMap) BlpPositionList() (*blproto.BlpPositionList, error)

BlpPositionList returns the current position of all the players

func (*BinlogPlayerMap) RefreshMap 

func (blm *BinlogPlayerMap) RefreshMap(tablet *topo.Tablet, keyspaceInfo *topo.KeyspaceInfo, shardInfo *topo.ShardInfo)

RefreshMap reads the right data from topo.Server and makes sure we're playing the right logs.

func (*BinlogPlayerMap) RunUntil 

func (blm *BinlogPlayerMap) RunUntil(blpPositionList *blproto.BlpPositionList, waitTimeout time.Duration) error

RunUntil will run all the players until they reach the given position. Holds the map lock during that exercise, shouldn't take long at all.

func (*BinlogPlayerMap) Start 

func (blm *BinlogPlayerMap) Start()

Start restarts the current players

func (*BinlogPlayerMap) Status 

func (blm *BinlogPlayerMap) Status() *BinlogPlayerMapStatus

Status returns the BinlogPlayerMapStatus for the BinlogPlayerMap

func (*BinlogPlayerMap) Stop 

func (blm *BinlogPlayerMap) Stop()

Stop stops the current players, but does not remove them from the map. Call 'Start' to restart the playback.

func (*BinlogPlayerMap) StopAllPlayersAndReset 

func (blm *BinlogPlayerMap) StopAllPlayersAndReset()

StopAllPlayersAndReset stops all the binlog players, and reset the map of players.

type BinlogPlayerMapStatus 

type BinlogPlayerMapStatus struct {
	State       string
	Controllers BinlogPlayerControllerStatusList
}

BinlogPlayerMapStatus is the complete player status

type HealthRecord 

type HealthRecord struct {
	Error            error
	ReplicationDelay time.Duration
	Time             time.Time
}

HealthRecord records one run of the health checker

func (*HealthRecord) Class 

func (r *HealthRecord) Class() string

Class returns a human-readable one word version of the health state.

func (*HealthRecord) HTML 

func (r *HealthRecord) HTML() template.HTML

HTML returns a html version to be displayed on UIs

func (*HealthRecord) IsDuplicate 

func (r *HealthRecord) IsDuplicate(other interface{}) bool

IsDuplicate implements history.Deduplicable

type RPCAgent 

type RPCAgent interface {
	Ping(ctx context.Context, args string) string

	GetSchema(ctx context.Context, tables, excludeTables []string, includeViews bool) (*myproto.SchemaDefinition, error)

	GetPermissions(ctx context.Context) (*myproto.Permissions, error)

	SetReadOnly(ctx context.Context, rdonly bool) error

	ChangeType(ctx context.Context, tabletType topo.TabletType) error

	Scrap(ctx context.Context) error

	Sleep(ctx context.Context, duration time.Duration)

	ExecuteHook(ctx context.Context, hk *hook.Hook) *hook.HookResult

	RefreshState(ctx context.Context)

	RunHealthCheck(ctx context.Context, targetTabletType topo.TabletType)

	RegisterHealthStream(chan<- *actionnode.HealthStreamReply) (int, error)
	UnregisterHealthStream(int) error

	ReloadSchema(ctx context.Context)

	PreflightSchema(ctx context.Context, change string) (*myproto.SchemaChangeResult, error)

	ApplySchema(ctx context.Context, change *myproto.SchemaChange) (*myproto.SchemaChangeResult, error)

	ExecuteFetch(ctx context.Context, query string, maxrows int, wantFields, disableBinlogs bool) (*proto.QueryResult, error)

	SlaveStatus(ctx context.Context) (*myproto.ReplicationStatus, error)

	WaitSlavePosition(ctx context.Context, position myproto.ReplicationPosition, waitTimeout time.Duration) (*myproto.ReplicationStatus, error)

	MasterPosition(ctx context.Context) (myproto.ReplicationPosition, error)

	ReparentPosition(ctx context.Context, rp *myproto.ReplicationPosition) (*actionnode.RestartSlaveData, error)

	StopSlave(ctx context.Context) error

	StopSlaveMinimum(ctx context.Context, position myproto.ReplicationPosition, waitTime time.Duration) (*myproto.ReplicationStatus, error)

	StartSlave(ctx context.Context) error

	TabletExternallyReparented(ctx context.Context, externalID string) error

	GetSlaves(ctx context.Context) ([]string, error)

	WaitBlpPosition(ctx context.Context, blpPosition *blproto.BlpPosition, waitTime time.Duration) error

	StopBlp(ctx context.Context) (*blproto.BlpPositionList, error)

	StartBlp(ctx context.Context) error

	RunBlpUntil(ctx context.Context, bpl *blproto.BlpPositionList, waitTime time.Duration) (*myproto.ReplicationPosition, error)

	DemoteMaster(ctx context.Context) error

	PromoteSlave(ctx context.Context) (*actionnode.RestartSlaveData, error)

	SlaveWasPromoted(ctx context.Context) error

	RestartSlave(ctx context.Context, rsd *actionnode.RestartSlaveData) error

	SlaveWasRestarted(ctx context.Context, swrd *actionnode.SlaveWasRestartedArgs) error

	BreakSlaves(ctx context.Context) error

	Snapshot(ctx context.Context, args *actionnode.SnapshotArgs, logger logutil.Logger) (*actionnode.SnapshotReply, error)

	SnapshotSourceEnd(ctx context.Context, args *actionnode.SnapshotSourceEndArgs) error

	ReserveForRestore(ctx context.Context, args *actionnode.ReserveForRestoreArgs) error

	Restore(ctx context.Context, args *actionnode.RestoreArgs, logger logutil.Logger) error

	// RPC helpers
	RPCWrap(ctx context.Context, name string, args, reply interface{}, f func() error) error
	RPCWrapLock(ctx context.Context, name string, args, reply interface{}, verbose bool, f func() error) error
	RPCWrapLockAction(ctx context.Context, name string, args, reply interface{}, verbose bool, f func() error) error
}

RPCAgent defines the interface implemented by the Agent for RPCs. It is useful for RPC implementations to test their full stack.

type RegisterQueryService 

type RegisterQueryService func(*ActionAgent)

RegisterQueryService is used to delay registration of RPC servers until we have all the objects.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.