vtgate

package
v0.0.0-...-3712671 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 28, 2014 License: BSD-3-Clause Imports: 24 Imported by: 0

Documentation

Overview

Package vtgate provides query routing rpc services for vttablets.

Package vtgate provides query routing rpc services for vttablets.

Index

Constants

View Source 
const (
	EID_NODE = iota
	VALUE_NODE
	LIST_NODE
	OTHER_NODE
)

Variables

View Source 
var (
	RpcVTGate *VTGate

	QPSByOperation *stats.Rates
	QPSByKeyspace  *stats.Rates
	QPSByDbType    *stats.Rates

	ErrorsByOperation *stats.Rates
	ErrorsByKeyspace  *stats.Rates
	ErrorsByDbType    *stats.Rates
)
View Source 
var RegisterVTGates []RegisterVTGate

Functions

func GetShardList 

func GetShardList(sql string, bindVariables map[string]interface{}, tabletKeys []key.KeyspaceId) (shardlist []int, err error)

func Init 

func Init(serv SrvTopoServer, cell string, retryDelay time.Duration, retryCount int, timeout time.Duration)

func StrsEquals 

func StrsEquals(a, b []string) bool

StrsEquals compares contents of two string slices.

Types

type Balancer 

type Balancer struct {
	// contains filtered or unexported fields
}

Balancer is a simple round-robin load balancer. It allows you to temporarily mark down nodes that are non-functional.

func NewBalancer 

func NewBalancer(getEndPoints GetEndPointsFunc, retryDelay time.Duration) *Balancer

NewBalancer creates a Balancer. getAddreses is the function it will use to refresh the list of addresses if one of the nodes has been marked down. The list of addresses is shuffled. retryDelay specifies the minimum time a node will be marked down before it will be cleared for a retry.

func (*Balancer) Get 

func (blc *Balancer) Get() (endPoint topo.EndPoint, err error)

Get returns a single endpoint that was not recently marked down. If it finds an address that was down for longer than retryDelay, it refreshes the list of addresses and returns the next available node. If all addresses are marked down, it waits and retries. If a refresh fails, it returns an error.

func (*Balancer) MarkDown 

func (blc *Balancer) MarkDown(uid uint32)

MarkDown marks the specified address down. Such addresses will not be used by Balancer for the duration of retryDelay.

type EndPointsCacheStatus 

type EndPointsCacheStatus struct {
	Cell             string
	Keyspace         string
	Shard            string
	TabletType       topo.TabletType
	Value            *topo.EndPoints
	OriginalValue    *topo.EndPoints
	LastError        error
	LastErrorContext context.Context
}

EndPointsCacheStatus is the current value for an EndPoints object

func (*EndPointsCacheStatus) StatusAsHTML 

func (st *EndPointsCacheStatus) StatusAsHTML() template.HTML

StatusAsHTML returns an HTML version of our status. It works best if there is data in the cache.

type EndPointsCacheStatusList 

type EndPointsCacheStatusList []*EndPointsCacheStatus

EndPointsCacheStatusList is used for sorting

func (EndPointsCacheStatusList) Len 

func (epcsl EndPointsCacheStatusList) Len() int

Len is part of sort.Interface

func (EndPointsCacheStatusList) Less 

func (epcsl EndPointsCacheStatusList) Less(i, j int) bool

Less is part of sort.Interface

func (EndPointsCacheStatusList) Swap 

func (epcsl EndPointsCacheStatusList) Swap(i, j int)

Swap is part of sort.Interface

type GetEndPointsFunc 

type GetEndPointsFunc func() (*topo.EndPoints, error)

type RegisterVTGate 

type RegisterVTGate func(*VTGate)

registration mechanism

type ResilientSrvTopoServer 

type ResilientSrvTopoServer struct {
	// contains filtered or unexported fields
}

ResilientSrvTopoServer is an implementation of SrvTopoServer based on a topo.Server that uses a cache for two purposes: - limit the QPS to the underlying topo.Server - return the last known value of the data if there is an error

func NewResilientSrvTopoServer 

func NewResilientSrvTopoServer(base topo.Server, counterName string) *ResilientSrvTopoServer

NewResilientSrvTopoServer creates a new ResilientSrvTopoServer based on the provided SrvTopoServer.

func (*ResilientSrvTopoServer) CacheStatus 

func (server *ResilientSrvTopoServer) CacheStatus() *ResilientSrvTopoServerCacheStatus

CacheStatus returns a displayable version of the cache

func (*ResilientSrvTopoServer) DegradedEndpointCount 

func (server *ResilientSrvTopoServer) DegradedEndpointCount() map[string]int64

DegradedEndpointCount returns how many degraded endpoints we have in the cache (entries that are not 100% healthy, because they are behind on replication for instance)

func (*ResilientSrvTopoServer) EndpointCount 

func (server *ResilientSrvTopoServer) EndpointCount() map[string]int64

EndpointCount returns how many endpoints we have per keyspace/shard/dbtype.

func (*ResilientSrvTopoServer) GetEndPoints 

func (server *ResilientSrvTopoServer) GetEndPoints(context context.Context, cell, keyspace, shard string, tabletType topo.TabletType) (*topo.EndPoints, error)

func (*ResilientSrvTopoServer) GetSrvKeyspace 

func (server *ResilientSrvTopoServer) GetSrvKeyspace(context context.Context, cell, keyspace string) (*topo.SrvKeyspace, error)

func (*ResilientSrvTopoServer) GetSrvKeyspaceNames 

func (server *ResilientSrvTopoServer) GetSrvKeyspaceNames(context context.Context, cell string) ([]string, error)

type ResilientSrvTopoServerCacheStatus 

type ResilientSrvTopoServerCacheStatus struct {
	SrvKeyspaceNames SrvKeyspaceNamesCacheStatusList
	SrvKeyspaces     SrvKeyspaceCacheStatusList
	EndPoints        EndPointsCacheStatusList
}

ResilientSrvTopoServerCacheStatus has the full status of the cache

type Resolver 

type Resolver struct {
	// contains filtered or unexported fields
}

Resolver is the layer to resolve KeyspaceIds and KeyRanges to shards. It will try to re-resolve shards if ScatterConn returns retryable error, which may imply horizontal or vertical resharding happened.

func NewResolver 

func NewResolver(serv SrvTopoServer, statsName, cell string, retryDelay time.Duration, retryCount int, timeout time.Duration) *Resolver

NewResolver creates a new Resolver. All input parameters are passed through for creating ScatterConn.

func (*Resolver) Commit 

func (res *Resolver) Commit(context context.Context, inSession *proto.Session) error

Commit commits a transaction.

func (*Resolver) Execute 

func (res *Resolver) Execute(
	context context.Context,
	sql string,
	bindVars map[string]interface{},
	keyspace string,
	tabletType topo.TabletType,
	session *proto.Session,
	mapToShards func(string) (string, []string, error),
) (*mproto.QueryResult, error)

Execute executes a non-streaming query based on shards resolved by given func. It retries query if new keyspace/shards are re-resolved after a retryable error.

func (*Resolver) ExecuteBatch 

func (res *Resolver) ExecuteBatch(
	context context.Context,
	queries []tproto.BoundQuery,
	keyspace string,
	tabletType topo.TabletType,
	session *proto.Session,
	mapToShards func(string) (string, []string, error),
) (*tproto.QueryResultList, error)

ExecuteBatch executes a group of queries based on shards resolved by given func. It retries query if new keyspace/shards are re-resolved after a retryable error.

func (*Resolver) ExecuteBatchKeyspaceIds 

func (res *Resolver) ExecuteBatchKeyspaceIds(context context.Context, query *proto.KeyspaceIdBatchQuery) (*tproto.QueryResultList, error)

ExecuteBatchKeyspaceIds executes a group of queries based on KeyspaceIds. It retries query if new keyspace/shards are re-resolved after a retryable error.

func (*Resolver) ExecuteEntityIds 

func (res *Resolver) ExecuteEntityIds(
	context context.Context,
	query *proto.EntityIdsQuery,
) (*mproto.QueryResult, error)

ExecuteEntityIds executes a non-streaming query based on given KeyspaceId map. It retries query if new keyspace/shards are re-resolved after a retryable error.

func (*Resolver) ExecuteKeyRanges 

func (res *Resolver) ExecuteKeyRanges(context context.Context, query *proto.KeyRangeQuery) (*mproto.QueryResult, error)

ExecuteKeyRanges executes a non-streaming query based on KeyRanges. It retries query if new keyspace/shards are re-resolved after a retryable error.

func (*Resolver) ExecuteKeyspaceIds 

func (res *Resolver) ExecuteKeyspaceIds(context context.Context, query *proto.KeyspaceIdQuery) (*mproto.QueryResult, error)

ExecuteKeyspaceIds executes a non-streaming query based on KeyspaceIds. It retries query if new keyspace/shards are re-resolved after a retryable error.

func (*Resolver) Rollback 

func (res *Resolver) Rollback(context context.Context, inSession *proto.Session) error

Rollback rolls back a transaction.

func (*Resolver) StreamExecute 

func (res *Resolver) StreamExecute(
	context context.Context,
	sql string,
	bindVars map[string]interface{},
	keyspace string,
	tabletType topo.TabletType,
	session *proto.Session,
	mapToShards func(string) (string, []string, error),
	sendReply func(*mproto.QueryResult) error,
) error

StreamExecuteShard executes a streaming query on shards resolved by given func. This function currently temporarily enforces the restriction of executing on one shard since it cannot merge-sort the results to guarantee ordering of response which is needed for checkpointing.

func (*Resolver) StreamExecuteKeyRanges 

func (res *Resolver) StreamExecuteKeyRanges(context context.Context, query *proto.KeyRangeQuery, sendReply func(*mproto.QueryResult) error) error

StreamExecuteKeyRanges executes a streaming query on the specified KeyRanges. The KeyRanges are resolved to shards using the serving graph. This function currently temporarily enforces the restriction of executing on one shard since it cannot merge-sort the results to guarantee ordering of response which is needed for checkpointing. The api supports supplying multiple keyranges to make it future proof.

func (*Resolver) StreamExecuteKeyspaceIds 

func (res *Resolver) StreamExecuteKeyspaceIds(context context.Context, query *proto.KeyspaceIdQuery, sendReply func(*mproto.QueryResult) error) error

StreamExecuteKeyspaceIds executes a streaming query on the specified KeyspaceIds. The KeyspaceIds are resolved to shards using the serving graph. This function currently temporarily enforces the restriction of executing on one shard since it cannot merge-sort the results to guarantee ordering of response which is needed for checkpointing. The api supports supplying multiple KeyspaceIds to make it future proof.

type RoutingPlan 

type RoutingPlan struct {
	// contains filtered or unexported fields
}

type SafeSession 

type SafeSession struct {
	*proto.Session
	// contains filtered or unexported fields
}

func NewSafeSession 

func NewSafeSession(sessn *proto.Session) *SafeSession

func (*SafeSession) Append 

func (session *SafeSession) Append(shardSession *proto.ShardSession)

func (*SafeSession) Find 

func (session *SafeSession) Find(keyspace, shard string, tabletType topo.TabletType) int64

func (*SafeSession) InTransaction 

func (session *SafeSession) InTransaction() bool

func (*SafeSession) Reset 

func (session *SafeSession) Reset()

type ScatterConn 

type ScatterConn struct {
	// contains filtered or unexported fields
}

ScatterConn is used for executing queries across multiple ShardConn connections.

func NewScatterConn 

func NewScatterConn(serv SrvTopoServer, statsName, cell string, retryDelay time.Duration, retryCount int, timeout time.Duration) *ScatterConn

NewScatterConn creates a new ScatterConn. All input parameters are passed through for creating the appropriate ShardConn.

func (*ScatterConn) Close 

func (stc *ScatterConn) Close() error

Close closes the underlying ShardConn connections.

func (*ScatterConn) Commit 

func (stc *ScatterConn) Commit(context context.Context, session *SafeSession) (err error)

Commit commits the current transaction. There are no retries on this operation.

func (*ScatterConn) Execute 

func (stc *ScatterConn) Execute(
	context context.Context,
	query string,
	bindVars map[string]interface{},
	keyspace string,
	shards []string,
	tabletType topo.TabletType,
	session *SafeSession,
) (*mproto.QueryResult, error)

Execute executes a non-streaming query on the specified shards.

func (*ScatterConn) ExecuteBatch 

func (stc *ScatterConn) ExecuteBatch(
	context context.Context,
	queries []tproto.BoundQuery,
	keyspace string,
	shards []string,
	tabletType topo.TabletType,
	session *SafeSession,
) (qrs *tproto.QueryResultList, err error)

ExecuteBatch executes a batch of non-streaming queries on the specified shards.

func (*ScatterConn) ExecuteEntityIds 

func (stc *ScatterConn) ExecuteEntityIds(
	context context.Context,
	shards []string,
	sqls map[string]string,
	bindVars map[string]map[string]interface{},
	keyspace string,
	tabletType topo.TabletType,
	session *SafeSession,
) (*mproto.QueryResult, error)

func (*ScatterConn) Rollback 

func (stc *ScatterConn) Rollback(context context.Context, session *SafeSession) (err error)

Rollback rolls back the current transaction. There are no retries on this operation.

func (*ScatterConn) StreamExecute 

func (stc *ScatterConn) StreamExecute(
	context context.Context,
	query string,
	bindVars map[string]interface{},
	keyspace string,
	shards []string,
	tabletType topo.TabletType,
	session *SafeSession,
	sendReply func(reply *mproto.QueryResult) error,
) error

StreamExecute executes a streaming query on vttablet. The retry rules are the same.

type ShardConn 

type ShardConn struct {
	// contains filtered or unexported fields
}

ShardConn represents a load balanced connection to a group of vttablets that belong to the same shard. ShardConn can be concurrently used across goroutines. Such requests are interleaved on the same underlying connection.

func NewShardConn 

func NewShardConn(ctx context.Context, serv SrvTopoServer, cell, keyspace, shard string, tabletType topo.TabletType, retryDelay time.Duration, retryCount int, timeout time.Duration) *ShardConn

NewShardConn creates a new ShardConn. It creates a Balancer using serv, cell, keyspace, tabletType and retryDelay. retryCount is the max number of retries before a ShardConn returns an error on an operation.

func (*ShardConn) Begin 

func (sdc *ShardConn) Begin(ctx context.Context) (transactionID int64, err error)

Begin begins a transaction. The retry rules are the same as Execute.

func (*ShardConn) Close 

func (sdc *ShardConn) Close()

Close closes the underlying TabletConn. ShardConn can be reused after this because it opens connections on demand.

func (*ShardConn) Commit 

func (sdc *ShardConn) Commit(ctx context.Context, transactionID int64) (err error)

Commit commits the current transaction. The retry rules are the same as Execute.

func (*ShardConn) Execute 

func (sdc *ShardConn) Execute(ctx context.Context, query string, bindVars map[string]interface{}, transactionID int64) (qr *mproto.QueryResult, err error)

Execute executes a non-streaming query on vttablet. If there are connection errors, it retries retryCount times before failing. It does not retry if the connection is in the middle of a transaction.

func (*ShardConn) ExecuteBatch 

func (sdc *ShardConn) ExecuteBatch(ctx context.Context, queries []tproto.BoundQuery, transactionID int64) (qrs *tproto.QueryResultList, err error)

ExecuteBatch executes a group of queries. The retry rules are the same as Execute.

func (*ShardConn) Rollback 

func (sdc *ShardConn) Rollback(ctx context.Context, transactionID int64) (err error)

Rollback rolls back the current transaction. The retry rules are the same as Execute.

func (*ShardConn) StreamExecute 

func (sdc *ShardConn) StreamExecute(ctx context.Context, query string, bindVars map[string]interface{}, transactionID int64) (<-chan *mproto.QueryResult, tabletconn.ErrFunc)

StreamExecute executes a streaming query on vttablet. The retry rules are the same as Execute.

func (*ShardConn) WrapError 

func (sdc *ShardConn) WrapError(in error, conn tabletconn.TabletConn, inTransaction bool) (wrapped error)

WrapError returns ShardConnError which preserves the original error code if possible, adds the connection context and adds a bit to determine whether the keyspace/shard needs to be re-resolved for a potential sharding event.

type ShardConnError 

type ShardConnError struct {
	Code            int
	ShardIdentifier string
	InTransaction   bool
	Err             string
}

func (*ShardConnError) Error 

func (e *ShardConnError) Error() string

type SrvKeyspaceCacheStatus 

type SrvKeyspaceCacheStatus struct {
	Cell             string
	Keyspace         string
	Value            *topo.SrvKeyspace
	LastError        error
	LastErrorContext context.Context
}

SrvKeyspaceCacheStatus is the current value for a SrvKeyspace object

func (*SrvKeyspaceCacheStatus) StatusAsHTML 

func (st *SrvKeyspaceCacheStatus) StatusAsHTML() template.HTML

StatusAsHTML returns an HTML version of our status. It works best if there is data in the cache.

type SrvKeyspaceCacheStatusList 

type SrvKeyspaceCacheStatusList []*SrvKeyspaceCacheStatus

SrvKeyspaceCacheStatusList is used for sorting

func (SrvKeyspaceCacheStatusList) Len 

func (skcsl SrvKeyspaceCacheStatusList) Len() int

Len is part of sort.Interface

func (SrvKeyspaceCacheStatusList) Less 

func (skcsl SrvKeyspaceCacheStatusList) Less(i, j int) bool

Less is part of sort.Interface

func (SrvKeyspaceCacheStatusList) Swap 

func (skcsl SrvKeyspaceCacheStatusList) Swap(i, j int)

Swap is part of sort.Interface

type SrvKeyspaceNamesCacheStatus 

type SrvKeyspaceNamesCacheStatus struct {
	Cell             string
	Value            []string
	LastError        error
	LastErrorContext context.Context
}

SrvKeyspaceNamesCacheStatus is the current value for SrvKeyspaceNames

type SrvKeyspaceNamesCacheStatusList 

type SrvKeyspaceNamesCacheStatusList []*SrvKeyspaceNamesCacheStatus

SrvKeyspaceNamesCacheStatusList is used for sorting

func (SrvKeyspaceNamesCacheStatusList) Len 

func (skncsl SrvKeyspaceNamesCacheStatusList) Len() int

Len is part of sort.Interface

func (SrvKeyspaceNamesCacheStatusList) Less 

func (skncsl SrvKeyspaceNamesCacheStatusList) Less(i, j int) bool

Less is part of sort.Interface

func (SrvKeyspaceNamesCacheStatusList) Swap 

func (skncsl SrvKeyspaceNamesCacheStatusList) Swap(i, j int)

Swap is part of sort.Interface

type SrvTopoServer 

type SrvTopoServer interface {
	GetSrvKeyspaceNames(context context.Context, cell string) ([]string, error)

	GetSrvKeyspace(context context.Context, cell, keyspace string) (*topo.SrvKeyspace, error)

	GetEndPoints(context context.Context, cell, keyspace, shard string, tabletType topo.TabletType) (*topo.EndPoints, error)
}

SrvTopoServer is a subset of topo.Server that only contains the serving graph read-only calls used by clients to resolve serving addresses.

type VTGate 

type VTGate struct {
	// contains filtered or unexported fields
}

VTGate is the rpc interface to vtgate. Only one instance can be created.

func (*VTGate) Begin 

func (vtg *VTGate) Begin(context context.Context, outSession *proto.Session) error

Begin begins a transaction. It has to be concluded by a Commit or Rollback.

func (*VTGate) Commit 

func (vtg *VTGate) Commit(context context.Context, inSession *proto.Session) error

Commit commits a transaction.

func (*VTGate) ExecuteBatchKeyspaceIds 

func (vtg *VTGate) ExecuteBatchKeyspaceIds(context context.Context, query *proto.KeyspaceIdBatchQuery, reply *proto.QueryResultList) error

ExecuteBatchKeyspaceIds executes a group of queries based on the specified keyspace ids.

func (*VTGate) ExecuteBatchShard 

func (vtg *VTGate) ExecuteBatchShard(context context.Context, batchQuery *proto.BatchQueryShard, reply *proto.QueryResultList) error

ExecuteBatchShard executes a group of queries on the specified shards.

func (*VTGate) ExecuteEntityIds 

func (vtg *VTGate) ExecuteEntityIds(context context.Context, query *proto.EntityIdsQuery, reply *proto.QueryResult) error

ExecuteEntityIds excutes a non-streaming query based on given KeyspaceId map.

func (*VTGate) ExecuteKeyRanges 

func (vtg *VTGate) ExecuteKeyRanges(context context.Context, query *proto.KeyRangeQuery, reply *proto.QueryResult) error

ExecuteKeyRanges executes a non-streaming query based on the specified keyranges.

func (*VTGate) ExecuteKeyspaceIds 

func (vtg *VTGate) ExecuteKeyspaceIds(context context.Context, query *proto.KeyspaceIdQuery, reply *proto.QueryResult) error

ExecuteKeyspaceIds executes a non-streaming query based on the specified keyspace ids.

func (*VTGate) ExecuteShard 

func (vtg *VTGate) ExecuteShard(context context.Context, query *proto.QueryShard, reply *proto.QueryResult) error

ExecuteShard executes a non-streaming query on the specified shards.

func (*VTGate) Rollback 

func (vtg *VTGate) Rollback(context context.Context, inSession *proto.Session) error

Rollback rolls back a transaction.

func (*VTGate) StreamExecuteKeyRanges 

func (vtg *VTGate) StreamExecuteKeyRanges(context context.Context, query *proto.KeyRangeQuery, sendReply func(*proto.QueryResult) error) error

StreamExecuteKeyRanges executes a streaming query on the specified KeyRanges. The KeyRanges are resolved to shards using the serving graph. This function currently temporarily enforces the restriction of executing on one shard since it cannot merge-sort the results to guarantee ordering of response which is needed for checkpointing. The api supports supplying multiple keyranges to make it future proof.

func (*VTGate) StreamExecuteKeyspaceIds 

func (vtg *VTGate) StreamExecuteKeyspaceIds(context context.Context, query *proto.KeyspaceIdQuery, sendReply func(*proto.QueryResult) error) error

StreamExecuteKeyspaceIds executes a streaming query on the specified KeyspaceIds. The KeyspaceIds are resolved to shards using the serving graph. This function currently temporarily enforces the restriction of executing on one shard since it cannot merge-sort the results to guarantee ordering of response which is needed for checkpointing. The api supports supplying multiple KeyspaceIds to make it future proof.

func (*VTGate) StreamExecuteShard 

func (vtg *VTGate) StreamExecuteShard(context context.Context, query *proto.QueryShard, sendReply func(*proto.QueryResult) error) error

StreamExecuteShard executes a streaming query on the specified shards.

Source Files

View all Source files

Directories

Path Synopsis
Package gorpcvtgateservice provides to go rpc glue for vtgate
 Package gorpcvtgateservice provides to go rpc glue for vtgate

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.