cockroach: github.com/cockroachdb/cockroach/pkg/rpc Index | Files | Directories

package rpc

import "github.com/cockroachdb/cockroach/pkg/rpc"

Index

Package Files

breaker.go clock_offset.go connection_class.go context.go context_testutils.go heartbeat.go heartbeat.pb.go keepalive.go metrics.go snappy.go stats_handler.go

Constants

const (
    // DefaultClass is the default ConnectionClass and should be used for most
    // client traffic.
    DefaultClass ConnectionClass = iota
    // SystemClass is the ConnectionClass used for system traffic.
    SystemClass

    // NumConnectionClasses is the number of valid ConnectionClass values.
    NumConnectionClasses int = iota
)

Variables

var (
    ErrInvalidLengthHeartbeat = fmt.Errorf("proto: negative length found during unmarshaling")
    ErrIntOverflowHeartbeat   = fmt.Errorf("proto: integer overflow")
)
var ErrNotHeartbeated = errors.New("not yet heartbeated")

ErrNotHeartbeated is returned by ConnHealth when we have not yet performed the first heartbeat.

func IsLocal Uses

func IsLocal(iface roachpb.InternalClient) bool

IsLocal returns true if the given InternalClient is local.

func NewServer Uses

func NewServer(ctx *Context) *grpc.Server

NewServer is a thin wrapper around grpc.NewServer that registers a heartbeat service.

func NewServerWithInterceptor Uses

func NewServerWithInterceptor(
    ctx *Context, interceptor func(fullMethod string) error,
) *grpc.Server

NewServerWithInterceptor is like NewServer, but accepts an additional interceptor which is called before streaming and unary RPCs and may inject an error.

func RegisterHeartbeatServer Uses

func RegisterHeartbeatServer(s *grpc.Server, srv HeartbeatServer)

func RegisterTestingHeartbeatStreamServer Uses

func RegisterTestingHeartbeatStreamServer(s *grpc.Server, srv TestingHeartbeatStreamServer)

type Connection Uses

type Connection struct {
    // contains filtered or unexported fields
}

Connection is a wrapper around grpc.ClientConn. It prevents the underlying connection from being used until it has been validated via heartbeat.

func (*Connection) Connect Uses

func (c *Connection) Connect(ctx context.Context) (*grpc.ClientConn, error)

Connect returns the underlying grpc.ClientConn after it has been validated, or an error if dialing or validation fails.

func (*Connection) Health Uses

func (c *Connection) Health() error

Health returns an error indicating the success or failure of the connection's latest heartbeat. Returns ErrNotHeartbeated if the first heartbeat has not completed.

type ConnectionClass Uses

type ConnectionClass int8

ConnectionClass is the identifier of a group of RPC client sessions that are allowed to share an underlying TCP connections; RPC sessions with different connection classes are guaranteed to use separate gRPC client connections.

RPC sessions that share a connection class are arbitrated using the gRPC flow control logic, see google.golang.org/grpc/internal/transport. The lack of support of prioritization in the current gRPC implementation is the reason why we are separating different priority flows across separate TCP connections. Future gRPC improvements may enable further simplification here. See https://github.com/grpc/grpc-go/issues/1448 for progress on gRPC's adoption of HTTP2 priorities.

func ConnectionClassForKey Uses

func ConnectionClassForKey(key roachpb.RKey) ConnectionClass

ConnectionClassForKey determines the ConnectionClass which should be used for traffic addressed to the RKey.

type Context Uses

type Context struct {
    *base.Config

    AmbientCtx log.AmbientContext
    LocalClock *hlc.Clock

    Stopper      *stop.Stopper
    RemoteClocks *RemoteClockMonitor

    HeartbeatCB func()

    ClusterID base.ClusterIDContainer
    NodeID    base.NodeIDContainer

    // For unittesting.
    BreakerFactory func() *circuit.Breaker

    // For testing. See the comment on the same field in HeartbeatService.
    TestingAllowNamedRPCToAnonymousServer bool
    // contains filtered or unexported fields
}

Context contains the fields required by the rpc framework.

func NewContext Uses

func NewContext(
    ambient log.AmbientContext,
    baseCtx *base.Config,
    hlcClock *hlc.Clock,
    stopper *stop.Stopper,
    version *cluster.ExposedClusterVersion,
) *Context

NewContext creates an rpc Context with the supplied values.

func NewContextWithTestingKnobs Uses

func NewContextWithTestingKnobs(
    ambient log.AmbientContext,
    baseCtx *base.Config,
    hlcClock *hlc.Clock,
    stopper *stop.Stopper,
    version *cluster.ExposedClusterVersion,
    knobs ContextTestingKnobs,
) *Context

NewContextWithTestingKnobs creates an rpc Context with the supplied values.

func NewInsecureTestingContext Uses

func NewInsecureTestingContext(clock *hlc.Clock, stopper *stop.Stopper) *Context

NewInsecureTestingContext creates an insecure rpc Context suitable for tests.

func NewInsecureTestingContextWithClusterID Uses

func NewInsecureTestingContextWithClusterID(
    clock *hlc.Clock, stopper *stop.Stopper, clusterID uuid.UUID,
) *Context

NewInsecureTestingContextWithClusterID creates an insecure rpc Context suitable for tests. The context is given the provided cluster ID.

func NewInsecureTestingContextWithKnobs Uses

func NewInsecureTestingContextWithKnobs(
    clock *hlc.Clock, stopper *stop.Stopper, knobs ContextTestingKnobs,
) *Context

NewInsecureTestingContextWithKnobs creates an insecure rpc Context suitable for tests configured with the provided knobs.

func (*Context) ClusterName Uses

func (ctx *Context) ClusterName() string

ClusterName retrieves the configured cluster name.

func (*Context) GRPCDialNode Uses

func (ctx *Context) GRPCDialNode(
    target string, remoteNodeID roachpb.NodeID, class ConnectionClass,
) *Connection

GRPCDialNode calls grpc.Dial with options appropriate for the context and class (see the comment on ConnectionClass).

The remoteNodeID becomes a constraint on the expected node ID of the remote node; this is checked during heartbeats. The caller is responsible for ensuring the remote node ID is known prior to using this function.

func (*Context) GRPCDialOptions Uses

func (ctx *Context) GRPCDialOptions() ([]grpc.DialOption, error)

GRPCDialOptions returns the minimal `grpc.DialOption`s necessary to connect to a server created with `NewServer`.

At the time of writing, this is being used for making net.Pipe-based connections, so only those options that affect semantics are included. In particular, performance tuning options are omitted. Decompression is necessarily included to support compression-enabled servers, and compression is included for symmetry. These choices are admittedly subjective.

func (*Context) GRPCDialRaw Uses

func (ctx *Context) GRPCDialRaw(target string) (*grpc.ClientConn, <-chan struct{}, error)

GRPCDialRaw calls grpc.Dial with options appropriate for the context. Unlike GRPCDialNode, it does not start an RPC heartbeat to validate the connection. This connection will not be reconnected automatically; the returned channel is closed when a reconnection is attempted. This method implies a DefaultClass ConnectionClass for the returned ClientConn.

func (*Context) GRPCUnvalidatedDial Uses

func (ctx *Context) GRPCUnvalidatedDial(target string) *Connection

GRPCUnvalidatedDial uses GRPCDialNode and disables validation of the node ID between client and server. This function should only be used with the gossip client and CLI commands which can talk to any node. This method implies a SystemClass.

func (*Context) GetLocalInternalClientForAddr Uses

func (ctx *Context) GetLocalInternalClientForAddr(
    target string, nodeID roachpb.NodeID,
) roachpb.InternalClient

GetLocalInternalClientForAddr returns the context's internal batch client for target, if it exists.

func (*Context) GetStatsMap Uses

func (ctx *Context) GetStatsMap() *syncmap.Map

GetStatsMap returns a map of network statistics maintained by the internal stats handler. The map is from the remote network address (in string form) to an rpc.Stats object.

func (*Context) Metrics Uses

func (ctx *Context) Metrics() *Metrics

Metrics returns the Context's Metrics struct.

func (*Context) NewBreaker Uses

func (ctx *Context) NewBreaker(name string) *circuit.Breaker

NewBreaker creates a new circuit breaker properly configured for RPC connections. name is used internally for logging state changes of the returned breaker.

func (*Context) SetLocalInternalServer Uses

func (ctx *Context) SetLocalInternalServer(internalServer roachpb.InternalServer)

SetLocalInternalServer sets the context's local internal batch server.

type ContextTestingKnobs Uses

type ContextTestingKnobs struct {

    // UnaryClientInterceptor if non-nil will be called at dial time to provide
    // the base unary interceptor for client connections.
    // This function may return a nil interceptor to avoid injecting behavior
    // for a given target and class.
    UnaryClientInterceptor func(target string, class ConnectionClass) grpc.UnaryClientInterceptor

    // StreamClient if non-nil will be called at dial time to provide
    // the base stream interceptor for client connections.
    // This function may return a nil interceptor to avoid injecting behavior
    // for a given target and class.
    StreamClientInterceptor func(target string, class ConnectionClass) grpc.StreamClientInterceptor

    // ClusterID initializes the Context's ClusterID container to this value if
    // non-nil at construction time.
    ClusterID *uuid.UUID
}

ContextTestingKnobs provides hooks to aid in testing the system. The testing knob functions are called at various points in the Context life cycle if they are non-nil.

type HeartbeatClient Uses

type HeartbeatClient interface {
    Ping(ctx context.Context, in *PingRequest, opts ...grpc.CallOption) (*PingResponse, error)
}

HeartbeatClient is the client API for Heartbeat service.

For semantics around ctx use and closing/ending streaming RPCs, please refer to https://godoc.org/google.golang.org/grpc#ClientConn.NewStream.

func NewHeartbeatClient Uses

func NewHeartbeatClient(cc *grpc.ClientConn) HeartbeatClient

type HeartbeatServer Uses

type HeartbeatServer interface {
    Ping(context.Context, *PingRequest) (*PingResponse, error)
}

HeartbeatServer is the server API for Heartbeat service.

type HeartbeatService Uses

type HeartbeatService struct {
    // contains filtered or unexported fields
}

A HeartbeatService exposes a method to echo its request params. It doubles as a way to measure the offset of the server from other nodes. It uses the clock to return the server time every heartbeat. It also keeps track of remote clocks sent to it by storing them in the remoteClockMonitor.

func (*HeartbeatService) Ping Uses

func (hs *HeartbeatService) Ping(ctx context.Context, args *PingRequest) (*PingResponse, error)

Ping echos the contents of the request to the response, and returns the server's current clock value, allowing the requester to measure its clock. The requester should also estimate its offset from this server along with the requester's address.

type Metrics Uses

type Metrics struct {

    // HeartbeatLoopsStarted is a counter which tracks the number of heartbeat
    // loops which have been started.
    HeartbeatLoopsStarted *metric.Counter

    // HeartbeatLoopsExited is a counter which tracks the number of heartbeat
    // loops which have exited with an error. The only time a heartbeat loop
    // exits without an error is during server shutdown.
    HeartbeatLoopsExited *metric.Counter

    // HeartbeatsInitializing tracks the current number of heartbeat loops
    // which have not yet ever succeeded.
    HeartbeatsInitializing *metric.Gauge
    // HeartbeatsNominal tracks the current number of heartbeat loops which
    // succeeded on their previous attempt.
    HeartbeatsNominal *metric.Gauge
    // HeartbeatsNominal tracks the current number of heartbeat loops which
    // succeeded on their previous attempt.
    HeartbeatsFailed *metric.Gauge
}

Metrics is a metrics struct for Context metrics.

type PingRequest Uses

type PingRequest struct {
    // Echo this string with PingResponse.
    Ping string `protobuf:"bytes,1,opt,name=ping" json:"ping"`
    // The last offset the client measured with the server.
    Offset RemoteOffset `protobuf:"bytes,2,opt,name=offset" json:"offset"`
    // The address of the client.
    Addr string `protobuf:"bytes,3,opt,name=addr" json:"addr"`
    // The configured maximum clock offset (in nanoseconds) on the server.
    MaxOffsetNanos int64 `protobuf:"varint,4,opt,name=max_offset_nanos,json=maxOffsetNanos" json:"max_offset_nanos"`
    // Cluster ID to prevent connections between nodes in different clusters.
    ClusterID     *github_com_cockroachdb_cockroach_pkg_util_uuid.UUID `protobuf:"bytes,5,opt,name=cluster_id,json=clusterId,customtype=github.com/cockroachdb/cockroach/pkg/util/uuid.UUID" json:"cluster_id,omitempty"`
    ServerVersion roachpb.Version                                      `protobuf:"bytes,6,opt,name=server_version,json=serverVersion" json:"server_version"`
    // Node ID to prevent connections from being misrouted to an invalid node inside the cluster.
    NodeID github_com_cockroachdb_cockroach_pkg_roachpb.NodeID `protobuf:"varint,7,opt,name=node_id,json=nodeId,customtype=github.com/cockroachdb/cockroach/pkg/roachpb.NodeID" json:"node_id"`
}

A PingRequest specifies the string to echo in response. Fields are exported so that they will be serialized in the rpc call.

func (*PingRequest) Descriptor Uses

func (*PingRequest) Descriptor() ([]byte, []int)

func (*PingRequest) Marshal Uses

func (m *PingRequest) Marshal() (dAtA []byte, err error)

func (*PingRequest) MarshalTo Uses

func (m *PingRequest) MarshalTo(dAtA []byte) (int, error)

func (*PingRequest) ProtoMessage Uses

func (*PingRequest) ProtoMessage()

func (*PingRequest) Reset Uses

func (m *PingRequest) Reset()

func (*PingRequest) Size Uses

func (m *PingRequest) Size() (n int)

func (*PingRequest) String Uses

func (m *PingRequest) String() string

func (*PingRequest) Unmarshal Uses

func (m *PingRequest) Unmarshal(dAtA []byte) error

func (*PingRequest) XXX_DiscardUnknown Uses

func (m *PingRequest) XXX_DiscardUnknown()

func (*PingRequest) XXX_Marshal Uses

func (m *PingRequest) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*PingRequest) XXX_Merge Uses

func (dst *PingRequest) XXX_Merge(src proto.Message)

func (*PingRequest) XXX_Size Uses

func (m *PingRequest) XXX_Size() int

func (*PingRequest) XXX_Unmarshal Uses

func (m *PingRequest) XXX_Unmarshal(b []byte) error

type PingResponse Uses

type PingResponse struct {
    // An echo of value sent with PingRequest.
    Pong          string          `protobuf:"bytes,1,opt,name=pong" json:"pong"`
    ServerTime    int64           `protobuf:"varint,2,opt,name=server_time,json=serverTime" json:"server_time"`
    ServerVersion roachpb.Version `protobuf:"bytes,3,opt,name=server_version,json=serverVersion" json:"server_version"`
    // Cluster name to prevent joining a new node to the wrong cluster.
    ClusterName string `protobuf:"bytes,4,opt,name=cluster_name,json=clusterName" json:"cluster_name"`
    // Skip cluster name check if either side's name is empty / not configured.
    DisableClusterNameVerification bool `protobuf:"varint,5,opt,name=disable_cluster_name_verification,json=disableClusterNameVerification" json:"disable_cluster_name_verification"`
}

A PingResponse contains the echoed ping request string.

func (*PingResponse) Descriptor Uses

func (*PingResponse) Descriptor() ([]byte, []int)

func (*PingResponse) Marshal Uses

func (m *PingResponse) Marshal() (dAtA []byte, err error)

func (*PingResponse) MarshalTo Uses

func (m *PingResponse) MarshalTo(dAtA []byte) (int, error)

func (*PingResponse) ProtoMessage Uses

func (*PingResponse) ProtoMessage()

func (*PingResponse) Reset Uses

func (m *PingResponse) Reset()

func (*PingResponse) Size Uses

func (m *PingResponse) Size() (n int)

func (*PingResponse) String Uses

func (m *PingResponse) String() string

func (*PingResponse) Unmarshal Uses

func (m *PingResponse) Unmarshal(dAtA []byte) error

func (*PingResponse) XXX_DiscardUnknown Uses

func (m *PingResponse) XXX_DiscardUnknown()

func (*PingResponse) XXX_Marshal Uses

func (m *PingResponse) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*PingResponse) XXX_Merge Uses

func (dst *PingResponse) XXX_Merge(src proto.Message)

func (*PingResponse) XXX_Size Uses

func (m *PingResponse) XXX_Size() int

func (*PingResponse) XXX_Unmarshal Uses

func (m *PingResponse) XXX_Unmarshal(b []byte) error

type RemoteClockMetrics Uses

type RemoteClockMetrics struct {
    ClockOffsetMeanNanos   *metric.Gauge
    ClockOffsetStdDevNanos *metric.Gauge
    LatencyHistogramNanos  *metric.Histogram
}

RemoteClockMetrics is the collection of metrics for the clock monitor.

type RemoteClockMonitor Uses

type RemoteClockMonitor struct {
    // contains filtered or unexported fields
}

RemoteClockMonitor keeps track of the most recent measurements of remote offsets and round-trip latency from this node to connected nodes.

func (*RemoteClockMonitor) AllLatencies Uses

func (r *RemoteClockMonitor) AllLatencies() map[string]time.Duration

AllLatencies returns a map of all currently valid latency measurements.

func (*RemoteClockMonitor) Latency Uses

func (r *RemoteClockMonitor) Latency(addr string) (time.Duration, bool)

Latency returns the exponentially weighted moving average latency to the given node address. Returns true if the measurement is valid, or false if we don't have enough samples to compute a reliable average.

func (*RemoteClockMonitor) Metrics Uses

func (r *RemoteClockMonitor) Metrics() *RemoteClockMetrics

Metrics returns the metrics struct. Useful to examine individual metrics, or to add to the registry.

func (*RemoteClockMonitor) UpdateOffset Uses

func (r *RemoteClockMonitor) UpdateOffset(
    ctx context.Context, addr string, offset RemoteOffset, roundTripLatency time.Duration,
)

UpdateOffset is a thread-safe way to update the remote clock and latency measurements.

It only updates the offset for addr if one of the following cases holds: 1. There is no prior offset for that address. 2. The old offset for addr was measured long enough ago to be considered stale. 3. The new offset's error is smaller than the old offset's error.

Pass a roundTripLatency of 0 or less to avoid recording the latency.

func (*RemoteClockMonitor) VerifyClockOffset Uses

func (r *RemoteClockMonitor) VerifyClockOffset(ctx context.Context) error

VerifyClockOffset calculates the number of nodes to which the known offset is healthy (as defined by RemoteOffset.isHealthy). It returns nil iff more than half the known offsets are healthy, and an error otherwise. A non-nil return indicates that this node's clock is unreliable, and that the node should terminate.

type RemoteOffset Uses

type RemoteOffset struct {
    // The estimated offset from the remote server, in nanoseconds.
    Offset int64 `protobuf:"varint,1,opt,name=offset" json:"offset"`
    // The maximum error of the measured offset, in nanoseconds.
    Uncertainty int64 `protobuf:"varint,2,opt,name=uncertainty" json:"uncertainty"`
    // Measurement time, in nanoseconds from unix epoch.
    MeasuredAt int64 `protobuf:"varint,3,opt,name=measured_at,json=measuredAt" json:"measured_at"`
}

RemoteOffset keeps track of this client's estimate of its offset from a remote server. Uncertainty is the maximum error in the reading of this offset, so that the real offset should be in the interval [Offset - Uncertainty, Offset + Uncertainty]. If the last heartbeat timed out, Offset = 0.

Offset and Uncertainty are measured using the remote clock reading technique described in http://se.inf.tu-dresden.de/pubs/papers/SRDS1994.pdf, page 6.

func (*RemoteOffset) Descriptor Uses

func (*RemoteOffset) Descriptor() ([]byte, []int)

func (*RemoteOffset) Marshal Uses

func (m *RemoteOffset) Marshal() (dAtA []byte, err error)

func (*RemoteOffset) MarshalTo Uses

func (m *RemoteOffset) MarshalTo(dAtA []byte) (int, error)

func (*RemoteOffset) ProtoMessage Uses

func (*RemoteOffset) ProtoMessage()

func (*RemoteOffset) Reset Uses

func (m *RemoteOffset) Reset()

func (*RemoteOffset) Size Uses

func (m *RemoteOffset) Size() (n int)

func (RemoteOffset) String Uses

func (r RemoteOffset) String() string

String formats the RemoteOffset for human readability.

func (*RemoteOffset) Unmarshal Uses

func (m *RemoteOffset) Unmarshal(dAtA []byte) error

func (*RemoteOffset) XXX_DiscardUnknown Uses

func (m *RemoteOffset) XXX_DiscardUnknown()

func (*RemoteOffset) XXX_Marshal Uses

func (m *RemoteOffset) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*RemoteOffset) XXX_Merge Uses

func (dst *RemoteOffset) XXX_Merge(src proto.Message)

func (*RemoteOffset) XXX_Size Uses

func (m *RemoteOffset) XXX_Size() int

func (*RemoteOffset) XXX_Unmarshal Uses

func (m *RemoteOffset) XXX_Unmarshal(b []byte) error

type Stats Uses

type Stats struct {
    // contains filtered or unexported fields
}

Stats stores network statistics between this node and another.

func (*Stats) Count Uses

func (s *Stats) Count() int64

Count returns the total number of RPCs.

func (*Stats) Incoming Uses

func (s *Stats) Incoming() int64

Incoming returns the total bytes of incoming network traffic.

func (*Stats) Outgoing Uses

func (s *Stats) Outgoing() int64

Outgoing returns the total bytes of outgoing network traffic.

type StatsHandler Uses

type StatsHandler struct {
    // contains filtered or unexported fields
}

StatsHandler manages a map of Stats objects, one per connection. It implements grpc.stats.Handler, and is used directly for any incoming connections which connect to this node's server. It uses the newClient() method to return handlers for use with outgoing client connections from this node to remote nodes.

func (*StatsHandler) HandleConn Uses

func (sh *StatsHandler) HandleConn(context.Context, stats.ConnStats)

HandleConn implements the grpc.stats.Handler interface. This interface is used directly for server-side stats recording.

func (*StatsHandler) HandleRPC Uses

func (sh *StatsHandler) HandleRPC(ctx context.Context, rpcStats stats.RPCStats)

HandleRPC implements the grpc.stats.Handler interface. This interface is used directly for server-side stats recording. We consult the provided context for the remote address and use that to key into our stats map in order to properly update incoming and outgoing throughput for the implicated remote node.

func (*StatsHandler) TagConn Uses

func (sh *StatsHandler) TagConn(ctx context.Context, cti *stats.ConnTagInfo) context.Context

TagConn implements the grpc.stats.Handler interface. This interface is used directly for server-side stats recording. We tag the provided context with the remote address provided by the ConnTagInfo, and use that to properly update the Stats object belonging to that remote address.

func (*StatsHandler) TagRPC Uses

func (sh *StatsHandler) TagRPC(ctx context.Context, rti *stats.RPCTagInfo) context.Context

TagRPC implements the grpc.stats.Handler interface. This interface is used directly for server-side stats recording.

type TestingHeartbeatStreamClient Uses

type TestingHeartbeatStreamClient interface {
    PingStream(ctx context.Context, opts ...grpc.CallOption) (TestingHeartbeatStream_PingStreamClient, error)
}

TestingHeartbeatStreamClient is the client API for TestingHeartbeatStream service.

For semantics around ctx use and closing/ending streaming RPCs, please refer to https://godoc.org/google.golang.org/grpc#ClientConn.NewStream.

func NewTestingHeartbeatStreamClient Uses

func NewTestingHeartbeatStreamClient(cc *grpc.ClientConn) TestingHeartbeatStreamClient

type TestingHeartbeatStreamServer Uses

type TestingHeartbeatStreamServer interface {
    PingStream(TestingHeartbeatStream_PingStreamServer) error
}

TestingHeartbeatStreamServer is the server API for TestingHeartbeatStream service.

type TestingHeartbeatStream_PingStreamClient Uses

type TestingHeartbeatStream_PingStreamClient interface {
    Send(*PingRequest) error
    Recv() (*PingResponse, error)
    grpc.ClientStream
}

type TestingHeartbeatStream_PingStreamServer Uses

type TestingHeartbeatStream_PingStreamServer interface {
    Send(*PingResponse) error
    Recv() (*PingRequest, error)
    grpc.ServerStream
}

Directories

PathSynopsis
nodedialer

Package rpc imports 47 packages (graph) and is imported by 63 packages. Updated 2019-09-13. Refresh now. Tools for package owners.