dvara

package module
v0.0.0-...-95e0546 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 25, 2017 License: BSD-3-Clause Imports: 18 Imported by: 0

README

dvara Build Status

NOTE: dvara is no longer in use and we are not accepting new pull requests for it. Fork away and use it if it works for you!


dvara provides a connection pooling proxy for MongoDB. For more information look at the associated blog post: http://blog.parse.com/2014/06/23/dvara/.

To build from source you'll need Go. With it you can install it using:

go get github.com/facebookgo/dvara/cmd/dvara

Library documentation: https://godoc.org/github.com/facebookgo/dvara

Documentation

Overview

Package dvara provides a library to enable setting up a proxy server for mongo.

Index

Constants

View Source
const (
	OpReply       = OpCode(1)
	OpMessage     = OpCode(1000)
	OpUpdate      = OpCode(2001)
	OpInsert      = OpCode(2002)
	Reserved      = OpCode(2003)
	OpQuery       = OpCode(2004)
	OpGetMore     = OpCode(2005)
	OpDelete      = OpCode(2006)
	OpKillCursors = OpCode(2007)
)

The full set of known request op codes: http://docs.mongodb.org/meta-driver/latest/legacy/mongodb-wire-protocol/#request-opcodes

View Source
const (
	// ReplicaStatePrimary indicates the node is a primary.
	ReplicaStatePrimary = ReplicaState("PRIMARY")

	// ReplicaStateSecondary indicates the node is a secondary.
	ReplicaStateSecondary = ReplicaState("SECONDARY")

	// ReplicaStateArbiter indicates the node is an arbiter.
	ReplicaStateArbiter = ReplicaState("ARBITER")
)

Variables

This section is empty.

Functions

This section is empty.

Types

type GetLastErrorRewriter

type GetLastErrorRewriter struct {
	Log Logger `inject:""`
}

GetLastErrorRewriter handles getLastError requests and proxies, caches or sends cached responses as necessary.

func (*GetLastErrorRewriter) Rewrite

func (r *GetLastErrorRewriter) Rewrite(
	h *messageHeader,
	parts [][]byte,
	client io.ReadWriter,
	server io.ReadWriter,
	lastError *LastError,
) error

Rewrite handles getLastError requests.

type IsMasterResponseRewriter

type IsMasterResponseRewriter struct {
	Log                 Logger              `inject:""`
	ProxyMapper         ProxyMapper         `inject:""`
	ReplyRW             *ReplyRW            `inject:""`
	ReplicaStateCompare ReplicaStateCompare `inject:""`
}

IsMasterResponseRewriter rewrites the response for the "isMaster" query.

func (*IsMasterResponseRewriter) Rewrite

func (r *IsMasterResponseRewriter) Rewrite(client io.Writer, server io.Reader) error

Rewrite rewrites the response for the "isMaster" query.

type LastError

type LastError struct {
	// contains filtered or unexported fields
}

LastError holds the last known error.

func (*LastError) Exists

func (l *LastError) Exists() bool

Exists returns true if this instance contains a cached error.

func (*LastError) Reset

func (l *LastError) Reset()

Reset resets the stored error clearing it.

type Logger

type Logger interface {
	Error(args ...interface{})
	Errorf(format string, args ...interface{})
	Warn(args ...interface{})
	Warnf(format string, args ...interface{})
	Info(args ...interface{})
	Infof(format string, args ...interface{})
	Debug(args ...interface{})
	Debugf(format string, args ...interface{})
}

Logger allows for simple text logging.

type OpCode

type OpCode int32

OpCode allow identifying the type of operation:

http://docs.mongodb.org/meta-driver/latest/legacy/mongodb-wire-protocol/#request-opcodes

func (OpCode) HasResponse

func (c OpCode) HasResponse() bool

HasResponse tells us if the operation will have a response from the server.

func (OpCode) IsMutation

func (c OpCode) IsMutation() bool

IsMutation tells us if the operation will mutate data. These operations can be followed up by a getLastErr operation.

func (OpCode) String

func (c OpCode) String() string

String returns a human readable representation of the OpCode.

type Proxy

type Proxy struct {
	Log            Logger
	ReplicaSet     *ReplicaSet
	ClientListener net.Listener // Listener for incoming client connections
	ProxyAddr      string       // Address for incoming client connections
	MongoAddr      string       // Address for destination Mongo server
	// contains filtered or unexported fields
}

Proxy sends stuff from clients to mongo servers.

func (*Proxy) Start

func (p *Proxy) Start() error

Start the proxy.

func (*Proxy) Stop

func (p *Proxy) Stop() error

Stop the proxy.

func (*Proxy) String

func (p *Proxy) String() string

String representation for debugging.

type ProxyMapper

type ProxyMapper interface {
	Proxy(h string) (string, error)
}

ProxyMapper maps real mongo addresses to their corresponding proxy addresses.

type ProxyMapperError

type ProxyMapperError struct {
	RealHost string
	State    ReplicaState
}

ProxyMapperError occurs when a known host is being ignored and does not have a corresponding proxy address.

func (*ProxyMapperError) Error

func (p *ProxyMapperError) Error() string

type ProxyQuery

type ProxyQuery struct {
	Log                              Logger                            `inject:""`
	GetLastErrorRewriter             *GetLastErrorRewriter             `inject:""`
	IsMasterResponseRewriter         *IsMasterResponseRewriter         `inject:""`
	ReplSetGetStatusResponseRewriter *ReplSetGetStatusResponseRewriter `inject:""`
}

ProxyQuery proxies an OpQuery and a corresponding response.

func (*ProxyQuery) Proxy

func (p *ProxyQuery) Proxy(
	h *messageHeader,
	client io.ReadWriter,
	server io.ReadWriter,
	lastError *LastError,
) error

Proxy proxies an OpQuery and a corresponding response.

type ReplSetGetStatusResponseRewriter

type ReplSetGetStatusResponseRewriter struct {
	Log                 Logger              `inject:""`
	ProxyMapper         ProxyMapper         `inject:""`
	ReplyRW             *ReplyRW            `inject:""`
	ReplicaStateCompare ReplicaStateCompare `inject:""`
}

ReplSetGetStatusResponseRewriter rewrites the "replSetGetStatus" response.

func (*ReplSetGetStatusResponseRewriter) Rewrite

func (r *ReplSetGetStatusResponseRewriter) Rewrite(client io.Writer, server io.Reader) error

Rewrite rewrites the "replSetGetStatus" response.

type ReplicaSet

type ReplicaSet struct {
	Log                    Logger                  `inject:""`
	ReplicaSetStateCreator *ReplicaSetStateCreator `inject:""`
	ProxyQuery             *ProxyQuery             `inject:""`

	// Stats if provided will be used to record interesting stats.
	Stats stats.Client `inject:""`

	// Comma separated list of mongo addresses. This is the list of "seed"
	// servers, and one of two conditions must be met for each entry here -- it's
	// either alive and part of the same replica set as all others listed, or is
	// not reachable.
	Addrs string

	// PortStart and PortEnd define the port range within which proxies will be
	// allocated.
	PortStart int
	PortEnd   int

	// Maximum number of connections that will be established to each mongo node.
	MaxConnections uint

	// MinIdleConnections is the number of idle server connections we'll keep
	// around.
	MinIdleConnections uint

	// ServerIdleTimeout is the duration after which a server connection will be
	// considered idle.
	ServerIdleTimeout time.Duration

	// ServerClosePoolSize is the number of goroutines that will handle closing
	// server connections.
	ServerClosePoolSize uint

	// ClientIdleTimeout is how long until we'll consider a client connection
	// idle and disconnect and release it's resources.
	ClientIdleTimeout time.Duration

	// MaxPerClientConnections is how many client connections are allowed from a
	// single client.
	MaxPerClientConnections uint

	// GetLastErrorTimeout is how long we'll hold on to an acquired server
	// connection expecting a possibly getLastError call.
	GetLastErrorTimeout time.Duration

	// MessageTimeout is used to determine the timeout for a single message to be
	// proxied.
	MessageTimeout time.Duration

	// Name is the name of the replica set to connect to. Nodes that are not part
	// of this replica set will be ignored. If this is empty, the first replica set
	// will be used
	Name string
	// contains filtered or unexported fields
}

ReplicaSet manages the real => proxy address mapping. NewReplicaSet returns the ReplicaSet given the list of seed servers. It is required for the seed servers to be a strict subset of the actual members if they are reachable. That is, if two of the addresses are members of different replica sets, it will be considered an error.

func (*ReplicaSet) Proxy

func (r *ReplicaSet) Proxy(h string) (string, error)

Proxy returns the corresponding proxy address for the given real mongo address.

func (*ReplicaSet) ProxyMembers

func (r *ReplicaSet) ProxyMembers() []string

ProxyMembers returns the list of proxy members in this ReplicaSet.

func (*ReplicaSet) Restart

func (r *ReplicaSet) Restart()

Restart stops all the proxies and restarts them. This is used when we detect an RS config change, like when an election happens.

func (*ReplicaSet) SameIM

func (r *ReplicaSet) SameIM(o *isMasterResponse) bool

SameIM checks if the given isMasterResponse is the same as the last state.

func (*ReplicaSet) SameRS

func (r *ReplicaSet) SameRS(o *replSetGetStatusResponse) bool

SameRS checks if the given replSetGetStatusResponse is the same as the last state.

func (*ReplicaSet) Start

func (r *ReplicaSet) Start() error

Start starts proxies to support this ReplicaSet.

func (*ReplicaSet) Stop

func (r *ReplicaSet) Stop() error

Stop stops all the associated proxies for this ReplicaSet.

type ReplicaSetState

type ReplicaSetState struct {
	// contains filtered or unexported fields
}

ReplicaSetState is a snapshot of the RS configuration at some point in time.

func NewReplicaSetState

func NewReplicaSetState(addr string) (*ReplicaSetState, error)

NewReplicaSetState creates a new ReplicaSetState using the given address.

func (*ReplicaSetState) Addrs

func (r *ReplicaSetState) Addrs() []string

Addrs returns the addresses of members in primary or secondary state.

func (*ReplicaSetState) AssertEqual

func (r *ReplicaSetState) AssertEqual(o *ReplicaSetState) error

AssertEqual checks if the given ReplicaSetState equals this one. It returns a rich error message including the entire state for easier debugging.

func (*ReplicaSetState) Equal

func (r *ReplicaSetState) Equal(o *ReplicaSetState) bool

Equal returns true if the given ReplicaSetState is the same as this one.

func (*ReplicaSetState) SameIM

func (r *ReplicaSetState) SameIM(o *isMasterResponse) bool

SameIM checks if the given isMasterResponse is the same as the one we have.

func (*ReplicaSetState) SameRS

func (r *ReplicaSetState) SameRS(o *replSetGetStatusResponse) bool

SameRS checks if the given replSetGetStatusResponse is the same as the one we have.

type ReplicaSetStateCreator

type ReplicaSetStateCreator struct {
	Log Logger `inject:""`
}

ReplicaSetStateCreator allows for creating a ReplicaSetState from a given set of seed addresses.

func (*ReplicaSetStateCreator) FromAddrs

func (c *ReplicaSetStateCreator) FromAddrs(addrs []string, replicaSetName string) (*ReplicaSetState, error)

FromAddrs creates a ReplicaSetState from the given set of see addresses. It requires the addresses to be part of the same Replica Set.

type ReplicaState

type ReplicaState string

ReplicaState is the state of a node in the replica.

type ReplicaStateCompare

type ReplicaStateCompare interface {
	SameRS(o *replSetGetStatusResponse) bool
	SameIM(o *isMasterResponse) bool
}

ReplicaStateCompare provides the last ReplicaSetState and allows for checking if it has changed as we rewrite/proxy the isMaster & replSetGetStatus queries.

type ReplyRW

type ReplyRW struct {
	Log Logger `inject:""`
}

ReplyRW provides common helpers for rewriting replies from the server.

func (*ReplyRW) ReadOne

func (r *ReplyRW) ReadOne(server io.Reader, v interface{}) (*messageHeader, replyPrefix, int32, error)

ReadOne reads a 1 document response, from the server, unmarshals it into v and returns the various parts.

func (*ReplyRW) WriteOne

func (r *ReplyRW) WriteOne(client io.Writer, h *messageHeader, prefix replyPrefix, oldDocLen int32, v interface{}) error

WriteOne writes a rewritten response to the client.

Directories

Path Synopsis
cmd

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL