dvara

package module
v0.0.0-...-fa9593c Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jan 20, 2015 License: BSD-3-Clause Imports: 22 Imported by: 0

README

dvara Build Status

dvara provides a connection pooling proxy for MongoDB. For more information look at the associated blog post: http://blog.parse.com/2014/06/23/dvara/.

To build from source you'll need Go. With it you can install it using:

go get github.com/facebookgo/dvara/cmd/dvara

Library documentation: https://godoc.org/github.com/facebookgo/dvara

Documentation

Overview

Package dvara provides a library to enable setting up a proxy server for mongo.

Index

Constants

View Source
const (
	OpReply       = OpCode(1)
	OpMessage     = OpCode(1000)
	OpUpdate      = OpCode(2001)
	OpInsert      = OpCode(2002)
	Reserved      = OpCode(2003)
	OpQuery       = OpCode(2004)
	OpGetMore     = OpCode(2005)
	OpDelete      = OpCode(2006)
	OpKillCursors = OpCode(2007)
)

The full set of known request op codes: http://docs.mongodb.org/meta-driver/latest/legacy/mongodb-wire-protocol/#request-opcodes

View Source
const (
	// ReplicaStatePrimary indicates the node is a primary.
	ReplicaStatePrimary = ReplicaState("PRIMARY")

	// ReplicaStateSecondary indicates the node is a secondary.
	ReplicaStateSecondary = ReplicaState("SECONDARY")

	// ReplicaStateArbiter indicates the node is an arbiter.
	ReplicaStateArbiter = ReplicaState("ARBITER")
)

Variables

This section is empty.

Functions

This section is empty.

Types

type GetLastErrorRewriter

type GetLastErrorRewriter struct {
	Log Logger `inject:""`
}

GetLastErrorRewriter handles getLastError requests and proxies, caches or sends cached responses as necessary.

func (*GetLastErrorRewriter) Rewrite

func (r *GetLastErrorRewriter) Rewrite(
	h *messageHeader,
	parts [][]byte,
	client io.ReadWriter,
	server io.ReadWriter,
	lastError *LastError,
) error

Rewrite handles getLastError requests.

type IsMasterResponseRewriter

type IsMasterResponseRewriter struct {
	Log                 Logger              `inject:""`
	ProxyMapper         ProxyMapper         `inject:""`
	ReplyRW             *ReplyRW            `inject:""`
	ReplicaStateCompare ReplicaStateCompare `inject:""`
}

IsMasterResponseRewriter rewrites the response for the "isMaster" query.

func (*IsMasterResponseRewriter) Rewrite

func (r *IsMasterResponseRewriter) Rewrite(client io.Writer, server io.Reader) error

Rewrite rewrites the response for the "isMaster" query.

type LastError

type LastError struct {
	// contains filtered or unexported fields
}

LastError holds the last known error.

func (*LastError) Exists

func (l *LastError) Exists() bool

Exists returns true if this instance contains a cached error.

func (*LastError) Reset

func (l *LastError) Reset()

Reset resets the stored error clearing it.

type Logger

type Logger interface {
	Error(args ...interface{})
	Errorf(format string, args ...interface{})
	Warn(args ...interface{})
	Warnf(format string, args ...interface{})
	Info(args ...interface{})
	Infof(format string, args ...interface{})
	Debug(args ...interface{})
	Debugf(format string, args ...interface{})
}

Logger allows for simple text logging.

type OpCode

type OpCode int32

OpCode allow identifying the type of operation:

http://docs.mongodb.org/meta-driver/latest/legacy/mongodb-wire-protocol/#request-opcodes

func (OpCode) HasResponse

func (c OpCode) HasResponse() bool

HasResponse tells us if the operation will have a response from the server.

func (OpCode) IsMutation

func (c OpCode) IsMutation() bool

IsMutation tells us if the operation will mutate data. These operations can be followed up by a getLastErr operation.

func (OpCode) String

func (c OpCode) String() string

String returns a human readable representation of the OpCode.

type Proxy

type Proxy struct {
	Log            Logger
	ReplicaSet     *ReplicaSet
	ClientListener net.Listener // Listener for incoming client connections
	Username       string
	Password       string
	ProxyAddr      string // Address for incoming client connections
	MongoAddr      string // Address for destination Mongo server
	// contains filtered or unexported fields
}

Proxy sends stuff from clients to mongo servers.

func (*Proxy) Start

func (p *Proxy) Start() error

Start the proxy.

func (*Proxy) Stop

func (p *Proxy) Stop() error

Stop the proxy.

func (*Proxy) String

func (p *Proxy) String() string

String representation for debugging.

type ProxyMapper

type ProxyMapper interface {
	Proxy(h string) (string, error)
}

ProxyMapper maps real mongo addresses to their corresponding proxy addresses.

type ProxyMapperError

type ProxyMapperError struct {
	RealHost string
	State    ReplicaState
}

ProxyMapperError occurs when a known host is being ignored and does not have a corresponding proxy address.

func (*ProxyMapperError) Error

func (p *ProxyMapperError) Error() string

type ProxyQuery

type ProxyQuery struct {
	Log                              Logger                            `inject:""`
	GetLastErrorRewriter             *GetLastErrorRewriter             `inject:""`
	IsMasterResponseRewriter         *IsMasterResponseRewriter         `inject:""`
	ReplSetGetStatusResponseRewriter *ReplSetGetStatusResponseRewriter `inject:""`
}

ProxyQuery proxies an OpQuery and a corresponding response.

func (*ProxyQuery) Proxy

func (p *ProxyQuery) Proxy(
	h *messageHeader,
	client io.ReadWriter,
	server io.ReadWriter,
	lastError *LastError,
) error

Proxy proxies an OpQuery and a corresponding response.

type ReplSetGetStatusResponseRewriter

type ReplSetGetStatusResponseRewriter struct {
	Log                 Logger              `inject:""`
	ProxyMapper         ProxyMapper         `inject:""`
	ReplyRW             *ReplyRW            `inject:""`
	ReplicaStateCompare ReplicaStateCompare `inject:""`
}

ReplSetGetStatusResponseRewriter rewrites the "replSetGetStatus" response.

func (*ReplSetGetStatusResponseRewriter) Rewrite

func (r *ReplSetGetStatusResponseRewriter) Rewrite(client io.Writer, server io.Reader) error

Rewrite rewrites the "replSetGetStatus" response.

type ReplicaSet

type ReplicaSet struct {
	Log                    Logger                  `inject:""`
	ReplicaSetStateCreator *ReplicaSetStateCreator `inject:""`
	ProxyQuery             *ProxyQuery             `inject:""`

	// Stats if provided will be used to record interesting stats.
	Stats stats.Client `inject:""`

	// Comma separated list of mongo addresses. This is the list of "seed"
	// servers, and one of two conditions must be met for each entry here -- it's
	// either alive and part of the same replica set as all others listed, or is
	// not reachable.
	Addrs string

	// PortStart and PortEnd define the port range within which proxies will be
	// allocated.
	PortStart int
	PortEnd   int

	// Maximum number of connections that will be established to each mongo node.
	MaxConnections uint

	// MinIdleConnections is the number of idle server connections we'll keep
	// around.
	MinIdleConnections uint

	// ServerIdleTimeout is the duration after which a server connection will be
	// considered idle.
	ServerIdleTimeout time.Duration

	// ServerClosePoolSize is the number of goroutines that will handle closing
	// server connections.
	ServerClosePoolSize uint

	// ClientIdleTimeout is how long until we'll consider a client connection
	// idle and disconnect and release it's resources.
	ClientIdleTimeout time.Duration

	// MaxPerClientConnections is how many client connections are allowed from a
	// single client.
	MaxPerClientConnections uint

	// GetLastErrorTimeout is how long we'll hold on to an acquired server
	// connection expecting a possibly getLastError call.
	GetLastErrorTimeout time.Duration

	// MessageTimeout is used to determine the timeout for a single message to be
	// proxied.
	MessageTimeout time.Duration

	// CertFile points to the SSL .cert file.  If it is not the empty string,
	// KeyFile must also not be the empty string.
	CertFile string

	// KeyFile points to the SSL .key file.  If it is not the empty string,
	// CertFile must also not be the empty string.
	KeyFile string

	// Username is the username used to connect to the server for retrieving replica state.
	Username string

	// Password is the password used to connect to the server for retrieving replica state.
	Password string

	ClientsConnected metrics.Counter
	// contains filtered or unexported fields
}

ReplicaSet manages the real => proxy address mapping. NewReplicaSet returns the ReplicaSet given the list of seed servers. It is required for the seed servers to be a strict subset of the actual members if they are reachable. That is, if two of the addresses are members of different replica sets, it will be considered an error.

func (*ReplicaSet) Proxy

func (r *ReplicaSet) Proxy(h string) (string, error)

Proxy returns the corresponding proxy address for the given real mongo address.

func (*ReplicaSet) ProxyMembers

func (r *ReplicaSet) ProxyMembers() []string

ProxyMembers returns the list of proxy members in this ReplicaSet.

func (*ReplicaSet) RegisterMetrics

func (r *ReplicaSet) RegisterMetrics(registry *gangliamr.Registry)

RegisterMetrics registers the available metrics.

func (*ReplicaSet) Restart

func (r *ReplicaSet) Restart()

Restart stops all the proxies and restarts them. This is used when we detect an RS config change, like when an election happens.

func (*ReplicaSet) SameIM

func (r *ReplicaSet) SameIM(o *isMasterResponse) bool

SameIM checks if the given isMasterResponse is the same as the last state.

func (*ReplicaSet) SameRS

func (r *ReplicaSet) SameRS(o *replSetGetStatusResponse) bool

SameRS checks if the given replSetGetStatusResponse is the same as the last state.

func (*ReplicaSet) Start

func (r *ReplicaSet) Start() error

Start starts proxies to support this ReplicaSet.

func (*ReplicaSet) Stop

func (r *ReplicaSet) Stop() error

Stop stops all the associated proxies for this ReplicaSet.

type ReplicaSetState

type ReplicaSetState struct {
	// contains filtered or unexported fields
}

ReplicaSetState is a snapshot of the RS configuration at some point in time.

func NewReplicaSetState

func NewReplicaSetState(username, password, addr string) (*ReplicaSetState, error)

NewReplicaSetState creates a new ReplicaSetState using the given address.

func (*ReplicaSetState) Addrs

func (r *ReplicaSetState) Addrs() []string

Addrs returns the addresses of members in primary or secondary state.

func (*ReplicaSetState) AssertEqual

func (r *ReplicaSetState) AssertEqual(o *ReplicaSetState) error

AssertEqual checks if the given ReplicaSetState equals this one. It returns a rich error message including the entire state for easier debugging.

func (*ReplicaSetState) Equal

func (r *ReplicaSetState) Equal(o *ReplicaSetState) bool

Equal returns true if the given ReplicaSetState is the same as this one.

func (*ReplicaSetState) SameIM

func (r *ReplicaSetState) SameIM(o *isMasterResponse) bool

SameIM checks if the given isMasterResponse is the same as the one we have.

func (*ReplicaSetState) SameRS

func (r *ReplicaSetState) SameRS(o *replSetGetStatusResponse) bool

SameRS checks if the given replSetGetStatusResponse is the same as the one we have.

type ReplicaSetStateCreator

type ReplicaSetStateCreator struct {
	Log Logger `inject:""`
}

ReplicaSetStateCreator allows for creating a ReplicaSetState from a given set of seed addresses.

func (*ReplicaSetStateCreator) FromAddrs

func (c *ReplicaSetStateCreator) FromAddrs(username, password string, addrs []string) (*ReplicaSetState, error)

FromAddrs creates a ReplicaSetState from the given set of see addresses. It requires the addresses to be part of the same Replica Set.

type ReplicaState

type ReplicaState string

ReplicaState is the state of a node in the replica.

type ReplicaStateCompare

type ReplicaStateCompare interface {
	SameRS(o *replSetGetStatusResponse) bool
	SameIM(o *isMasterResponse) bool
}

ReplicaStateCompare provides the last ReplicaSetState and allows for checking if it has changed as we rewrite/proxy the isMaster & replSetGetStatus queries.

type ReplyRW

type ReplyRW struct {
	Log Logger `inject:""`
}

ReplyRW provides common helpers for rewriting replies from the server.

func (*ReplyRW) ReadOne

func (r *ReplyRW) ReadOne(server io.Reader, v interface{}) (*messageHeader, replyPrefix, int32, error)

ReadOne reads a 1 document response, from the server, unmarshals it into v and returns the various parts.

func (*ReplyRW) WriteOne

func (r *ReplyRW) WriteOne(client io.Writer, h *messageHeader, prefix replyPrefix, oldDocLen int32, v interface{}) error

WriteOne writes a rewritten response to the client.

Directories

Path Synopsis
Godeps
_workspace/src/github.com/davecgh/go-spew/spew
Package spew implements a deep pretty printer for Go data structures to aid in debugging.
Package spew implements a deep pretty printer for Go data structures to aid in debugging.
_workspace/src/github.com/facebookgo/ensure
Package ensure provides utilities for testing to ensure the given conditions are met and Fatal if they aren't satisified.
Package ensure provides utilities for testing to ensure the given conditions are met and Fatal if they aren't satisified.
_workspace/src/github.com/facebookgo/flag.addrs
Package addrs provides flags to define one or an array of net.Addr.
Package addrs provides flags to define one or an array of net.Addr.
_workspace/src/github.com/facebookgo/freeport
Package freeport provides an API to find a free port to bind to.
Package freeport provides an API to find a free port to bind to.
_workspace/src/github.com/facebookgo/ganglia/gmetric
Package gmetric provides a client for the ganglia gmetric API.
Package gmetric provides a client for the ganglia gmetric API.
_workspace/src/github.com/facebookgo/ganglia/gmetric/gmc
Package gmc is a light weight clone of the gmetric CLI.
Package gmc is a light weight clone of the gmetric CLI.
_workspace/src/github.com/facebookgo/gangliamr
Package gangliamr provides metrics backed by Ganglia.
Package gangliamr provides metrics backed by Ganglia.
_workspace/src/github.com/facebookgo/gangliamr/gmrtest
Package gmrtest provides a sample application to experiment with gangliamr.
Package gmrtest provides a sample application to experiment with gangliamr.
_workspace/src/github.com/facebookgo/inject
Package inject provides a reflect based injector.
Package inject provides a reflect based injector.
_workspace/src/github.com/facebookgo/metrics
Package metrics provides various measuring instruments.
Package metrics provides various measuring instruments.
_workspace/src/github.com/facebookgo/mgotest
Package mgotest provides standalone test instances of mongo sutable for use in tests.
Package mgotest provides standalone test instances of mongo sutable for use in tests.
_workspace/src/github.com/facebookgo/rpool
Package rpool provides a resource pool.
Package rpool provides a resource pool.
_workspace/src/github.com/facebookgo/stack
Package stack provides utilities to capture and pass around stack traces.
Package stack provides utilities to capture and pass around stack traces.
_workspace/src/github.com/facebookgo/stackerr
Package stackerr provides a way to augment errors with one or more stack traces to allow for easier debugging.
Package stackerr provides a way to augment errors with one or more stack traces to allow for easier debugging.
_workspace/src/github.com/facebookgo/startstop
Package startstop provides automatic Start/Stop for inject eliminating the necessity for manual ordering.
Package startstop provides automatic Start/Stop for inject eliminating the necessity for manual ordering.
_workspace/src/github.com/facebookgo/stats
Package stats defines a lightweight interface for collecting statistics.
Package stats defines a lightweight interface for collecting statistics.
_workspace/src/github.com/facebookgo/structtag
Package structtag provides parsing of the defacto struct tag style.
Package structtag provides parsing of the defacto struct tag style.
_workspace/src/github.com/facebookgo/subset
Package subset implements logic to check if a value is a subset of another using reflect.
Package subset implements logic to check if a value is a subset of another using reflect.
_workspace/src/github.com/facebookgo/testname
Package testname tries to figure out the current test name from the stack
Package testname tries to figure out the current test name from the stack
_workspace/src/github.com/facebookgo/waitout
Package waitout makes it easy to wait for some output.
Package waitout makes it easy to wait for some output.
_workspace/src/gopkg.in/mgo.v2
Package mgo offers a rich MongoDB driver for Go.
Package mgo offers a rich MongoDB driver for Go.
_workspace/src/gopkg.in/mgo.v2/bson
Package bson is an implementation of the BSON specification for Go: http://bsonspec.org It was created as part of the mgo MongoDB driver for Go, but is standalone and may be used on its own without the driver.
Package bson is an implementation of the BSON specification for Go: http://bsonspec.org It was created as part of the mgo MongoDB driver for Go, but is standalone and may be used on its own without the driver.
_workspace/src/gopkg.in/mgo.v2/internal/scram
Pacakage scram implements a SCRAM-{SHA-1,etc} client per RFC5802.
Pacakage scram implements a SCRAM-{SHA-1,etc} client per RFC5802.
_workspace/src/gopkg.in/mgo.v2/txn
The txn package implements support for multi-document transactions.
The txn package implements support for multi-document transactions.
cmd

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL