ringpop

package module
v0.6.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 27, 2016 License: MIT Imports: 23 Imported by: 0

README

ringpop-go Build Status Coverage Status

Ringpop is a library that brings cooperation and coordination to distributed applications. It maintains a consistent hash ring on top of a membership protocol and provides request forwarding as a routing convenience. It can be used to shard your application in a way that's scalable and fault tolerant.

Getting started

To install ringpop-go:

go get github.com/uber/ringpop-go

Developing

First make certain that thrift (OSX: brew install thrift) and glide are in your path (above). Then,

make setup

to install remaining golang dependencies and install the pre-commit hook.

Finally, run the tests by doing:

make test

Documentation

Interested in where to go from here? Read the docs at ringpop.readthedocs.org

Documentation

Overview

Package ringpop is a library that maintains a consistent hash ring atop a gossip-based membership protocol. It can be used by applications to arbitrarily shard data in a scalable and fault-tolerant manner.

Index

Constants

View Source 
const StatPeriodDefault = time.Duration(5 * time.Second)

StatPeriodDefault defines the default emission period for a periodic stat.

View Source 
const StatPeriodNever = time.Duration(-1)

StatPeriodNever defines a "period" which disables a periodic stat emission.

Variables

View Source 
var (
	// ErrNotBootstrapped is returned by public methods which require the ring to
	// be bootstrapped before they can operate correctly.
	ErrNotBootstrapped = errors.New("ringpop is not bootstrapped")

	// ErrEphemeralIdentity is returned by the identity resolver if TChannel is
	// using port 0 and is not listening (and thus has not been assigned a port by
	// the OS).
	ErrEphemeralIdentity = errors.New("unable to resolve this node's identity from channel that is not yet listening")
)

Functions

func DeserializeThrift 

func DeserializeThrift(b []byte, s athrift.TStruct) error

DeserializeThrift takes a byte slice and attempts to write it into the given thrift struct using the thrift binary protocol. This is a temporary measure before frames can forwarded directly past the endpoint to the proper destinaiton.

func SerializeThrift 

func SerializeThrift(s athrift.TStruct) ([]byte, error)

SerializeThrift takes a thrift struct and returns the serialized bytes of that struct using the thrift binary protocol. This is a temporary measure before frames can forwarded directly past the endpoint to the proper destinaiton.

Types

type Arg 

type Arg struct{}

Arg is a blank arg

type IdentityResolver 

type IdentityResolver func() (string, error)

IdentityResolver is a function that returns the listen interface/port that Ringpop should identify as.

type Interface 

type Interface interface {
	Destroy()
	App() string
	WhoAmI() (string, error)
	Uptime() (time.Duration, error)
	RegisterListener(l events.EventListener)
	Bootstrap(opts *swim.BootstrapOptions) ([]string, error)
	Checksum() (uint32, error)
	Lookup(key string) (string, error)
	LookupN(key string, n int) ([]string, error)
	GetReachableMembers() ([]string, error)
	CountReachableMembers() (int, error)

	HandleOrForward(key string, request []byte, response *[]byte, service, endpoint string, format tchannel.Format, opts *forward.Options) (bool, error)
	Forward(dest string, keys []string, request []byte, service, endpoint string, format tchannel.Format, opts *forward.Options) ([]byte, error)
}

Interface specifies the public facing methods a user of ringpop is able to use.

type Option 

type Option func(*Ringpop) error

An Option is a modifier functions that configure/modify a real Ringpop object.

There are typically two types of runtime options you can provide: flags (functions that modify the object) and value options (functions the accept user-specific arguments and then return a function that modifies the object).

For more information, see: http://commandcenter.blogspot.com/2014/01/self-referential-functions-and-design.html

func Channel 

func Channel(ch shared.TChannel) Option

Channel is used to provide a TChannel instance that Ringpop should use for all communication.

Example: 

rp, err := ringpop.New("my-app", ringpop.Channel(myChannel))

Channel is a required option. The constructor will throw an error if this option is not present.

func Clock added in v0.3.0 

func Clock(c clock.Clock) Option

Clock is used to set the Clock mechanism. Testing harnesses will typically replace this with a mocked clock.

func FaultyPeriod added in v0.4.0 

func FaultyPeriod(period time.Duration) Option

FaultyPeriod configures the period Ringpop keeps a faulty node in its memberlist. Even though the node will not receive any traffic it is still present in the list in case it will come back online later. After this timeout ringpop will remove the node from its membership list permanently. If a node happens to come back after it has been removed from the membership Ringpop still allows it to join and take its old position in the hashring. To remove the node from the distributed membership it will mark it as a tombstone which can be removed from every members membership list independently.

func HashRingConfig 

func HashRingConfig(c *hashring.Configuration) Option

HashRingConfig takes a `HashRingConfiguration` struct that can be used to configure the hash ring.

Example: 

rp, err := ringpop.New("my-app",
    ringpop.Channel(myChannel),
    ringpop.HashRingConfig(&HashRingConfiguration{
        ReplicaPoints: 100,
    }),
)

See documentation on the `HashRingConfiguration` struct for more information about what options are available.

func Identity 

func Identity(hostport string) Option

Identity is used to specify a static hostport string as this Ringpop instance's identity.

Example: 

ringpop.New("my-app",
    ringpop.Channel(myChannel),
    ringpop.Identity("10.32.12.2:21130"),
)

You should make sure the identity matches the listening address of the TChannel object.

By default, you do not need to provide an identity. If you do not provide one, the identity will be resolved automatically by the default resolver.

func IdentityResolverFunc 

func IdentityResolverFunc(resolver IdentityResolver) Option

IdentityResolverFunc is used to specify a function that will called when the Ringpop instance needs to resolve its identity (typically, on bootstrap).

func LogLevels added in v0.3.0 

func LogLevels(levels map[string]logging.Level) Option

LogLevels is used to set the severity log level for all Ringpop named loggers.

func Logger 

func Logger(l log.Logger) Option

Logger is used to specify a bark-compatible logger that will be used for all Ringpop logging. If a logger is not provided, one will be created automatically.

func MembershipChecksumStatPeriod added in v0.3.0 

func MembershipChecksumStatPeriod(period time.Duration) Option

MembershipChecksumStatPeriod configures the period between emissions of the stat 'membership.checksum-periodic'. Using a value <=0 (or StatPeriodNever) will disable emission of this stat. Using a value in (0, 10ms) will return an error, as that value is unrealistically small. Normal values must therefore be >=10ms. StatPeriodDefault defines the default.

func RingChecksumStatPeriod added in v0.3.0 

func RingChecksumStatPeriod(period time.Duration) Option

RingChecksumStatPeriod configures the period between emissions of the stat 'ring.checksum-periodic'. Using a value <=0 (or StatPeriodNever) will disable emission of this stat. Using a value in (0, 10ms) will return an error, as that value is unrealistically small. Normal values must therefore be >=10ms. StatPeriodDefault defines the default.

func Statter 

func Statter(s log.StatsReporter) Option

Statter is used to specify a bark-compatible (bark.StatsReporter) stats reporter that will be used to record ringpop stats. If a statter is not provided, stats will be emitted to a null stats-reporter.

func SuspectPeriod added in v0.4.0 

func SuspectPeriod(period time.Duration) Option

SuspectPeriod configures the period it takes ringpop to declare a node faulty after ringpop has first detected the node to be unresponsive to a healthcheck. When a node is declared faulty it is removed from the consistent hashring and stops forwarding traffic to that node. All keys previously routed to that node will then be routed to the new owner of the key

func TombstonePeriod added in v0.4.0 

func TombstonePeriod(period time.Duration) Option

TombstonePeriod configures the period of the last time of the lifecycle in of a node in the membership list. This period should give the gossip protocol the time it needs to disseminate this change. If configured too short the node in question might show up again in faulty state in the distributed memberlist of Ringpop.

type Ringpop 

type Ringpop struct {
	// contains filtered or unexported fields
}

Ringpop is a consistent hashring that uses a gossip protocol to disseminate changes around the ring.

func New 

func New(app string, opts ...Option) (*Ringpop, error)

New returns a new Ringpop instance.

func (*Ringpop) App 

func (rp *Ringpop) App() string

App returns the name of the application this Ringpop instance belongs to. The application name is set in the constructor when the Ringpop instance is created.

func (*Ringpop) Bootstrap 

func (rp *Ringpop) Bootstrap(bootstrapOpts *swim.BootstrapOptions) ([]string, error)

Bootstrap starts communication for this Ringpop instance.

When Bootstrap is called, this Ringpop instance will attempt to contact other instances from the DiscoverProvider.

If no seed hosts are provided, a single-node cluster will be created.

func (*Ringpop) Checksum 

func (rp *Ringpop) Checksum() (uint32, error)

Checksum returns the current checksum of this Ringpop instance's hashring.

func (*Ringpop) CountReachableMembers 

func (rp *Ringpop) CountReachableMembers() (int, error)

CountReachableMembers returns the number of members currently in this instance's membership list that aren't faulty.

func (*Ringpop) Destroy 

func (rp *Ringpop) Destroy()

Destroy stops all communication. Note that this does not close the TChannel instance that was passed to Ringpop in the constructor. Once an instance is destroyed, it cannot be restarted.

func (*Ringpop) Forward 

func (rp *Ringpop) Forward(dest string, keys []string, request []byte, service, endpoint string,
	format tchannel.Format, opts *forward.Options) ([]byte, error)

Forward forwards the request to given destination host and returns the response.

func (*Ringpop) GetReachableMembers 

func (rp *Ringpop) GetReachableMembers() ([]string, error)

GetReachableMembers returns a slice of members currently in this instance's membership list that aren't faulty.

func (*Ringpop) HandleEvent 

func (rp *Ringpop) HandleEvent(event events.Event)

HandleEvent is used to satisfy the swim.EventListener interface. No touchy.

func (*Ringpop) HandleOrForward 

func (rp *Ringpop) HandleOrForward(key string, request []byte, response *[]byte, service, endpoint string,
	format tchannel.Format, opts *forward.Options) (bool, error)

HandleOrForward returns true if the request should be handled locally, or false if it should be forwarded to a different node. If false is returned, forwarding is taken care of internally by the method, and, if no error has occured, the response is written in the provided response field.

func (*Ringpop) Lookup 

func (rp *Ringpop) Lookup(key string) (string, error)

Lookup returns the address of the server in the ring that is responsible for the specified key. It returns an error if the Ringpop instance is not yet initialized/bootstrapped.

func (*Ringpop) LookupN 

func (rp *Ringpop) LookupN(key string, n int) ([]string, error)

LookupN returns the addresses of all the servers in the ring that are responsible for the specified key. It returns an error if the Ringpop instance is not yet initialized/bootstrapped.

func (*Ringpop) Ready 

func (rp *Ringpop) Ready() bool

Ready returns whether or not ringpop is bootstrapped and ready to receive requests.

func (*Ringpop) RegisterListener 

func (rp *Ringpop) RegisterListener(l events.EventListener)

RegisterListener adds a listener to the ringpop. The listener's HandleEvent method should be thread safe.

func (*Ringpop) Uptime 

func (rp *Ringpop) Uptime() (time.Duration, error)

Uptime returns the amount of time that this Ringpop instance has been bootstrapped for.

func (*Ringpop) WhoAmI 

func (rp *Ringpop) WhoAmI() (string, error)

WhoAmI returns the address of the current/local Ringpop node. It returns an error if Ringpop is not yet initialized/bootstrapped.

Source Files

View all Source files

Directories

Path Synopsis
jsonfile
statichosts
test/mocks
examples
ping-json
ping-thrift
ping-thrift-gen
ping-thrift-gen/gen-go/ping
Package ping is generated code used to make or handle TChannel calls using Thrift.
 Package ping is generated code used to make or handle TChannel calls using Thrift.
ping-thrift/gen-go/ping
Package ping is generated code used to make or handle TChannel calls using Thrift.
 Package ping is generated code used to make or handle TChannel calls using Thrift.
Package forward provides a mechanism to forward TChannel requests.
 Package forward provides a mechanism to forward TChannel requests.
Package hashring provides a hashring implementation that uses a red-black Tree.
 Package hashring provides a hashring implementation that uses a red-black Tree.
Package replica extends Ringpop functionality by providing a mechanism to replicate a request to multiple nodes in the ring.
 Package replica extends Ringpop functionality by providing a mechanism to replicate a request to multiple nodes in the ring.
scripts
testpop
test/mocks
test
mocks
thrift/pingpong
Package pingpong is generated code used to make or handle TChannel calls using Thrift.
 Package pingpong is generated code used to make or handle TChannel calls using Thrift.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.