choices

package module
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 16, 2017 License: Apache-2.0 Imports: 16 Imported by: 9

README

GoDoc Go Report Card

choices

A way to choose things.

Building

In order to build you need go installed.

You will need go1.7.1 if you are deploying to kubernetes.

To build choices library and all the included binaries run the following in a terminal.

go get -u github.com/Nordstrom/choices/...

If you are only interested in the library you can run the following in a terminal.

go get -u github.com/Nordstrom/choices

Running locally

There are two main components of choices. The storage server (bolt-store or mongo-store) and the frontend experiment server (elwin). You should first start the storage server. You will need a local version of running, bolt-store will be the easiest to setup. It only requires a local file similar to sqlite.

cd bolt-store && go build
./bolt-store

You can skip to starting the experiment server if you are using the bolt-store.

If you want to use mongo refer to this guide Mongo Installation. If you are using docker you can run the following docker cmd.

docker run -d --name mongo-storage -p 27017:27017 mongo

With mongo running you can start the storage server.

# from the choice/cmd/mongo-store directory run
./mongo-store

This creates the storage service on port 8080

Now that the storage server is up you can start the experiment server (elwin).

# from the choices/cmd/elwin directory run
JSON_ADDRESS=":8181" GRPC_ADDRESS=":8282" MONGO_ADDRESS="localhost:8080" MONGO_DATABASE="elwin" MONGO_COLLECTION="staging" ./elwin

The MONGO_ADDRESS MONGO_DATABASE and MONGO_COLLECTION are an artifact of previously only supporting mongo. These may change in an upcoming version.

The experiment store loads some example data into the staging environment. You should be able to open a web browser and navigate to localhost:8181?label=test&userid=1234 and see some results. You can also run the following command from a terminal if you have curl installed.

The bolt-store does not load sample data.

curl "localhost:8181?label=test&userid=1234"

Documentation

Overview

Package choices provides a library for simple a/b and multivariate testing.

choices uses hashing to uniquely assign users to experiments and decide the values the user is assigned. This allows us to quickly assign a user to an experiment with out having to look up what they were assigned previously. Most of the ideas in this package are based off of Facebook's Planout.

In choices there are three main concepts. Namespaces, Experiments, and Params. Namespaces split traffic between the experiments they contain. Experiments are the element you are testing. Experiments are made up of one or more Params. Params are a key value pair. The value can be either a Uniform Choice value or a Weighted Choice value.

In most cases you will want to create one namespace per experiment. If you have experiments that might have interactions you can use namespaces to split the traffic between them. For example, if you are running a test on a banner and another test that takes over the whole page, you will want to split the traffic between these two tests. Another example, is if you want to run a test on a small percent of traffic. Namespaces contain a list of TeamID's. These TeamID's are used to deterime which Experiments to return to the caller.

Experiments contain the Params for a running experiment. When a caller queries Namespaces, Namespaces will hash the user into a segment, It will then check if the segment is contained in an Experiment. If the segment is contained in the experiment then the experiment will be evaluated. An experiment will in turn evaluate each of it's Params.

Params are key-value pairs. They Options for value are Uniform choice or Weighted choice. Uniform choices will be selected in a uniform fashion. Weighted choices will be selected based on the proportions supplied in the weights.

Index

Constants

View Source 
const (
	StorageEnvironmentBad = iota
	StorageEnvironmentDev
	StorageEnvironmentProd
)

constants for storage environments. So far only support a staging and production.

Variables

View Source 
var ErrBadStorageEnvironment = errors.New("bad storage environment")

ErrBadStorageEnvironment is an error for when the storage environment is not set correctly.

View Source 
var (

	// ErrGlobalSaltAlreadySet is the error returned when a SetGlobalSalt
	// has been called more than once.
	ErrGlobalSaltAlreadySet = errors.New("global salt already set")
)
View Source 
var (
	// ErrSegmentNotInExperiment occurs when a user is hashed into a
	// segment that has not been claimed by an experiment.
	ErrSegmentNotInExperiment = errors.New("Segment is not assigned to an experiment")
)
View Source 
var (
	// ErrSegmentUnavailable is thrown when you request an a segment set to
	// 0, an unavailable segment.
	ErrSegmentUnavailable = errors.New("segment unavailable")
)

Functions

func HashExperience 

func HashExperience(namespace, experiment, param, userID string) (uint64, error)

HashExperience takes the supplied arguments and returns the hashed uint64 that can be used for determining a segment.

func InSegment 

func InSegment(namespace, userID string, s segments) bool

InSegment takes a namespace name, userID and segments and returns whether the userID is in a segment that is claimed. Typical use case for this is determining whether or not a user is in a given experiment.

func SetGlobalSalt 

func SetGlobalSalt(s string) error

SetGlobalSalt this sets the global salt used for hashing users. It should only ever be called once. It returns an error if it is called more than once.

Types

type Config 

type Config struct {
	ErrChan chan error
	// contains filtered or unexported fields
}

Config is the configuration struct used in an elwin server.

func NewChoices 

func NewChoices(ctx context.Context, opts ...ConfigOpt) (*Config, error)

NewChoices sets the storage engine. It starts a ticker that will call s.Update() until the context is cancelled. To change the tick interval call SetUpdateInterval(d time.Duration). Must cancel the context before calling NewChoices again otherwise you will leak go routines.

func (*Config) IsHealthy 

func (c *Config) IsHealthy() error

IsHealthy returns an error if the time since the last successful update is longer than the max allowed time to fail. The interval for updates can be set by WithUpdateInterval. The time until failure can be set with WithMaxUpdateFailTime.

func (*Config) Namespaces 

func (ec *Config) Namespaces(userID string, selector labels.Selector) ([]ExperimentResponse, error)

Namespaces determines the assignments for the a given user's id based on the current set of namespaces and experiments. It returns a []ExperimentResponse if it is successful or an error if something went wrong.

type ConfigOpt 

type ConfigOpt func(*Config) error

ConfigOpt is a type that modifies Config. It is used when calling NewChoices to configure choices.

func WithGlobalSalt 

func WithGlobalSalt(s string) ConfigOpt

WithGlobalSalt is a configuration option for Config that sets the globalSalt to something other than the default.

func WithMaxUpdateFailTime 

func WithMaxUpdateFailTime(dur time.Duration) ConfigOpt

WithMaxUpdateFailTime changes the max duration allowed for failing updates.

func WithStorageConfig 

func WithStorageConfig(addr string, env int, updateInterval time.Duration) ConfigOpt

WithStorageConfig is where you set the address and environment you'd like to point. This is used as a ConfigOpt in NewChoices.

func WithUpdateInterval 

func WithUpdateInterval(dur time.Duration) ConfigOpt

WithUpdateInterval changes the update interval for Storage. Must call SetStorage after this or cancel context of the current Storage and call SetStorage again.

type ErrUpdateStorage 

type ErrUpdateStorage struct {
	// contains filtered or unexported fields
}

ErrUpdateStorage is an error type that is returned when storage fails to update.

func (ErrUpdateStorage) Error 

func (e ErrUpdateStorage) Error() string

Error is to implement the error interface

type Experiment 

type Experiment struct {
	Name     string
	Labels   labels.Set
	Params   []Param
	Segments segments
}

Experiment is a structure that represents a single experiment in elwin. It can contain multiple parameters. Experiments are evaluated through the call to Namespaces.

func FromExperiment 

func FromExperiment(s *storage.Experiment) Experiment

FromExperiment converts a *storage.Experiment into an Experiment

func NewExperiment 

func NewExperiment(name string) *Experiment

NewExperiment creates an experiment with the supplied name and no segments cliamed. In order for any traffic to be assigned to this experiment you will need to call Experiment.SetSegments or Experiment.SampleSegments.

func (*Experiment) MarshalJSON 

func (e *Experiment) MarshalJSON() ([]byte, error)

MarshalJSON implements the json.Marshaler interface for Experiments.

func (*Experiment) SampleSegments 

func (e *Experiment) SampleSegments(ns *Namespace, num int) *Experiment

SampleSegments takes a namespace and an amount of segments you want in your experiment and returns a random sample of the unclaimed segments from the namespace.

func (*Experiment) SetSegments 

func (e *Experiment) SetSegments(seg segments) *Experiment

SetSegments copies the segments supplied to the experiment.

func (*Experiment) ToExperiment 

func (e *Experiment) ToExperiment() *storage.Experiment

ToExperiment is a helper function that converts an Experiment into a *storage.Experiment.

type ExperimentResponse 

type ExperimentResponse struct {
	Name      string
	Namespace string
	Params    []ParamValue
}

ExperimentResponse holds the data for an evaluated expeiment.

type Namespace 

type Namespace struct {
	Name        string
	Segments    segments
	Experiments []Experiment
}

Namespace is a container for experiments. Segments in the namespace divide traffic. Units are the keys that will hash experiments.

func FromNamespace 

func FromNamespace(s *storage.Namespace) (Namespace, error)

FromNamespace converts a *storage.Namespace into a Namespace.

func NewNamespace 

func NewNamespace(name string) *Namespace

NewNamespace creates a new namespace with all segments available. It returns an error if no units are given.

func (*Namespace) MarshalJSON 

func (n *Namespace) MarshalJSON() ([]byte, error)

MarshalJSON implements the json.Marshaler interface for Namespaces.

func (*Namespace) ToNamespace 

func (n *Namespace) ToNamespace() *storage.Namespace

ToNamespace is a helper function that converts a Namespace into a *storage.Namespace.

type Param 

type Param struct {
	Name  string
	Value Value
}

Param is a struct that represents a single parameter in an experiment. Param is evaluated through the call to Namespaces.

func FromParam 

func FromParam(s *storage.Param) Param

FromParam converts a *storage.Param into a Param

func (*Param) MarshalJSON 

func (p *Param) MarshalJSON() ([]byte, error)

MarshalJSON implements the json.Marshaler interface for Params.

func (*Param) ToParam 

func (p *Param) ToParam() *storage.Param

ToParam is a helper function that converts a Param into a *storage.Param.

type ParamValue 

type ParamValue struct {
	Name  string
	Value string
}

ParamValue is a key value pair returned from an evalated experiment parameter.

type Uniform 

type Uniform struct {
	Choices []string
}

Uniform is a way to select from a list of Choices with uniform probability.

func (*Uniform) MarshalJSON 

func (u *Uniform) MarshalJSON() ([]byte, error)

MarshalJSON implements the json.Marshaler interface for Uniform choices.

func (*Uniform) Value 

func (u *Uniform) Value(i uint64) (string, error)

Value implements the Value interface for Uniform choices.

type Value 

type Value interface {
	Value(i uint64) (string, error)
}

Value is the interface Param Values must implement. They take a hash value and return the string that represents the value or an error.

type ValueType 

type ValueType int

ValueType are the different types of Values a Param can have. This is used for parsing params in storage. 

const (
	ValueTypeBad ValueType = iota
	ValueTypeUniform
	ValueTypeWeighted
)

Constants for ValueTypes

type Weighted 

type Weighted struct {
	Choices []string
	Weights []float64
}

Weighted is a way to select from a list of Choices with probability ratio supplied in Weights.

func (*Weighted) MarshalJSON 

func (w *Weighted) MarshalJSON() ([]byte, error)

MarshalJSON implements the json.Marshaler interface for Weighted choices.

func (*Weighted) Value 

func (w *Weighted) Value(i uint64) (string, error)

Value implements the Value interface for Weighted choices.

Source Files

View all Source files

Directories

Path Synopsis
cmd
bolt-store
elwin
gen
houston
json-gateway
mongo-store
storage
mongo
mongo/internal/types

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.