sequencer

package
v0.0.0-...-4dcfcdd Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 2, 2024 License: Apache-2.0 Imports: 8 Imported by: 0

Documentation

Overview

Package sequencer contains a number of sub-packages that implement various strategies for applying staged mutations.

Index

Constants

View Source
const (
	DefaultFlushPeriod     = 1 * time.Second
	DefaultFlushSize       = 1_000
	DefaultParallelism     = 16
	DefaultQuiescentPeriod = 10 * time.Second
	DefaultRetireOffset    = 24 * time.Hour
	DefaultScanSize        = 10_000
	DefaultTimestampLimit  = 1_000
)

Defaults for flag bindings.

Variables

This section is empty.

Functions

func CommonProgress

func CommonProgress(s Stat) hlc.Range

CommonProgress returns the minimum progress across all tables within the [Stat.Group]. If no progress has been made for one or more tables in the group, hlc.RangeEmpty will be returned.

func IsMarking

func IsMarking(acc types.MultiAcceptor) bool

IsMarking returns true if the acceptor implements MarkingAcceptor or MarkingAcceptor is implemented by an acceptor somewhere in a delegate chain.

Types

type Config

type Config struct {
	Chaos           float32       // Set by tests to inject errors.
	FlushPeriod     time.Duration // Don't queue mutations for longer than this.
	FlushSize       int           // Ideal target database transaction size
	Parallelism     int           // The number of concurrent connections to use.
	QuiescentPeriod time.Duration // How often to sweep for queued mutations.
	RetireOffset    time.Duration // Delay removal of applied mutations.
	ScanSize        int           // Limit on staging-table read queries.
	TimestampLimit  int           // The maximum number of timestamps to operate on.
}

Config is an injection point common to sequencer implementations. Not all sequencers necessarily respond to all configuration options.

func (*Config) Bind

func (c *Config) Bind(flags *pflag.FlagSet)

Bind adds configuration flags to the set.

func (*Config) Preflight

func (c *Config) Preflight() error

Preflight ensure that the configuration has sane defaults.

type MarkingAcceptor

type MarkingAcceptor interface {
	// IsMarking should return true.
	IsMarking() bool
}

MarkingAcceptor is a marker interface to indicate that a types.MultiAcceptor will assume responsibility for calling types.Stager.MarkApplied.

type Sequencer

type Sequencer interface {
	// Start any goroutines necessary for operating within a
	// [types.TableGroup] and return an acceptor for admitting mutations
	// into that group. The Sequencer will monitor the
	// [StartOptions.Bounds] for updates and periodically emit a Stat
	// which reflects the Sequencer's current state.  It is reasonable
	// to extend the maximum value of the bounds over time, but the
	// minimum value should not be advanced until all tables in the Stat
	// have advanced to at least some common minimum value. When
	// switching between different Sequencers at runtime, callers should
	// discontinue updates to the bounds and ideally wait for the Stat
	// variable to catch up.
	Start(ctx *stopper.Context, opts *StartOptions) (types.MultiAcceptor, *notify.Var[Stat], error)
}

A Sequencer implements a lifecycle strategy for mutations. The various strategies may include immediate, best-effort, or fully-transactional behaviors.

The Sequencer type should also allow for composition of behaviors, e.g.: userscript dispatch.

type Shim

type Shim interface {
	// Wrap will modify the delegate's behavior. The shim should call
	// [StartOptions.Copy] if the options passed to [Sequencer.Start]
	// are to be modified before passing them to the delegate.
	Wrap(ctx *stopper.Context, delegate Sequencer) (Sequencer, error)
}

A Shim allows the behaviors of another Sequencer to be modified.

type StartOptions

type StartOptions struct {
	Bounds      *notify.Var[hlc.Range] // Control the range of eligible timestamps.
	Delegate    types.MultiAcceptor    // The acceptor to use when continuing to process mutations.
	Group       *types.TableGroup      // The tables that should be operated on.
	MaxDeferred int                    // Back off after deferring this many mutations.
}

StartOptions is passed to [Sequencer.Start].

func (*StartOptions) Copy

func (o *StartOptions) Copy() *StartOptions

Copy returns a deep copy of the options.

type Stat

type Stat interface {
	Copy() Stat                           // Copy returns a deep copy of the Stat.
	Group() *types.TableGroup             // The TableGroup that was passed to [Sequencer.Start].
	Progress() *ident.TableMap[hlc.Range] // The times to which the members of the group have advanced.
}

Stat is an interface to allow for more specialized types to aid in testing.

func NewStat

func NewStat(group *types.TableGroup, progress *ident.TableMap[hlc.Range]) Stat

NewStat constructs a basic Stat.

Directories

Path Synopsis
Package besteffort contains a best-effort implementation of [types.MultiAcceptor].
Package besteffort contains a best-effort implementation of [types.MultiAcceptor].
Package chaos allows errors to be introduced in a sequencer stack.
Package chaos allows errors to be introduced in a sequencer stack.
Package core contains a sequencer implementation that preserves source transactions and relative timestamps.
Package core contains a sequencer implementation that preserves source transactions and relative timestamps.
Package immediate contains a trivial [sequencer.Sequencer] implementation which writes data directly to the configured acceptor.
Package immediate contains a trivial [sequencer.Sequencer] implementation which writes data directly to the configured acceptor.
Package retire contains a utility for removing old mutations.
Package retire contains a utility for removing old mutations.
Package scheduler contains Sequencer-specific utilities for ensuring ordered access to rows.
Package scheduler contains Sequencer-specific utilities for ensuring ordered access to rows.
Package script provides an API shim that integrates the userscript into the [sequencer.Sequencer] API.
Package script provides an API shim that integrates the userscript into the [sequencer.Sequencer] API.
Package seqtest provides a test fixture for instantiating sequencers and other general-purpose test helpers.
Package seqtest provides a test fixture for instantiating sequencers and other general-purpose test helpers.
Package sequtil contains sequencer utility methods.
Package sequtil contains sequencer utility methods.
Package switcher contains a meta-Sequencer that switches between various modes of operation.
Package switcher contains a meta-Sequencer that switches between various modes of operation.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL