cockroach: github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/cdctest Index | Files

package cdctest

import "github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/cdctest"

Index

Package Files

nemeses.go testfeed.go validator.go

func ParseJSONValueTimestamps Uses

func ParseJSONValueTimestamps(v []byte) (updated, resolved hlc.Timestamp, err error)

ParseJSONValueTimestamps returns the updated or resolved timestamp set in the provided `format=json` value. Exported for acceptance testing.

func ReformatJSON Uses

func ReformatJSON(j interface{}) ([]byte, error)

ReformatJSON marshals a golang stdlib based JSON into a byte slice preserving whitespace in accordance with the crdb json library.

type CountValidator Uses

type CountValidator struct {
    NumRows, NumResolved                 int
    NumResolvedRows, NumResolvedWithRows int
    // contains filtered or unexported fields
}

CountValidator wraps a Validator and keeps count of how many rows and resolved timestamps have been seen.

func MakeCountValidator Uses

func MakeCountValidator(v Validator) *CountValidator

MakeCountValidator returns a CountValidator wrapping the given Validator.

func (*CountValidator) Failures Uses

func (v *CountValidator) Failures() []string

Failures implements the Validator interface.

func (*CountValidator) NoteResolved Uses

func (v *CountValidator) NoteResolved(partition string, resolved hlc.Timestamp) error

NoteResolved implements the Validator interface.

func (*CountValidator) NoteRow Uses

func (v *CountValidator) NoteRow(partition string, key, value string, updated hlc.Timestamp) error

NoteRow implements the Validator interface.

type TableFeed Uses

type TableFeed struct {
    // contains filtered or unexported fields
}

TableFeed is a TestFeed implementation using the `experimental-sql` sink.

func (*TableFeed) Close Uses

func (c *TableFeed) Close() error

Close implements the TestFeed interface.

func (*TableFeed) Details Uses

func (f *TableFeed) Details() (*jobspb.ChangefeedDetails, error)

func (*TableFeed) Next Uses

func (c *TableFeed) Next() (*TestFeedMessage, error)

Next implements the TestFeed interface.

func (*TableFeed) Partitions Uses

func (c *TableFeed) Partitions() []string

Partitions implements the TestFeed interface.

func (*TableFeed) Pause Uses

func (f *TableFeed) Pause() error

func (*TableFeed) Resume Uses

func (f *TableFeed) Resume() error

type TestFeed Uses

type TestFeed interface {
    // Partitions returns the domain of values that may be returned as a partition
    // by Next.
    Partitions() []string
    // Next returns the next message. Within a given topic+partition, the order is
    // preserved, but not otherwise. Either len(key) and len(value) will be
    // greater than zero (a row updated) or len(payload) will be (a resolved
    // timestamp).
    Next() (*TestFeedMessage, error)
    // Pause stops the feed from running. Next will continue to return any results
    // that were queued before the pause, eventually blocking or erroring once
    // they've all been drained.
    Pause() error
    // Resume restarts the feed from the last changefeed-wide resolved timestamp.
    Resume() error
    // Close shuts down the changefeed and releases resources.
    Close() error
}

TestFeed abstracts over reading from the various types of changefeed sinks.

type TestFeedFactory Uses

type TestFeedFactory interface {
    // Feed creates a new TestFeed.
    Feed(create string, args ...interface{}) (TestFeed, error)
    // Server returns the raw underlying TestServer, if applicable.
    Server() serverutils.TestServerInterface
}

TestFeedFactory is an interface to create changefeeds.

func MakeCloudFeedFactory Uses

func MakeCloudFeedFactory(
    s serverutils.TestServerInterface, db *gosql.DB, dir string, flushCh chan struct{},
) TestFeedFactory

MakeCloudFeedFactory returns a TestFeedFactory implementation using the cloud storage sink.

func MakeSinklessFeedFactory Uses

func MakeSinklessFeedFactory(s serverutils.TestServerInterface, sink url.URL) TestFeedFactory

MakeSinklessFeedFactory returns a TestFeedFactory implementation using the `experimental-sql` sink.

func MakeTableFeedFactory Uses

func MakeTableFeedFactory(
    s serverutils.TestServerInterface, db *gosql.DB, flushCh chan struct{}, sink url.URL,
) TestFeedFactory

MakeTableFeedFactory returns a TestFeedFactory implementation using the `experimental-sql` sink.

type TestFeedMessage Uses

type TestFeedMessage struct {
    Topic, Partition string
    Key, Value       []byte
    Resolved         []byte
}

TestFeedMessage represents one row update or resolved timestamp message from a changefeed.

func (TestFeedMessage) String Uses

func (m TestFeedMessage) String() string

type Validator Uses

type Validator interface {
    // NoteRow accepts a changed row entry.
    NoteRow(partition string, key, value string, updated hlc.Timestamp) error
    // NoteResolved accepts a resolved timestamp entry.
    NoteResolved(partition string, resolved hlc.Timestamp) error
    // Failures returns any violations seen so far.
    Failures() []string
}

Validator checks for violations of our changefeed ordering and delivery guarantees in a single table.

func NewBeforeAfterValidator Uses

func NewBeforeAfterValidator(sqlDB *gosql.DB, table string) (Validator, error)

NewBeforeAfterValidator returns a Validator verifies that the "before" and "after" fields in each row agree with the source table when performing AS OF SYSTEM TIME lookups before and at the row's timestamp.

func NewFingerprintValidator Uses

func NewFingerprintValidator(
    sqlDB *gosql.DB, origTable, fprintTable string, partitions []string, maxTestColumnCount int,
) (Validator, error)

NewFingerprintValidator returns a new FingerprintValidator that uses `fprintTable` as scratch space to recreate `origTable`. `fprintTable` must exist before calling this constructor. `maxTestColumnCount` indicates the maximum number of columns that can be expected in `origTable` due to test-related schema changes. This fingerprint validator will modify `fprint`'s schema to add `maxTestColumnCount` columns to avoid having to accommodate schema changes on the fly.

func NewOrderValidator Uses

func NewOrderValidator(topic string) Validator

NewOrderValidator returns a Validator that checks the row and resolved timestamp ordering guarantees. It also asserts that keys have an affinity to a single partition.

Once a row with has been emitted with some timestamp, no previously unseen versions of that row will be emitted with a lower timestamp.

Once a resolved timestamp has been emitted, no previously unseen rows with a lower update timestamp will be emitted on that partition.

func RunNemesis Uses

func RunNemesis(f TestFeedFactory, db *gosql.DB, isSinkless bool) (Validator, error)

RunNemesis runs a jepsen-style validation of whether a changefeed meets our user-facing guarantees. It's driven by a state machine with various nemeses: txn begin/commit/rollback, job pause/unpause.

Changefeeds have a set of user-facing guarantees about ordering and duplicates, which the two cdctest.Validator implementations verify for the real output of a changefeed. The output rows and resolved timestamps of the tested feed are fed into them to check for anomalies.

type Validators Uses

type Validators []Validator

Validators abstracts over running multiple `Validator`s at once on the same feed.

func (Validators) Failures Uses

func (vs Validators) Failures() []string

Failures implements the Validator interface.

func (Validators) NoteResolved Uses

func (vs Validators) NoteResolved(partition string, resolved hlc.Timestamp) error

NoteResolved implements the Validator interface.

func (Validators) NoteRow Uses

func (vs Validators) NoteRow(partition string, key, value string, updated hlc.Timestamp) error

NoteRow implements the Validator interface.

Package cdctest imports 33 packages (graph) and is imported by 1 packages. Updated 2020-02-16. Refresh now. Tools for package owners.