cockroach: github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl Index | Files | Directories

package changefeedccl

import "github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl"

Index

Package Files

avro.go buffer.go changefeed.go changefeed_dist.go changefeed_processors.go changefeed_stmt.go encoder.go errors.go metrics.go name.go poller.go rowfetcher_cache.go sink.go sink_cloudstorage.go span_frontier.go table_history.go testing_knobs.go

func AvroNameToSQLName Uses

func AvroNameToSQLName(s string) string

AvroNameToSQLName is the inverse of SQLNameToAvroName.

func IsRetryableError Uses

func IsRetryableError(err error) bool

IsRetryableError returns true if the supplied error, or any of its parent causes, is a IsRetryableError.

func KafkaNameToSQLName Uses

func KafkaNameToSQLName(s string) string

KafkaNameToSQLName is the inverse of SQLNameToKafkaName except when SQLNameToKafkaName had to truncate.

func MakeMetrics Uses

func MakeMetrics(histogramWindow time.Duration) metric.Struct

MakeMetrics makes the metrics for changefeed monitoring.

func MarkRetryableError Uses

func MarkRetryableError(e error) error

MarkRetryableError wraps the given error, marking it as retryable to changefeeds.

func MaybeStripRetryableErrorMarker Uses

func MaybeStripRetryableErrorMarker(err error) error

MaybeStripRetryableErrorMarker performs some minimal attempt to clean the RetryableError marker out. This won't do anything if the RetryableError itself has been wrapped, but that's okay, we'll just have an uglier string.

func SQLNameToAvroName Uses

func SQLNameToAvroName(s string) string

SQLNameToAvroName escapes a sql table name into a valid avro record or field name. This is reversible by AvroNameToSQLName.

Avro allows names matching `[a-zA-Z_][a-zA-Z0-9_]*`.

Runes are escaped with _u<hex>_ in an attempt to look like U+0021. For example `!` escapes to `_u0021_`.

func SQLNameToKafkaName Uses

func SQLNameToKafkaName(s string) string

SQLNameToKafkaName escapes a sql table name into a valid kafka topic name. This is reversible by KafkaNameToSQLName except when the escaped string is longer than kafka's length limit.

Kafka allows names matching `[a-zA-Z0-9\._\-]{1,249}` excepting `.` and `..`.

Runes are escaped with _u<hex>_ in an attempt to look like U+0021. For example `!` escapes to `_u0021_`.

type Encoder Uses

type Encoder interface {
    // EncodeKey encodes the primary key of the given row. The columns of the
    // datums are expected to match 1:1 with the `Columns` field of the
    // `TableDescriptor`, but only the primary key fields will be used. The
    // returned bytes are only valid until the next call to Encode*.
    EncodeKey(encodeRow) ([]byte, error)
    // EncodeValue encodes the primary key of the given row. The columns of the
    // datums are expected to match 1:1 with the `Columns` field of the
    // `TableDescriptor`. The returned bytes are only valid until the next call
    // to Encode*.
    EncodeValue(encodeRow) ([]byte, error)
    // EncodeResolvedTimestamp encodes a resolved timestamp payload for the
    // given topic name. The returned bytes are only valid until the next call
    // to Encode*.
    EncodeResolvedTimestamp(string, hlc.Timestamp) ([]byte, error)
}

Encoder turns a row into a serialized changefeed key, value, or resolved timestamp. It represents one of the `format=` changefeed options.

type Metrics Uses

type Metrics struct {
    EmittedMessages  *metric.Counter
    EmittedBytes     *metric.Counter
    Flushes          *metric.Counter
    ErrorRetries     *metric.Counter
    BufferEntriesIn  *metric.Counter
    BufferEntriesOut *metric.Counter

    PollRequestNanosHist *metric.Histogram
    ProcessingNanos      *metric.Counter
    TableMetadataNanos   *metric.Counter
    EmitNanos            *metric.Counter
    FlushNanos           *metric.Counter

    MaxBehindNanos *metric.Gauge
    // contains filtered or unexported fields
}

Metrics are for production monitoring of changefeeds.

func (*Metrics) MetricStruct Uses

func (*Metrics) MetricStruct()

MetricStruct implements the metric.Struct interface.

type Sink Uses

type Sink interface {
    // EmitRow enqueues a row message for asynchronous delivery on the sink. An
    // error may be returned if a previously enqueued message has failed.
    EmitRow(
        ctx context.Context,
        table *sqlbase.TableDescriptor,
        key, value []byte,
        updated hlc.Timestamp,
    ) error
    // EmitResolvedTimestamp enqueues a resolved timestamp message for
    // asynchronous delivery on every topic that has been seen by EmitRow. An
    // error may be returned if a previously enqueued message has failed.
    EmitResolvedTimestamp(ctx context.Context, encoder Encoder, resolved hlc.Timestamp) error
    // Flush blocks until every message enqueued by EmitRow and
    // EmitResolvedTimestamp has been acknowledged by the sink. If an error is
    // returned, no guarantees are given about which messages have been
    // delivered or not delivered.
    Flush(ctx context.Context) error
    // Close does not guarantee delivery of outstanding messages.
    Close() error
}

Sink is an abstraction for anything that a changefeed may emit into.

type TestingKnobs Uses

type TestingKnobs struct {
    // BeforeEmitRow is called before every sink emit row operation.
    BeforeEmitRow func(context.Context) error
    // AfterSinkFlush is called after a sink flush operation has returned without
    // error.
    AfterSinkFlush func() error
    // MemBufferCapacity, if non-zero, overrides memBufferDefaultCapacity.
    MemBufferCapacity int64
}

TestingKnobs are the testing knobs for changefeed.

func (*TestingKnobs) ModuleTestingKnobs Uses

func (*TestingKnobs) ModuleTestingKnobs()

ModuleTestingKnobs is part of the base.ModuleTestingKnobs interface.

Directories

PathSynopsis
cdctest

Package changefeedccl imports 81 packages (graph) and is imported by 2 packages. Updated 2019-09-18. Refresh now. Tools for package owners.