cockroach: Index | Files

package kvserverbase

import ""


Package Files

base.go bulk_adder.go knobs.go


const TxnCleanupThreshold = time.Hour

TxnCleanupThreshold is the threshold after which a transaction is considered abandoned and fit for removal, as measured by the maximum of its last heartbeat and timestamp. Abort spans for the transaction are cleaned up at the same time.

TODO(tschottdorf): need to enforce at all times that this is much larger than the heartbeat interval used by the coordinator.


var MergeQueueEnabled = settings.RegisterBoolSetting(
    "whether the automatic merge queue is enabled",

MergeQueueEnabled is a setting that controls whether the merge queue is enabled.

func ContainsKey Uses

func ContainsKey(desc *roachpb.RangeDescriptor, key roachpb.Key) bool

ContainsKey returns whether this range contains the specified key.

func ContainsKeyRange Uses

func ContainsKeyRange(desc *roachpb.RangeDescriptor, start, end roachpb.Key) bool

ContainsKeyRange returns whether this range contains the specified key range from start to end.

func IntersectSpan Uses

func IntersectSpan(
    span roachpb.Span, desc *roachpb.RangeDescriptor,
) (middle *roachpb.Span, outside []roachpb.Span)

IntersectSpan takes an span and a descriptor. It then splits the span into up to three pieces: A first piece which is contained in the Range, and a slice of up to two further spans which are outside of the key range. An span for which [Key, EndKey) is empty does not result in any spans; thus intersectIntent only applies to span ranges.

A range-local span range is never split: It's returned as either belonging to or outside of the descriptor's key range, and passing an span which begins range-local but ends non-local results in a panic.

TODO(tschottdorf): move to proto, make more gen-purpose - kv.truncate does some similar things.

type ApplyFilterArgs Uses

type ApplyFilterArgs struct {
    CmdID   CmdIDKey
    RangeID roachpb.RangeID
    StoreID roachpb.StoreID

ApplyFilterArgs groups the arguments to a ReplicaApplyFilter.

type BatchEvalTestingKnobs Uses

type BatchEvalTestingKnobs struct {
    // TestingEvalFilter is called before evaluating each command.
    TestingEvalFilter ReplicaCommandFilter

    // TestingPostEvalFilter is called after evaluating each command.
    TestingPostEvalFilter ReplicaCommandFilter

    // NumKeysEvaluatedForRangeIntentResolution is set by the stores to the
    // number of keys evaluated for range intent resolution.
    NumKeysEvaluatedForRangeIntentResolution *int64

    // RecoverIndeterminateCommitsOnFailedPushes will propagate indeterminate
    // commit errors to trigger transaction recovery even if the push that
    // discovered the indeterminate commit was going to fail. This increases
    // the chance that conflicting transactions will prevent parallel commit
    // attempts from succeeding.
    RecoverIndeterminateCommitsOnFailedPushes bool

BatchEvalTestingKnobs contains testing helpers that are used during batch evaluation.

type BulkAdder Uses

type BulkAdder interface {
    // Add adds a KV pair to the adder's buffer, potentially flushing if needed.
    Add(ctx context.Context, key roachpb.Key, value []byte) error
    // Flush explicitly flushes anything remaining in the adder's buffer.
    Flush(ctx context.Context) error
    // IsEmpty returns whether or not this BulkAdder has data buffered.
    IsEmpty() bool
    // CurrentBufferFill returns how full the configured buffer is.
    CurrentBufferFill() float32
    // GetSummary returns a summary of rows/bytes/etc written by this batcher.
    GetSummary() roachpb.BulkOpSummary
    // Close closes the underlying buffers/writers.
    Close(ctx context.Context)
    // SetOnFlush sets a callback function called after flushing the buffer.

BulkAdder describes a bulk-adding helper that can be used to add lots of KVs.

type BulkAdderFactory Uses

type BulkAdderFactory func(
    ctx context.Context, db *kv.DB, timestamp hlc.Timestamp, opts BulkAdderOptions,
) (BulkAdder, error)

BulkAdderFactory describes a factory function for BulkAdders.

type BulkAdderOptions Uses

type BulkAdderOptions struct {
    // Name is used in logging messages to identify this adder or the process on
    // behalf of which it is adding data.
    Name string

    // SSTSize is the size at which an SST will be flushed and a new one started.
    // SSTs are also split during a buffer flush to avoid spanning range bounds so
    // they may be smaller than this limit.
    SSTSize func() int64

    // SplitAndScatterAfter is the number of bytes which if added without hitting
    // an existing split will cause the adder to split and scatter the next span.
    SplitAndScatterAfter func() int64

    // MinBufferSize is the initial size of the BulkAdder buffer. It indicates the
    // amount of memory we require to be able to buffer data before flushing for
    // SST creation.
    MinBufferSize int64

    // BufferSize is the maximum size we can grow the BulkAdder buffer to.
    MaxBufferSize func() int64

    // StepBufferSize is the increment in which we will attempt to grow the
    // BulkAdder buffer if the memory monitor permits.
    StepBufferSize int64

    // SkipLocalDuplicates configures handling of duplicate keys within a local
    // sorted batch. When true if the same key/value pair is added more than once
    // subsequent additions will be ignored instead of producing an error. If an
    // attempt to add the same key has a differnet value, it is always an error.
    // Once a batch is flushed – explicitly or automatically – local duplicate
    // detection does not apply.
    SkipDuplicates bool

    // DisallowShadowing controls whether shadowing of existing keys is permitted
    // when the SSTables produced by this adder are ingested.
    DisallowShadowing bool

BulkAdderOptions is used to configure the behavior of a BulkAdder.

type CmdIDKey Uses

type CmdIDKey string

CmdIDKey is a Raft command id.

type DuplicateKeyError Uses

type DuplicateKeyError struct {
    Key   roachpb.Key
    Value []byte

DuplicateKeyError represents a failed attempt to ingest the same key twice using a BulkAdder within the same batch.

func (*DuplicateKeyError) Error Uses

func (d *DuplicateKeyError) Error() string

type FilterArgs Uses

type FilterArgs struct {
    Ctx   context.Context
    CmdID CmdIDKey
    Index int
    Sid   roachpb.StoreID
    Req   roachpb.Request
    Hdr   roachpb.Header
    Err   error // only used for TestingPostEvalFilter

FilterArgs groups the arguments to a ReplicaCommandFilter.

func (*FilterArgs) InRaftCmd Uses

func (f *FilterArgs) InRaftCmd() bool

InRaftCmd returns true if the filter is running in the context of a Raft command (it could be running outside of one, for example for a read).

type IntentResolverTestingKnobs Uses

type IntentResolverTestingKnobs struct {
    // DisableAsyncIntentResolution disables the async intent resolution
    // path (but leaves synchronous resolution). This can avoid some
    // edge cases in tests that start and stop servers.
    DisableAsyncIntentResolution bool

    // ForceSyncIntentResolution forces all asynchronous intent resolution to be
    // performed synchronously. It is equivalent to setting IntentResolverTaskLimit
    // to -1.
    ForceSyncIntentResolution bool

    // MaxGCBatchSize overrides the maximum number of transaction record gc
    // requests which can be sent in a single batch.
    MaxGCBatchSize int

    // MaxIntentResolutionBatchSize overrides the maximum number of intent
    // resolution requests which can be sent in a single batch.
    MaxIntentResolutionBatchSize int

IntentResolverTestingKnobs contains testing helpers that are used during intent resolution.

type ProposalFilterArgs Uses

type ProposalFilterArgs struct {
    Ctx   context.Context
    Cmd   kvserverpb.RaftCommand
    CmdID CmdIDKey
    Req   roachpb.BatchRequest

ProposalFilterArgs groups the arguments to ReplicaProposalFilter.

type ReplicaApplyFilter Uses

type ReplicaApplyFilter func(args ApplyFilterArgs) (int, *roachpb.Error)

A ReplicaApplyFilter can be used in testing to influence the error returned from proposals after they apply. The returned int is treated as a storage.proposalReevaluationReason and will only take an effect when it is nonzero and the existing reason is zero. Similarly, the error is only applied if there's no error so far.

type ReplicaCommandFilter Uses

type ReplicaCommandFilter func(args FilterArgs) *roachpb.Error

ReplicaCommandFilter may be used in tests through the StoreTestingKnobs to intercept the handling of commands and artificially generate errors. Return nil to continue with regular processing or non-nil to terminate processing with the returned error.

type ReplicaConcurrencyRetryFilter Uses

type ReplicaConcurrencyRetryFilter func(context.Context, roachpb.BatchRequest, *roachpb.Error)

ReplicaConcurrencyRetryFilter can be used to examine a concurrency retry error before it is handled and its batch is re-evaluated.

type ReplicaProposalFilter Uses

type ReplicaProposalFilter func(args ProposalFilterArgs) *roachpb.Error

ReplicaProposalFilter can be used in testing to influence the error returned from proposals after a request is evaluated but before it is proposed.

type ReplicaRangefeedFilter Uses

type ReplicaRangefeedFilter func(
    args *roachpb.RangeFeedRequest, stream roachpb.Internal_RangeFeedServer,
) *roachpb.Error

ReplicaRangefeedFilter is used in unit tests to modify the request, inject responses, or return errors from rangefeeds.

type ReplicaRequestFilter Uses

type ReplicaRequestFilter func(context.Context, roachpb.BatchRequest) *roachpb.Error

ReplicaRequestFilter can be used in testing to influence the error returned from a request before it is evaluated. Return nil to continue with regular processing or non-nil to terminate processing with the returned error.

type ReplicaResponseFilter Uses

type ReplicaResponseFilter func(context.Context, roachpb.BatchRequest, *roachpb.BatchResponse) *roachpb.Error

ReplicaResponseFilter is used in unittests to modify the outbound response returned to a waiting client after a replica command has been processed. This filter is invoked only by the command proposer.

Package kvserverbase imports 10 packages (graph) and is imported by 29 packages. Updated 2020-08-12. Refresh now. Tools for package owners.