cockroach: github.com/cockroachdb/cockroach/pkg/storage/storagebase Index | Files

package storagebase

import "github.com/cockroachdb/cockroach/pkg/storage/storagebase"

Index

Package Files

base.go bulk_adder.go knobs.go

Constants

const TxnCleanupThreshold = time.Hour

TxnCleanupThreshold is the threshold after which a transaction is considered abandoned and fit for removal, as measured by the maximum of its last heartbeat and timestamp. Abort spans for the transaction are cleaned up at the same time.

TODO(tschottdorf): need to enforce at all times that this is much larger than the heartbeat interval used by the coordinator.

Variables

var MergeQueueEnabled = settings.RegisterBoolSetting(
    "kv.range_merge.queue_enabled",
    "whether the automatic merge queue is enabled",
    true,
)

MergeQueueEnabled is a setting that controls whether the merge queue is enabled.

func ContainsKey Uses

func ContainsKey(desc roachpb.RangeDescriptor, key roachpb.Key) bool

ContainsKey returns whether this range contains the specified key.

func ContainsKeyRange Uses

func ContainsKeyRange(desc roachpb.RangeDescriptor, start, end roachpb.Key) bool

ContainsKeyRange returns whether this range contains the specified key range from start to end.

func IntersectSpan Uses

func IntersectSpan(
    span roachpb.Span, desc roachpb.RangeDescriptor,
) (middle *roachpb.Span, outside []roachpb.Span)

IntersectSpan takes an intent and a descriptor. It then splits the intent's range into up to three pieces: A first piece which is contained in the Range, and a slice of up to two further intents which are outside of the key range. An intent for which [Key, EndKey) is empty does not result in any intents; thus intersectIntent only applies to intent ranges. A range-local intent range is never split: It's returned as either belonging to or outside of the descriptor's key range, and passing an intent which begins range-local but ends non-local results in a panic. TODO(tschottdorf): move to proto, make more gen-purpose - kv.truncate does some similar things.

type ApplyFilterArgs Uses

type ApplyFilterArgs struct {
    storagepb.ReplicatedEvalResult
    CmdID   CmdIDKey
    RangeID roachpb.RangeID
    StoreID roachpb.StoreID
}

ApplyFilterArgs groups the arguments to a ReplicaApplyFilter.

type BatchEvalTestingKnobs Uses

type BatchEvalTestingKnobs struct {
    // TestingEvalFilter is called before evaluating each command. The
    // number of times this callback is run depends on the propEvalKV
    // setting, and it is therefore deprecated in favor of either
    // TestingProposalFilter (which runs only on the lease holder) or
    // TestingApplyFilter (which runs on each replica). If your filter is
    // not idempotent, consider wrapping it in a
    // ReplayProtectionFilterWrapper.
    // TODO(bdarnell,tschottdorf): Migrate existing tests which use this
    // to one of the other filters. See #10493
    // TODO(andrei): Provide guidance on what to use instead for trapping reads.
    TestingEvalFilter ReplicaCommandFilter
    // NumKeysEvaluatedForRangeIntentResolution is set by the stores to the
    // number of keys evaluated for range intent resolution.
    NumKeysEvaluatedForRangeIntentResolution *int64
    // RecoverIndeterminateCommitsOnFailedPushes will propagate indeterminate
    // commit errors to trigger transaction recovery even if the push that
    // discovered the indeterminate commit was going to fail. This increases
    // the chance that conflicting transactions will prevent parallel commit
    // attempts from succeeding.
    RecoverIndeterminateCommitsOnFailedPushes bool
}

BatchEvalTestingKnobs contains testing helpers that are used during batch evaluation.

type BulkAdder Uses

type BulkAdder interface {
    // Add adds a KV pair to the adder's buffer, potentially flushing if needed.
    Add(ctx context.Context, key roachpb.Key, value []byte) error
    // Flush explicitly flushes anything remaining in the adder's buffer.
    Flush(ctx context.Context) error
    // IsEmpty returns whether or not this BulkAdder has data buffered.
    IsEmpty() bool
    // CurrentBufferFill returns how full the configured buffer is.
    CurrentBufferFill() float32
    // GetSummary returns a summary of rows/bytes/etc written by this batcher.
    GetSummary() roachpb.BulkOpSummary
    // Close closes the underlying buffers/writers.
    Close(ctx context.Context)
    // SetOnFlush sets a callback function called after flushing the buffer.
    SetOnFlush(func())
}

BulkAdder describes a bulk-adding helper that can be used to add lots of KVs.

type BulkAdderFactory Uses

type BulkAdderFactory func(
    ctx context.Context, db *client.DB, timestamp hlc.Timestamp, opts BulkAdderOptions,
) (BulkAdder, error)

BulkAdderFactory describes a factory function for BulkAdders.

type BulkAdderOptions Uses

type BulkAdderOptions struct {
    // Name is used in logging messages to identify this adder or the process on
    // behalf of which it is adding data.
    Name string

    // SSTSize is the size at which an SST will be flushed and a new one started.
    // SSTs are also split during a buffer flush to avoid spanning range bounds so
    // they may be smaller than this limit.
    SSTSize func() int64

    // SplitAndScatterAfter is the number of bytes which if added without hitting
    // an existing split will cause the adder to split and scatter the next span.
    SplitAndScatterAfter func() int64

    // MinBufferSize is the initial size of the BulkAdder buffer. It indicates the
    // amount of memory we require to be able to buffer data before flushing for
    // SST creation.
    MinBufferSize int64

    // BufferSize is the maximum size we can grow the BulkAdder buffer to.
    MaxBufferSize func() int64

    // StepBufferSize is the increment in which we will attempt to grow the
    // BulkAdder buffer if the memory monitor permits.
    StepBufferSize int64

    // SkipLocalDuplicates configures handling of duplicate keys within a local
    // sorted batch. When true if the same key/value pair is added more than once
    // subsequent additions will be ignored instead of producing an error. If an
    // attempt to add the same key has a differnet value, it is always an error.
    // Once a batch is flushed – explicitly or automatically – local duplicate
    // detection does not apply.
    SkipDuplicates bool

    // DisallowShadowing controls whether shadowing of existing keys is permitted
    // when the SSTables produced by this adder are ingested.
    DisallowShadowing bool
}

BulkAdderOptions is used to configure the behavior of a BulkAdder.

type CmdIDKey Uses

type CmdIDKey string

CmdIDKey is a Raft command id.

type DuplicateKeyError Uses

type DuplicateKeyError struct {
    Key   roachpb.Key
    Value []byte
}

DuplicateKeyError represents a failed attempt to ingest the same key twice using a BulkAdder within the same batch.

func (DuplicateKeyError) Error Uses

func (d DuplicateKeyError) Error() string

type FilterArgs Uses

type FilterArgs struct {
    Ctx   context.Context
    CmdID CmdIDKey
    Index int
    Sid   roachpb.StoreID
    Req   roachpb.Request
    Hdr   roachpb.Header
}

FilterArgs groups the arguments to a ReplicaCommandFilter.

func (*FilterArgs) InRaftCmd Uses

func (f *FilterArgs) InRaftCmd() bool

InRaftCmd returns true if the filter is running in the context of a Raft command (it could be running outside of one, for example for a read).

type IntentResolverTestingKnobs Uses

type IntentResolverTestingKnobs struct {
    // DisableAsyncIntentResolution disables the async intent resolution
    // path (but leaves synchronous resolution). This can avoid some
    // edge cases in tests that start and stop servers.
    DisableAsyncIntentResolution bool

    // ForceSyncIntentResolution forces all asynchronous intent resolution to be
    // performed synchronously. It is equivalent to setting IntentResolverTaskLimit
    // to -1.
    ForceSyncIntentResolution bool

    // MaxIntentResolutionBatchSize overrides the maximum number of intent
    // resolution requests which can be sent in a single batch.
    MaxIntentResolutionBatchSize int
}

IntentResolverTestingKnobs contains testing helpers that are used during intent resolution.

type ProposalFilterArgs Uses

type ProposalFilterArgs struct {
    Ctx   context.Context
    Cmd   storagepb.RaftCommand
    CmdID CmdIDKey
    Req   roachpb.BatchRequest
}

ProposalFilterArgs groups the arguments to ReplicaProposalFilter.

type ReplicaApplyFilter Uses

type ReplicaApplyFilter func(args ApplyFilterArgs) (int, *roachpb.Error)

A ReplicaApplyFilter can be used in testing to influence the error returned from proposals after they apply. The returned int is treated as a storage.proposalReevaluationReason and will only take an effect when it is nonzero and the existing reason is zero. Similarly, the error is only applied if there's no error so far.

type ReplicaCommandFilter Uses

type ReplicaCommandFilter func(args FilterArgs) *roachpb.Error

ReplicaCommandFilter may be used in tests through the StoreTestingKnobs to intercept the handling of commands and artificially generate errors. Return nil to continue with regular processing or non-nil to terminate processing with the returned error.

type ReplicaProposalFilter Uses

type ReplicaProposalFilter func(args ProposalFilterArgs) *roachpb.Error

ReplicaProposalFilter can be used in testing to influence the error returned from proposals after a request is evaluated but before it is proposed.

type ReplicaRequestFilter Uses

type ReplicaRequestFilter func(roachpb.BatchRequest) *roachpb.Error

ReplicaRequestFilter can be used in testing to influence the error returned from a request before it is evaluated. Return nil to continue with regular processing or non-nil to terminate processing with the returned error.

type ReplicaResponseFilter Uses

type ReplicaResponseFilter func(roachpb.BatchRequest, *roachpb.BatchResponse) *roachpb.Error

ReplicaResponseFilter is used in unittests to modify the outbound response returned to a waiting client after a replica command has been processed. This filter is invoked only by the command proposer.

Package storagebase imports 10 packages (graph) and is imported by 96 packages. Updated 2019-11-19. Refresh now. Tools for package owners.