cockroach: github.com/cockroachdb/cockroach/pkg/sql/row Index | Files

package row

import "github.com/cockroachdb/cockroach/pkg/sql/row"

Index

Package Files

cascader.go deleter.go errors.go fetcher.go fk_existence_base.go fk_existence_batch.go fk_existence_check.go fk_existence_delete.go fk_existence_insert.go fk_existence_prepare.go fk_existence_update.go fk_spans.go fk_table_lookup.go helper.go inserter.go kv_batch_fetcher.go kv_fetcher.go row_converter.go updater.go writer.go

Constants

const (
    // UpdaterDefault indicates that an Updater should update everything
    // about a row, including secondary indexes.
    UpdaterDefault rowUpdaterType = 0
    // UpdaterOnlyColumns indicates that an Updater should only update the
    // columns of a row.
    UpdaterOnlyColumns rowUpdaterType = 1
)
const (
    // CheckFKs can be passed to row writers to check fk validity.
    CheckFKs checkFKConstraints = true
    // SkipFKs can be passed to row writer to skip fk validity checks.
    SkipFKs checkFKConstraints = false
)
const DebugRowFetch = false

DebugRowFetch can be used to turn on some low-level debugging logs. We use this to avoid using log.V in the hot path.

func ColIDtoRowIndexFromCols Uses

func ColIDtoRowIndexFromCols(cols []sqlbase.ColumnDescriptor) map[sqlbase.ColumnID]int

ColIDtoRowIndexFromCols groups a slice of ColumnDescriptors by their ID field, returning a map from ID to ColumnDescriptor. It assumes there are no duplicate descriptors in the input.

func ConvertBatchError Uses

func ConvertBatchError(
    ctx context.Context, tableDesc *sqlbase.ImmutableTableDescriptor, b *client.Batch,
) error

ConvertBatchError returns a user friendly constraint violation error.

func GenerateInsertRow Uses

func GenerateInsertRow(
    defaultExprs []tree.TypedExpr,
    computeExprs []tree.TypedExpr,
    insertCols []sqlbase.ColumnDescriptor,
    computedCols []sqlbase.ColumnDescriptor,
    evalCtx *tree.EvalContext,
    tableDesc *sqlbase.ImmutableTableDescriptor,
    rowVals tree.Datums,
    rowContainerForComputedVals *sqlbase.RowIndexedVarContainer,
) (tree.Datums, error)

GenerateInsertRow prepares a row tuple for insertion. It fills in default expressions, verifies non-nullable columns, and checks column widths.

The result is a row tuple providing values for every column in insertCols. This results contains:

- the values provided by rowVals, the tuple of source values. The

caller ensures this provides values 1-to-1 to the prefix of
insertCols that was specified explicitly in the INSERT statement.

- the default values for any additional columns in insertCols that

have default values in defaultExprs.

- the computed values for any additional columns in insertCols

that are computed. The mapping in rowContainerForComputedCols
maps the indexes of the comptuedCols/computeExpr slices
back into indexes in the result row tuple.

func NewUniquenessConstraintViolationError Uses

func NewUniquenessConstraintViolationError(
    ctx context.Context,
    tableDesc *sqlbase.ImmutableTableDescriptor,
    key roachpb.Key,
    value *roachpb.Value,
) error

NewUniquenessConstraintViolationError creates an error that represents a violation of a UNIQUE constraint.

func NoCheckPrivilege Uses

func NoCheckPrivilege(_ context.Context, _ sqlbase.DescriptorProto, _ privilege.Kind) error

NoCheckPrivilege is a stub that can be used to not actually verify privileges. This can be used when the FK work is initialized from a pre-populated FkTableMetadata map.

func SetKVBatchSize Uses

func SetKVBatchSize(val int64) func()

SetKVBatchSize changes the kvBatchFetcher batch size, and returns a function that restores it.

type CheckPrivilegeFunction Uses

type CheckPrivilegeFunction func(context.Context, sqlbase.DescriptorProto, privilege.Kind) error

CheckPrivilegeFunction is the function type used by MakeFkMetadata that will check the privileges of the current user to access specific tables.

type DatumRowConverter Uses

type DatumRowConverter struct {
    // current row buf
    Datums []tree.Datum

    // kv destination and current batch
    KvCh     chan<- KVBatch
    KvBatch  KVBatch
    BatchCap int

    // Tracks which column indices in the set of visible columns are part of the
    // user specified target columns. This can be used before populating Datums
    // to filter out unwanted column data.
    IsTargetCol map[int]struct{}

    EvalCtx *tree.EvalContext

    VisibleCols     []sqlbase.ColumnDescriptor
    VisibleColTypes []*types.T

    // FractionFn is used to set the progress header in KVBatches.
    CompletedRowFn func() uint64
    FractionFn     func() float32
    // contains filtered or unexported fields
}

DatumRowConverter converts Datums into kvs and streams it to the destination channel.

func NewDatumRowConverter Uses

func NewDatumRowConverter(
    tableDesc *sqlbase.TableDescriptor,
    targetColNames tree.NameList,
    evalCtx *tree.EvalContext,
    kvCh chan<- KVBatch,
) (*DatumRowConverter, error)

NewDatumRowConverter returns an instance of a DatumRowConverter.

func (*DatumRowConverter) Row Uses

func (c *DatumRowConverter) Row(ctx context.Context, sourceID int32, rowIndex int64) error

Row inserts kv operations into the current kv batch, and triggers a SendBatch if necessary.

func (*DatumRowConverter) SendBatch Uses

func (c *DatumRowConverter) SendBatch(ctx context.Context) error

SendBatch streams kv operations from the current KvBatch to the destination channel, and resets the KvBatch to empty.

type Deleter Uses

type Deleter struct {
    Helper               rowHelper
    FetchCols            []sqlbase.ColumnDescriptor
    FetchColIDtoRowIndex map[sqlbase.ColumnID]int
    Fks                  fkExistenceCheckForDelete
    // contains filtered or unexported fields
}

Deleter abstracts the key/value operations for deleting table rows.

func MakeDeleter Uses

func MakeDeleter(
    txn *client.Txn,
    tableDesc *sqlbase.ImmutableTableDescriptor,
    fkTables FkTableMetadata,
    requestedCols []sqlbase.ColumnDescriptor,
    checkFKs checkFKConstraints,
    evalCtx *tree.EvalContext,
    alloc *sqlbase.DatumAlloc,
) (Deleter, error)

MakeDeleter creates a Deleter for the given table.

The returned Deleter contains a FetchCols field that defines the expectation of which values are passed as values to DeleteRow. Any column passed in requestedCols will be included in FetchCols.

func (*Deleter) DeleteIndexRow Uses

func (rd *Deleter) DeleteIndexRow(
    ctx context.Context,
    b *client.Batch,
    idx *sqlbase.IndexDescriptor,
    values []tree.Datum,
    traceKV bool,
) error

DeleteIndexRow adds to the batch the kv operations necessary to delete a table row from the given index.

func (*Deleter) DeleteRow Uses

func (rd *Deleter) DeleteRow(
    ctx context.Context,
    b *client.Batch,
    values []tree.Datum,
    checkFKs checkFKConstraints,
    traceKV bool,
) error

DeleteRow adds to the batch the kv operations necessary to delete a table row with the given values. It also will cascade as required and check for orphaned rows. The bytesMonitor is only used if cascading/fk checking and can be nil if not.

type FKCheckType Uses

type FKCheckType int

FKCheckType indicates the type of mutation that triggers FK work (delete, insert, or both).

const (
    // CheckDeletes checks if rows reference a changed value.
    CheckDeletes FKCheckType = iota
    // CheckInserts checks if a new value references an existing row.
    CheckInserts
    // CheckUpdates checks all references (CheckDeletes+CheckInserts).
    CheckUpdates
)

type Fetcher Uses

type Fetcher struct {
    // contains filtered or unexported fields
}

Fetcher handles fetching kvs and forming table rows for an arbitrary number of tables. Usage:

var rf Fetcher
err := rf.Init(..)
// Handle err
err := rf.StartScan(..)
// Handle err
for {
   res, err := rf.NextRow()
   // Handle err
   if res.row == nil {
      // Done
      break
   }
   // Process res.row
}

func (*Fetcher) GetBytesRead Uses

func (rf *Fetcher) GetBytesRead() int64

GetBytesRead returns total number of bytes read by the underlying KVFetcher.

func (*Fetcher) GetRangesInfo Uses

func (rf *Fetcher) GetRangesInfo() []roachpb.RangeInfo

GetRangesInfo returns information about the ranges where the rows came from. The RangeInfo's are deduped and not ordered.

func (*Fetcher) Init Uses

func (rf *Fetcher) Init(
    reverse, returnRangeInfo bool,
    isCheck bool,
    alloc *sqlbase.DatumAlloc,
    tables ...FetcherTableArgs,
) error

Init sets up a Fetcher for a given table and index. If we are using a non-primary index, tables.ValNeededForCol can only refer to columns in the index.

func (*Fetcher) Key Uses

func (rf *Fetcher) Key() roachpb.Key

Key returns the next key (the key that follows the last returned row). Key returns nil when there are no more rows.

func (*Fetcher) NextKey Uses

func (rf *Fetcher) NextKey(ctx context.Context) (rowDone bool, err error)

NextKey retrieves the next key/value and sets kv/kvEnd. Returns whether a row has been completed.

func (*Fetcher) NextRow Uses

func (rf *Fetcher) NextRow(
    ctx context.Context,
) (
    row sqlbase.EncDatumRow,
    table *sqlbase.TableDescriptor,
    index *sqlbase.IndexDescriptor,
    err error,
)

NextRow processes keys until we complete one row, which is returned as an EncDatumRow. The row contains one value per table column, regardless of the index used; values that are not needed (as per neededCols) are nil. The EncDatumRow should not be modified and is only valid until the next call. When there are no more rows, the EncDatumRow is nil. The error returned may be a scrub.ScrubError, which the caller is responsible for unwrapping. It also returns the table and index descriptor associated with the row (relevant when more than one table is specified during initialization).

func (*Fetcher) NextRowDecoded Uses

func (rf *Fetcher) NextRowDecoded(
    ctx context.Context,
) (
    datums tree.Datums,
    table *sqlbase.TableDescriptor,
    index *sqlbase.IndexDescriptor,
    err error,
)

NextRowDecoded calls NextRow and decodes the EncDatumRow into a Datums. The Datums should not be modified and is only valid until the next call. When there are no more rows, the Datums is nil. It also returns the table and index descriptor associated with the row (relevant when more than one table is specified during initialization).

func (*Fetcher) NextRowWithErrors Uses

func (rf *Fetcher) NextRowWithErrors(ctx context.Context) (sqlbase.EncDatumRow, error)

NextRowWithErrors calls NextRow to fetch the next row and also run additional additional logic for physical checks. The Datums should not be modified and are only valid until the next call. When there are no more rows, the Datums is nil. The checks executed include:

- k/v data round-trips, i.e. it decodes and re-encodes to the same
  value.
- There is no extra unexpected or incorrect data encoded in the k/v
  pair.
- Decoded keys follow the same ordering as their encoding.

func (*Fetcher) PartialKey Uses

func (rf *Fetcher) PartialKey(nCols int) (roachpb.Key, error)

PartialKey returns a partial slice of the next key (the key that follows the last returned row) containing nCols columns, without the ending column family. Returns nil when there are no more rows.

func (*Fetcher) ReadIndexKey Uses

func (rf *Fetcher) ReadIndexKey(key roachpb.Key) (remaining []byte, ok bool, err error)

ReadIndexKey decodes an index key for a given table. It returns whether or not the key is for any of the tables initialized in Fetcher, and the remaining part of the key if it is.

func (*Fetcher) Reset Uses

func (rf *Fetcher) Reset()

Reset resets this Fetcher, preserving the memory capacity that was used for the tables slice, and the slices within each of the tableInfo objects within tables. This permits reuse of this objects without forcing total reallocation of all of those slice fields.

func (*Fetcher) RowIsDeleted Uses

func (rf *Fetcher) RowIsDeleted() bool

RowIsDeleted may only be called after NextRow has returned a non-nil row and returns true if that row was most recently deleted. This method is only meaningful when the configured kvBatchFetcher returns deletion tombstones, which the normal one (via `StartScan`) does not.

func (*Fetcher) RowLastModified Uses

func (rf *Fetcher) RowLastModified() hlc.Timestamp

RowLastModified may only be called after NextRow has returned a non-nil row and returns the timestamp of the last modification to that row.

func (*Fetcher) StartInconsistentScan Uses

func (rf *Fetcher) StartInconsistentScan(
    ctx context.Context,
    db *client.DB,
    initialTimestamp hlc.Timestamp,
    maxTimestampAge time.Duration,
    spans roachpb.Spans,
    limitBatches bool,
    limitHint int64,
    traceKV bool,
) error

StartInconsistentScan initializes and starts an inconsistent scan, where each KV batch can be read at a different historical timestamp.

The scan uses the initial timestamp, until it becomes older than maxTimestampAge; at this time the timestamp is bumped by the amount of time that has passed. See the documentation for TableReaderSpec for more details.

Can be used multiple times.

func (*Fetcher) StartScan Uses

func (rf *Fetcher) StartScan(
    ctx context.Context,
    txn *client.Txn,
    spans roachpb.Spans,
    limitBatches bool,
    limitHint int64,
    traceKV bool,
) error

StartScan initializes and starts the key-value scan. Can be used multiple times.

func (*Fetcher) StartScanFrom Uses

func (rf *Fetcher) StartScanFrom(ctx context.Context, f kvBatchFetcher) error

StartScanFrom initializes and starts a scan from the given kvBatchFetcher. Can be used multiple times.

type FetcherTableArgs Uses

type FetcherTableArgs struct {
    // The spans of keys to return for the given table. Fetcher
    // ignores keys outside these spans.
    // This is irrelevant if Fetcher is initialize with only one
    // table.
    Spans            roachpb.Spans
    Desc             *sqlbase.ImmutableTableDescriptor
    Index            *sqlbase.IndexDescriptor
    ColIdxMap        map[sqlbase.ColumnID]int
    IsSecondaryIndex bool
    Cols             []sqlbase.ColumnDescriptor
    // The indexes (0 to # of columns - 1) of the columns to return.
    ValNeededForCol util.FastIntSet
}

FetcherTableArgs are the arguments passed to Fetcher.Init for a given table that includes descriptors and row information.

type FkTableMetadata Uses

type FkTableMetadata map[TableID]TableEntry

FkTableMetadata maps table IDs to looked up descriptors or, for tables that exist but are not yet public/leasable, entries with just the IsAdding flag.

This is populated by the lookup queue (below) and used as input to the FK existence checkers and cascading actions.

TODO(knz): the redundancy between this struct and the code in other packages (sql, sqlbase) is troubling! Some of this should be factored.

func MakeFkMetadata Uses

func MakeFkMetadata(
    ctx context.Context,
    mutatedTable *sqlbase.ImmutableTableDescriptor,
    startUsage FKCheckType,
    tblLookupFn TableLookupFunction,
    privCheckFn CheckPrivilegeFunction,
    analyzeExprFn sqlbase.AnalyzeExprFunction,
    checkHelper *sqlbase.CheckHelper,
) (FkTableMetadata, error)

MakeFkMetadata populates a map of FkTableMetadata for all the TableDescriptors that might be needed when performing FK checking for delete and/or insert operations. It uses the passed in lookup function to perform the actual lookup. The caller is expected to create the CheckHelper for the given mutatedTable. However, if cascading is required, then the FK code will need to create additional CheckHelpers. It does this using the AnalyzeExprFunction, TableLookupFunction, and CheckPrivilegeFunction functions, so these must be provided if there's a possibility of a cascading operation.

type Inserter Uses

type Inserter struct {
    Helper                rowHelper
    InsertCols            []sqlbase.ColumnDescriptor
    InsertColIDtoRowIndex map[sqlbase.ColumnID]int
    Fks                   fkExistenceCheckForInsert
    // contains filtered or unexported fields
}

Inserter abstracts the key/value operations for inserting table rows.

func MakeInserter Uses

func MakeInserter(
    txn *client.Txn,
    tableDesc *sqlbase.ImmutableTableDescriptor,
    fkTables FkTableMetadata,
    insertCols []sqlbase.ColumnDescriptor,
    checkFKs checkFKConstraints,
    alloc *sqlbase.DatumAlloc,
) (Inserter, error)

MakeInserter creates a Inserter for the given table.

insertCols must contain every column in the primary key.

func (*Inserter) InsertRow Uses

func (ri *Inserter) InsertRow(
    ctx context.Context,
    b putter,
    values []tree.Datum,
    overwrite bool,
    checkFKs checkFKConstraints,
    traceKV bool,
) error

InsertRow adds to the batch the kv operations necessary to insert a table row with the given values.

type KVBatch Uses

type KVBatch struct {
    // Source is where the row data in the batch came from.
    Source int32
    // LastRow is the index of the last converted row in source in this batch.
    LastRow uint64
    // Progress represents the fraction of the input that generated this row.
    Progress float32
    // KVs is the actual converted KV data.
    KVs []roachpb.KeyValue
}

KVBatch represents a batch of KVs generated from converted rows.

type KVFetcher Uses

type KVFetcher struct {
    Span roachpb.Span
    // contains filtered or unexported fields
}

KVFetcher wraps kvBatchFetcher, providing a NextKV interface that returns the next kv from its input.

func NewKVFetcher Uses

func NewKVFetcher(
    txn *client.Txn,
    spans roachpb.Spans,
    reverse bool,
    useBatchLimit bool,
    firstBatchLimit int64,
    returnRangeInfo bool,
) (*KVFetcher, error)

NewKVFetcher creates a new KVFetcher.

func (*KVFetcher) NextKV Uses

func (f *KVFetcher) NextKV(
    ctx context.Context,
) (ok bool, kv roachpb.KeyValue, newSpan bool, err error)

NextKV returns the next kv from this fetcher. Returns false if there are no more kvs to fetch, the kv that was fetched, and any errors that may have occurred.

type KVInserter Uses

type KVInserter func(roachpb.KeyValue)

KVInserter implements the putter interface.

func (KVInserter) CPut Uses

func (i KVInserter) CPut(key, value interface{}, expValue *roachpb.Value)

CPut is not implmented.

func (KVInserter) Del Uses

func (i KVInserter) Del(key ...interface{})

Del is not implemented.

func (KVInserter) InitPut Uses

func (i KVInserter) InitPut(key, value interface{}, failOnTombstones bool)

InitPut method of the putter interface.

func (KVInserter) Put Uses

func (i KVInserter) Put(key, value interface{})

Put method of the putter interface.

type SpanKVFetcher Uses

type SpanKVFetcher struct {
    KVs []roachpb.KeyValue
}

SpanKVFetcher is a kvBatchFetcher that returns a set slice of kvs.

func (*SpanKVFetcher) GetRangesInfo Uses

func (f *SpanKVFetcher) GetRangesInfo() []roachpb.RangeInfo

GetRangesInfo implements the kvBatchFetcher interface.

type TableEntry Uses

type TableEntry struct {
    // Desc is the descriptor of the table. This can be nil if eg.
    // the table is not public.
    Desc *sqlbase.ImmutableTableDescriptor

    // IsAdding indicates the descriptor is being created.
    IsAdding bool

    // CheckHelper is the utility responsible for CHECK constraint
    // checks. The lookup function (see TableLookupFunction below) needs
    // not populate this field; this is populated by the lookup queue
    // below.
    CheckHelper *sqlbase.CheckHelper
}

TableEntry is the value type of FkTableMetadata: An optional table descriptor, populated when the table is public/leasable, and an IsAdding flag.

This also includes an optional CheckHelper for the table (for CHECK constraints). This is needed for FK work because CASCADE actions can modify rows, and CHECK constraints must be applied to rows modified by CASCADE.

func NoLookup Uses

func NoLookup(_ context.Context, _ TableID) (TableEntry, error)

NoLookup is a stub that can be used to not actually fetch metadata. This can be used when the FK work is initialized from a pre-populated FkTableMetadata map.

type TableID Uses

type TableID = sqlbase.ID

TableID is an alias for sqlbase.TableID (table IDs).

type TableLookupFunction Uses

type TableLookupFunction func(context.Context, TableID) (TableEntry, error)

TableLookupFunction is the function type used by MakeFkMetadata that will perform the actual lookup of table metadata.

type Updater Uses

type Updater struct {
    Helper                rowHelper
    DeleteHelper          *rowHelper
    FetchCols             []sqlbase.ColumnDescriptor
    FetchColIDtoRowIndex  map[sqlbase.ColumnID]int
    UpdateCols            []sqlbase.ColumnDescriptor
    UpdateColIDtoRowIndex map[sqlbase.ColumnID]int

    Fks fkExistenceCheckForUpdate
    // contains filtered or unexported fields
}

Updater abstracts the key/value operations for updating table rows.

func MakeUpdater Uses

func MakeUpdater(
    txn *client.Txn,
    tableDesc *sqlbase.ImmutableTableDescriptor,
    fkTables FkTableMetadata,
    updateCols []sqlbase.ColumnDescriptor,
    requestedCols []sqlbase.ColumnDescriptor,
    updateType rowUpdaterType,
    checkFKs checkFKConstraints,
    evalCtx *tree.EvalContext,
    alloc *sqlbase.DatumAlloc,
) (Updater, error)

MakeUpdater creates a Updater for the given table.

UpdateCols are the columns being updated and correspond to the updateValues that will be passed to UpdateRow.

The returned Updater contains a FetchCols field that defines the expectation of which values are passed as oldValues to UpdateRow. All the columns passed in requestedCols will be included in FetchCols at the beginning.

func (*Updater) IsColumnOnlyUpdate Uses

func (ru *Updater) IsColumnOnlyUpdate() bool

IsColumnOnlyUpdate returns true if this Updater is only updating column data (in contrast to updating the primary key or other indexes).

func (*Updater) UpdateRow Uses

func (ru *Updater) UpdateRow(
    ctx context.Context,
    b *client.Batch,
    oldValues []tree.Datum,
    updateValues []tree.Datum,
    checkFKs checkFKConstraints,
    traceKV bool,
) ([]tree.Datum, error)

UpdateRow adds to the batch the kv operations necessary to update a table row with the given values.

The row corresponding to oldValues is updated with the ones in updateValues. Note that updateValues only contains the ones that are changing.

The return value is only good until the next call to UpdateRow.

Package row imports 29 packages (graph) and is imported by 17 packages. Updated 2019-09-19. Refresh now. Tools for package owners.