cockroach: github.com/cockroachdb/cockroach/pkg/sql/rowcontainer Index | Files

package rowcontainer

import "github.com/cockroachdb/cockroach/pkg/sql/rowcontainer"

Index

Package Files

datum_row_container.go disk_row_container.go hash_row_container.go row_container.go

Constants

const (
    // SizeOfDatum is the memory size of a Datum reference.
    SizeOfDatum = int64(unsafe.Sizeof(tree.Datum(nil)))
    // SizeOfDatums is the memory size of a Datum slice.
    SizeOfDatums = int64(unsafe.Sizeof(tree.Datums(nil)))
)

type AllRowsIterator Uses

type AllRowsIterator struct {
    RowIterator
    // contains filtered or unexported fields
}

AllRowsIterator iterates over all rows in HashDiskBackedRowContainer which should be initialized to not do marking. This iterator will be recreated in-place if the container spills to disk.

func (*AllRowsIterator) Close Uses

func (i *AllRowsIterator) Close()

Close implements RowIterator interface.

type DiskBackedIndexedRowContainer Uses

type DiskBackedIndexedRowContainer struct {
    *DiskBackedRowContainer

    // DisableCache is intended for testing only. It can be set to true to
    // disable reading and writing from the row cache.
    DisableCache bool
    // contains filtered or unexported fields
}

DiskBackedIndexedRowContainer is a wrapper around DiskBackedRowContainer that adds an index to each row added in the order of addition of those rows by storing an extra int column at the end of each row. These indices can be thought of as ordinals of the rows.

Note: although DiskRowContainer appends unique rowIDs to the keys that the rows are put at, MemRowContainer doesn't do something like that, so the code that utilizes internal rowIDs of DiskRowContainer ends up being worse than having this specialized container.

func NewDiskBackedIndexedRowContainer Uses

func NewDiskBackedIndexedRowContainer(
    ordering sqlbase.ColumnOrdering,
    typs []types.T,
    evalCtx *tree.EvalContext,
    engine diskmap.Factory,
    memoryMonitor *mon.BytesMonitor,
    diskMonitor *mon.BytesMonitor,
    rowCapacity int,
) *DiskBackedIndexedRowContainer

NewDiskBackedIndexedRowContainer creates a DiskBackedIndexedRowContainer with the given engine as the underlying store that rows are stored on when it spills to disk. Arguments:

- ordering is the output ordering; the order in which rows should be sorted.
- types is the schema of rows that will be added to this container.
- evalCtx defines the context in which to evaluate comparisons, only used
  when storing rows in memory.
- engine is the underlying store that rows are stored on when the container
  spills to disk.
- memoryMonitor is used to monitor this container's memory usage.
- diskMonitor is used to monitor this container's disk usage.
- rowCapacity (if not 0) specifies the number of rows in-memory container
  should be preallocated for.

func (*DiskBackedIndexedRowContainer) AddRow Uses

func (f *DiskBackedIndexedRowContainer) AddRow(ctx context.Context, row sqlbase.EncDatumRow) error

AddRow implements SortableRowContainer.

func (*DiskBackedIndexedRowContainer) Close Uses

func (f *DiskBackedIndexedRowContainer) Close(ctx context.Context)

Close implements SortableRowContainer.

func (*DiskBackedIndexedRowContainer) GetRow Uses

func (f *DiskBackedIndexedRowContainer) GetRow(
    ctx context.Context, pos int,
) (tree.IndexedRow, error)

GetRow implements tree.IndexedRows.

Getting a row by index is fast from an in-memory row container but is a lot slower from a disk-backed one. In order to mitigate the impact we add optimizations of maintaining a cache of tree.IndexedRow's and storing a disk iterator along with the index of the row it currently points at.

func (*DiskBackedIndexedRowContainer) Reorder Uses

func (f *DiskBackedIndexedRowContainer) Reorder(
    ctx context.Context, ordering sqlbase.ColumnOrdering,
) error

Reorder implements ReorderableRowContainer.

func (*DiskBackedIndexedRowContainer) UnsafeReset Uses

func (f *DiskBackedIndexedRowContainer) UnsafeReset(ctx context.Context) error

UnsafeReset resets the underlying container (if it is using disk, it will be reset to using memory).

type DiskBackedRowContainer Uses

type DiskBackedRowContainer struct {
    // contains filtered or unexported fields
}

DiskBackedRowContainer is a ReorderableRowContainer that uses a MemRowContainer to store rows and spills back to disk automatically if memory usage exceeds a given budget.

func (*DiskBackedRowContainer) AddRow Uses

func (f *DiskBackedRowContainer) AddRow(ctx context.Context, row sqlbase.EncDatumRow) error

AddRow is part of the SortableRowContainer interface.

func (*DiskBackedRowContainer) Close Uses

func (f *DiskBackedRowContainer) Close(ctx context.Context)

Close is part of the SortableRowContainer interface.

func (*DiskBackedRowContainer) Init Uses

func (f *DiskBackedRowContainer) Init(
    ordering sqlbase.ColumnOrdering,
    types []types.T,
    evalCtx *tree.EvalContext,
    engine diskmap.Factory,
    memoryMonitor *mon.BytesMonitor,
    diskMonitor *mon.BytesMonitor,
    rowCapacity int,
)

Init initializes a DiskBackedRowContainer. Arguments:

- ordering is the output ordering; the order in which rows should be sorted.
- types is the schema of rows that will be added to this container.
- evalCtx defines the context in which to evaluate comparisons, only used
  when storing rows in memory.
- engine is the store used for rows when spilling to disk.
- memoryMonitor is used to monitor the DiskBackedRowContainer's memory usage.
  If this monitor denies an allocation, the DiskBackedRowContainer will
  spill to disk.
- diskMonitor is used to monitor the DiskBackedRowContainer's disk usage if
  and when it spills to disk.
- rowCapacity (if not 0) indicates the number of rows that the underlying
  in-memory container should be preallocated for.

func (*DiskBackedRowContainer) InitTopK Uses

func (f *DiskBackedRowContainer) InitTopK()

InitTopK is part of the SortableRowContainer interface.

func (*DiskBackedRowContainer) Len Uses

func (f *DiskBackedRowContainer) Len() int

Len is part of the SortableRowContainer interface.

func (*DiskBackedRowContainer) MaybeReplaceMax Uses

func (f *DiskBackedRowContainer) MaybeReplaceMax(
    ctx context.Context, row sqlbase.EncDatumRow,
) error

MaybeReplaceMax is part of the SortableRowContainer interface.

func (*DiskBackedRowContainer) NewFinalIterator Uses

func (f *DiskBackedRowContainer) NewFinalIterator(ctx context.Context) RowIterator

NewFinalIterator is part of the SortableRowContainer interface.

func (*DiskBackedRowContainer) NewIterator Uses

func (f *DiskBackedRowContainer) NewIterator(ctx context.Context) RowIterator

NewIterator is part of the SortableRowContainer interface.

func (*DiskBackedRowContainer) Reorder Uses

func (f *DiskBackedRowContainer) Reorder(
    ctx context.Context, ordering sqlbase.ColumnOrdering,
) error

Reorder implements ReorderableRowContainer.

func (*DiskBackedRowContainer) Sort Uses

func (f *DiskBackedRowContainer) Sort(ctx context.Context)

Sort is part of the SortableRowContainer interface.

func (*DiskBackedRowContainer) SpillToDisk Uses

func (f *DiskBackedRowContainer) SpillToDisk(ctx context.Context) error

SpillToDisk creates a disk row container, injects all the data from the in-memory container into it, and clears the in-memory one afterwards.

func (*DiskBackedRowContainer) Spilled Uses

func (f *DiskBackedRowContainer) Spilled() bool

Spilled returns whether or not the DiskBackedRowContainer spilled to disk in its lifetime.

func (*DiskBackedRowContainer) UnsafeReset Uses

func (f *DiskBackedRowContainer) UnsafeReset(ctx context.Context) error

UnsafeReset resets the container for reuse. The DiskBackedRowContainer will reset to use memory if it is using disk.

func (*DiskBackedRowContainer) UsingDisk Uses

func (f *DiskBackedRowContainer) UsingDisk() bool

UsingDisk returns whether or not the DiskBackedRowContainer is currently using disk.

type DiskRowContainer Uses

type DiskRowContainer struct {
    // contains filtered or unexported fields
}

DiskRowContainer is a SortableRowContainer that stores rows on disk according to the ordering specified in DiskRowContainer.ordering. The underlying store is a SortedDiskMap so the sorting itself is delegated. Use an iterator created through NewIterator() to read the rows in sorted order.

func MakeDiskRowContainer Uses

func MakeDiskRowContainer(
    diskMonitor *mon.BytesMonitor,
    types []types.T,
    ordering sqlbase.ColumnOrdering,
    e diskmap.Factory,
) DiskRowContainer

MakeDiskRowContainer creates a DiskRowContainer with the given engine as the underlying store that rows are stored on. Arguments:

- diskMonitor is used to monitor this DiskRowContainer's disk usage.
- types is the schema of rows that will be added to this container.
- ordering is the output ordering; the order in which rows should be sorted.
- e is the underlying store that rows are stored on.

func (*DiskRowContainer) AddRow Uses

func (d *DiskRowContainer) AddRow(ctx context.Context, row sqlbase.EncDatumRow) error

AddRow is part of the SortableRowContainer interface.

Note: if key calculation changes, computeKey() of hashMemRowIterator should be changed accordingly.

func (*DiskRowContainer) Close Uses

func (d *DiskRowContainer) Close(ctx context.Context)

Close is part of the SortableRowContainer interface.

func (*DiskRowContainer) InitTopK Uses

func (d *DiskRowContainer) InitTopK()

InitTopK limits iterators to read the first k rows.

func (*DiskRowContainer) Len Uses

func (d *DiskRowContainer) Len() int

Len is part of the SortableRowContainer interface.

func (*DiskRowContainer) MaybeReplaceMax Uses

func (d *DiskRowContainer) MaybeReplaceMax(ctx context.Context, row sqlbase.EncDatumRow) error

MaybeReplaceMax adds row to the DiskRowContainer. The SortedDiskMap will sort this row into the top k if applicable.

func (*DiskRowContainer) NewFinalIterator Uses

func (d *DiskRowContainer) NewFinalIterator(ctx context.Context) RowIterator

NewFinalIterator returns an iterator that reads rows exactly once throughout the lifetime of a DiskRowContainer. Rows are not actually discarded from the DiskRowContainer, but the lastReadKey is kept track of in order to serve as the start key for future diskRowFinalIterators. NOTE: Don't use NewFinalIterator if you passed in an ordering for the rows and will be adding rows between iterations. New rows could sort before the current row.

func (*DiskRowContainer) NewIterator Uses

func (d *DiskRowContainer) NewIterator(ctx context.Context) RowIterator

NewIterator is part of the SortableRowContainer interface.

func (*DiskRowContainer) Reorder Uses

func (d *DiskRowContainer) Reorder(ctx context.Context, ordering sqlbase.ColumnOrdering) error

Reorder implements ReorderableRowContainer. It creates a new DiskRowContainer with the requested ordering and adds a row one by one from the current DiskRowContainer, the latter is closed at the end.

func (*DiskRowContainer) Sort Uses

func (d *DiskRowContainer) Sort(context.Context)

Sort is a noop because the use of a SortedDiskMap as the underlying store keeps the rows in sorted order.

func (*DiskRowContainer) UnsafeReset Uses

func (d *DiskRowContainer) UnsafeReset(ctx context.Context) error

UnsafeReset is part of the SortableRowContainer interface.

type HashDiskBackedRowContainer Uses

type HashDiskBackedRowContainer struct {
    // contains filtered or unexported fields
}

HashDiskBackedRowContainer is a hashRowContainer that uses a HashMemRowContainer to store rows and spills to disk automatically if memory usage exceeds a given budget. When spilled to disk, the rows are stored with an extra boolean column to keep track of that row's mark.

func NewHashDiskBackedRowContainer Uses

func NewHashDiskBackedRowContainer(
    mrc *MemRowContainer,
    evalCtx *tree.EvalContext,
    memoryMonitor *mon.BytesMonitor,
    diskMonitor *mon.BytesMonitor,
    engine diskmap.Factory,
) *HashDiskBackedRowContainer

NewHashDiskBackedRowContainer makes a HashDiskBackedRowContainer. mrc (the first argument) can either be nil (in which case HashMemRowContainer will be built upon an empty MemRowContainer) or non-nil (in which case mrc is used as underlying MemRowContainer under HashMemRowContainer). The latter case is used by the hashJoiner since when initializing HashDiskBackedRowContainer it will have accumulated rows from both sides of the join in MemRowContainers, and we can reuse one of them.

func (*HashDiskBackedRowContainer) AddRow Uses

func (h *HashDiskBackedRowContainer) AddRow(ctx context.Context, row sqlbase.EncDatumRow) error

AddRow adds a row to the HashDiskBackedRowContainer. This row is unmarked by default.

func (*HashDiskBackedRowContainer) Close Uses

func (h *HashDiskBackedRowContainer) Close(ctx context.Context)

Close implements the hashRowContainer interface.

func (*HashDiskBackedRowContainer) Init Uses

func (h *HashDiskBackedRowContainer) Init(
    ctx context.Context, shouldMark bool, types []types.T, storedEqCols columns, encodeNull bool,
) error

Init implements the hashRowContainer interface.

func (*HashDiskBackedRowContainer) NewAllRowsIterator Uses

func (h *HashDiskBackedRowContainer) NewAllRowsIterator(
    ctx context.Context,
) (*AllRowsIterator, error)

NewAllRowsIterator creates AllRowsIterator that can iterate over all rows (equivalent to an unmarked iterator when the container doesn't do marking) and will be recreated if the spilling to disk occurs.

func (*HashDiskBackedRowContainer) NewBucketIterator Uses

func (h *HashDiskBackedRowContainer) NewBucketIterator(
    ctx context.Context, row sqlbase.EncDatumRow, probeEqCols columns,
) (RowMarkerIterator, error)

NewBucketIterator implements the hashRowContainer interface.

func (*HashDiskBackedRowContainer) NewUnmarkedIterator Uses

func (h *HashDiskBackedRowContainer) NewUnmarkedIterator(ctx context.Context) RowIterator

NewUnmarkedIterator implements the hashRowContainer interface.

func (*HashDiskBackedRowContainer) ReserveMarkMemoryMaybe Uses

func (h *HashDiskBackedRowContainer) ReserveMarkMemoryMaybe(ctx context.Context) error

ReserveMarkMemoryMaybe attempts to reserve memory for marks if we're using an in-memory container at the moment. If there is not enough memory left, it spills to disk.

func (*HashDiskBackedRowContainer) Sort Uses

func (h *HashDiskBackedRowContainer) Sort(ctx context.Context)

Sort sorts the underlying row container based on stored equality columns which forces all rows from the same hash bucket to be contiguous.

func (*HashDiskBackedRowContainer) SpillToDisk Uses

func (h *HashDiskBackedRowContainer) SpillToDisk(ctx context.Context) error

SpillToDisk creates a disk row container, injects all the data from the in-memory container into it, and clears the in-memory one afterwards.

func (*HashDiskBackedRowContainer) UnsafeReset Uses

func (h *HashDiskBackedRowContainer) UnsafeReset(ctx context.Context) error

UnsafeReset resets the container for reuse. The HashDiskBackedRowContainer will reset to using memory if it is using disk.

func (*HashDiskBackedRowContainer) UsingDisk Uses

func (h *HashDiskBackedRowContainer) UsingDisk() bool

UsingDisk returns whether or not the HashDiskBackedRowContainer is currently using disk.

type HashDiskRowContainer Uses

type HashDiskRowContainer struct {
    DiskRowContainer
    // contains filtered or unexported fields
}

HashDiskRowContainer is an on-disk implementation of a HashRowContainer. The rows are stored in an underlying DiskRowContainer with an extra boolean column to keep track of that row's mark.

func MakeHashDiskRowContainer Uses

func MakeHashDiskRowContainer(
    diskMonitor *mon.BytesMonitor, e diskmap.Factory,
) HashDiskRowContainer

MakeHashDiskRowContainer creates a HashDiskRowContainer with the given engine as the underlying store that rows are stored on. shouldMark specifies whether the HashDiskRowContainer should set itself up to mark rows.

func (*HashDiskRowContainer) AddRow Uses

func (h *HashDiskRowContainer) AddRow(ctx context.Context, row sqlbase.EncDatumRow) error

AddRow adds a row to the HashDiskRowContainer. This row is unmarked by default.

func (*HashDiskRowContainer) Init Uses

func (h *HashDiskRowContainer) Init(
    _ context.Context, shouldMark bool, typs []types.T, storedEqCols columns, encodeNull bool,
) error

Init implements the HashRowContainer interface.

func (*HashDiskRowContainer) NewBucketIterator Uses

func (h *HashDiskRowContainer) NewBucketIterator(
    ctx context.Context, row sqlbase.EncDatumRow, probeEqCols columns,
) (RowMarkerIterator, error)

NewBucketIterator implements the HashRowContainer interface.

func (*HashDiskRowContainer) NewUnmarkedIterator Uses

func (h *HashDiskRowContainer) NewUnmarkedIterator(ctx context.Context) RowIterator

NewUnmarkedIterator implements the HashRowContainer interface.

type HashMemRowContainer Uses

type HashMemRowContainer struct {
    *MemRowContainer
    // contains filtered or unexported fields
}

HashMemRowContainer is an in-memory implementation of a HashRowContainer. The rows are stored in an underlying MemRowContainer and an accompanying map stores the mapping from equality column encodings to indices in the MemRowContainer corresponding to matching rows. NOTE: Once a row is marked, adding more rows to the HashMemRowContainer results in undefined behavior. It is not necessary to do otherwise for the current usage of HashMemRowContainer.

func MakeHashMemRowContainer Uses

func MakeHashMemRowContainer(rowContainer *MemRowContainer) HashMemRowContainer

MakeHashMemRowContainer creates a HashMemRowContainer from the given rowContainer. This rowContainer must still be Close()d by the caller.

func (*HashMemRowContainer) AddRow Uses

func (h *HashMemRowContainer) AddRow(ctx context.Context, row sqlbase.EncDatumRow) error

AddRow adds a row to the HashMemRowContainer. This row is unmarked by default.

func (*HashMemRowContainer) Close Uses

func (h *HashMemRowContainer) Close(ctx context.Context)

Close implements the HashRowContainer interface.

func (*HashMemRowContainer) Init Uses

func (h *HashMemRowContainer) Init(
    ctx context.Context, shouldMark bool, _ []types.T, storedEqCols columns, encodeNull bool,
) error

Init implements the HashRowContainer interface. types is ignored because the schema is inferred from the MemRowContainer.

func (*HashMemRowContainer) NewBucketIterator Uses

func (h *HashMemRowContainer) NewBucketIterator(
    ctx context.Context, row sqlbase.EncDatumRow, probeEqCols columns,
) (RowMarkerIterator, error)

NewBucketIterator implements the HashRowContainer interface.

func (*HashMemRowContainer) NewUnmarkedIterator Uses

func (h *HashMemRowContainer) NewUnmarkedIterator(ctx context.Context) RowIterator

NewUnmarkedIterator implements the HashRowContainer interface.

func (*HashMemRowContainer) ReserveMarkMemoryMaybe Uses

func (h *HashMemRowContainer) ReserveMarkMemoryMaybe(ctx context.Context) error

ReserveMarkMemoryMaybe is a utility function to grow the HashMemRowContainer's memory account by the memory needed to mark all rows. It is a noop if h.markMemoryReserved is true.

type HashRowContainer Uses

type HashRowContainer interface {
    // Init initializes the HashRowContainer with the given equality columns.
    //	- shouldMark specifies whether the caller cares about marking rows. If
    //	  not, the HashRowContainer will not perform any row marking logic. This
    //	  is meant to optimize space usage and runtime.
    //	- types is the schema of rows that will be added to this container.
    //	- storedEqCols are the equality columns of rows stored in this
    // 	  container.
    // 	  i.e. when adding a row, the columns specified by storedEqCols are used
    // 	  to get the bucket that the row should be added to.
    //	- encodeNull indicates whether rows with NULL equality columns should be
    //	  stored or skipped.
    Init(
        ctx context.Context, shouldMark bool, types []types.T, storedEqCols columns,
        encodeNull bool,
    ) error
    AddRow(context.Context, sqlbase.EncDatumRow) error

    // NewBucketIterator returns a RowMarkerIterator that iterates over a bucket
    // of rows that match the given row on equality columns. This iterator can
    // also be used to mark rows.
    // Rows are marked because of the use of this interface by the hashJoiner.
    // Given a row, the hashJoiner does not necessarily want to emit all rows
    // that match on equality columns. There is an additional `ON` clause that
    // specifies an arbitrary expression that matching rows must pass to be
    // emitted. For full/outer joins, this is tracked through marking rows if
    // they match and then iterating over all unmarked rows to emit those that
    // did not match.
    // 	- probeEqCols are the equality columns of the given row that are used to
    // 	  get the bucket of matching rows.
    NewBucketIterator(
        ctx context.Context, row sqlbase.EncDatumRow, probeEqCols columns,
    ) (RowMarkerIterator, error)

    // NewUnmarkedIterator returns a RowIterator that iterates over unmarked
    // rows. If shouldMark was false in Init(), this iterator iterates over all
    // rows.
    NewUnmarkedIterator(context.Context) RowIterator

    // Close frees up resources held by the HashRowContainer.
    Close(context.Context)
}

HashRowContainer is a container used to store rows according to an encoding of given equality columns. The stored rows can then be probed to return a bucket of matching rows. Additionally, each stored row can be marked and all rows that are unmarked can be iterated over. An example of where this is useful is in full/outer joins. The caller can mark all matched rows and iterate over the unmarked rows to produce a result.

type IndexedRow Uses

type IndexedRow struct {
    Idx int
    Row sqlbase.EncDatumRow
}

IndexedRow is a row with a corresponding index.

func (IndexedRow) GetDatum Uses

func (ir IndexedRow) GetDatum(colIdx int) (tree.Datum, error)

GetDatum implements tree.IndexedRow interface.

func (IndexedRow) GetDatums Uses

func (ir IndexedRow) GetDatums(startColIdx, endColIdx int) (tree.Datums, error)

GetDatums implements tree.IndexedRow interface.

func (IndexedRow) GetIdx Uses

func (ir IndexedRow) GetIdx() int

GetIdx implements tree.IndexedRow interface.

type IndexedRowContainer Uses

type IndexedRowContainer interface {
    ReorderableRowContainer

    // GetRow returns a row at the given index or an error.
    GetRow(ctx context.Context, idx int) (tree.IndexedRow, error)
}

IndexedRowContainer is a ReorderableRowContainer which also implements tree.IndexedRows. It allows retrieving a row at a particular index.

type MemRowContainer Uses

type MemRowContainer struct {
    RowContainer
    // contains filtered or unexported fields
}

MemRowContainer is the wrapper around rowcontainer.RowContainer that provides more functionality, especially around converting to/from EncDatumRows and facilitating sorting.

func (*MemRowContainer) AddRow Uses

func (mc *MemRowContainer) AddRow(ctx context.Context, row sqlbase.EncDatumRow) error

AddRow adds a row to the container.

func (*MemRowContainer) EncRow Uses

func (mc *MemRowContainer) EncRow(idx int) sqlbase.EncDatumRow

EncRow returns the idx-th row as an EncDatumRow. The slice itself is reused so it is only valid until the next call to EncRow.

func (*MemRowContainer) GetRow Uses

func (mc *MemRowContainer) GetRow(ctx context.Context, pos int) (tree.IndexedRow, error)

GetRow implements IndexedRowContainer.

func (*MemRowContainer) Init Uses

func (mc *MemRowContainer) Init(
    ordering sqlbase.ColumnOrdering, types []types.T, evalCtx *tree.EvalContext,
)

Init initializes the MemRowContainer. The MemRowContainer uses evalCtx.Mon to track memory usage.

func (*MemRowContainer) InitTopK Uses

func (mc *MemRowContainer) InitTopK()

InitTopK rearranges the rows in the MemRowContainer into a Max-Heap.

func (*MemRowContainer) InitWithMon Uses

func (mc *MemRowContainer) InitWithMon(
    ordering sqlbase.ColumnOrdering,
    types []types.T,
    evalCtx *tree.EvalContext,
    mon *mon.BytesMonitor,
    rowCapacity int,
)

InitWithMon initializes the MemRowContainer with an explicit monitor. Only use this if the default MemRowContainer.Init() function is insufficient.

func (*MemRowContainer) Less Uses

func (mc *MemRowContainer) Less(i, j int) bool

Less is part of heap.Interface and is only meant to be used internally.

func (*MemRowContainer) MaybeReplaceMax Uses

func (mc *MemRowContainer) MaybeReplaceMax(ctx context.Context, row sqlbase.EncDatumRow) error

MaybeReplaceMax replaces the maximum element with the given row, if it is smaller. Assumes InitTopK was called.

func (*MemRowContainer) NewFinalIterator Uses

func (mc *MemRowContainer) NewFinalIterator(_ context.Context) RowIterator

NewFinalIterator returns an iterator that can be used to iterate over a MemRowContainer. Note that this iterator doesn't iterate over a snapshot of MemRowContainer and that it deletes rows as soon as they are iterated over.

func (*MemRowContainer) NewIterator Uses

func (mc *MemRowContainer) NewIterator(_ context.Context) RowIterator

NewIterator returns an iterator that can be used to iterate over a MemRowContainer. Note that this iterator doesn't iterate over a snapshot of MemRowContainer.

func (*MemRowContainer) Pop Uses

func (mc *MemRowContainer) Pop() interface{}

Pop is part of heap.Interface.

func (*MemRowContainer) Push Uses

func (mc *MemRowContainer) Push(_ interface{})

Push is part of heap.Interface.

func (*MemRowContainer) Reorder Uses

func (mc *MemRowContainer) Reorder(_ context.Context, ordering sqlbase.ColumnOrdering) error

Reorder implements ReorderableRowContainer. We don't need to create a new MemRowContainer and can just change the ordering on-the-fly.

func (*MemRowContainer) Sort Uses

func (mc *MemRowContainer) Sort(ctx context.Context)

Sort is part of the SortableRowContainer interface.

func (*MemRowContainer) Types Uses

func (mc *MemRowContainer) Types() []types.T

Types returns the MemRowContainer's types.

type ReorderableRowContainer Uses

type ReorderableRowContainer interface {
    SortableRowContainer

    // Reorder changes the ordering on which the rows are sorted. In order for
    // new ordering to take effect, Sort() must be called. It returns an error if
    // it occurs.
    Reorder(context.Context, sqlbase.ColumnOrdering) error
}

ReorderableRowContainer is a SortableRowContainer that can change the ordering on which the rows are sorted.

type RowContainer Uses

type RowContainer struct {
    // contains filtered or unexported fields
}

RowContainer is a container for rows of Datums which tracks the approximate amount of memory allocated for row data. Rows must be added using AddRow(); once the work is done the Close() method must be called to release the allocated memory.

TODO(knz): this does not currently track the amount of memory used for the outer array of Datums references.

func NewRowContainer Uses

func NewRowContainer(acc mon.BoundAccount, ti sqlbase.ColTypeInfo, rowCapacity int) *RowContainer

NewRowContainer allocates a new row container.

The acc argument indicates where to register memory allocations by this row container. Should probably be created by Session.makeBoundAccount() or Session.TxnState.makeBoundAccount().

The rowCapacity argument indicates how many rows are to be expected; it is used to pre-allocate the outer array of row references, in the fashion of Go's capacity argument to the make() function.

Note that we could, but do not (yet), report the size of the row container itself to the monitor in this constructor. This is because the various planNodes are not (yet) equipped to call Close() upon encountering errors in their constructor (all nodes initializing a RowContainer there) and SetLimitHint() (for sortNode which initializes a RowContainer there). This would be rather error-prone to implement consistently and hellishly difficult to test properly. The trade-off is that very large table schemas or column selections could cause unchecked and potentially dangerous memory growth.

func (*RowContainer) AddRow Uses

func (c *RowContainer) AddRow(ctx context.Context, row tree.Datums) (tree.Datums, error)

AddRow attempts to insert a new row in the RowContainer. The row slice is not used directly: the Datum values inside the Datums are copied to internal storage. Returns an error if the allocation was denied by the MemoryMonitor.

func (*RowContainer) At Uses

func (c *RowContainer) At(i int) tree.Datums

At accesses a row at a specific index.

func (*RowContainer) Clear Uses

func (c *RowContainer) Clear(ctx context.Context)

Clear resets the container and releases the associated memory. This allows the RowContainer to be reused.

func (*RowContainer) Close Uses

func (c *RowContainer) Close(ctx context.Context)

Close releases the memory associated with the RowContainer.

func (*RowContainer) Init Uses

func (c *RowContainer) Init(acc mon.BoundAccount, ti sqlbase.ColTypeInfo, rowCapacity int)

Init can be used instead of NewRowContainer if we have a RowContainer that is already part of an on-heap structure.

func (*RowContainer) Len Uses

func (c *RowContainer) Len() int

Len reports the number of rows currently held in this RowContainer.

func (*RowContainer) MemUsage Uses

func (c *RowContainer) MemUsage() int64

MemUsage returns the current accounted memory usage.

func (*RowContainer) NumCols Uses

func (c *RowContainer) NumCols() int

NumCols reports the number of columns for each row in the container.

func (*RowContainer) PopFirst Uses

func (c *RowContainer) PopFirst()

PopFirst discards the first row in the RowContainer.

func (*RowContainer) Replace Uses

func (c *RowContainer) Replace(ctx context.Context, i int, newRow tree.Datums) error

Replace substitutes one row for another. This does query the MemoryMonitor to determine whether the new row fits the allowance.

func (*RowContainer) Swap Uses

func (c *RowContainer) Swap(i, j int)

Swap exchanges two rows. Used for sorting.

func (*RowContainer) UnsafeReset Uses

func (c *RowContainer) UnsafeReset(ctx context.Context) error

UnsafeReset resets the container without releasing the associated memory. This allows the RowContainer to be reused, but keeps the previously-allocated buffers around for reuse. This is desirable if this RowContainer will be used and reset many times in the course of a computation before eventually being discarded. It's unsafe because it immediately renders all previously allocated rows unsafe - they might be overwritten without notice. This is only safe to use if it's guaranteed that all previous rows retrieved by At have been copied or otherwise not retained.

type RowIterator Uses

type RowIterator interface {
    // Rewind seeks to the first row.
    Rewind()
    // Valid must be called after any call to Rewind() or Next(). It returns
    // (true, nil) if the iterator points to a valid row and (false, nil) if the
    // iterator has moved past the last row.
    // If an error has occurred, the returned bool is invalid.
    Valid() (bool, error)
    // Next advances the iterator to the next row in the iteration.
    Next()
    // Row returns the current row. The returned row is only valid until the
    // next call to Rewind() or Next().
    Row() (sqlbase.EncDatumRow, error)

    // Close frees up resources held by the iterator.
    Close()
}

RowIterator is a simple iterator used to iterate over sqlbase.EncDatumRows. Example use:

var i RowIterator
for i.Rewind(); ; i.Next() {
	if ok, err := i.Valid(); err != nil {
		// Handle error.
	} else if !ok {
		break
	}
	row, err := i.Row()
	if err != nil {
		// Handle error.
	}
	// Do something.
}

type RowMarkerIterator Uses

type RowMarkerIterator interface {
    RowIterator
    // Reset resets this iterator to point at a bucket that matches the given
    // row. This will cause RowIterator.Rewind to rewind to the front of the
    // input row's bucket.
    Reset(ctx context.Context, row sqlbase.EncDatumRow) error
    Mark(ctx context.Context, mark bool) error
    IsMarked(ctx context.Context) bool
}

RowMarkerIterator is a RowIterator that can be used to mark rows.

type SortableRowContainer Uses

type SortableRowContainer interface {
    Len() int
    AddRow(context.Context, sqlbase.EncDatumRow) error
    // Sort sorts the rows according to the current ordering (the one set either
    // at initialization or by the last call of Reorder() - if the container is
    // ReorderableRowContainer).
    Sort(context.Context)
    // NewIterator returns a RowIterator that can be used to iterate over
    // the rows.
    NewIterator(context.Context) RowIterator
    // NewFinalIterator returns a RowIterator that can be used to iterate over the
    // rows, possibly freeing resources along the way. Subsequent calls to
    // NewIterator or NewFinalIterator are not guaranteed to return any rows.
    NewFinalIterator(context.Context) RowIterator

    // UnsafeReset resets the container, allowing for reuse. It renders all
    // previously allocated rows unsafe.
    UnsafeReset(context.Context) error

    // InitTopK enables optimizations in cases where the caller cares only about
    // the top k rows where k is the size of the SortableRowContainer when
    // InitTopK is called. Once InitTopK is called, callers should not call
    // AddRow. Iterators created after calling InitTopK are guaranteed to read the
    // top k rows only.
    InitTopK()
    // MaybeReplaceMax checks whether the given row belongs in the top k rows,
    // potentially evicting a row in favor of the given row.
    MaybeReplaceMax(context.Context, sqlbase.EncDatumRow) error

    // Close frees up resources held by the SortableRowContainer.
    Close(context.Context)
}

SortableRowContainer is a container used to store rows and optionally sort these.

Package rowcontainer imports 18 packages (graph) and is imported by 12 packages. Updated 2019-09-19. Refresh now. Tools for package owners.