cockroach: github.com/cockroachdb/cockroach/pkg/sql/colmem Index | Files

package colmem

import "github.com/cockroachdb/cockroach/pkg/sql/colmem"

Index

Package Files

allocator.go

Variables

var SizeOfBatchSizeSelVector = coldata.BatchSize() * sizeOfInt

SizeOfBatchSizeSelVector is the size (in bytes) of a selection vector of coldata.BatchSize() length.

func EstimateBatchSizeBytes Uses

func EstimateBatchSizeBytes(vecTypes []*types.T, batchLength int) int

EstimateBatchSizeBytes returns an estimated amount of bytes needed to store a batch in memory that has column types vecTypes. WARNING: This only is correct for fixed width types, and returns an estimate for non fixed width types. In future it might be possible to remove the need for estimation by specifying batch sizes in terms of bytes.

func GetProportionalBatchMemSize Uses

func GetProportionalBatchMemSize(b coldata.Batch, length int64) int64

GetProportionalBatchMemSize returns the memory size of the batch that is proportional to given 'length'. This method returns the estimated memory footprint *only* of the first 'length' tuples in 'b'.

type Allocator Uses

type Allocator struct {
    // contains filtered or unexported fields
}

Allocator is a memory management tool for vectorized components. It provides new batches (and appends to existing ones) within a fixed memory budget. If the budget is exceeded, it will panic with an error.

In the future this can also be used to pool coldata.Vec allocations.

func NewAllocator Uses

func NewAllocator(
    ctx context.Context, acc *mon.BoundAccount, factory coldata.ColumnFactory,
) *Allocator

NewAllocator constructs a new Allocator instance.

func (*Allocator) AdjustMemoryUsage Uses

func (a *Allocator) AdjustMemoryUsage(delta int64)

AdjustMemoryUsage adjusts the number of bytes currently allocated through this allocator by delta bytes (which can be both positive or negative).

func (*Allocator) MaybeAppendColumn Uses

func (a *Allocator) MaybeAppendColumn(b coldata.Batch, t *types.T, colIdx int)

MaybeAppendColumn might append a newly allocated coldata.Vec of the given type to b at position colIdx. Behavior of the function depends on how colIdx compares to the width of b: 1. if colIdx < b.Width(), then we expect that correctly-typed vector is already present in position colIdx. If that's not the case, we will panic. 2. if colIdx == b.Width(), then we will append a newly allocated coldata.Vec of the given type. 3. if colIdx > b.Width(), then we will panic because such condition indicates an error in setting up vector type enforcers during the planning stage. NOTE: b must be non-zero length batch.

func (*Allocator) NewMemBatch Uses

func (a *Allocator) NewMemBatch(typs []*types.T) coldata.Batch

NewMemBatch allocates a new in-memory coldata.Batch.

func (*Allocator) NewMemBatchNoCols Uses

func (a *Allocator) NewMemBatchNoCols(types []*types.T, size int) coldata.Batch

NewMemBatchNoCols creates a "skeleton" of new in-memory coldata.Batch. It allocates memory for the selection vector but does *not* allocate any memory for the column vectors - those will have to be added separately.

func (*Allocator) NewMemBatchWithSize Uses

func (a *Allocator) NewMemBatchWithSize(typs []*types.T, size int) coldata.Batch

NewMemBatchWithSize allocates a new in-memory coldata.Batch with the given column size.

func (*Allocator) NewMemColumn Uses

func (a *Allocator) NewMemColumn(t *types.T, n int) coldata.Vec

NewMemColumn returns a new coldata.Vec, initialized with a length.

func (*Allocator) PerformOperation Uses

func (a *Allocator) PerformOperation(destVecs []coldata.Vec, operation func())

PerformOperation executes 'operation' (that somehow modifies 'destVecs') and updates the memory account accordingly. NOTE: if some columnar vectors are not modified, they should not be included in 'destVecs' to reduce the performance hit of memory accounting.

func (*Allocator) ReleaseBatch Uses

func (a *Allocator) ReleaseBatch(b coldata.Batch)

ReleaseBatch releases the size of the batch from the memory account. This shouldn't need to be used regularly, since all accounts are closed by Flow.Cleanup. Use this if you want to explicitly manage the memory used. An example of a use case is releasing a batch before writing it to disk. NOTE: when calculating memory footprint, this method looks at the capacities of the vectors and does *not* pay attention to the length of the batch.

func (*Allocator) ReleaseMemory Uses

func (a *Allocator) ReleaseMemory(size int64)

ReleaseMemory reduces the number of bytes currently allocated through this allocator by (at most) size bytes. size must be non-negative.

func (*Allocator) RetainBatch Uses

func (a *Allocator) RetainBatch(b coldata.Batch)

RetainBatch adds the size of the batch to the memory account. This shouldn't need to be used regularly, since most memory accounting necessary is done through PerformOperation. Use this if you want to explicitly manage the memory accounted for. NOTE: when calculating memory footprint, this method looks at the capacities of the vectors and does *not* pay attention to the length of the batch.

func (*Allocator) Used Uses

func (a *Allocator) Used() int64

Used returns the number of bytes currently allocated through this allocator.

Package colmem imports 12 packages (graph) and is imported by 9 packages. Updated 2020-08-12. Refresh now. Tools for package owners.