Documentation
¶
Index ¶
- Constants
- func Copy(t flux.Table) (flux.BufferedTable, error)
- func Diff(want, got flux.TableIterator, opts ...DiffOption) string
- func FromBuffer(cr flux.ColReader) flux.Table
- func Sort(tables flux.TableIterator) (flux.TableIterator, error)
- func Stringify(table flux.Table) string
- func Values(cr flux.ColReader, j int) array.Array
- type BufferedBuilder
- type BufferedTable
- type Builder
- type BuilderCache
- func (d *BuilderCache) DiscardTable(key flux.GroupKey)
- func (d *BuilderCache) ExpireTable(key flux.GroupKey)
- func (d *BuilderCache) ForEach(f func(key flux.GroupKey, builder Builder) error) error
- func (d *BuilderCache) Get(key flux.GroupKey, b interface{}) bool
- func (d *BuilderCache) Table(key flux.GroupKey) (flux.Table, bool, error)
- type Chunk
- func (v Chunk) Bools(j int) *array.Boolean
- func (v Chunk) Buffer() arrow.TableBuffer
- func (v Chunk) Col(j int) flux.ColMeta
- func (v Chunk) Cols() []flux.ColMeta
- func (v Chunk) Floats(j int) *array.Float
- func (v Chunk) HasCol(label string) bool
- func (v Chunk) Index(label string) int
- func (v Chunk) Ints(j int) *array.Int
- func (v Chunk) Key() flux.GroupKey
- func (v Chunk) Len() int
- func (v Chunk) NCols() int
- func (v Chunk) Release()
- func (v Chunk) Retain()
- func (v Chunk) Stringify() string
- func (v Chunk) Strings(j int) *array.String
- func (v Chunk) Uints(j int) *array.Uint
- func (v Chunk) Values(j int) array.Array
- type DiffOption
- type Iterator
- type KeyLookup
- type ProfilerResult
Constants ¶
const BufferSize = 1000
BufferSize represents a constant buffer size to be used by flux that buffer data by the number of rows.
This isn't a required size, but a recommended one that can be shared as a constant around the various standard library functions so that they are more likely to not reorganize data.
This number was chosen because it is the same buffer size that the influxdb storage engine uses when buffering table data. In the future, we may want to make it possible for different sources to report their own buffer sizes so influxdb isn't given an unfair advantage just because these constants are set to the same value.
Variables ¶
This section is empty.
Functions ¶
func Copy ¶
func Copy(t flux.Table) (flux.BufferedTable, error)
Copy returns a buffered copy of the table and consumes the input table. If the input table is already buffered, it "consumes" the input and returns the same table.
The buffered table can then be copied additional times using the BufferedTable.Copy method.
This method should be used sparingly if at all. It will retain each of the buffers of data coming out of a table so the entire table is materialized in memory. For large datasets, this could potentially cause a problem. The allocator is meant to catch when this happens and prevent it.
func Diff ¶
func Diff(want, got flux.TableIterator, opts ...DiffOption) string
Diff will perform a diff between two table iterators. This will sort the tables within the table iterators and produce a diff of the full output.
func FromBuffer ¶
FromBuffer constructs a flux.Table from a single flux.ColReader.
func Sort ¶
func Sort(tables flux.TableIterator) (flux.TableIterator, error)
Sort will read a TableIterator and produce another TableIterator where the keys are sorted.
This method will buffer all of the data since it needs to ensure all of the tables are read to avoid any deadlocks. Be careful using this method in performance sensitive areas.
Types ¶
type BufferedBuilder ¶
type BufferedBuilder struct {
GroupKey flux.GroupKey
Columns []flux.ColMeta
Buffers []*arrow.TableBuffer
Allocator memory.Allocator
}
BufferedBuilder is a table builder that constructs a BufferedTable with zero or more buffers.
func GetBufferedBuilder ¶
func GetBufferedBuilder(key flux.GroupKey, cache *BuilderCache) (builder *BufferedBuilder, created bool)
GetBufferedBuilder is a convenience method for retrieving a BufferedBuilder from the BuilderCache.
func NewBufferedBuilder ¶
func NewBufferedBuilder(key flux.GroupKey, mem memory.Allocator) *BufferedBuilder
NewBufferedBuilder constructs a new BufferedBuilder.
func (*BufferedBuilder) AppendBuffer ¶
func (b *BufferedBuilder) AppendBuffer(cr flux.ColReader) error
AppendBuffer will append a new buffer to this table builder. It ensures the schemas are compatible and will backfill previous buffers with nil for new columns that didn't previously exist.
func (*BufferedBuilder) AppendTable ¶
func (b *BufferedBuilder) AppendTable(tbl flux.Table) error
AppendTable will append all of the table buffers inside of a table to this BufferedBuilder.
This method will take care of normalizing the schema in the case where there is an empty table with no buffers.
func (*BufferedBuilder) Release ¶
func (b *BufferedBuilder) Release()
type BufferedTable ¶
type BufferedTable struct {
GroupKey flux.GroupKey
Columns []flux.ColMeta
Buffers []flux.ColReader
// contains filtered or unexported fields
}
BufferedTable represents a table of buffered column readers.
func (*BufferedTable) Cols ¶
func (b *BufferedTable) Cols() []flux.ColMeta
func (*BufferedTable) Done ¶
func (b *BufferedTable) Done()
func (*BufferedTable) Empty ¶
func (b *BufferedTable) Empty() bool
func (*BufferedTable) Key ¶
func (b *BufferedTable) Key() flux.GroupKey
type Builder ¶
type Builder interface {
// Table will construct a Table from the existing contents.
// Invoking this method should reset the builder and all allocated
// memory will be owned by the returned flux.Table.
Table() (flux.Table, error)
// Release will release the buffered contents from the builder.
// This method is unnecessary if Table is called.
Release()
}
Builder is the minimum interface for constructing a Table.
type BuilderCache ¶
type BuilderCache struct {
// New will be called to construct a new Builder
// when a GroupKey that hasn't been seen before is
// requested. The returned Builder should be empty.
New func(key flux.GroupKey) Builder
// Tables contains the cached builders.
// This can be set before use to customize the
// method for storing data. If this is null,
// the default execute.GroupLookup is initialized
// when the cache is first used.
Tables KeyLookup
}
BuilderCache hold a mapping of group keys to Builder. When a Builder is requested for a specific group key, the BuilderCache will return a Builder that is unique for that GroupKey.
func (*BuilderCache) DiscardTable ¶
func (d *BuilderCache) DiscardTable(key flux.GroupKey)
func (*BuilderCache) ExpireTable ¶
func (d *BuilderCache) ExpireTable(key flux.GroupKey)
func (*BuilderCache) Get ¶
func (d *BuilderCache) Get(key flux.GroupKey, b interface{}) bool
Get retrieves the Builder for this group key. If one doesn't exist, it will invoke the New function and store it within the Builder. If the builder was newly created, this method returns true for the second parameter. The interface must be a pointer to the type that is created from the New method. This method will use reflection to set the value of the pointer.
type Chunk ¶
type Chunk struct {
// contains filtered or unexported fields
}
Chunk is a horizontal partition of a Table. It is a subset of rows and contains a set of columns known as the group key. It may not contain all columns that have been seen associated with that group key so transformations should verify the existence of columns for each chunk independently.
func ChunkFromBuffer ¶
func ChunkFromBuffer(buf arrow.TableBuffer) Chunk
ChunkFromBuffer will create a Chunk from the TableBuffer.
This function takes ownership of the arrow.TableBuffer and the Chunk goes out of scope at the same time as the arrow.TableBuffer unless Retain is called.
func ChunkFromReader ¶
ChunkFromReader will create a Chunk from the ColReader.
This function borrows a reference to the data in the ColReader and will go out of scope at the same time as the ColReader unless Retain is called.
func (Chunk) Buffer ¶
func (v Chunk) Buffer() arrow.TableBuffer
Buffer returns the underlying TableBuffer used for this Chunk. This is exposed for use by another package, but this method should never be invoked in normal code.
func (Chunk) Stringify ¶
Stringify turns the current chunk into a stringified representation for easier debugging.
type DiffOption ¶
type DiffOption interface {
// contains filtered or unexported methods
}
func DiffContext ¶
func DiffContext(n int) DiffOption
type KeyLookup ¶
type KeyLookup interface {
// Lookup will retrieve the value associated with the given key if it exists.
Lookup(key flux.GroupKey) (interface{}, bool)
// LookupOrCreate will retrieve the value associated with the given key or,
// if it does not exist, will invoke the function to create one and set
// it in the group lookup.
LookupOrCreate(key flux.GroupKey, fn func() interface{}) interface{}
// Set will set the value for the given key.
// It will overwrite an existing value.
Set(key flux.GroupKey, value interface{})
// Delete will remove the key from this KeyLookup.
// It will return the same thing as a call to Lookup.
Delete(key flux.GroupKey) (v interface{}, found bool)
// Range will iterate over all groups keys in a stable ordering.
// Range must not be called within another call to Range.
// It is safe to call Set/Delete while ranging.
Range(f func(key flux.GroupKey, value interface{}) error) error
// Clear will clear the lookup and reset it to contain nothing.
Clear()
}
KeyLookup is an interface for storing and retrieving items by their group key.
type ProfilerResult ¶
type ProfilerResult struct {
// contains filtered or unexported fields
}
func NewProfilerResult ¶
func NewProfilerResult(tables ...flux.Table) ProfilerResult
func (*ProfilerResult) Name ¶
func (r *ProfilerResult) Name() string
func (*ProfilerResult) Tables ¶
func (r *ProfilerResult) Tables() flux.TableIterator
Source Files
Directories