bigslice: github.com/grailbio/bigslice/frame Index | Files

package frame

import "github.com/grailbio/bigslice/frame"

Package frame implements a typed, columnar data structure that represents data vectors throughout Bigslice.

The package contains the definition of Frame as well as a set of index-based operators that amortize runtime safety overhead.

Index

Package Files

codec.go frame.go ops.go ops_builtin.go unsafe.go

Variables

var Empty = Frame{/* contains filtered or unexported fields */}

Empty is the empty frame.

func CanCompare Uses

func CanCompare(typ reflect.Type) bool

CanCompare returns whether values of the provided type are comparable.

func CanHash Uses

func CanHash(typ reflect.Type) bool

CanHash returns whether values of the provided type can be hashed.

func Compatible Uses

func Compatible(f, g Frame) bool

Compatible reports whether frames f and g are assignment compatible: that is, they have the same number of columns and the same column types.

func Copy Uses

func Copy(dst, src Frame) (n int)

Copy copies the contents of src until either dst has been filled or src exhausted. It returns the number of elements copied.

func RegisterOps Uses

func RegisterOps(make interface{})

RegisterOps registers an ops implementation. The provided argument make should be a function of the form

func(slice []t) Ops

returning operations for a t-typed slice. RegisterOps panics if the argument does not have the required shape or if operations have already been registered for type t.

type Decoder Uses

type Decoder interface {
    Session
    // Decode decodes a value from the underlying stream using
    // Gob.
    Decode(v interface{}) error
}

type Encoder Uses

type Encoder interface {
    Session
    // Encode encodes the provided value into the
    // encoder's stream using Gob.
    Encode(v interface{}) error
}

An Encoder manages transmission of data over a connection or file.

type Frame Uses

type Frame struct {
    // contains filtered or unexported fields
}

A Frame is a collection of 0 or more typed, equal-length columns that form a logical table. Each column is represented by a Go slice. Frames can be sliced efficiently, and the package provides computed operators that can perform efficient index-based operations on the Frame.

func AppendFrame Uses

func AppendFrame(dst, src Frame) Frame

AppendFrame appends src to dst, growing dst if needed.

func Make Uses

func Make(types slicetype.Type, len, cap int) Frame

Make returns a new frame with the provided type, length, and capacity.

func Slices Uses

func Slices(cols ...interface{}) Frame

Slices returns a new Frame constructed from a set of Go slices, each representing a column. The slices must have the same length, or Slices panics.

func Values Uses

func Values(cols []reflect.Value) Frame

Values returns a new Frame constructed from a set of reflect.Values, each representing a column. The slices must have the same length, or Values panics.

func (Frame) Cap Uses

func (f Frame) Cap() int

Cap returns the Frame's capacity.

func (Frame) Decode Uses

func (f Frame) Decode(col int, d Decoder) error

Decode decodes a column col as encoded by Frame.Encode. may only Decode be invoked for columns where HasCodec is true.

func (Frame) Encode Uses

func (f Frame) Encode(col int, e Encoder) error

Encode encodes column col of this frame. The given scratch buffer may be used by the encode function to avoid extra allocation. Encode may only be invoked for columns where HasCodec is true.

func (Frame) Ensure Uses

func (f Frame) Ensure(n int) Frame

Ensure Slice(0, n), growing the frame as needed.

func (Frame) Grow Uses

func (f Frame) Grow(n int) Frame

Grow returns a Frame with at least n extra capacity. The returned frame will have length f.Len()+n.

func (Frame) HasCodec Uses

func (f Frame) HasCodec(col int) bool

HasCodec returns whether column col has a type-specific codec.

func (Frame) Hash Uses

func (f Frame) Hash(i int) uint32

Hash returns a 32-bit hash of the prefix columns of frame f with a seed of 0.

func (Frame) HashWithSeed Uses

func (f Frame) HashWithSeed(i int, seed uint32) uint32

HashWithSeed returns a 32-bit seeded hash of the prefix columns of frame f.

func (Frame) Index Uses

func (f Frame) Index(col, i int) reflect.Value

Index returns the i'th row of col'th column as a reflect.Value.

func (Frame) Interface Uses

func (f Frame) Interface(i int) interface{}

Interface returns the i'th column as an empty interface.

func (Frame) Interfaces Uses

func (f Frame) Interfaces() []interface{}

Interfaces returns the frame's columns as empty interfaces.

func (Frame) IsZero Uses

func (f Frame) IsZero() bool

IsZero tells whether this frame is zero-valued.

func (Frame) Len Uses

func (f Frame) Len() int

Len returns the Frame's length.

func (Frame) Less Uses

func (f Frame) Less(i, j int) bool

Less reports whether the row with index i should sort before the element with index j. Less operates on the frame's prefix columns, and is available only if the operation is defined for those column types. See RegisterOps for more details.

TODO(marius): this method presents an unnecessary indirection; provide a way to get at a sort.Interface directly.

func (Frame) NumOut Uses

func (f Frame) NumOut() int

NumOut implements slicetype.Type

func (Frame) Out Uses

func (f Frame) Out(i int) reflect.Type

Out implements slicetype.Type.

func (Frame) Prefix Uses

func (f Frame) Prefix() int

Prefix implements slicetype.Type.

func (Frame) Prefixed Uses

func (f Frame) Prefixed(prefix int) Frame

Prefixed returns f with the given prefix.

func (Frame) Slice Uses

func (f Frame) Slice(i, j int) Frame

Slice returns the frame f[i:j]. It panics if indices are out of bounds.

func (Frame) SliceHeader Uses

func (f Frame) SliceHeader(i int) reflect.SliceHeader

SliceHeader returns the slice header for column i. As with other uses of SliceHeader, the user must ensure that a reference to the frame is maintained so that the underlying slice is not garbage collected while (unsafely) using the slice header.

func (Frame) String Uses

func (f Frame) String() string

String returns a descriptive string of the frame.

func (Frame) Swap Uses

func (f Frame) Swap(i, j int)

Swap swaps rows i and j in frame f.

func (Frame) TabString Uses

func (f Frame) TabString() string

TabString returns a string representing the frame in tabular format.

func (Frame) UnsafeIndexAddr Uses

func (f Frame) UnsafeIndexAddr(col, i int) uintptr

UnsafeIndexAddr returns the address of the i'th row of the col'th column. This can be used by advanced clients that import the unsafe package. Clients are responsible for managing reference lifetimes so that the underlying objects will not be garbage collected while an address returned from this method may still be used.

func (Frame) Value Uses

func (f Frame) Value(i int) reflect.Value

Value returns the ith column as a reflect.Value.

func (Frame) Values Uses

func (f Frame) Values() []reflect.Value

Values returns the frame's columns as reflect.Values.

func (Frame) WriteTab Uses

func (f Frame) WriteTab(w io.Writer)

WriteTab writes the frame in tabular format to the provided io.Writer.

func (Frame) Zero Uses

func (f Frame) Zero()

Zero zeros the memory all columnns.

type Key Uses

type Key uint64

Key represents a key that can be used to store session specific state.

func FreshKey Uses

func FreshKey() Key

FreshKey returns a unique key that can be used to store state in sessions.

type Ops Uses

type Ops struct {
    // Less compares two indices of a slice.
    Less func(i, j int) bool
    // HashWithSeed computes a 32-bit hash, given a seed, of an index
    // of a slice.
    HashWithSeed func(i int, seed uint32) uint32

    // Encode encodes a slice of the underlying vector. Encode is
    // optional, and overrides the default encoding (gob) used when
    // serializing and deserializing frames. Encode and decode must
    // always be specified together.
    Encode func(enc Encoder, i, j int) error

    // Decode decodes a slice encoded by Encode(i, j).
    Decode func(dec Decoder, i, j int) error
    // contains filtered or unexported fields
}

Ops represents a set of operations on a single frame instance. Ops are instantiated from implementations registered with RegisterOps and are managed by the frame instance.

Note that not all types may support all operations.

type Session Uses

type Session interface {
    // State retrieves the session state for the provided key
    // into the pointer state. If state is not a pointer, then
    // State will panic. State returns true the first time the key
    // is encountered in the session. This is typically used by
    // user code to initialize the state.
    //
    // If the state value is a pointer, then the first call to State
    // sets the state pointer (a pointer to a pointer) to a newly
    // allocated value. Otherwise it is set to a zero value of the
    // value type.
    State(key Key, state interface{}) bool
}

Session is a key-value store that is used by frame codecs to store session-specific state.

Package frame imports 14 packages (graph) and is imported by 5 packages. Updated 2019-10-09. Refresh now. Tools for package owners.