arrowtools

package module
v0.0.0-...-409bde9 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 19, 2019 License: BSD-3-Clause Imports: 8 Imported by: 0

README

Build Status Go Report Card codecov GoDoc

arrowtools is a Go package for manipulating Apache arrow data containers. For more about support for Arrow in Go see here.

Documentation

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func AppendColumns 

func AppendColumns(tbl array.Table, cols ...array.Column) array.Table

AppendColumns returns a table that contains the columns of the provided table, and appends the given additional columns. These new columns must not have the same name as any existing columns.

func ColumnsEqual 

func ColumnsEqual(col1, col2 *array.Column) (bool, string)

ColumnsEqual returns a boolean indicating whether the data in the two given columns are equal. If the data are not equal, a brief message describing the difference is returned.

func DropColumns 

func DropColumns(tbl array.Table, names ...string) array.Table

DropColumns returns a table that contains the columns of the provided table, except for the columns with the given names.

func DropNA 

func DropNA(tbl array.Table) array.Table

DropNA drops all rows from the table that contain at least one null value.

Example 
var sh SliceHelper

col1 := sh.Float64Column([][]float64{{0, 1, 2, 3}},
	[][]bool{{true, true, false, true}}, "x")
col2 := sh.Int16Column([][]int16{{0, 1, 2, 3}},
	[][]bool{{false, true, false, true}}, "y")

tbl1 := TableFromColumns([]array.Column{*col1, *col2})
tbl2 := DropNA(tbl1)

fmt.Printf("%d %d", tbl2.NumRows(), tbl2.NumCols())

Output:

2 2

func FilterRows 

func FilterRows(tbl array.Table, selector SelectFunc) array.Table

FilterRows selects a subset of the rows of a table. Rows where the selector function is false are dropped.

func ReadCSV 

func ReadCSV(r io.Reader, fields []arrow.Field, opts ...Option) array.Table

ReadCSV is a CSV reader that produces a arrow Table from the data in a CSV source. If a header row is present, the provided Schema can contain only a subset of the columns in the file, and only those columns are retained. If there is no header, then the length of the schema should match the number of columns in the source file.

func ReplaceColumn 

func ReplaceColumn(tbl array.Table, col array.Column) array.Table

ReplaceColumn returns a new table in which one column is replaced with a given column. The name of the given column must match the name of one column in the provided table.

func SelectColumns 

func SelectColumns(tbl array.Table, keep ...string) array.Table

SelectColumns returns a Table whose columns are a subset of the columns of the provided table. The retained columns are shallow copies of the source columns.

func TableFromColumns 

func TableFromColumns(cols []array.Column) array.Table

TableFromColumns creates an array.Table from the given columns.

Example 
var sh SliceHelper

col1 := sh.Float32Column([][]float32{{4, 1, 3}, {2, 5}}, nil, "col1")
col2 := sh.Float32Column([][]float32{{0, 1, 2}, {3, 4}}, nil, "col2")
col3 := sh.Float32Column([][]float32{{5, 2, 3}, {4, 1}}, nil, "col3")

tbl := TableFromColumns([]array.Column{*col1, *col2, *col3})
fmt.Printf("%d %d", tbl.NumRows(), tbl.NumCols())

Output:

5 3

func TablesEqual 

func TablesEqual(tbl1, tbl2 array.Table) (bool, string)

TablesEqual returns a boolean indicating whether the two given tables contain equal data. A message describing any differences is also returned.

Types

type ColumnHelper 

type ColumnHelper struct {
	// contains filtered or unexported fields
}

func NewColumnHelper 

func NewColumnHelper(col *array.Column) *ColumnHelper

NewColumnHelper creates a ColumnHelper for the given column. It can be used to extract data from the column as raw slices.

func (*ColumnHelper) Float32Slices 

func (ch *ColumnHelper) Float32Slices() [][]float32

Float32Column returns a slice of slices holding the data from the given column.

func (*ColumnHelper) Float64Slices 

func (ch *ColumnHelper) Float64Slices() [][]float64

Float64Column returns a slice of slices holding the data from the given column.

func (*ColumnHelper) Int16Slices 

func (ch *ColumnHelper) Int16Slices() [][]int16

Int16Column returns a slice of slices holding the data from the given column.

func (*ColumnHelper) Int32Slices 

func (ch *ColumnHelper) Int32Slices() [][]int32

Int32Column returns a slice of slices holding the data from the given column.

func (*ColumnHelper) Int64Slices 

func (ch *ColumnHelper) Int64Slices() [][]int64

Int64Column returns a slice of slices holding the data from the given column.

func (*ColumnHelper) Int8Slices 

func (ch *ColumnHelper) Int8Slices() [][]int8

Int8Column returns a slice of slices holding the data from the given column.

func (*ColumnHelper) Uint16Slices 

func (ch *ColumnHelper) Uint16Slices() [][]uint16

Uint16Column returns a slice of slices holding the data from the given column.

func (*ColumnHelper) Uint32Slices 

func (ch *ColumnHelper) Uint32Slices() [][]uint32

Uint32Column returns a slice of slices holding the data from the given column.

func (*ColumnHelper) Uint64Slices 

func (ch *ColumnHelper) Uint64Slices() [][]uint64

Uint64Column returns a slice of slices holding the data from the given column.

func (*ColumnHelper) Uint8Slices 

func (ch *ColumnHelper) Uint8Slices() [][]uint8

Uint8Column returns a slice of slices holding the data from the given column.

type Option 

type Option func(*reader)

Option provides a configuration option for the CSV reader.

func WithComma 

func WithComma(c rune) Option

WithComma defines a delimiter to the underlying csv reader. If not set, the delimiter is a comma.

func WithComment 

func WithComment(c rune) Option

WithComment provides a comment symbol to the underlying csv reader.

func WithHeader 

func WithHeader() Option

WithHeader indicates that the CSV file has a header row.

type RecordHelper 

type RecordHelper struct {
	// contains filtered or unexported fields
}

func NewRecordHelper 

func NewRecordHelper(rec array.Record) *RecordHelper

NewRecordHelper returns a RecordHelper that can obtain data slices from the given Record.

func (*RecordHelper) Float32Slice 

func (rh *RecordHelper) Float32Slice() []float32

GetFloat32SliceFromRecord returns a slice corresponding to the given column position in the given record.

func (*RecordHelper) Float64Slice 

func (rh *RecordHelper) Float64Slice() []float64

GetFloat64SliceFromRecord returns a slice corresponding to the given column position in the given record.

func (*RecordHelper) Int16Slice 

func (rh *RecordHelper) Int16Slice() []int16

GetInt16SliceFromRecord returns a slice corresponding to the given column position in the given record.

func (*RecordHelper) Int32Slice 

func (rh *RecordHelper) Int32Slice() []int32

GetInt32SliceFromRecord returns a slice corresponding to the given column position in the given record.

func (*RecordHelper) Int64Slice 

func (rh *RecordHelper) Int64Slice() []int64

GetInt64SliceFromRecord returns a slice corresponding to the given column position in the given record.

func (*RecordHelper) Int8Slice 

func (rh *RecordHelper) Int8Slice() []int8

GetInt8SliceFromRecord returns a slice corresponding to the given column position in the given record.

func (*RecordHelper) SetName 

func (rh *RecordHelper) SetName(name string)

SetName sets the column name from which the next SliceFromRecord call will return data.

func (*RecordHelper) SetPos 

func (rh *RecordHelper) SetPos(pos int)

SetPos sets the column position from which the next SliceFromRecord call will return data.

func (*RecordHelper) StringSlice 

func (rh *RecordHelper) StringSlice() []string

func (*RecordHelper) Uint16Slice 

func (rh *RecordHelper) Uint16Slice() []uint16

GetUint16SliceFromRecord returns a slice corresponding to the given column position in the given record.

func (*RecordHelper) Uint32Slice 

func (rh *RecordHelper) Uint32Slice() []uint32

GetUint32SliceFromRecord returns a slice corresponding to the given column position in the given record.

func (*RecordHelper) Uint64Slice 

func (rh *RecordHelper) Uint64Slice() []uint64

GetUint64SliceFromRecord returns a slice corresponding to the given column position in the given record.

func (*RecordHelper) Uint8Slice 

func (rh *RecordHelper) Uint8Slice() []uint8

GetUint8SliceFromRecord returns a slice corresponding to the given column position in the given record.

type SelectFunc 

type SelectFunc func([]array.Interface, []bool)

SelectFunc is a function that uses the provided, pre-allocated boolean array to specify which rows of the data chunk are selected.

type SliceHelper 

type SliceHelper struct {
}

func (*SliceHelper) Float32Column 

func (sh *SliceHelper) Float32Column(x [][]float32, valid [][]bool, name string) *array.Column

ColumnFromFloat32Slices returns a pointer to an array.Column value that holds the given float32 data.

func (*SliceHelper) Float64Column 

func (sh *SliceHelper) Float64Column(x [][]float64, valid [][]bool, name string) *array.Column

ColumnFromFloat64Slices returns a pointer to an array.Column value that holds the given float64 data.

func (*SliceHelper) Int16Column 

func (sh *SliceHelper) Int16Column(x [][]int16, valid [][]bool, name string) *array.Column

ColumnFromInt16Slices returns a pointer to an array.Column value that holds the given int16 data.

func (*SliceHelper) Int32Column 

func (sh *SliceHelper) Int32Column(x [][]int32, valid [][]bool, name string) *array.Column

ColumnFromInt32Slices returns a pointer to an array.Column value that holds the given int32 data.

func (*SliceHelper) Int64Column 

func (sh *SliceHelper) Int64Column(x [][]int64, valid [][]bool, name string) *array.Column

ColumnFromInt64Slices returns a pointer to an array.Column value that holds the given int64 data.

func (*SliceHelper) Int8Column 

func (sh *SliceHelper) Int8Column(x [][]int8, valid [][]bool, name string) *array.Column

ColumnFromInt8Slices returns a pointer to an array.Column value that holds the given int8 data.

func (*SliceHelper) Uint16Column 

func (sh *SliceHelper) Uint16Column(x [][]uint16, valid [][]bool, name string) *array.Column

ColumnFromUint16Slices returns a pointer to an array.Column value that holds the given uint16 data.

func (*SliceHelper) Uint32Column 

func (sh *SliceHelper) Uint32Column(x [][]uint32, valid [][]bool, name string) *array.Column

ColumnFromUint32Slices returns a pointer to an array.Column value that holds the given uint32 data.

func (*SliceHelper) Uint64Column 

func (sh *SliceHelper) Uint64Column(x [][]uint64, valid [][]bool, name string) *array.Column

ColumnFromUint64Slices returns a pointer to an array.Column value that holds the given uint64 data.

func (*SliceHelper) Uint8Column 

func (sh *SliceHelper) Uint8Column(x [][]uint8, valid [][]bool, name string) *array.Column

ColumnFromUint8Slices returns a pointer to an array.Column value that holds the given uint8 data.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.