data

package
v0.0.0-...-040724e Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 6, 2019 License: BSD-3-Clause, GPL-2.0, BSD-3-Clause, + 1 more Imports: 13 Imported by: 2

Documentation

Overview

Package data provides lazy data tables with file format support.

Overview

The aim of this package is to make it possible to efficiently manipulate tabular data with arbitrary schemas in Go (and Antha). It is intended to handle large datasets efficiently by operating lazily where possible.

An example usecase is to load a dataset, project it dynamically to some subset of columns, calculate a new column and write the combined output to a new file.

Tables

The main data type is Table. Typically we will start by reading from a data source such as a Parquet file: 

import 	"github.com/antha-lang/antha/antha/anthalib/data/parquet"
table, err := parquet.TableFromBytes(myBuffer)

Tables are lazy and will not typically load all data until requested to.

We can materialize the Table into a slice of Row objects. This is useful for printing it, so that: 

fmt.Println(table.ToRows())

might print: 

5 Row(s):
| |Quantity|Capacity| label|
| | float64| float64|string|
----------------------------
|0|      11|      30|     A|
|1|      12|      30|     B|
|2|    10.5|      50|     A|
|3|   <nil>|      30|     C|
|4|     5.5|      30|     C|

A Table collects Series values as columns. These are not really useful on their own but their types are exposed via the table Schema, so that: 

fmt.Printf("%+v\n", table.Schema().Columns)

might print: 

[{Name:quantity Type:float64} {Name:label Type:string}]

Each column in a Table has a Go datatype, but may also be nil, even for scalar types such as string. nil is not thre same as zero or NaN.

Data import

In addition to reading tables from Parquet files (see above), there are several other ways to create tables.

It is possible to construct a table "column-wise" by creating Series instances from a Go slice and a nullability mask, like this: 

column1 := data.Must().NewSeriesFromSlice("quantity", []float64{11.0, 12.0, 10.5, 0, 5.5}, []bool{true, true, true, false, true})
column2 := data.Must().NewSeriesFromSlice("label", []string{"A", "B", "A", "C", "C"}, nil)
table := data.NewTable(column1, column2)

If it is needeed to create a table "row-wise" rather than "column-wise", the easiest way is to read it from a slice of structs: 

type myType struct {
	Capacity float64
	Label    string
}
myData := []myType{{30, "A"}, {50, "B"}}
table := data.Must().NewTableFromStructs(myData)

Another way to create a table "row-wise", especially when its schema is not known at compile time, is building a table with a Builder: 

columns := []data.Column{
	{Name: "Id", Type: reflect.TypeOf(0)},
	{Name: "Label", Type: reflect.TypeOf("")},
}
builder := data.Must().NewTableBuilder(columns)
builder.Append([]interface{}{30, "A"})
builder.Append([]interface{}{40, nil})
table := builder.Build()

Data manipulation

Tables can be transformed with methods that return a new Table. Typically the returned Table is lazy. This means that an expression like: 

callback := func(vals ...float64) float64 { return 100 * vals[0] / vals[1] }
table.Extend("quantity_as_percent").On("quantity", "Capacity").Float64(callback)

does not evaluate the callback. It will be evaluated when the quantity_as_percent value is needed by a dependent Table.

Here the value of the Table would be: 

5 Row(s):
| |quantity|Capacity| Label|quantity_as_percent|
| | float64| float64|string|            float64|
------------------------------------------------
|0|      11|      30|     A| 36.666666666666664|
|1|      12|      30|     B|                 40|
|2|    10.5|      50|     A|                 21|
|3|   <nil>|      30|     C|              <nil>|
|4|     5.5|      30|     C| 18.333333333333332|

Sort is a special case, as it is eager: it materializes the entire table.

Other data manipulation methods include Project, Slice, Sort, Filter, Distinct, Pivot, and Join.

Calling Cache on a Table returns a fully materialized copy, which is useful if the data needs to be used for more than one subsequent operation.

Data export

We can write data back to files, such as Parquet, using parquet.TableToBytes.

Also it is possible to populate a slice of structs with a table data: 

type myType struct {
	Capacity float64
	Label    string
}
structs := []myType{}
table.ToStructs(&structs)
fmt.Printf("%+v\n", structs)

This might print: 

[{Capacity:30 Label:A} {Capacity:30 Label:B} {Capacity:50 Label:A} {Capacity:30 Label:C} {Capacity:30 Label:C}]

Or, alternatively, table data can be exported manually using iteration.

Iteration

If necessary the whole table can be processed by a callback: 

for record := range table.IterAll() {
	m, _ := record.Value("Quantity")
	if m.IsNull() {
		fmt.Println("quantity=null at index", record.Index)
	} else {
		fmt.Println("quantity=", m.MustFloat64())
	}
}

which here might print: 

quantity= 11
quantity= 12
quantity= 10.5
quantity=null at index 3
quantity= 5.5

Future features

The main features still to implement include:

Aggregation, such as group by, along with aggregate functions such as average, max, etc.

Relational operations - concat, union, join.

Complete Parquet and CSV read/write support.

There is the possibility of extending this to an event processing dataflow framework (like Apache Flink) for more 'realtime' usecases on unbounded datasets.

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func Eq 

func Eq(expected ...interface{}) (MatchInterface, SchemaAssertion)

Eq retuns a function matching the selected column(s) where equal to expected value(s), after any required type conversion.

Types

type AppendSelection 

type AppendSelection struct {
	// contains filtered or unexported fields
}

func Append 

func Append(t ...*Table) *AppendSelection

Append concatenates several tables vertically. Use the returned object to set append mode.

Example (Exact) 
morePirateBooty, _ := Append(pirateBooty, NewTable(
	Must().NewSeriesFromSlice("Name", []string{"piastre"}, nil),
	Must().NewSeriesFromSlice("Price", []float64{1.5}, nil),
	Must().NewSeriesFromSlice("Quantity", []int64{1000}, nil),
), NewTable(
	Must().NewSeriesFromSlice("Name", []string{"guinea"}, nil),
	Must().NewSeriesFromSlice("Price", []float64{0.5}, nil),
	Must().NewSeriesFromSlice("Quantity", []int64{100}, nil),
)).Exact()
fmt.Println(morePirateBooty.ToRows())

Output:

6 Row(s):
| |    Name|  Price|Quantity|
| |  string|float64|   int64|
-----------------------------
|0|doubloon|      1|    1200|
|1|    grog|  <nil>|      44|
|2| cutlass|    5.5|      30|
|3|   chest|    600|       2|
|4| piastre|    1.5|    1000|
|5|  guinea|    0.5|     100|
Example (Inner) 
morePirateBootyNames, _ := Append(pirateBooty, NewTable(
	Must().NewSeriesFromSlice("Name", []string{"piastre"}, nil),
	Must().NewSeriesFromSlice("Price", []float64{1.5}, nil),
), NewTable(
	Must().NewSeriesFromSlice("Name", []string{"guinea"}, nil),
)).Inner()
fmt.Println(morePirateBootyNames.ToRows())

Output:

6 Row(s):
| |    Name|
| |  string|
------------
|0|doubloon|
|1|    grog|
|2| cutlass|
|3|   chest|
|4| piastre|
|5|  guinea|
Example (Outer) 
morePirateBooty, _ := Append(pirateBooty, NewTable(
	Must().NewSeriesFromSlice("Name", []string{"piastre"}, nil),
	Must().NewSeriesFromSlice("Price", []float64{1.5}, nil),
), NewTable(
	Must().NewSeriesFromSlice("Name", []string{"guinea"}, nil),
)).Outer()
fmt.Println(morePirateBooty.ToRows())

Output:

6 Row(s):
| |    Name|  Price|Quantity|
| |  string|float64|   int64|
-----------------------------
|0|doubloon|      1|    1200|
|1|    grog|  <nil>|      44|
|2| cutlass|    5.5|      30|
|3|   chest|    600|       2|
|4| piastre|    1.5|   <nil>|
|5|  guinea|  <nil>|   <nil>|
Example (Positional) 
morePirateBooty, _ := Append(pirateBooty, NewTable(
	Must().NewSeriesFromSlice("Name", []string{"piastre"}, nil),
	Must().NewSeriesFromSlice("Price", []float64{1.5}, nil),
	Must().NewSeriesFromSlice("Quantity", []int64{1000}, nil),
), NewTable(
	Must().NewSeriesFromSlice("Name", []string{"guinea"}, nil),
	Must().NewSeriesFromSlice("PriceModified", []float64{0.5}, nil),
	Must().NewSeriesFromSlice("Quantity", []int64{100}, nil),
)).Positional()
fmt.Println(morePirateBooty.ToRows())

Output:

6 Row(s):
| |    Name|  Price|Quantity|
| |  string|float64|   int64|
-----------------------------
|0|doubloon|      1|    1200|
|1|    grog|  <nil>|      44|
|2| cutlass|    5.5|      30|
|3|   chest|    600|       2|
|4| piastre|    1.5|    1000|
|5|  guinea|    0.5|     100|

func (*AppendSelection) Exact 

func (s *AppendSelection) Exact() (*Table, error)

Exact append: appends tables vertically, matches their columns by names, types and ordinals. All the source tables must have the same schemas (except the columns order).

func (*AppendSelection) Inner 

func (s *AppendSelection) Inner() (*Table, error)

Inner append: appends tables vertically, matches their columns by names, types and ordinals. The resulting table contains intersecting columns only. Returns an error only when called with an empty tables list.

func (*AppendSelection) Outer 

func (s *AppendSelection) Outer() (*Table, error)

Outer append: appends tables vertically, matches their columns by names, types and ordinals. The resulting table contains union of columns. Returns an error only when called with an empty tables list.

func (*AppendSelection) Positional 

func (s *AppendSelection) Positional() (*Table, error)

Positional append: appends tables vertically, matches their columns by positions. All the source tables must have the same schemas (except the columns names).

type Column 

type Column struct {
	Name ColumnName
	Type reflect.Type
}

Column is Series metadata

func (*Column) MarshalJSON 

func (c *Column) MarshalJSON() ([]byte, error)

MarshalJSON encodes type info only for well-known types.

func (*Column) UnmarshalJSON 

func (c *Column) UnmarshalJSON(bytes []byte) error

UnmarshalJSON decodes type info only for well-known types.

type ColumnKey 

type ColumnKey struct {
	Column ColumnName
	Asc    bool
}

ColumnKey defines ordering on a single column

func (ColumnKey) Equal 

func (ck ColumnKey) Equal(other ColumnKey) bool

Equal checks if two key entries are equal

type ColumnName 

type ColumnName string

ColumnName is a type for column names in schema

type DistinctSelection 

type DistinctSelection struct {
	// contains filtered or unexported fields
}

DistinctSelection contains a table to apply distinct.

func (*DistinctSelection) On 

func (ds *DistinctSelection) On(cols ...ColumnName) (*Table, error)

On selects distinct rows by columns specified.

type ExtendOn 

type ExtendOn struct {
	// contains filtered or unexported fields
}

ExtendOn enables extensions using specific column values as function inputs

func (*ExtendOn) Bool 

func (e *ExtendOn) Bool(f func(v ...bool) (bool, bool)) (*Table, error)

Bool adds a bool col using bool inputs. Null on any null inputs. Returns error if any column cannot be assigned to bool; no conversions are performed.

func (*ExtendOn) Float64 

func (e *ExtendOn) Float64(f func(v ...float64) (float64, bool)) (*Table, error)

Float64 adds a float64 col using float64 inputs. Null on any null inputs. Returns error if any column cannot be assigned to float64; no conversions are performed.

func (*ExtendOn) Int 

func (e *ExtendOn) Int(f func(v ...int) (int, bool)) (*Table, error)

Int adds a int col using int inputs. Null on any null inputs. Returns error if any column cannot be assigned to int; no conversions are performed.

func (*ExtendOn) Int64 

func (e *ExtendOn) Int64(f func(v ...int64) (int64, bool)) (*Table, error)

Int64 adds a int64 col using int64 inputs. Null on any null inputs. Returns error if any column cannot be assigned to int64; no conversions are performed.

func (*ExtendOn) Interface 

func (e *ExtendOn) Interface(f func(v ...interface{}) interface{}, newType reflect.Type) (*Table, error)

Interface adds a column of an arbitrary type using inputs of arbitrary types.

func (*ExtendOn) InterfaceBool 

func (e *ExtendOn) InterfaceBool(f func(v ...interface{}) (bool, bool)) (*Table, error)

InterfaceBool adds a bool col using arbitrary (interface{}) inputs.

func (*ExtendOn) InterfaceFloat64 

func (e *ExtendOn) InterfaceFloat64(f func(v ...interface{}) (float64, bool)) (*Table, error)

InterfaceFloat64 adds a float64 col using arbitrary (interface{}) inputs.

func (*ExtendOn) InterfaceInt 

func (e *ExtendOn) InterfaceInt(f func(v ...interface{}) (int, bool)) (*Table, error)

InterfaceInt adds a int col using arbitrary (interface{}) inputs.

func (*ExtendOn) InterfaceInt64 

func (e *ExtendOn) InterfaceInt64(f func(v ...interface{}) (int64, bool)) (*Table, error)

InterfaceInt64 adds a int64 col using arbitrary (interface{}) inputs.

func (*ExtendOn) InterfaceString 

func (e *ExtendOn) InterfaceString(f func(v ...interface{}) (string, bool)) (*Table, error)

InterfaceString adds a string col using arbitrary (interface{}) inputs.

func (*ExtendOn) InterfaceTimestampMicros 

func (e *ExtendOn) InterfaceTimestampMicros(f func(v ...interface{}) (TimestampMicros, bool)) (*Table, error)

InterfaceTimestampMicros adds a TimestampMicros col using arbitrary (interface{}) inputs.

func (*ExtendOn) InterfaceTimestampMillis 

func (e *ExtendOn) InterfaceTimestampMillis(f func(v ...interface{}) (TimestampMillis, bool)) (*Table, error)

InterfaceTimestampMillis adds a TimestampMillis col using arbitrary (interface{}) inputs.

func (*ExtendOn) String 

func (e *ExtendOn) String(f func(v ...string) (string, bool)) (*Table, error)

String adds a string col using string inputs. Null on any null inputs. Returns error if any column cannot be assigned to string; no conversions are performed.

func (*ExtendOn) TimestampMicros 

func (e *ExtendOn) TimestampMicros(f func(v ...TimestampMicros) (TimestampMicros, bool)) (*Table, error)

TimestampMicros adds a TimestampMicros col using TimestampMicros inputs. Null on any null inputs. Returns error if any column cannot be assigned to TimestampMicros; no conversions are performed.

func (*ExtendOn) TimestampMillis 

func (e *ExtendOn) TimestampMillis(f func(v ...TimestampMillis) (TimestampMillis, bool)) (*Table, error)

TimestampMillis adds a TimestampMillis col using TimestampMillis inputs. Null on any null inputs. Returns error if any column cannot be assigned to TimestampMillis; no conversions are performed.

type Extension 

type Extension struct {
	// contains filtered or unexported fields
}

Extension is the fluent interface for adding calculated columns.

func (*Extension) By 

func (e *Extension) By(f func(r Row) interface{}, newType reflect.Type) *Table

By performs extension using the whole table row as input. For wide tables this might be inefficient, consider using On(...) instead.

func (*Extension) Constant 

func (e *Extension) Constant(value interface{}) *Table

Constant adds a constant column with the given value to the table. This column has the dynamic type of the given value.

func (*Extension) ConstantType 

func (e *Extension) ConstantType(value interface{}, typ reflect.Type) (*Table, error)

ConstantType adds a constant column with the given value to the table. This column has the given type (boxing nil for example). Returns error if a non-nil value cannot be converted to the required type.

func (*Extension) On 

func (e *Extension) On(cols ...ColumnName) *ExtendOn

On selects a subset of columns to use as an extension source. If duplicate columns exist, the first so named is used. Note this does not panic yet, even if the columns do not exist. (However subsequent calls to the returned object will error.)

type FilterOn 

type FilterOn struct {
	// contains filtered or unexported fields
}

FilterOn filters by named columns. TODO(twoodwark): this is needlessly eager with respect to all underlying columns.

func (*FilterOn) Bool 

func (o *FilterOn) Bool(fn MatchBool, assertions ...SchemaAssertion) (*Table, error)

Bool matches the named column values as bool arguments. If any column is nil the filter is automatically false. If given any SchemaAssertions, they are called now and may have side effects.

func (*FilterOn) Float64 

func (o *FilterOn) Float64(fn MatchFloat64, assertions ...SchemaAssertion) (*Table, error)

Float64 matches the named column values as float64 arguments. If any column is nil the filter is automatically false. If given any SchemaAssertions, they are called now and may have side effects.

func (*FilterOn) Int 

func (o *FilterOn) Int(fn MatchInt, assertions ...SchemaAssertion) (*Table, error)

Int matches the named column values as int arguments. If any column is nil the filter is automatically false. If given any SchemaAssertions, they are called now and may have side effects.

func (*FilterOn) Int64 

func (o *FilterOn) Int64(fn MatchInt64, assertions ...SchemaAssertion) (*Table, error)

Int64 matches the named column values as int64 arguments. If any column is nil the filter is automatically false. If given any SchemaAssertions, they are called now and may have side effects.

func (*FilterOn) Interface 

func (o *FilterOn) Interface(fn MatchInterface, assertions ...SchemaAssertion) (*Table, error)

Interface matches the named column values as interface{} arguments, including nil. If given any SchemaAssertions, they are called now and may have side effects.

func (*FilterOn) String 

func (o *FilterOn) String(fn MatchString, assertions ...SchemaAssertion) (*Table, error)

String matches the named column values as string arguments. If any column is nil the filter is automatically false. If given any SchemaAssertions, they are called now and may have side effects.

func (*FilterOn) TimestampMicros 

func (o *FilterOn) TimestampMicros(fn MatchTimestampMicros, assertions ...SchemaAssertion) (*Table, error)

TimestampMicros matches the named column values as TimestampMicros arguments. If any column is nil the filter is automatically false. If given any SchemaAssertions, they are called now and may have side effects.

func (*FilterOn) TimestampMillis 

func (o *FilterOn) TimestampMillis(fn MatchTimestampMillis, assertions ...SchemaAssertion) (*Table, error)

TimestampMillis matches the named column values as TimestampMillis arguments. If any column is nil the filter is automatically false. If given any SchemaAssertions, they are called now and may have side effects.

type FilterSelection 

type FilterSelection struct {
	// contains filtered or unexported fields
}

FilterSelection is the fluent interface for filtering rows.

func (*FilterSelection) By 

func (fs *FilterSelection) By(fn MatchRow) *Table

By accepts a filter function that operates on rows of the whole table. The filtered table contains rows where this function returns true.

func (*FilterSelection) On 

func (fs *FilterSelection) On(cols ...ColumnName) *FilterOn

On selects columns for filtering. Note this does not panic yet, even if the columns do not exist. (However subsequent calls to the returned object will error.)

type ForeachKeySelection 

type ForeachKeySelection struct {
	// contains filtered or unexported fields
}

ForeachKeySelection encapsulates a table to iterate with ForeachKey and a key to iterate with.

func (*ForeachKeySelection) By 

func (fks *ForeachKeySelection) By(fn func(key Row, partition *Table)) error

By performs ForeachKey on the whole table rows. For wide tables this might be inefficient, consider using Project before By.

type ForeachOn 

type ForeachOn struct {
	// contains filtered or unexported fields
}

ForeachOn encapsulated the table and the columns to iterate.

func (*ForeachOn) Bool 

func (o *ForeachOn) Bool(fn func(...bool), assertions ...SchemaAssertion) error

Bool invokes a user-supplied function passing the named column values as bool arguments. An error is returned and execution is aborted at the first null value encountered. If given any SchemaAssertions, they are called in the beginning and may have side effects.

func (*ForeachOn) Float64 

func (o *ForeachOn) Float64(fn func(...float64), assertions ...SchemaAssertion) error

Float64 invokes a user-supplied function passing the named column values as float64 arguments. An error is returned and execution is aborted at the first null value encountered. If given any SchemaAssertions, they are called in the beginning and may have side effects.

func (*ForeachOn) Int 

func (o *ForeachOn) Int(fn func(...int), assertions ...SchemaAssertion) error

Int invokes a user-supplied function passing the named column values as int arguments. An error is returned and execution is aborted at the first null value encountered. If given any SchemaAssertions, they are called in the beginning and may have side effects.

func (*ForeachOn) Int64 

func (o *ForeachOn) Int64(fn func(...int64), assertions ...SchemaAssertion) error

Int64 invokes a user-supplied function passing the named column values as int64 arguments. An error is returned and execution is aborted at the first null value encountered. If given any SchemaAssertions, they are called in the beginning and may have side effects.

func (*ForeachOn) Interface 

func (o *ForeachOn) Interface(fn func(v ...interface{}), assertions ...SchemaAssertion) error

Interface invokes a user-supplied function passing the named column values as interface{} arguments, including nil. If given any SchemaAssertions, they are called in the beginning and may have side effects.

func (*ForeachOn) String 

func (o *ForeachOn) String(fn func(...string), assertions ...SchemaAssertion) error

String invokes a user-supplied function passing the named column values as string arguments. An error is returned and execution is aborted at the first null value encountered. If given any SchemaAssertions, they are called in the beginning and may have side effects.

func (*ForeachOn) TimestampMicros 

func (o *ForeachOn) TimestampMicros(fn func(...TimestampMicros), assertions ...SchemaAssertion) error

TimestampMicros invokes a user-supplied function passing the named column values as TimestampMicros arguments. An error is returned and execution is aborted at the first null value encountered. If given any SchemaAssertions, they are called in the beginning and may have side effects.

func (*ForeachOn) TimestampMillis 

func (o *ForeachOn) TimestampMillis(fn func(...TimestampMillis), assertions ...SchemaAssertion) error

TimestampMillis invokes a user-supplied function passing the named column values as TimestampMillis arguments. An error is returned and execution is aborted at the first null value encountered. If given any SchemaAssertions, they are called in the beginning and may have side effects.

type ForeachSelection 

type ForeachSelection struct {
	// contains filtered or unexported fields
}

ForeachSelection encapsulates the table to iterate with Foreach.

func (*ForeachSelection) By 

func (fs *ForeachSelection) By(fn func(r Row))

By performs Foreach on the whole table rows. For wide tables this might be inefficient, consider using On(...) instead.

func (*ForeachSelection) On 

func (fs *ForeachSelection) On(cols ...ColumnName) *ForeachOn

On selects columns for iterating on. Note this does not panic yet, even if the columns do not exist. (However subsequent calls to the returned object will error.)

type Index 

type Index int

Index is a type for rows numbers

type JoinOn 

type JoinOn struct {
	// contains filtered or unexported fields
}

JoinOn contains a left table and its columns to apply join.

func (*JoinOn) Inner 

func (jo *JoinOn) Inner(t *Table, cols ...ColumnName) (*Table, error)

Inner sets a right table and columns for inner join and performs the join itself.

func (*JoinOn) LeftOuter 

func (jo *JoinOn) LeftOuter(t *Table, cols ...ColumnName) (*Table, error)

LeftOuter sets a right table and columns for left outer join and performs the join itself.

type JoinSelection 

type JoinSelection struct {
	// contains filtered or unexported fields
}

JoinSelection contains a left table to apply join.

func (*JoinSelection) NaturalInner 

func (js *JoinSelection) NaturalInner(t *Table) (*Table, error)

NaturalInner performs inner join between js.t and t by columns having the same names.

func (*JoinSelection) NaturalLeftOuter 

func (js *JoinSelection) NaturalLeftOuter(t *Table) (*Table, error)

NaturalLeftOuter performs left outer join between js.t and t by columns having the same names.

func (*JoinSelection) On 

func (js *JoinSelection) On(cols ...ColumnName) *JoinOn

On sets left table columns for a join.

type Key 

type Key []ColumnKey

Key defines sorting columns (and directions)

func (Key) Equal 

func (key Key) Equal(other Key) bool

Equal checks if two keys are equal

func (Key) HasPrefix 

func (key Key) HasPrefix(other Key) bool

HasPrefix checks if other key is a prefix of k

type MatchBool 

type MatchBool func(...bool) bool

MatchBool implements a filter on bool columns.

type MatchFloat64 

type MatchFloat64 func(...float64) bool

MatchFloat64 implements a filter on float64 columns.

type MatchInt 

type MatchInt func(...int) bool

MatchInt implements a filter on int columns.

type MatchInt64 

type MatchInt64 func(...int64) bool

MatchInt64 implements a filter on int64 columns.

type MatchInterface 

type MatchInterface func(...interface{}) bool

MatchInterface implements a filter on interface{} columns. Note that this can receive nil values.

type MatchRow 

type MatchRow func(r Row) bool

MatchRow implements a filter on entire table rows.

type MatchString 

type MatchString func(...string) bool

MatchString implements a filter on string columns.

type MatchTimestampMicros 

type MatchTimestampMicros func(...TimestampMicros) bool

MatchTimestampMicros implements a filter on TimestampMicros columns.

type MatchTimestampMillis 

type MatchTimestampMillis func(...TimestampMillis) bool

MatchTimestampMillis implements a filter on TimestampMillis columns.

type MustAppendSelection 

type MustAppendSelection struct {
	*AppendSelection
}

MustAppendSelection is a proxy for AppendSelection.

func (*MustAppendSelection) Exact 

func (s *MustAppendSelection) Exact() *Table

Exact append: appends tables vertically, matches their columns by names, types and ordinals. All the source tables must have the same schemas (except the columns order).

func (*MustAppendSelection) Inner 

func (s *MustAppendSelection) Inner() *Table

Inner append: appends tables vertically, matches their columns by names, types and ordinals. The resulting table contains intersecting columns only.

func (*MustAppendSelection) Outer 

func (s *MustAppendSelection) Outer() *Table

Outer append: appends tables vertically, matches their columns by names, types and ordinals. The resulting table contains union of columns.

func (*MustAppendSelection) Positional 

func (s *MustAppendSelection) Positional() *Table

Positional append: appends tables vertically, matches their columns by positions. All the source tables must have the same schemas (except the columns names).

type MustDistinctSelection 

type MustDistinctSelection struct {
	*DistinctSelection
}

MustDistinctSelection panics on any error when creating derived tables.

func (*MustDistinctSelection) On 

func (s *MustDistinctSelection) On(cols ...ColumnName) *Table

On selects distinct rows by columns specified. Panics on errors.

type MustExtendOn 

type MustExtendOn struct {
	*ExtendOn
}

MustExtendOn panics on any error when creating derived tables.

func (*MustExtendOn) Bool 

func (m *MustExtendOn) Bool(f func(v ...bool) (bool, bool)) *Table

Bool adds a bool col using bool inputs. Null on any null inputs. Panics on error.

func (*MustExtendOn) Float64 

func (m *MustExtendOn) Float64(f func(v ...float64) (float64, bool)) *Table

Float64 adds a float64 col using float64 inputs. Null on any null inputs. Panics on error.

func (*MustExtendOn) Int 

func (m *MustExtendOn) Int(f func(v ...int) (int, bool)) *Table

Int adds a int col using int inputs. Null on any null inputs. Panics on error.

func (*MustExtendOn) Int64 

func (m *MustExtendOn) Int64(f func(v ...int64) (int64, bool)) *Table

Int64 adds a int64 col using int64 inputs. Null on any null inputs. Panics on error.

func (*MustExtendOn) Interface 

func (on *MustExtendOn) Interface(f func(v ...interface{}) interface{}, newType reflect.Type) *Table

Interface panics on any error when creating derived tables.

func (*MustExtendOn) InterfaceBool 

func (m *MustExtendOn) InterfaceBool(f func(v ...interface{}) (bool, bool)) *Table

InterfaceBool adds a bool col using arbitrary (interface{}) inputs. Panics on error.

func (*MustExtendOn) InterfaceFloat64 

func (m *MustExtendOn) InterfaceFloat64(f func(v ...interface{}) (float64, bool)) *Table

InterfaceFloat64 adds a float64 col using arbitrary (interface{}) inputs. Panics on error.

func (*MustExtendOn) InterfaceInt 

func (m *MustExtendOn) InterfaceInt(f func(v ...interface{}) (int, bool)) *Table

InterfaceInt adds a int col using arbitrary (interface{}) inputs. Panics on error.

func (*MustExtendOn) InterfaceInt64 

func (m *MustExtendOn) InterfaceInt64(f func(v ...interface{}) (int64, bool)) *Table

InterfaceInt64 adds a int64 col using arbitrary (interface{}) inputs. Panics on error.

func (*MustExtendOn) InterfaceString 

func (m *MustExtendOn) InterfaceString(f func(v ...interface{}) (string, bool)) *Table

InterfaceString adds a string col using arbitrary (interface{}) inputs. Panics on error.

func (*MustExtendOn) InterfaceTimestampMicros 

func (m *MustExtendOn) InterfaceTimestampMicros(f func(v ...interface{}) (TimestampMicros, bool)) *Table

InterfaceTimestampMicros adds a TimestampMicros col using arbitrary (interface{}) inputs. Panics on error.

func (*MustExtendOn) InterfaceTimestampMillis 

func (m *MustExtendOn) InterfaceTimestampMillis(f func(v ...interface{}) (TimestampMillis, bool)) *Table

InterfaceTimestampMillis adds a TimestampMillis col using arbitrary (interface{}) inputs. Panics on error.

func (*MustExtendOn) String 

func (m *MustExtendOn) String(f func(v ...string) (string, bool)) *Table

String adds a string col using string inputs. Null on any null inputs. Panics on error.

func (*MustExtendOn) TimestampMicros 

func (m *MustExtendOn) TimestampMicros(f func(v ...TimestampMicros) (TimestampMicros, bool)) *Table

TimestampMicros adds a TimestampMicros col using TimestampMicros inputs. Null on any null inputs. Panics on error.

func (*MustExtendOn) TimestampMillis 

func (m *MustExtendOn) TimestampMillis(f func(v ...TimestampMillis) (TimestampMillis, bool)) *Table

TimestampMillis adds a TimestampMillis col using TimestampMillis inputs. Null on any null inputs. Panics on error.

type MustExtension 

type MustExtension struct {
	*Extension
}

MustExtension panics on any error when creating derived tables.

func (*MustExtension) ConstantType 

func (e *MustExtension) ConstantType(value interface{}, typ reflect.Type) *Table

ConstantType panics unless *Extension.ConstantType.

func (*MustExtension) On 

func (e *MustExtension) On(cols ...ColumnName) *MustExtendOn

On returns a proxy for *Extension.On.

type MustFilterOn 

type MustFilterOn struct {
	*FilterOn
}

MustFilterOn panics on any error when creating derived tables.

func (*MustFilterOn) Bool 

func (o *MustFilterOn) Bool(m MatchBool, assertions ...SchemaAssertion) *Table

Bool matches the named column values as bool arguments.

func (*MustFilterOn) Float64 

func (o *MustFilterOn) Float64(m MatchFloat64, assertions ...SchemaAssertion) *Table

Float64 matches the named column values as float64 arguments.

func (*MustFilterOn) Int 

func (o *MustFilterOn) Int(m MatchInt, assertions ...SchemaAssertion) *Table

Int matches the named column values as int arguments.

func (*MustFilterOn) Int64 

func (o *MustFilterOn) Int64(m MatchInt64, assertions ...SchemaAssertion) *Table

Int64 matches the named column values as int64 arguments.

func (*MustFilterOn) Interface 

func (o *MustFilterOn) Interface(m MatchInterface, assertions ...SchemaAssertion) *Table

Interface matches the named column values as interface{} arguments.

func (*MustFilterOn) String 

func (o *MustFilterOn) String(m MatchString, assertions ...SchemaAssertion) *Table

String matches the named column values as string arguments.

func (*MustFilterOn) TimestampMicros 

func (o *MustFilterOn) TimestampMicros(m MatchTimestampMicros, assertions ...SchemaAssertion) *Table

TimestampMicros matches the named column values as TimestampMicros arguments.

func (*MustFilterOn) TimestampMillis 

func (o *MustFilterOn) TimestampMillis(m MatchTimestampMillis, assertions ...SchemaAssertion) *Table

TimestampMillis matches the named column values as TimestampMillis arguments.

type MustFilterSelection 

type MustFilterSelection struct {
	*FilterSelection
}

MustFilterSelection panics on any error when creating derived tables.

func (*MustFilterSelection) On 

func (s *MustFilterSelection) On(cols ...ColumnName) *MustFilterOn

On returns a proxy for *FilterSelection.On.

type MustForeachKeySelection 

type MustForeachKeySelection struct {
	// contains filtered or unexported fields
}

MustForeachKeySelection is a proxy for ForeachKeySelection.

func (*MustForeachKeySelection) By 

func (fks *MustForeachKeySelection) By(fn func(key Row, data *Table))

By performs ForeachKey on the whole table rows. For wide tables this might be inefficient, consider using Project before By.

type MustForeachOn 

type MustForeachOn struct {
	// contains filtered or unexported fields
}

MustForeachOn is a proxy for ForeachOn.

func (*MustForeachOn) Bool 

func (on *MustForeachOn) Bool(fn func(...bool), assertions ...SchemaAssertion)

Bool invokes a user-supplied function passing the named column values as bool arguments. If a null value is encountered, the function fails. If given any SchemaAssertions, they are called in the beginning and may have side effects.

func (*MustForeachOn) Float64 

func (on *MustForeachOn) Float64(fn func(...float64), assertions ...SchemaAssertion)

Float64 invokes a user-supplied function passing the named column values as float64 arguments. If a null value is encountered, the function fails. If given any SchemaAssertions, they are called in the beginning and may have side effects.

func (*MustForeachOn) Int 

func (on *MustForeachOn) Int(fn func(...int), assertions ...SchemaAssertion)

Int invokes a user-supplied function passing the named column values as int arguments. If a null value is encountered, the function fails. If given any SchemaAssertions, they are called in the beginning and may have side effects.

func (*MustForeachOn) Int64 

func (on *MustForeachOn) Int64(fn func(...int64), assertions ...SchemaAssertion)

Int64 invokes a user-supplied function passing the named column values as int64 arguments. If a null value is encountered, the function fails. If given any SchemaAssertions, they are called in the beginning and may have side effects.

func (*MustForeachOn) Interface 

func (on *MustForeachOn) Interface(fn func(v ...interface{}), assertions ...SchemaAssertion)

Interface invokes a user-supplied function passing the named column values as interface{} arguments, including nil. If given any SchemaAssertions, they are called in the beginning and may have side effects.

func (*MustForeachOn) String 

func (on *MustForeachOn) String(fn func(...string), assertions ...SchemaAssertion)

String invokes a user-supplied function passing the named column values as string arguments. If a null value is encountered, the function fails. If given any SchemaAssertions, they are called in the beginning and may have side effects.

func (*MustForeachOn) TimestampMicros 

func (on *MustForeachOn) TimestampMicros(fn func(...TimestampMicros), assertions ...SchemaAssertion)

TimestampMicros invokes a user-supplied function passing the named column values as TimestampMicros arguments. If a null value is encountered, the function fails. If given any SchemaAssertions, they are called in the beginning and may have side effects.

func (*MustForeachOn) TimestampMillis 

func (on *MustForeachOn) TimestampMillis(fn func(...TimestampMillis), assertions ...SchemaAssertion)

TimestampMillis invokes a user-supplied function passing the named column values as TimestampMillis arguments. If a null value is encountered, the function fails. If given any SchemaAssertions, they are called in the beginning and may have side effects.

type MustForeachSelection 

type MustForeachSelection struct {
	// contains filtered or unexported fields
}

MustForeachSelection is a proxy for ForeachSelection.

func (*MustForeachSelection) By 

func (fs *MustForeachSelection) By(fn func(r Row))

By performs Foreach on the whole table rows. For wide tables this might be inefficient, consider using On(...) instead.

func (*MustForeachSelection) On 

func (fs *MustForeachSelection) On(cols ...ColumnName) *MustForeachOn

On selects columns for iterating on.

type MustGlobal 

type MustGlobal struct{}

MustGlobal panics on any error when calling global functions.

func Must 

func Must() MustGlobal

Must returns a proxy object that asserts errors are nil.

func (MustGlobal) Append 

func (m MustGlobal) Append(tables ...*Table) *MustAppendSelection

Append returns a proxy for (global) Append.

func (MustGlobal) NewSeriesFromSlice 

func (m MustGlobal) NewSeriesFromSlice(col ColumnName, values interface{}, notNull []bool) *Series

NewSeriesFromSlice is a errors handling wrapper for NewSeriesFromSlice.

func (MustGlobal) NewTableBuilder 

func (m MustGlobal) NewTableBuilder(columns []Column) *TableBuilder

NewTableBuilder is a errors handling wrapper for NewTableBuilder.

func (MustGlobal) NewTableFromStructs 

func (m MustGlobal) NewTableFromStructs(structs interface{}) *Table

NewTableFromStructs is a errors handling wrapper for NewTableFromStructs.

type MustJoinOn 

type MustJoinOn struct {
	// contains filtered or unexported fields
}

MustJoinOn is a proxy for JoinOn.

func (*MustJoinOn) Inner 

func (jo *MustJoinOn) Inner(t *Table, cols ...ColumnName) *Table

Inner sets a right table and columns for inner join and performs the join itself.

func (*MustJoinOn) LeftOuter 

func (jo *MustJoinOn) LeftOuter(t *Table, cols ...ColumnName) *Table

LeftOuter sets a right table and columns for left outer join and performs the join itself.

type MustJoinSelection 

type MustJoinSelection struct {
	// contains filtered or unexported fields
}

MustJoinSelection is a proxy for JoinSelection.

func (*MustJoinSelection) NaturalInner 

func (js *MustJoinSelection) NaturalInner(t *Table) *Table

NaturalInner performs inner join between js.t and t by columns having the same names.

func (*MustJoinSelection) NaturalLeftOuter 

func (js *MustJoinSelection) NaturalLeftOuter(t *Table) *Table

NaturalLeftOuter performs left outer join between js.t and t by columns having the same names.

func (*MustJoinSelection) On 

func (js *MustJoinSelection) On(cols ...ColumnName) *MustJoinOn

On sets left table columns for a join.

type MustPivotKey 

type MustPivotKey struct {
	*PivotKey
}

MustPivotKey panics on any error when creating derived tables.

func (*MustPivotKey) Columns 

func (mpk *MustPivotKey) Columns(pivot ColumnName, value ColumnName) *Table

Columns panics on any error when creating derived tables.

type MustPivotSelection 

type MustPivotSelection struct {
	*PivotSelection
}

MustPivotSelection is a proxy for PivotSelection.

func (*MustPivotSelection) Key 

func (mps *MustPivotSelection) Key(key ...ColumnName) *MustPivotKey

Key returns a proxy for PivotKey.

type MustTable 

type MustTable struct {
	*Table
}

MustTable panics on any error when creating derived tables.

func (MustTable) Append 

func (m MustTable) Append(other *Table) *MustAppendSelection

Append returns a proxy for *Table.Append.

func (MustTable) Cache 

func (m MustTable) Cache() *Table

Cache panics unless *Table.Cache.

func (MustTable) Convert 

func (m MustTable) Convert(col ColumnName, typ reflect.Type) *Table

Convert panics unless *Table.Convert.

func (MustTable) Distinct 

func (m MustTable) Distinct() *MustDistinctSelection

Distinct returns a proxy for *Table.Distinct.

func (MustTable) Extend 

func (m MustTable) Extend(newCol ColumnName) *MustExtension

Extend returns a proxy for *Table.Extend.

func (MustTable) Filter 

func (m MustTable) Filter() *MustFilterSelection

Filter returns a proxy for *Table.Filter.

func (MustTable) Foreach 

func (m MustTable) Foreach() *MustForeachSelection

Foreach returns a proxy for *Table.Foreach.

func (MustTable) ForeachKey 

func (m MustTable) ForeachKey(key ...ColumnName) *MustForeachKeySelection

ForeachKey returns a proxy for *Table.ForeachKey.

func (MustTable) Join 

func (m MustTable) Join() *MustJoinSelection

Join returns a proxy for *Table.Join.

func (MustTable) Pivot 

func (m MustTable) Pivot() *MustPivotSelection

Pivot returns a proxy for *Table.Pivot.

func (MustTable) Project 

func (m MustTable) Project(columns ...ColumnName) *Table

Project panics unless *Table.Project.

func (MustTable) Sort 

func (m MustTable) Sort(key Key) *Table

Sort panics unless *Table.Sort.

func (MustTable) SortByFunc 

func (m MustTable) SortByFunc(f SortFunc) *Table

SortByFunc panics unless *Table.SortByFunc.

func (MustTable) Update 

func (m MustTable) Update(col ColumnName) *MustUpdateSelection

Update returns a proxy for *Table.Update.

type MustUpdateOn 

type MustUpdateOn struct {
	*UpdateOn
}

MustUpdateOn panics on any error when creating derived tables.

func (*MustUpdateOn) Bool 

func (on *MustUpdateOn) Bool(fn func(v ...bool) (bool, bool)) *Table

Bool updates a bool col using bool inputs. Null on any null inputs. Returns error if any column cannot be assigned to bool; no conversions are performed.

func (*MustUpdateOn) Float64 

func (on *MustUpdateOn) Float64(fn func(v ...float64) (float64, bool)) *Table

Float64 updates a float64 col using float64 inputs. Null on any null inputs. Returns error if any column cannot be assigned to float64; no conversions are performed.

func (*MustUpdateOn) Int 

func (on *MustUpdateOn) Int(fn func(v ...int) (int, bool)) *Table

Int updates a int col using int inputs. Null on any null inputs. Returns error if any column cannot be assigned to int; no conversions are performed.

func (*MustUpdateOn) Int64 

func (on *MustUpdateOn) Int64(fn func(v ...int64) (int64, bool)) *Table

Int64 updates a int64 col using int64 inputs. Null on any null inputs. Returns error if any column cannot be assigned to int64; no conversions are performed.

func (*MustUpdateOn) Interface 

func (on *MustUpdateOn) Interface(fn func(v ...interface{}) interface{}) *Table

Interface updates a column of an arbitrary type using a subset of source columns of arbitrary types.

func (*MustUpdateOn) InterfaceBool 

func (on *MustUpdateOn) InterfaceBool(fn func(v ...interface{}) (bool, bool)) *Table

InterfaceBool updates a bool col using interface{} inputs.

func (*MustUpdateOn) InterfaceFloat64 

func (on *MustUpdateOn) InterfaceFloat64(fn func(v ...interface{}) (float64, bool)) *Table

InterfaceFloat64 updates a float64 col using interface{} inputs.

func (*MustUpdateOn) InterfaceInt 

func (on *MustUpdateOn) InterfaceInt(fn func(v ...interface{}) (int, bool)) *Table

InterfaceInt updates a int col using interface{} inputs.

func (*MustUpdateOn) InterfaceInt64 

func (on *MustUpdateOn) InterfaceInt64(fn func(v ...interface{}) (int64, bool)) *Table

InterfaceInt64 updates a int64 col using interface{} inputs.

func (*MustUpdateOn) InterfaceString 

func (on *MustUpdateOn) InterfaceString(fn func(v ...interface{}) (string, bool)) *Table

InterfaceString updates a string col using interface{} inputs.

func (*MustUpdateOn) InterfaceTimestampMicros 

func (on *MustUpdateOn) InterfaceTimestampMicros(fn func(v ...interface{}) (TimestampMicros, bool)) *Table

InterfaceTimestampMicros updates a TimestampMicros col using interface{} inputs.

func (*MustUpdateOn) InterfaceTimestampMillis 

func (on *MustUpdateOn) InterfaceTimestampMillis(fn func(v ...interface{}) (TimestampMillis, bool)) *Table

InterfaceTimestampMillis updates a TimestampMillis col using interface{} inputs.

func (*MustUpdateOn) String 

func (on *MustUpdateOn) String(fn func(v ...string) (string, bool)) *Table

String updates a string col using string inputs. Null on any null inputs. Returns error if any column cannot be assigned to string; no conversions are performed.

func (*MustUpdateOn) TimestampMicros 

func (on *MustUpdateOn) TimestampMicros(fn func(v ...TimestampMicros) (TimestampMicros, bool)) *Table

TimestampMicros updates a TimestampMicros col using TimestampMicros inputs. Null on any null inputs. Returns error if any column cannot be assigned to TimestampMicros; no conversions are performed.

func (*MustUpdateOn) TimestampMillis 

func (on *MustUpdateOn) TimestampMillis(fn func(v ...TimestampMillis) (TimestampMillis, bool)) *Table

TimestampMillis updates a TimestampMillis col using TimestampMillis inputs. Null on any null inputs. Returns error if any column cannot be assigned to TimestampMillis; no conversions are performed.

type MustUpdateSelection 

type MustUpdateSelection struct {
	*UpdateSelection
}

MustUpdateSelection panics on any error when creating derived tables.

func (*MustUpdateSelection) By 

func (us *MustUpdateSelection) By(fn func(r Row) interface{}) *Table

By performs Update using the whole table row as input. For wide tables this might be inefficient, consider using On(...) instead.

func (*MustUpdateSelection) Constant 

func (us *MustUpdateSelection) Constant(value interface{}) *Table

Constant makes column a constant column with the given value, but of the same type. Returns error if a non-nil value cannot be converted to the column type.

func (*MustUpdateSelection) On 

func (us *MustUpdateSelection) On(cols ...ColumnName) *MustUpdateOn

On selects a subset of columns to use as an extension source. If duplicate columns exist, the first so named is used. Note this does not panic yet, even if the columns do not exist. (However subsequent calls to the returned object will error.)

type PivotKey 

type PivotKey struct {
	// contains filtered or unexported fields
}

PivotKey contains a table to apply pivot and a key.

func (*PivotKey) Columns 

func (pk *PivotKey) Columns(pivot ColumnName, value ColumnName) (*Table, error)

Columns set pivot and value column names for the ongoing pivot operation

type PivotSelection 

type PivotSelection struct {
	// contains filtered or unexported fields
}

PivotSelection contains a table to apply pivot.

func (*PivotSelection) Key 

func (ps *PivotSelection) Key(key ...ColumnName) *PivotKey

Key set key columns names for the ongoing pivot operation

type Row 

type Row struct {
	// contains filtered or unexported fields
}

Row represents a materialized record.

func (Row) Index 

func (r Row) Index() Index

Index returns a row index.

func (Row) Interface 

func (r Row) Interface() []interface{}

Interface returns the row values in the form of []interface{}.

func (Row) MustValue 

func (r Row) MustValue(c ColumnName) Value

MustValue accesses a column value by column name. Panics on errors

func (Row) Schema 

func (r Row) Schema() Schema

Schema returns a schema of the table which contains the Row.

func (Row) String 

func (r Row) String() string

String formats the Row as the Rows containing a single row

func (Row) ToStruct 

func (r Row) ToStruct(structPtr interface{}) error

ToStruct copies to the exported struct fields by name, ignoring unmapped columns (column names that do not correspond to a struct field). structPtr must be of a *struct{} type.

See documentation on (t *Table) ToStructs.

Note this is much less efficient than handling a slice at a time.

Example 
fmt.Println()
for row := range pirateBooty.IterAll() {
	// get a struct representation of the row.
	asStruct := &struct {
		Pr   float64 `table:"Price"`
		Idx  int     `table:",index"`
		Name string
	}{}
	err := row.ToStruct(asStruct)
	if err != nil {
		panic(err)
	}
	fmt.Printf("row # %2d has price: %.1f\n", asStruct.Idx, asStruct.Pr)
}

Output:

row #  0 has price: 1.0
row #  1 has price: 0.0
row #  2 has price: 5.5
row #  3 has price: 600.0

func (Row) Value 

func (r Row) Value(c ColumnName) (Value, error)

Value accesses a column value by column name.

func (Row) ValueAt 

func (r Row) ValueAt(colIndex int) Value

ValueAt accesses a column value by column index.

func (Row) Values 

func (r Row) Values() []Value

Values retrieves all the columns values from a Row.

type Rows 

type Rows struct {
	Data   []Row
	Schema Schema
}

Rows are materialized table data, suitable for printing for example.

func (Rows) String 

func (r Rows) String() string

String formats the rows as an ascii art table

type Schema 

type Schema struct {
	Columns []Column
	// contains filtered or unexported fields
}

Schema is intended as an immutable representation of table metadata

func NewSchema 

func NewSchema(columns []Column) *Schema

NewSchema creates a new schema by a columns list

func (Schema) Check 

func (s Schema) Check(assertions ...SchemaAssertion) error

Check checks arbitrary assertions about the schema

func (Schema) CheckColumnsExist 

func (s Schema) CheckColumnsExist(colNames ...ColumnName) error

CheckColumnsExist checks that all given columns exist

func (Schema) Col 

func (s Schema) Col(col ColumnName) (Column, error)

Col gets the column by name, first matched

func (Schema) ColIndex 

func (s Schema) ColIndex(col ColumnName) (int, error)

ColIndex gets the column index by name, first matched

func (Schema) Equal 

func (s Schema) Equal(other Schema) bool

Equal returns true if order, name, and types match

func (Schema) MustCol 

func (s Schema) MustCol(colName ColumnName) Column

MustCol gets the column by name, first matched

func (Schema) MustColIndex 

func (s Schema) MustColIndex(col ColumnName) int

MustColIndex gets the column index by name, first matched; panics on error

func (Schema) MustProject 

func (s Schema) MustProject(colNames ...ColumnName) *Schema

MustProject projects a schema by given column names; panics on error

func (Schema) NumColumns 

func (s Schema) NumColumns() int

NumColumns is number of columns

func (Schema) Project 

func (s Schema) Project(colNames ...ColumnName) (*Schema, error)

Project projects a schema by given column names

func (Schema) String 

func (s Schema) String() string

String formats the schema as a list of columns names and types (each from a new line).

type SchemaAssertion 

type SchemaAssertion func(Schema) error

SchemaAssertion is an interface for arbitrary assertions which might be required from schema. It should return nil if the schema is acceptable.

type Series 

type Series struct {
	// contains filtered or unexported fields
}

Series is a named sequence of values. for larger datasets this sequence may be loaded lazily (eg memory map) or may even be unbounded

func NewConstantSeries 

func NewConstantSeries(col ColumnName, value interface{}) *Series

NewConstantSeries returns an unbounded repetition of the same value, using the dynamic type of the given value. Panics if the value is null.

func NewNullSeries 

func NewNullSeries(col ColumnName, typ reflect.Type) *Series

NewNullSeries returns an unbounded series containing null values of the given type.

func NewSeriesFromSlice 

func NewSeriesFromSlice(col ColumnName, values interface{}, notNull []bool) (*Series, error)

NewSeriesFromSlice converts a slice of scalars to a new Series. notNull is a bit mask indicating which values are not null; notNull == nil implies that all values are not null.

type SortFunc 

type SortFunc func(r1 Row, r2 Row) bool

SortFunc should return true if r1 is less than r2.

type Table 

type Table struct {
	// contains filtered or unexported fields
}

Table is an immutable container of Series.

Example 
// create a table
pirateBooty = NewTable(
	Must().NewSeriesFromSlice("Name", []string{"doubloon", "grog", "cutlass", "chest"}, nil),
	Must().NewSeriesFromSlice("Price", []float64{1.0, 0, 5.5, 600.0}, []bool{true, false, true, true}),
	Must().NewSeriesFromSlice("Quantity", []int64{1200, 44, 30, 2}, []bool{true, true, true, true}),
)

fmt.Println(pirateBooty.ToRows())

Output:

4 Row(s):
| |    Name|  Price|Quantity|
| |  string|float64|   int64|
-----------------------------
|0|doubloon|      1|    1200|
|1|    grog|  <nil>|      44|
|2| cutlass|    5.5|      30|
|3|   chest|    600|       2|

func NewTable 

func NewTable(series ...*Series) *Table

NewTable constructs a Table from the given Series. If at least one of the given Series is bounded, then the returned Table is bounded at that size. If multiple different-sized bounded series are provided, we panic.

Example 
// create a table
pirateBooty = NewTable(
	Must().NewSeriesFromSlice("Name", []string{"doubloon", "grog", "cutlass", "chest"}, nil),
	Must().NewSeriesFromSlice("Price", []float64{1.0, 0, 5.5, 600.0}, []bool{true, false, true, true}),
	Must().NewSeriesFromSlice("Quantity", []int64{1200, 44, 30, 2}, []bool{true, true, true, true}),
)

fmt.Println(pirateBooty.ToRows())

Output:

4 Row(s):
| |    Name|  Price|Quantity|
| |  string|float64|   int64|
-----------------------------
|0|doubloon|      1|    1200|
|1|    grog|  <nil>|      44|
|2| cutlass|    5.5|      30|
|3|   chest|    600|       2|

func NewTableFromStructs 

func NewTableFromStructs(structs interface{}) (*Table, error)

NewTableFromStructs copies exported struct fields to a new table. structs must be of a []struct{} or []*struct{} type. A nil entry in a []*struct{} is mapped to zero values for each column (not nulls!).

Use the struct tag `table:"column name"` to bind the field to a column with a different name.

Use the struct tag `table:"-"` to ignore a field.

Example 
type price struct {
	Pr   float64 `table:"Price"`
	Name string
	Who  string `table:"-"` // ignored
}
priceStructs := []*price{
	{Pr: 1.5, Name: "scrimshaw"},
	{Pr: 10000, Name: "yer mortal soul"},
}
newPrices := Must().NewTableFromStructs(priceStructs)
fmt.Println(newPrices.ToRows())

Output:

2 Row(s):
| |  Price|           Name|
| |float64|         string|
---------------------------
|0|    1.5|      scrimshaw|
|1|  10000|yer mortal soul|

func (*Table) Append 

func (t *Table) Append(other *Table) *AppendSelection

Append concatenates two tables vertically. Use the returned object to set append mode.

Example (Exact) 
morePirateBooty, _ := pirateBooty.Append(NewTable(
	Must().NewSeriesFromSlice("Price", []float64{1.5}, nil),
	Must().NewSeriesFromSlice("Name", []string{"piastre"}, nil),
	Must().NewSeriesFromSlice("Quantity", []int64{1000}, nil),
)).Exact()
fmt.Println(morePirateBooty.ToRows())

Output:

5 Row(s):
| |    Name|  Price|Quantity|
| |  string|float64|   int64|
-----------------------------
|0|doubloon|      1|    1200|
|1|    grog|  <nil>|      44|
|2| cutlass|    5.5|      30|
|3|   chest|    600|       2|
|4| piastre|    1.5|    1000|
Example (Inner) 
morePirateBootyNames, _ := pirateBooty.Append(NewTable(
	Must().NewSeriesFromSlice("Name", []string{"piastre"}, nil),
)).Inner()
fmt.Println(morePirateBootyNames.ToRows())

Output:

5 Row(s):
| |    Name|
| |  string|
------------
|0|doubloon|
|1|    grog|
|2| cutlass|
|3|   chest|
|4| piastre|
Example (Outer) 
morePirateBooty, _ := pirateBooty.Append(NewTable(
	Must().NewSeriesFromSlice("Name", []string{"piastre"}, nil),
	Must().NewSeriesFromSlice("Quantity", []int64{1000}, nil),
)).Outer()
fmt.Println(morePirateBooty.ToRows())

Output:

5 Row(s):
| |    Name|  Price|Quantity|
| |  string|float64|   int64|
-----------------------------
|0|doubloon|      1|    1200|
|1|    grog|  <nil>|      44|
|2| cutlass|    5.5|      30|
|3|   chest|    600|       2|
|4| piastre|  <nil>|    1000|
Example (Positional) 
morePirateBooty, _ := pirateBooty.Append(NewTable(
	Must().NewSeriesFromSlice("Name", []string{"piastre"}, nil),
	Must().NewSeriesFromSlice("Price", []float64{1.5}, nil),
	Must().NewSeriesFromSlice("QuantityRenamed", []int64{1000}, nil),
)).Positional()
fmt.Println(morePirateBooty.ToRows())

Output:

5 Row(s):
| |    Name|  Price|Quantity|
| |  string|float64|   int64|
-----------------------------
|0|doubloon|      1|    1200|
|1|    grog|  <nil>|      44|
|2| cutlass|    5.5|      30|
|3|   chest|    600|       2|
|4| piastre|    1.5|    1000|

func (*Table) Cache 

func (t *Table) Cache() (*Table, error)

Cache converts a lazy table to one that is fully materialized.

func (*Table) Convert 

func (t *Table) Convert(col ColumnName, typ reflect.Type) (*Table, error)

Convert lazily converts all columns of the given name to the assigned type. Returns non-nil error (and nil Table) if any column is not convertible. Note that if no column name matches, the same table is returned.

Example 
// convert column values from int to float
converted, _ := pirateBooty.Convert("Quantity", reflect.TypeOf(float64(0)))
fmt.Println(converted.ToRows())

Output:

4 Row(s):
| |    Name|  Price|Quantity|
| |  string|float64| float64|
-----------------------------
|0|doubloon|      1|    1200|
|1|    grog|  <nil>|      44|
|2| cutlass|    5.5|      30|
|3|   chest|    600|       2|

func (*Table) Distinct 

func (t *Table) Distinct() *DistinctSelection

Distinct filters the table retaining only unique rows. Use the returned object to construct a derived table.

Example (OnSpecifiedColumnsByEq) 
// distinct by equality of the specified columns.
distinctByName, _ := pirateBooty.Distinct().On("Name")
fmt.Println(distinctByName.ToRows())

Output:

4 Row(s):
| |    Name|  Price|Quantity|
| |  string|float64|   int64|
-----------------------------
|0|doubloon|      1|    1200|
|1|    grog|  <nil>|      44|
|2| cutlass|    5.5|      30|
|3|   chest|    600|       2|

func (*Table) Equal 

func (t *Table) Equal(other *Table) bool

Equal is true if the other table has the same schema (in the same order) and exactly equal series values.

func (*Table) Extend 

func (t *Table) Extend(newCol ColumnName) *Extension

Extend adds a column called newCol by applying a function. Use the returned object to construct a derived table.

Example (Constant) 
// add a constant column value.
withSource := pirateBooty.Extend("Source").
	Constant("BOOTY")
fmt.Println(withSource.ToRows())

Output:

4 Row(s):
| |    Name|  Price|Quantity|Source|
| |  string|float64|   int64|string|
------------------------------------
|0|doubloon|      1|    1200| BOOTY|
|1|    grog|  <nil>|      44| BOOTY|
|2| cutlass|    5.5|      30| BOOTY|
|3|   chest|    600|       2| BOOTY|
Example (DynamicProjection) 
// calculate new column value on a projection using dynamic types.
type Stuff struct {
	description string
}
withCustomCol, _ := pirateBooty.Extend("Stuff").On("Name", "Price").
	Interface(func(v ...interface{}) interface{} {
		if v[1] == nil {
			return nil
		}
		return Stuff{description: fmt.Sprintf("%v that costs $%.2f", v[0].(string), v[1].(float64))}
	}, reflect.TypeOf(Stuff{}))
fmt.Println(withCustomCol.ToRows())

Output:

4 Row(s):
| |    Name|  Price|Quantity|                                  Stuff|
| |  string|float64|   int64|                             data.Stuff|
---------------------------------------------------------------------
|0|doubloon|      1|    1200|{description:doubloon that costs $1.00}|
|1|    grog|  <nil>|      44|                                  <nil>|
|2| cutlass|    5.5|      30| {description:cutlass that costs $5.50}|
|3|   chest|    600|       2| {description:chest that costs $600.00}|
Example (DynamicTypes) 
// calculate new column value on the whole row using dynamic types.
totals := pirateBooty.Extend("Total").
	By(func(r Row) interface{} {
		q, _ := r.Value("Quantity")
		p, _ := r.Value("Price")
		if p.IsNull() {
			return nil
		}
		return p.MustFloat64() * float64(q.MustInt64())
	}, reflect.TypeOf(float64(0)))
fmt.Println(totals.ToRows())

Output:

4 Row(s):
| |    Name|  Price|Quantity|  Total|
| |  string|float64|   int64|float64|
-------------------------------------
|0|doubloon|      1|    1200|   1200|
|1|    grog|  <nil>|      44|  <nil>|
|2| cutlass|    5.5|      30|    165|
|3|   chest|    600|       2|   1200|
Example (OnSelectedColsDynamicTypes) 
// calculate new column value on selected columns using dynamic types.
salePrices, _ := pirateBooty.Extend("Reduced Price").On("Price").
	Interface(func(v ...interface{}) interface{} {
		// 25% off
		if v[0] == nil {
			return nil
		}
		return v[0].(float64) * 0.75
	}, reflect.TypeOf(float64(0)))
fmt.Println(salePrices.ToRows())

Output:

4 Row(s):
| |    Name|  Price|Quantity|Reduced Price|
| |  string|float64|   int64|      float64|
-------------------------------------------
|0|doubloon|      1|    1200|         0.75|
|1|    grog|  <nil>|      44|        <nil>|
|2| cutlass|    5.5|      30|        4.125|
|3|   chest|    600|       2|          450|
Example (OnSelectedColsMixedTypes) 
// calculate new column value on selected columns using dynamic types for inputs and static type for output.
salePrices, _ := pirateBooty.Extend("Reduced Price").On("Price").
	InterfaceFloat64(func(v ...interface{}) (float64, bool) {
		// 25% off
		if v[0] == nil {
			return float64(0), false
		}
		return v[0].(float64) * 0.75, true
	})
fmt.Println(salePrices.ToRows())

Output:

4 Row(s):
| |    Name|  Price|Quantity|Reduced Price|
| |  string|float64|   int64|      float64|
-------------------------------------------
|0|doubloon|      1|    1200|         0.75|
|1|    grog|  <nil>|      44|        <nil>|
|2| cutlass|    5.5|      30|        4.125|
|3|   chest|    600|       2|          450|
Example (OnSelectedColsStaticType) 
// calculate new column value on selected columns using static type.
salePrices, _ := pirateBooty.Extend("Reduced Price").On("Price").
	Float64(func(v ...float64) (float64, bool) {
		// 25% off
		return v[0] * 0.75, true
	})
fmt.Println(salePrices.ToRows())

Output:

4 Row(s):
| |    Name|  Price|Quantity|Reduced Price|
| |  string|float64|   int64|      float64|
-------------------------------------------
|0|doubloon|      1|    1200|         0.75|
|1|    grog|  <nil>|      44|        <nil>|
|2| cutlass|    5.5|      30|        4.125|
|3|   chest|    600|       2|          450|

func (*Table) Filter 

func (t *Table) Filter() *FilterSelection

Filter selects some records lazily. Use the returned object to construct a derived table.

Example (DynamicType) 
// any column value can be filtered on.
startsWithC := pirateBooty.Filter().
	By(func(r Row) bool {
		name := r.ValueAt(0)
		return !name.IsNull() && strings.HasPrefix(name.MustString(), "c")
	})
fmt.Println(startsWithC.ToRows())

Output:

2 Row(s):
| |   Name|  Price|Quantity|
| | string|float64|   int64|
----------------------------
|0|cutlass|    5.5|      30|
|1|  chest|    600|       2|
Example (Equal) 
// equality filter is provided as a function.
grog, _ := pirateBooty.Filter().On("Name").
	Interface(Eq("grog"))
fmt.Println(grog.ToRows())

Output:

1 Row(s):
| |  Name|  Price|Quantity|
| |string|float64|   int64|
---------------------------
|0|  grog|  <nil>|      44|
Example (StaticType) 
// notice that rows with Price=nil are not passed to the filter.
priceLessThan100, _ := pirateBooty.Filter().On("Price").
	Float64(func(v ...float64) bool {
		price := v[0]
		return price < 100
	})
fmt.Println(priceLessThan100.ToRows())

Output:

2 Row(s):
| |    Name|  Price|Quantity|
| |  string|float64|   int64|
-----------------------------
|0|doubloon|      1|    1200|
|1| cutlass|    5.5|      30|

func (*Table) Foreach 

func (t *Table) Foreach() *ForeachSelection

Foreach eagerly iterates over a table using a user-defined function. May be useful for:

- reading table contents into some user-defined data struct (e.g. a map)

- computing some scalar aggregate (e.g. MIN(column))

Use the returned object to specify the columns and the function.

Example (Generic) 
cost := 0.
_ = pirateBooty.Foreach().On("Price", "Quantity").Interface(func(v ...interface{}) {
	if v[0] == nil {
		return
	}
	cost += v[0].(float64) * float64(v[1].(int64))
})
fmt.Printf("Total cost: %f\n", cost)

Output:

Total cost: 2565.000000
Example (Specialized) 
quantity := int64(0)
_ = pirateBooty.Foreach().On("Quantity").Int64(func(v ...int64) {
	quantity += v[0]
})
fmt.Printf("Total quantity: %d\n", quantity)

Output:

Total quantity: 1276
Example (Wholerow) 
nonNullCount := 0
pirateBooty.Foreach().By(func(r Row) {
	for _, v := range r.Values() {
		if !v.IsNull() {
			nonNullCount++
		}
	}
})
fmt.Printf("Non null count: %d\n", nonNullCount)

Output:

Non null count: 11

func (*Table) ForeachKey 

func (t *Table) ForeachKey(key ...ColumnName) *ForeachKeySelection

ForeachKey eagerly iterates over a table grouping it by a given key: for every single key value, Foreach().By() calls the callback and supplies it with the key value and the "partition" table (the table containing other values from corresponding rows of the source table).

For the time being, it is implemented on the top of Sort() - therefore only orderable keys are supported (currently only supported types are considered orderable).

Example 
_ = pirateBooty.ForeachKey("Name").By(func(key Row, partition *Table) {
	fmt.Printf("Key: %v\n", key)
	fmt.Printf("Partition: %v\n", partition.ToRows())
})

Output:

Key: 1 Row(s):
| |  Name|
| |string|
----------
|0|  grog|

Partition: 1 Row(s):
| |  Price|Quantity|
| |float64|   int64|
--------------------
|0|  <nil>|      44|

Key: 1 Row(s):
| |    Name|
| |  string|
------------
|1|doubloon|

Partition: 1 Row(s):
| |  Price|Quantity|
| |float64|   int64|
--------------------
|0|      1|    1200|

Key: 1 Row(s):
| |   Name|
| | string|
-----------
|2|cutlass|

Partition: 1 Row(s):
| |  Price|Quantity|
| |float64|   int64|
--------------------
|0|    5.5|      30|

Key: 1 Row(s):
| |  Name|
| |string|
----------
|3| chest|

Partition: 1 Row(s):
| |  Price|Quantity|
| |float64|   int64|
--------------------
|0|    600|       2|

func (*Table) Head 

func (t *Table) Head(count int) *Table

Head is a lazy subset of the first count records (but may return fewer).

Example 
// first 2 rows
top := pirateBooty.Head(2)
fmt.Println(top.ToRows())

Output:

2 Row(s):
| |    Name|  Price|Quantity|
| |  string|float64|   int64|
-----------------------------
|0|doubloon|      1|    1200|
|1|    grog|  <nil>|      44|

func (*Table) Iter 

func (t *Table) Iter() (rows <-chan Row, done func())

Iter iterates over the table, no buffer. call done() to release resources after a partial read.

func (*Table) IterAll 

func (t *Table) IterAll() <-chan Row

IterAll iterates over the entire table, no buffer. Use when ranging over all rows is required.

func (*Table) Join 

func (t *Table) Join() *JoinSelection

Join performs a "horizontal" join of two tables by some key. E.g. if the left table is 

| |LeftKeyColumn|OtherLeftColumn|
| |      keyType|  otherLeftType|
---------------------------------
|1|         key1| someLeftValue1|
|2|         key2| someLeftValue2|
|3|         key2| someLeftValue3|
|4|         key3| someLeftValue4|

and the right table is 

| |RightKeyColumn|OtherRightColumn|
| |       keyType|  otherRightType|
-----------------------------------
|1|          key2| someRightValue1|
|2|          key3| someRightValue2|

then if we inner join them on LeftKeyColumn and RightKeyColumn columns correspondingly 

resultTable, err := leftTable.Join().On("LeftKeyColumn").Inner(rightTable, "RightKeyColumn")

the result would be: 

| |LeftKeyColumn|OtherLeftColumn|RightKeyColumn|OtherRightColumn|
| |      keyType|  otherLeftType|       keyType|  otherRightType|
-----------------------------------------------------------------
|2|         key2| someLeftValue2|          key2| someRightValue1|
|3|         key2| someLeftValue3|          key2| someRightValue1|
|4|         key3| someLeftValue4|          key3| someRightValue2|
Example (Inner) 
// create another table to join
currency := NewTable(
	Must().NewSeriesFromSlice("CurrencyName", []string{"doubloon", "piastre"}, nil),
	Must().NewSeriesFromSlice("CurrencyExchangeRate", []float64{0.5, 2.3}, nil),
)

// inner join
currencyBooty, _ := pirateBooty.Join().On("Name").Inner(currency, "CurrencyName")
fmt.Println(currencyBooty.ToRows())

Output:

1 Row(s):
| |    Name|  Price|Quantity|CurrencyName|CurrencyExchangeRate|
| |  string|float64|   int64|      string|             float64|
---------------------------------------------------------------
|0|doubloon|      1|    1200|    doubloon|                 0.5|
Example (NaturalInner) 
// create another table to join
currency := NewTable(
	Must().NewSeriesFromSlice("Name", []string{"doubloon", "piastre"}, nil),
	Must().NewSeriesFromSlice("ExchangeRate", []float64{0.5, 2.3}, nil),
)

// natural inner join
currencyBooty, _ := pirateBooty.Join().NaturalInner(currency)
fmt.Println(currencyBooty.ToRows())

Output:

1 Row(s):
| |    Name|  Price|Quantity|    Name|ExchangeRate|
| |  string|float64|   int64|  string|     float64|
---------------------------------------------------
|0|doubloon|      1|    1200|doubloon|         0.5|

func (*Table) MarshalJSON 

func (t *Table) MarshalJSON() ([]byte, error)

MarshalJSON saves the table as an object with a header array (for the schema) followed by a row-major data array. This is supported for scalar types only.

func (*Table) Must 

func (t *Table) Must() MustTable

Must returns a proxy object that asserts errors are nil.

func (*Table) Pivot 

func (t *Table) Pivot() *PivotSelection

Pivot takes a "narrow" table containing "column names" and "column values", for instance: 

| |    Key|    Pivot|    Value|
| |keyType|   string|valueType|
-------------------------------
|1|   key1|"column1"|   value1|
|2|   key1|"column2"|   value2|
|3|   key1|"column3"|   value3|
|4|   key2|"column1"|   value4|
|5|   key2|"column2"|   value5|

and transforms it into a "wide" table, for instance: 

| |    Key|  column1|  column2|  column3|
| |keyType|valueType|valueType|valueType|
-----------------------------------------
|1|   key1|   value1|   value2|   value3|
|2|   key2|   value4|   value5|    <nil>|
Example 
// create a table
pirateBootyNarrow := NewTable(
	Must().NewSeriesFromSlice("Name", []string{"doubloon", "doubloon", "grog", "cutlass", "cutlass", "chest", "chest"}, nil),
	Must().NewSeriesFromSlice("PropertyName", []string{"Price", "Quantity", "Quantity", "Price", "Quantity", "Price", "Quantity"}, nil),
	Must().NewSeriesFromSlice("PropertyValue", []float64{1.0, 1200, 44, 5.5, 30, 600.0, 2}, nil),
)

// make a wide table from pirateBootyNarrow.
pirateBootyWide, _ := pirateBootyNarrow.Pivot().Key("Name").Columns("PropertyName", "PropertyValue")
fmt.Println(pirateBootyWide.ToRows())

Output:

4 Row(s):
| |    Name|  Price|Quantity|
| |  string|float64| float64|
-----------------------------
|0|   chest|    600|       2|
|1| cutlass|    5.5|      30|
|2|doubloon|      1|    1200|
|3|    grog|  <nil>|      44|

func (*Table) Project 

func (t *Table) Project(columns ...ColumnName) (*Table, error)

Project reorders and/or takes a subset of columns. On duplicate columns, only the first so named is taken. Returns error, and nil table, if any column is missing.

Example 
// two columns by name
projected, _ := pirateBooty.Project("Quantity", "Name")
fmt.Println(projected.ToRows())

Output:

4 Row(s):
| |Quantity|    Name|
| |   int64|  string|
---------------------
|0|    1200|doubloon|
|1|      44|    grog|
|2|      30| cutlass|
|3|       2|   chest|

func (*Table) ProjectAllBut 

func (t *Table) ProjectAllBut(columns ...ColumnName) *Table

ProjectAllBut discards the named columns, which may not exist in the schema.

Example 
// drop some columns.
justQuantity := pirateBooty.ProjectAllBut("Price", "Name", "No such column")
fmt.Println(justQuantity.ToRows())

Output:

4 Row(s):
| |Quantity|
| |   int64|
------------
|0|    1200|
|1|      44|
|2|      30|
|3|       2|

func (*Table) Rename 

func (t *Table) Rename(old, new ColumnName) *Table

Rename updates all columns of the old name to the new name. Does nothing if none match in the schema.

Example 
// rename a column.
renameQuantity := pirateBooty.Rename("Quantity", "QuantityBeforeSinking")
fmt.Println(renameQuantity.ToRows())

Output:

4 Row(s):
| |    Name|  Price|QuantityBeforeSinking|
| |  string|float64|                int64|
------------------------------------------
|0|doubloon|      1|                 1200|
|1|    grog|  <nil>|                   44|
|2| cutlass|    5.5|                   30|
|3|   chest|    600|                    2|

func (*Table) Schema 

func (t *Table) Schema() Schema

Schema returns the type information for the Table.

Example 
fmt.Println(pirateBooty.Schema())

Output:

Name, string
Price, float64
Quantity, int64

func (*Table) Size 

func (t *Table) Size() int

Size returns the number of rows contained in the table, if known. Unbounded, or lazy (non materialized) tables can return -1.

Example 
fmt.Println(pirateBooty.Size())

Output:

4

func (*Table) Slice 

func (t *Table) Slice(start, end Index) *Table

Slice is a lazy subset of records between the start index and the end (exclusive). Unlike go slices, if the end index is out of range then fewer records are returned rather than receiving an error.

Example 
// last 2 rows
bottom := pirateBooty.Slice(2, 4)
fmt.Println(bottom.ToRows())

Output:

2 Row(s):
| |   Name|  Price|Quantity|
| | string|float64|   int64|
----------------------------
|0|cutlass|    5.5|      30|
|1|  chest|    600|       2|

func (*Table) Sort 

func (t *Table) Sort(key Key) (*Table, error)

Sort produces a Table sorted by the columns defined by the Key. This is eager, not lazy; it materializes the whole table.

Example 
// in ascending order of Name.
byNameAsc, _ := pirateBooty.Sort(Key{{"Name", true}})
fmt.Println(byNameAsc.ToRows())

Output:

4 Row(s):
| |    Name|  Price|Quantity|
| |  string|float64|   int64|
-----------------------------
|0|   chest|    600|       2|
|1| cutlass|    5.5|      30|
|2|doubloon|      1|    1200|
|3|    grog|  <nil>|      44|

func (*Table) SortByFunc 

func (t *Table) SortByFunc(f SortFunc) (*Table, error)

SortByFunc sorts a table by an arbitrary user-defined function. This is eager, not lazy; it materializes the whole table. Slower than Table.Sort(). For better performance, use Table.Sort() instead (+ extension if needed).

Example 
// in ascending order of length of Name.
byNameLenAsc, _ := pirateBooty.SortByFunc(func(r1 Row, r2 Row) bool {
	return len(r1.ValueAt(0).MustString()) < len(r2.ValueAt(0).MustString())
})
fmt.Println(byNameLenAsc.ToRows())

Output:

4 Row(s):
| |    Name|  Price|Quantity|
| |  string|float64|   int64|
-----------------------------
|0|    grog|  <nil>|      44|
|1|   chest|    600|       2|
|2| cutlass|    5.5|      30|
|3|doubloon|      1|    1200|

func (*Table) ToRows 

func (t *Table) ToRows() Rows

ToRows materializes all table data into a Rows object. This is useful if you want to print the table data.

Example 
rows := pirateBooty.ToRows()
fmt.Println(rows)

Output:

4 Row(s):
| |    Name|  Price|Quantity|
| |  string|float64|   int64|
-----------------------------
|0|doubloon|      1|    1200|
|1|    grog|  <nil>|      44|
|2| cutlass|    5.5|      30|
|3|   chest|    600|       2|

func (*Table) ToStructs 

func (t *Table) ToStructs(structsPtr interface{}) error

ToStructs copies to the exported struct fields by name, ignoring unmapped columns (column names that do not correspond to a struct field). structsPtr must be of a *[]struct{} or *[]*struct{} type. Returns error if the struct fields are not assignable from the input data.

When columns have null value, the struct field receives the zero value for that type instead.

Use the struct tag `table:"column name"` to bind the field to a column with a different name.

Use the struct tag `table:"-"` to ignore a field.

Use the struct tag `table:",index"` will mark that field as receiving the row index value (it must be an int field).

Example 
type price struct {
	Price float64
	Name  string
}
structSlice := make([]*price, 0)
err := pirateBooty.ToStructs(&structSlice)
if err != nil {
	panic(err)
}
fmt.Println()
for _, p := range structSlice {
	fmt.Printf("%#v\n", p)
}

Output:

&data.price{Price:1, Name:"doubloon"}
&data.price{Price:0, Name:"grog"}
&data.price{Price:5.5, Name:"cutlass"}
&data.price{Price:600, Name:"chest"}
Example (WithTags) 
type price struct {
	Pr   float64 `table:"Price"`
	Idx  int     `table:",index"`
	Name int     `table:"-"` // ignored
}
structSlice := make([]*price, 0)
err := pirateBooty.ToStructs(&structSlice)
if err != nil {
	panic(err)
}
fmt.Println()
for _, p := range structSlice {
	fmt.Printf("%#v\n", p)
}

Output:

&data.price{Pr:1, Idx:0, Name:0}
&data.price{Pr:0, Idx:1, Name:0}
&data.price{Pr:5.5, Idx:2, Name:0}
&data.price{Pr:600, Idx:3, Name:0}

func (*Table) UnmarshalJSON 

func (t *Table) UnmarshalJSON(b []byte) (err error)

UnmarshalJSON eagerly deserializes to a Table. This is supported for scalar types only.

func (*Table) Update 

func (t *Table) Update(col ColumnName) *UpdateSelection

Update replaces the values in the column 'col'. Use the returned object to construct a derived table.

Example (Constant) 
// add a constant column value.
withSource, _ := pirateBooty.Update("Quantity").
	Constant(int64(0))
fmt.Println(withSource.ToRows())

Output:

4 Row(s):
| |    Name|  Price|Quantity|
| |  string|float64|   int64|
-----------------------------
|0|doubloon|      1|       0|
|1|    grog|  <nil>|       0|
|2| cutlass|    5.5|       0|
|3|   chest|    600|       0|
Example (DynamicTypes) 
// calculate updated column value on the whole row using dynamic types.
totals, _ := pirateBooty.Update("Quantity").
	By(func(r Row) interface{} {
		q, _ := r.Value("Quantity")
		p, _ := r.Value("Price")
		if p.IsNull() {
			return 0
		}
		return q.MustInt64()
	})
fmt.Println(totals.ToRows())

Output:

4 Row(s):
| |    Name|  Price|Quantity|
| |  string|float64|   int64|
-----------------------------
|0|doubloon|      1|    1200|
|1|    grog|  <nil>|       0|
|2| cutlass|    5.5|      30|
|3|   chest|    600|       2|
Example (OnSelectedColsDynamicTypes) 
// calculate updated column value on selected columns using dynamic types.
salePrices, _ := pirateBooty.Update("Price").On("Price").
	Interface(func(v ...interface{}) interface{} {
		// 25% off
		if v[0] == nil {
			return nil
		}
		return v[0].(float64) * 0.75
	})
fmt.Println(salePrices.ToRows())

Output:

4 Row(s):
| |    Name|  Price|Quantity|
| |  string|float64|   int64|
-----------------------------
|0|doubloon|   0.75|    1200|
|1|    grog|  <nil>|      44|
|2| cutlass|  4.125|      30|
|3|   chest|    450|       2|
Example (OnSelectedColsMixedTypes) 
// calculate new column value on selected columns using dynamic types for inputs and static type for output.
salePrices, _ := pirateBooty.Update("Price").On("Price").
	InterfaceFloat64(func(v ...interface{}) (float64, bool) {
		// 25% off
		if v[0] == nil {
			return float64(0), false
		}
		return v[0].(float64) * 0.75, true
	})
fmt.Println(salePrices.ToRows())

Output:

4 Row(s):
| |    Name|  Price|Quantity|
| |  string|float64|   int64|
-----------------------------
|0|doubloon|   0.75|    1200|
|1|    grog|  <nil>|      44|
|2| cutlass|  4.125|      30|
|3|   chest|    450|       2|
Example (OnSelectedColsStaticType) 
// calculate new column value on selected columns using static type.
salePrices, _ := pirateBooty.Update("Price").On("Price").
	Float64(func(v ...float64) (float64, bool) {
		// 25% off
		return v[0] * 0.75, true
	})
fmt.Println(salePrices.ToRows())

Output:

4 Row(s):
| |    Name|  Price|Quantity|
| |  string|float64|   int64|
-----------------------------
|0|doubloon|   0.75|    1200|
|1|    grog|  <nil>|      44|
|2| cutlass|  4.125|      30|
|3|   chest|    450|       2|

type TableBuilder 

type TableBuilder struct {
	// contains filtered or unexported fields
}

TableBuilder builds a materialized Table from external source row by row

func NewTableBuilder 

func NewTableBuilder(columns []Column) (*TableBuilder, error)

NewTableBuilder creates a new default table builder by a columns list

func (*TableBuilder) Append 

func (b *TableBuilder) Append(row []interface{})

Append appends a row (nil element denotes null)

func (*TableBuilder) Build 

func (b *TableBuilder) Build() *Table

Build builds a table

func (*TableBuilder) Reserve 

func (b *TableBuilder) Reserve(capacity int)

Reserve reserves extra buffer space

type TimestampMicros 

type TimestampMicros arrow.Timestamp

TimestampMicros is timestamp measured in us

type TimestampMillis 

type TimestampMillis arrow.Timestamp

TimestampMillis is timestamp measured in ms

type UpdateOn 

type UpdateOn struct {
	// contains filtered or unexported fields
}

UpdateOn stores a table, its column to update and the source columns list.

func (*UpdateOn) Bool 

func (on *UpdateOn) Bool(fn func(v ...bool) (bool, bool)) (*Table, error)

Bool updates a bool col using bool inputs. Null on any null inputs. Returns error if any column cannot be assigned to bool; no conversions are performed.

func (*UpdateOn) Float64 

func (on *UpdateOn) Float64(fn func(v ...float64) (float64, bool)) (*Table, error)

Float64 updates a float64 col using float64 inputs. Null on any null inputs. Returns error if any column cannot be assigned to float64; no conversions are performed.

func (*UpdateOn) Int 

func (on *UpdateOn) Int(fn func(v ...int) (int, bool)) (*Table, error)

Int updates a int col using int inputs. Null on any null inputs. Returns error if any column cannot be assigned to int; no conversions are performed.

func (*UpdateOn) Int64 

func (on *UpdateOn) Int64(fn func(v ...int64) (int64, bool)) (*Table, error)

Int64 updates a int64 col using int64 inputs. Null on any null inputs. Returns error if any column cannot be assigned to int64; no conversions are performed.

func (*UpdateOn) Interface 

func (on *UpdateOn) Interface(fn func(v ...interface{}) interface{}) (*Table, error)

Interface updates a column of an arbitrary type using a subset of source columns of arbitrary types.

func (*UpdateOn) InterfaceBool 

func (on *UpdateOn) InterfaceBool(fn func(v ...interface{}) (bool, bool)) (*Table, error)

InterfaceBool updates a bool col using interface{} inputs.

func (*UpdateOn) InterfaceFloat64 

func (on *UpdateOn) InterfaceFloat64(fn func(v ...interface{}) (float64, bool)) (*Table, error)

InterfaceFloat64 updates a float64 col using interface{} inputs.

func (*UpdateOn) InterfaceInt 

func (on *UpdateOn) InterfaceInt(fn func(v ...interface{}) (int, bool)) (*Table, error)

InterfaceInt updates a int col using interface{} inputs.

func (*UpdateOn) InterfaceInt64 

func (on *UpdateOn) InterfaceInt64(fn func(v ...interface{}) (int64, bool)) (*Table, error)

InterfaceInt64 updates a int64 col using interface{} inputs.

func (*UpdateOn) InterfaceString 

func (on *UpdateOn) InterfaceString(fn func(v ...interface{}) (string, bool)) (*Table, error)

InterfaceString updates a string col using interface{} inputs.

func (*UpdateOn) InterfaceTimestampMicros 

func (on *UpdateOn) InterfaceTimestampMicros(fn func(v ...interface{}) (TimestampMicros, bool)) (*Table, error)

InterfaceTimestampMicros updates a TimestampMicros col using interface{} inputs.

func (*UpdateOn) InterfaceTimestampMillis 

func (on *UpdateOn) InterfaceTimestampMillis(fn func(v ...interface{}) (TimestampMillis, bool)) (*Table, error)

InterfaceTimestampMillis updates a TimestampMillis col using interface{} inputs.

func (*UpdateOn) String 

func (on *UpdateOn) String(fn func(v ...string) (string, bool)) (*Table, error)

String updates a string col using string inputs. Null on any null inputs. Returns error if any column cannot be assigned to string; no conversions are performed.

func (*UpdateOn) TimestampMicros 

func (on *UpdateOn) TimestampMicros(fn func(v ...TimestampMicros) (TimestampMicros, bool)) (*Table, error)

TimestampMicros updates a TimestampMicros col using TimestampMicros inputs. Null on any null inputs. Returns error if any column cannot be assigned to TimestampMicros; no conversions are performed.

func (*UpdateOn) TimestampMillis 

func (on *UpdateOn) TimestampMillis(fn func(v ...TimestampMillis) (TimestampMillis, bool)) (*Table, error)

TimestampMillis updates a TimestampMillis col using TimestampMillis inputs. Null on any null inputs. Returns error if any column cannot be assigned to TimestampMillis; no conversions are performed.

type UpdateSelection 

type UpdateSelection struct {
	// contains filtered or unexported fields
}

UpdateSelection stores a table and its column to update.

func (*UpdateSelection) By 

func (us *UpdateSelection) By(fn func(r Row) interface{}) (*Table, error)

By performs Update using the whole table row as input. For wide tables this might be inefficient, consider using On(...) instead.

func (*UpdateSelection) Constant 

func (us *UpdateSelection) Constant(value interface{}) (*Table, error)

Constant replaces a column with a constant column having the given value, which must be of the same type. Returns error if a non-nil value cannot be converted to the column type.

func (*UpdateSelection) On 

func (us *UpdateSelection) On(cols ...ColumnName) *UpdateOn

On selects a subset of columns to use as an extension source. If duplicate columns exist, the first so named is used. Note this does not panic yet, even if the columns do not exist. (However subsequent calls to the returned object will error.)

type Value 

type Value struct {
	// contains filtered or unexported fields
}

Value holds an arbitrary, nullable column value.

func (Value) Bool 

func (v Value) Bool() (bool, bool)

Bool extracts a value of type bool from a Value. Returns false if the value is null. Panics if the types mismatch.

func (Value) ColIndex 

func (v Value) ColIndex() int

Column returns the column of the value.

func (Value) Column 

func (v Value) Column() Column

Column returns the column of the value.

func (Value) Float64 

func (v Value) Float64() (float64, bool)

Float64 extracts a value of type float64 from a Value. Returns false if the value is null. Panics if the types mismatch.

func (Value) Int 

func (v Value) Int() (int, bool)

Int extracts a value of type int from a Value. Returns false if the value is null. Panics if the types mismatch.

func (Value) Int64 

func (v Value) Int64() (int64, bool)

Int64 extracts a value of type int64 from a Value. Returns false if the value is null. Panics if the types mismatch.

func (Value) Interface 

func (v Value) Interface() interface{}

Interface returns the underlying representation of the value.

func (Value) IsNull 

func (v Value) IsNull() bool

IsNull returns true if the value is null.

func (Value) MustBool 

func (v Value) MustBool() bool

MustBool extracts a value of type bool from a Value. Panics if the value is null or the types mismatch.

func (Value) MustFloat64 

func (v Value) MustFloat64() float64

MustFloat64 extracts a value of type float64 from a Value. Panics if the value is null or the types mismatch.

func (Value) MustInt 

func (v Value) MustInt() int

MustInt extracts a value of type int from a Value. Panics if the value is null or the types mismatch.

func (Value) MustInt64 

func (v Value) MustInt64() int64

MustInt64 extracts a value of type int64 from a Value. Panics if the value is null or the types mismatch.

func (Value) MustString 

func (v Value) MustString() string

MustString extracts a value of type string from a Value. Panics if the value is null or the types mismatch.

func (Value) MustTimestampMicros 

func (v Value) MustTimestampMicros() TimestampMicros

MustTimestampMicros extracts a value of type TimestampMicros from a Value. Panics if the value is null or the types mismatch.

func (Value) MustTimestampMillis 

func (v Value) MustTimestampMillis() TimestampMillis

MustTimestampMillis extracts a value of type TimestampMillis from a Value. Panics if the value is null or the types mismatch.

func (Value) String 

func (v Value) String() (string, bool)

String extracts a value of type string from a Value. Returns false if the value is null. Panics if the types mismatch.

func (Value) TimestampMicros 

func (v Value) TimestampMicros() (TimestampMicros, bool)

TimestampMicros extracts a value of type TimestampMicros from a Value. Returns false if the value is null. Panics if the types mismatch.

func (Value) TimestampMillis 

func (v Value) TimestampMillis() (TimestampMillis, bool)

TimestampMillis extracts a value of type TimestampMillis from a Value. Returns false if the value is null. Panics if the types mismatch.

Source Files

View all Source files

Directories

Path Synopsis
csv
Package csv provides tools for data tables seriaization to/from CSV files.
 Package csv provides tools for data tables seriaization to/from CSV files.
example
Package parquet provides tools for data tables serialization to and from Parquet files - in the form of files on disk, memory buffer or io.Reader/io.Writer.
 Package parquet provides tools for data tables serialization to and from Parquet files - in the form of files on disk, memory buffer or io.Reader/io.Writer.
example

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.