Documentation
¶
Index ¶
- func Read(buffer []byte, desiredSchema typeof.Schema) (column.Columns, error)
- func Transform(filter *typeof.Schema, computed ...computed.Computed) applyFunc
- type Block
- func FromBatchBy(batch *talaria.Batch, partitionBy string, filter *typeof.Schema, ...) ([]Block, error)
- func FromBuffer(b []byte) (block Block, err error)
- func FromCSVBy(input []byte, partitionBy string, filter *typeof.Schema, apply applyFunc) ([]Block, error)
- func FromColumns(key string, columns column.Columns) (blk Block, err error)
- func FromOrcBy(payload []byte, partitionBy string, filter *typeof.Schema, apply applyFunc) ([]Block, error)
- func FromParquetBy(payload []byte, partitionBy string, filter *typeof.Schema, apply applyFunc) ([]Block, error)
- func FromRequestBy(request *talaria.IngestRequest, partitionBy string, filter *typeof.Schema, ...) ([]Block, error)
- func FromRequestWithTable(request *talaria.IngestWithTableRequest, partitionBy string, ...) ([]Block, error)
- func FromURLBy(uri string, partitionBy string, filter *typeof.Schema, apply applyFunc) ([]Block, error)
- type Row
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
Types ¶
type Block ¶
type Block struct {
Size int64 // The unencoded size of the block
Key nocopy.String // The key of the block
Columns nocopy.ByteMap // The encoded column metadata
Data nocopy.Bytes // The set of columnar data
Expires int64 // The expiration time for the block, in unix seconds
// contains filtered or unexported fields
}
Block represents a serialized block
func FromBatchBy ¶
func FromBatchBy(batch *talaria.Batch, partitionBy string, filter *typeof.Schema, apply applyFunc) ([]Block, error)
FromBatchBy creates a block from a talaria protobuf-encoded batch. It repartitions the batch by a given partition key at the same time.
func FromBuffer ¶
FromBuffer unmarshals a block from a in-memory buffer.
func FromCSVBy ¶ added in v1.1.6
func FromCSVBy(input []byte, partitionBy string, filter *typeof.Schema, apply applyFunc) ([]Block, error)
FromCSVBy creates a block from a comma-separated file. It repartitions the batch by a given partition key at the same time.
func FromColumns ¶
FromColumns creates a block from a set of presto named columns
func FromOrcBy ¶
func FromOrcBy(payload []byte, partitionBy string, filter *typeof.Schema, apply applyFunc) ([]Block, error)
FromOrcBy decodes a set of blocks from an orc file and repartitions it by the specified partition key.
func FromParquetBy ¶ added in v1.2.6
func FromParquetBy(payload []byte, partitionBy string, filter *typeof.Schema, apply applyFunc) ([]Block, error)
FromParquetBy decodes a set of blocks from a Parquet file and repartitions it by the specified partition key.
func FromRequestBy ¶
func FromRequestBy(request *talaria.IngestRequest, partitionBy string, filter *typeof.Schema, funcs ...applyFunc) ([]Block, error)
FromRequestBy creates a block from a talaria protobuf-encoded request. It repartitions the batch by a given partition key at the same time.
func FromRequestWithTable ¶ added in v1.6.1
func FromRequestWithTable(request *talaria.IngestWithTableRequest, partitionBy string, filter *typeof.Schema, funcs ...applyFunc) ([]Block, error)
FromRequestByTable creates a block from a talaria protobuf-encoded request. It repartitions the batch by a given partition key at the same time.
func FromURLBy ¶ added in v1.1.6
func FromURLBy(uri string, partitionBy string, filter *typeof.Schema, apply applyFunc) ([]Block, error)
FromURLBy creates a block from a remote url which should be loaded. It repartitions the batch by a given partition key at the same time.
type Row ¶ added in v1.2.5
Row represents a single row on which we can perform transformations
func FromBlockBy ¶ added in v1.6.0
FromBlockBy creates and returns a list of new block.Row for a block.
Source Files