recordwriter

package
v2.1.7 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 8, 2023 License: Apache-2.0 Imports: 9 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func NewLineJSON

func NewLineJSON[T any](
	it iterator.RecordIterator[T],
	w io.Writer,
) error

NewLineJSON writes all the records from the records iterator as newline json to the writer. returns the first error from either the record iterator or the json encoding.

func NewLineJSONPartitionedBySize

func NewLineJSONPartitionedBySize[T any](
	it iterator.RecordIterator[T],
	wf writerfactory.WriterFactory,
	maxBytesParBatch int,
	baseName string,
	gz bool,
) error

NewLineJSON writes all the records from the records iterator as newline json to the writer. returns the first error from either the record iterator or the json encoding. if gz is true, the output will be gzipped and the size limit will be based on gz records will be put in batches of maxBytesParBatch bytes into the writer factory with a name of $basename_%06d$index.ndjson(.gz)

Types

This section is empty.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL