container

package
v5.2.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 8, 2018 License: MIT Imports: 7 Imported by: 0

Documentation

Overview

Container provides a Writer which is capable of serializing gogen-avro structs and writing them in the Avro Object Container File (OCF) format

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type AvroRecord

type AvroRecord interface {
	Serialize(io.Writer) error
	Schema() string
}

AvroRecord is an interface fulfilled by the structs generated by gogen-avro for Avro records.

type CloseableResettableWriter

type CloseableResettableWriter interface {
	Close() error
	Reset(io.Writer)
}

type Codec

type Codec string

A Codec specifies how the blocks within a container file should be compressed.

const (
	// No compression
	Null Codec = "null"
	// Deflate compression
	Deflate Codec = "deflate"
	// Snappy compression
	Snappy Codec = "snappy"
)

type Writer

type Writer struct {
	// contains filtered or unexported fields
}

Writer wraps an io.Writer and writes the file and block-level framing required for an OCF file

func NewWriter

func NewWriter(writer io.Writer, codec Codec, recordsPerBlock int64, schema string) (*Writer, error)

Create a new Writer wrapping the provided io.Writer with the given Codec and number of records per block. The Writer will lazily write the container file header when WriteRecord is called the first time. You must call Flush on the Writer before closing the underlying io.Writer, to ensure the final block is written. A schema string must be passed to ensure that a correct header is written even if no records are written. This is required to produce valid empty Avro container files.

func (*Writer) Flush

func (avroWriter *Writer) Flush() error

Write the current block to the file if it has been filled. It is best-practise to always call this before the underlying io.Writer is closed.

func (*Writer) WriteRecord

func (avroWriter *Writer) WriteRecord(record AvroRecord) error

Write an AvroRecord to the container file. All gogen-avro generated structs fulfill the AvroRecord interface. Note that all records in a given container file must be of the same Avro type.

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL