arrjson

package
v0.0.0-...-bc21918 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 12, 2021 License: Apache-2.0, BSD-2-Clause, BSD-3-Clause, + 8 more Imports: 18 Imported by: 0

Documentation

Overview

Package arrjson provides types and functions to encode and decode ARROW types and data to and from JSON files.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Array

type Array struct {
	Name     string        `json:"name"`
	Count    int           `json:"count"`
	Valids   []int         `json:"VALIDITY,omitempty"`
	Data     []interface{} `json:"DATA,omitempty"`
	Offset   []int32       `json:"OFFSET,omitempty"`
	Children []Array       `json:"children,omitempty"`
}

type Field

type Field struct {
	Name string `json:"name"`

	// leave this as a json RawMessage in order to partially unmarshal as needed
	// during marshal/unmarshal time so we can determine what the structure is
	// actually expected to be.
	Type     json.RawMessage `json:"type"`
	Nullable bool            `json:"nullable"`
	Children []FieldWrapper  `json:"children"`

	Metadata []metaKV `json:"metadata,omitempty"`
	// contains filtered or unexported fields
}

type FieldWrapper

type FieldWrapper struct {
	Field
}

FieldWrapper gets used in order to hook into the JSON marshalling and unmarshalling without creating an infinite loop when dealing with the children fields.

func (FieldWrapper) MarshalJSON

func (f FieldWrapper) MarshalJSON() ([]byte, error)

func (*FieldWrapper) UnmarshalJSON

func (f *FieldWrapper) UnmarshalJSON(data []byte) error

type Option

type Option func(*config)

Option is a functional option to configure opening or creating Arrow files and streams.

func WithAllocator

func WithAllocator(mem memory.Allocator) Option

WithAllocator specifies the Arrow memory allocator used while building records.

func WithSchema

func WithSchema(schema *arrow.Schema) Option

WithSchema specifies the Arrow schema to be used for reading or writing.

type Reader

type Reader struct {
	// contains filtered or unexported fields
}

func NewReader

func NewReader(r io.Reader, opts ...Option) (*Reader, error)

func (*Reader) NumRecords

func (r *Reader) NumRecords() int

func (*Reader) Read

func (r *Reader) Read() (array.Record, error)

func (*Reader) Release

func (r *Reader) Release()

Release decreases the reference count by 1. When the reference count goes to zero, the memory is freed. Release may be called simultaneously from multiple goroutines.

func (*Reader) Retain

func (r *Reader) Retain()

Retain increases the reference count by 1. Retain may be called simultaneously from multiple goroutines.

func (*Reader) Schema

func (r *Reader) Schema() *arrow.Schema

type Record

type Record struct {
	Count   int64   `json:"count"`
	Columns []Array `json:"columns"`
}

type Schema

type Schema struct {
	Fields []FieldWrapper `json:"fields"`

	Metadata []metaKV `json:"metadata,omitempty"`
	// contains filtered or unexported fields
}

func (Schema) MarshalJSON

func (s Schema) MarshalJSON() ([]byte, error)

func (*Schema) UnmarshalJSON

func (s *Schema) UnmarshalJSON(data []byte) error

type Writer

type Writer struct {
	// contains filtered or unexported fields
}

func NewWriter

func NewWriter(w io.Writer, schema *arrow.Schema) (*Writer, error)

func (*Writer) Close

func (w *Writer) Close() error

func (*Writer) Write

func (w *Writer) Write(rec array.Record) error

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL