yson

package
v0.0.17 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 14, 2024 License: Apache-2.0 Imports: 18 Imported by: 6

Documentation

Overview

Package yson implements encoding and decoding of YSON.

See https://wiki.yandex-team.ru/yt/userdoc/yson/ for introduction to YSON.

Package provides interface similar to encoding/json package.

type myValue struct {
     A int    `yson:"a"`
     B string `yson:"b"`
}

func Example() error {
    var v myValue
    b := []byte("{a=1;b=c;}")

    if err := yson.Unmarshal(b, &v); err != nil {
        return err
    }

    if err, b = yson.Marshal(v); err != nil {
        return err
    }

    return nil
}

Index

Examples

Constants

This section is empty.

Variables

View Source
var ErrIntegerOverflow = errors.New("yson: integer overflow")
View Source
var ErrInvalidNesting = errors.New("invalid YSON nesting")

Functions

func AttrsOf

func AttrsOf(value any) map[string]any

AttrsOf is a helper function to work with ValueWithAttrs. If value is a pointer to ValueWithAttrs, returns attributes. Otherwise returns nil.

func InferAttrs

func InferAttrs(v any) ([]string, error)

InferAttrs infers attribute names from struct.

Function returns attribute names of struct fields including exported fields of yson-untagged anonymous fields

func IsTrue

func IsTrue(value any) bool

IsTrue is a helper function for compatibility with C++ ConvertTo<bool>().

It returns true, if value is bool(true) or string("true").

Early versions of YSON was missing separate wire type for bool. Instead, string "true" was used to specify true value. C++ code still accepts string value where bool is expected, for compatibility reasons.

func Marshal

func Marshal(value any) ([]byte, error)

Marshal returns YSON encoding of value.

Marshal traverses value recursively.

If value implements Marshaler interface, Marshal calls its MarshalYSON method to produce YSON.

If value implements StreamMarshaler interface, Marshal calls its MarshalYSON method to produce YSON.

Otherwise, the following default encoding is used.

Boolean values are encoded as YSON booleans.

Floating point values are encoded as YSON float64.

Unsigned integer types uint8, uint16, uint32, uint64 and uint are encoded as unsigned YSON integers.

Signed integer types int8, int16, int32, int64 and int are encoded as signed YSON integers.

string and []byte values are encoded as YSON strings. Note that YSON strings are always binary.

Slice values are encoded as YSON lists (with exception of []byte).

Struct values are encoded as YSON maps by default. Encoding of each struct field can be customized by format string stored under the "yson" key in the field's tag.

// Field appears in YSON under key "my_field".
Field int `yson:"my_field"`

// Field appears as attribute with name "my_attr".
Field int `yson:"my_attr,attr"`

// Field encoding completely replaces encoding of the whole struct.
// Other fields annotated as ",attr" are encoded as attributes preceding the value.
// All other fields are ignored.
Field int `yson:",value"`

// Field is skipped if empty.
Field int `yson:",omitempty"'

// Field is ignored by this package
Field int `yson:"-"`

// Field appears in YSON under key "-"
Field int `yson:"-,"`

Map values are encoded as YSON maps. The map's key type must either be a string, implement encoding.TextMarshaler, or implement encoding.BinaryMarshaler. The map keys are sorted and used as YSON map keys by applying the following rules:

  • keys of any string type are used directly
  • encoding.TextMarshalers are marshaled
  • encoding.BinaryMarshalers are marshaled

Pointer values are encoded as the value pointed to. A nil pointer encodes as the YSON entity value.

Values implementing encoding.TextMarshaler and encoding.BinaryMarshaler interface are encoded as YSON strings.

Interface values are encoded as the value contained in the interface. A nil interface value encodes as the YSON entity value.

func MarshalFormat

func MarshalFormat(value any, format Format) ([]byte, error)

func MarshalOptions

func MarshalOptions(value any, opts *EncoderOptions) ([]byte, error)

func MarshalTime

func MarshalTime(t Time) (s string, err error)

MarshalTime encodes time to YT-specific time format.

Zero time is encoded into entity.

func MustInferAttrs

func MustInferAttrs(v any) []string

MustInferAttrs is like InferAttrs but panics in case of an error.

Example
package main

import (
	"fmt"
	"strings"

	"go.ytsaurus.tech/yt/go/yson"
)

type Inner struct {
	G bool `yson:"g,attr"`
}

type anonymous struct {
	F *int `yson:"f,attr"`
	*Inner
}

type Anonymous struct {
	F *int `yson:"f,attr"`
	*Inner
}

type MyStruct struct {
	Value  string  `yson:",value"`
	Attr   int64   `yson:"attr,attr"`
	NoName float64 `yson:",attr"`
	NoAttr string  `yson:"no_attr"`
	NoTag  string
	Inner  *Inner `yson:"inner,attr"`
	anonymous
	Anonymous `yson:"a,attr"`
}

var Attrs = yson.MustInferAttrs(&MyStruct{})

func main() {
	fmt.Println(strings.Join(Attrs, ","))

}
Output:

attr,NoName,inner,f,g,a

func RegisterInterfaceDecoder

func RegisterInterfaceDecoder(iface any, decoder DecoderFn)

func SliceYPathAttrs

func SliceYPathAttrs(ypath []byte) (n int, err error)

SliceYPathAttrs splits ypath into attributes and path.

SliceYPathAttrs does not validate that path is a correct ypath.

func SliceYPathString

func SliceYPathString(ypath []byte, str *[]byte) (n int, err error)

func SliceYPathValue

func SliceYPathValue(ypath []byte, value *any) (n int, err error)

SliceYPathValue decodes single YSON value from beginning of ypath.

Returned value might point into the buffer.

func Unmarshal

func Unmarshal(data []byte, v any) error

Unmarshal parses YSON-encoded data and stores result in the value pointed by v. If v is nil or not a pointer, Unmarshal returns UnsupportedTypeError.

Unmarshal works similar to encoding/json package.

Mapping between YSON types and go objects is the same as in Marshal().

func UnmarshalOptions

func UnmarshalOptions(data []byte, v any, opts *DecoderOptions) error

func Valid

func Valid(data []byte) error

Valid checks that byte sequence is a valid YSON node.

func ValidListFragment

func ValidListFragment(data []byte) error

ValidListFragment checks that byte sequence is a valid YSON list fragment.

func ValidMapFragment

func ValidMapFragment(data []byte) error

ValidMapFragment checks that byte sequence is a valid YSON map fragment.

func ValueOf

func ValueOf(value any) any

ValueOf is a helper function to work with ValueWithAttrs. If value is a pointer to ValueWithAttrs, returns inner value. Otherwise returns value directly.

Types

type Decoder

type Decoder struct {
	R *Reader
	// contains filtered or unexported fields
}

func NewDecoder

func NewDecoder(r io.Reader) *Decoder

func NewDecoderFromBytes

func NewDecoderFromBytes(b []byte) *Decoder

func (*Decoder) CheckFinish

func (d *Decoder) CheckFinish() error

func (*Decoder) Decode

func (d *Decoder) Decode(v any) error

type DecoderFn

type DecoderFn func(*Reader, any) error

type DecoderOptions

type DecoderOptions struct {
	SupportYPAPIMaps bool
}

type Duration

type Duration time.Duration

Duration is an alias for time.Duration with YT specific time representation format.

type Encoder

type Encoder struct {
	// contains filtered or unexported fields
}

Encoder writes YSON to output stream.

func NewEncoder

func NewEncoder(w io.Writer) *Encoder

func NewEncoderWriter

func NewEncoderWriter(w *Writer) *Encoder

func (*Encoder) Encode

func (e *Encoder) Encode(value any) (err error)

type EncoderOptions

type EncoderOptions struct {
	SupportYPAPIMaps bool
}

type Event

type Event int
const (
	EventBeginList Event = iota
	EventEndList
	EventBeginAttrs
	EventEndAttrs
	EventBeginMap
	EventEndMap
	EventLiteral
	EventKey
	EventEOF
)

type Format

type Format int

Format is YSON representation.

const (
	FormatBinary Format = iota
	FormatText
	FormatPretty
)

func (Format) String

func (f Format) String() string

type Marshaler

type Marshaler interface {
	MarshalYSON() ([]byte, error)
}

Marshaler is an interface implemented by types that can encode themselves to YSON.

type RawValue

type RawValue []byte

type Reader

type Reader struct {
	// contains filtered or unexported fields
}

Reader provides streaming access to YSON node.

func NewReader

func NewReader(r io.Reader) *Reader

func NewReaderFromBytes

func NewReaderFromBytes(b []byte) *Reader

func NewReaderKind

func NewReaderKind(r io.Reader, kind StreamKind) *Reader

func NewReaderKindFromBytes

func NewReaderKindFromBytes(b []byte, kind StreamKind) *Reader

func (*Reader) Bool

func (r *Reader) Bool() bool

func (*Reader) Bytes

func (r *Reader) Bytes() []byte

func (*Reader) CheckFinish

func (r *Reader) CheckFinish() error

CheckFinish consumes the rest of input.

func (*Reader) Float64

func (r *Reader) Float64() float64

func (*Reader) Int64

func (r *Reader) Int64() int64

func (*Reader) Next

func (r *Reader) Next(skipAttributes bool) (Event, error)

Next returns next event from yson stream.

func (*Reader) NextKey

func (r *Reader) NextKey() (ok bool, err error)

NextKey returns true if next event in the stream is a key.

If next event is a key, NextKey consumes it.

func (*Reader) NextListItem

func (r *Reader) NextListItem() (ok bool, err error)

NextListItem returns true if next event in the stream is end of list.

func (*Reader) NextRawValue

func (r *Reader) NextRawValue() ([]byte, error)

NextRawValue reads next YSON value from stream.

Returned slice is valid only until next call to the reader.

func (*Reader) String

func (r *Reader) String() string

func (*Reader) Type

func (r *Reader) Type() Type

Type returns type of the last read literal.

func (*Reader) Uint64

func (r *Reader) Uint64() uint64

func (*Reader) Undo

func (r *Reader) Undo(event Event)

Undo last call to Next.

It is not possible to undo call to Next(true).

type StreamKind

type StreamKind int

StreamKind is kind of top level value.

const (
	StreamNode StreamKind = iota
	StreamListFragment
	StreamMapFragment
)

func (StreamKind) String

func (s StreamKind) String() string

type StreamMarshaler

type StreamMarshaler interface {
	MarshalYSON(*Writer) error
}

StreamMarshaler is an interface implemented by types that can encode themselves to YSON.

type StreamUnmarshaler

type StreamUnmarshaler interface {
	UnmarshalYSON(*Reader) error
}

StreamUnmarshaler is an interface implemented by types that can unmarshal themselves from YSON reader.

type SyntaxError

type SyntaxError struct {
	Message string
}

func (*SyntaxError) Error

func (e *SyntaxError) Error() string

type Tag

type Tag struct {
	Name string

	Omitempty bool
	Value     bool
	Attr      bool
	Attrs     bool
	Key       bool
}

func ParseTag

func ParseTag(fieldName string, fieldTag reflect.StructTag) (tag *Tag, skip bool)

ParseTag parses yson annotation for struct field tag.

type Time

type Time time.Time

Time is an alias for time.Time with YT specific time representation format.

func UnmarshalTime

func UnmarshalTime(in string) (t Time, err error)

UnmarshalTime decodes time from YT-specific time format.

Entity is decoded into zero time.

func (Time) IsZero added in v0.0.13

func (t Time) IsZero() bool

func (Time) MarshalText

func (t Time) MarshalText() (text []byte, err error)

func (*Time) UnmarshalText

func (t *Time) UnmarshalText(text []byte) error

type Type

type Type int

Type is a logical YSON type.

const (
	// TypeEntity is YSON equivalent to nil.
	TypeEntity Type = iota
	TypeBool
	TypeString
	TypeInt64
	TypeUint64
	TypeFloat64
	TypeList
	TypeMap
)

func (Type) String

func (t Type) String() string

type TypeError

type TypeError struct {
	UserType reflect.Type
	YSONType Type

	Struct string
	Field  string
}

func (*TypeError) Error

func (e *TypeError) Error() string

type Unmarshaler

type Unmarshaler interface {
	UnmarshalYSON([]byte) error
}

Unmarshaler is an interface implemented by types that can unmarshal themselves from YSON. Input can be assumed to be a valid YSON value.

type UnsupportedTypeError

type UnsupportedTypeError struct {
	UserType reflect.Type
}

func (*UnsupportedTypeError) Error

func (e *UnsupportedTypeError) Error() string

type ValueWithAttrs

type ValueWithAttrs struct {
	Attrs map[string]any `yson:",attrs"`
	Value any            `yson:",value"`
}

ValueWithAttrs is a generic representation of YSON node with attached attributes.

type Writer

type Writer struct {
	// contains filtered or unexported fields
}

Writer writes YSON stream to underlying byte stream.

Example usage:

var buf bytes.Buffer
w := yson.NewWriter(&buf)

w.BeginMap()
w.MapKeyString("a")
w.Bool(false)
w.EndMap()

if err := w.Flush(); err != nil {
    return err
}

yson := buf.Bytes()

Most users should not use Writer directly. Use higher level Marshal() function instead.

Writer ignores all calls after the first error.

func NewWriter

func NewWriter(inner io.Writer) *Writer

NewWriter returns new writer configured with text format.

func NewWriterConfig

func NewWriterConfig(inner io.Writer, c WriterConfig) (w *Writer)

func NewWriterFormat

func NewWriterFormat(inner io.Writer, format Format) (w *Writer)

func (*Writer) Any

func (w *Writer) Any(v any)

func (*Writer) BeginAttrs

func (w *Writer) BeginAttrs()

func (*Writer) BeginList

func (w *Writer) BeginList()

func (*Writer) BeginMap

func (w *Writer) BeginMap()

func (*Writer) Bool

func (w *Writer) Bool(b bool)

func (*Writer) Bytes

func (w *Writer) Bytes(b []byte)

func (*Writer) EndAttrs

func (w *Writer) EndAttrs()

func (*Writer) EndList

func (w *Writer) EndList()

func (*Writer) EndMap

func (w *Writer) EndMap()

func (*Writer) Entity

func (w *Writer) Entity()

func (*Writer) Err

func (w *Writer) Err() error

func (*Writer) Finish

func (w *Writer) Finish() error

func (*Writer) Float64

func (w *Writer) Float64(f float64)

func (*Writer) Int64

func (w *Writer) Int64(i int64)

func (*Writer) MapKeyBytes

func (w *Writer) MapKeyBytes(k []byte)

func (*Writer) MapKeyString

func (w *Writer) MapKeyString(k string)

func (*Writer) RawNode

func (w *Writer) RawNode(raw []byte)

func (*Writer) String

func (w *Writer) String(s string)

func (*Writer) Uint64

func (w *Writer) Uint64(u uint64)

type WriterConfig

type WriterConfig struct {
	Format Format
	Kind   StreamKind
}

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL