jreader

package
v1.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 3, 2021 License: Apache-2.0 Imports: 8 Imported by: 5

Documentation

Overview

Package jreader provides an efficient mechanism for reading JSON data sequentially.

The high-level API for this package, Writer, is designed to facilitate writing custom JSON marshaling logic concisely and reliably. Output is buffered in memory. 

import (
    "gopkg.in/launchdarkly/jsonstream.v1/jreader"
)

type myStruct struct {
    value int
}

func (s *myStruct) ReadFromJSONReader(r *jreader.Reader) {
    // reading a JSON object structure like {"value":2}
    for obj := r.Object(); obj.Next; {
        if string(obj.Name()) == "value" {
            s.value = r.Int()
        }
    }
}

func ParseMyStructJSON() {
    var s myStruct
    r := jreader.NewReader([]byte(`{"value":2}`))
    s.ReadFromJSONReader(&r)
    fmt.Printf("%+v\n", s)
}

The underlying low-level token parsing mechanism has two available implementations. The default implementation has no external dependencies. For interoperability with the easyjson library (https://github.com/mailru/easyjson), there is also an implementation that delegates to the easyjson streaming parser; this is enabled by setting the build tag "launchdarkly_easyjson". Be aware that by default, easyjson uses Go's "unsafe" package (https://pkg.go.dev/unsafe), which may not be available on all platforms.

Setting the "launchdarkly_easyjson" tag also adds a new constructor function, NewReaderFromEasyJSONLexer, allowing Reader-based code to read directly from an existing EasyJSON jlexer.Lexer. This may be desirable in order to define common unmarshaling logic that may be used with or without EasyJSON. For example: 

import (
    "github.com/mailru/easyjson/jlexer"
)

func (s *myStruct) UnmarshalEasyJSON(lexer *jlexer.Lexer) {
    r := jreader.NewReaderFromEasyJSONLexer(lexer)
    s.ReadFromJSONReader(&r)
}
Example 
r := NewReader([]byte(`"a \"good\" string"`))

s := r.String()

if err := r.Error(); err != nil {
	fmt.Println("error:", err.Error())
} else {
	fmt.Println(s)
}

Output:

a "good" string

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func ToJSONError 

func ToJSONError(err error, target interface{}) error

ToJSONError converts errors defined by the jreader package into the corresponding error types defined by the encoding/json package, if any. The target parameter, if not nil, is used to determine the target value type for json.UnmarshalTypeError.

func UnmarshalJSONWithReader 

func UnmarshalJSONWithReader(data []byte, readable Readable) error

UnmarshalJSONWithReader is a convenience method for implementing json.Marshaler to unmarshal from a byte slice with the default TokenReader implementation. If an error occurs, it is converted to the corresponding error type defined by the encoding/json package when applicable.

This method will generally be less efficient than writing the exact same logic inline in a custom UnmarshalJSON method for the object's specific type, because casting a pointer to an interface (Readable) will force the object to be allocated on the heap if it was not already.

Types

type AnyValue 

type AnyValue struct {
	// Kind describes the type of the JSON value.
	Kind ValueKind

	// Bool is the value if the JSON value is a boolean, or false otherwise.
	Bool bool

	// Number is the value if the JSON value is a number, or zero otherwise.
	Number float64

	// String is the value if the JSON value is a string, or an empty string otherwise.
	String string

	// Array is an ArrayState that can be used to iterate through the array elements if the JSON
	// value is an array, or an uninitialized ArrayState{} otherwise.
	Array ArrayState

	// Object is an ObjectState that can be used to iterate through the object properties if the
	// JSON value is an object, or an uninitialized ObjectState{} otherwise.
	Object ObjectState
}

AnyValue is returned by Reader.Any() to represent a JSON value of an arbitrary type.

type ArrayState 

type ArrayState struct {
	// contains filtered or unexported fields
}

ArrayState is returned by Reader's Array and ArrayOrNull methods. Use it in conjunction with Reader to iterate through a JSON array. To read the value of each array element, you will still use the Reader's methods.

This example reads an array of strings; if there is a null instead of an array, it behaves the same as for an empty array. Note that it is not necessary to check for an error result before iterating over the ArrayState, or to break out of the loop if String causes an error, because the ArrayState's Next method will return false if the Reader has had any errors. 

var values []string
for arr := r.ArrayOrNull(); arr.Next(); {
    if s := r.String(); r.Error() == nil {
        values = append(values, s)
    }
}

func (*ArrayState) IsDefined 

func (arr *ArrayState) IsDefined() bool

IsDefined returns true if the ArrayState represents an actual array, or false if it was parsed from a null value or was the result of an error. If IsDefined is false, Next will always return false. The zero value ArrayState{} returns false for IsDefined.

func (*ArrayState) Next 

func (arr *ArrayState) Next() bool

Next checks whether an array element is available and returns true if so. It returns false if the Reader has reached the end of the array, or if any previous Reader operation failed, or if the array was empty or null.

If Next returns true, you can then use Reader methods such as Bool or String to read the element value. If you do not care about the value, simply calling Next again without calling a Reader method will discard the value, just as if you had called SkipValue on the reader.

See ArrayState for example code.

type ObjectState 

type ObjectState struct {
	// contains filtered or unexported fields
}

ObjectState is returned by Reader's Object and ObjectOrNull methods. Use it in conjunction with Reader to iterate through a JSON object. To read the value of each object property, you will still use the Reader's methods. Properties may appear in any order.

This example reads an object whose values are strings; if there is a null instead of an object, it behaves the same as for an empty object. Note that it is not necessary to check for an error result before iterating over the ObjectState, or to break out of the loop if String causes an error, because the ObjectState's Next method will return false if the Reader has had any errors. 

values := map[string]string
for obj := r.ObjectOrNull(); obj.Next(); {
    key := string(obj.Name())
    if s := r.String(); r.Error() == nil {
        values[key] = s
    }
}

The next example reads an object with two expected property names, "a" and "b". Any unrecognized properties are ignored. 

var result struct {
    a int
    b int
}
for obj := r.ObjectOrNull(); obj.Next(); {
    switch string(obj.Name()) {
    case "a":
        result.a = r.Int()
    case "b":
        result.b = r.Int()
    }
}

If the schema requires certain properties to always be present, the WithRequiredProperties method is a convenient way to enforce this.

func (*ObjectState) IsDefined 

func (obj *ObjectState) IsDefined() bool

IsDefined returns true if the ObjectState represents an actual object, or false if it was parsed from a null value or was the result of an error. If IsDefined is false, Next will always return false. The zero value ObjectState{} returns false for IsDefined.

func (*ObjectState) Name 

func (obj *ObjectState) Name() []byte

Name returns the name of the current object property, or nil if there is no current property (that is, if Next returned false or if Next was never called).

For efficiency, to avoid allocating a string for each property name, the name is returned as a byte slice which may refer directly to the source data. Casting this to a string within a simple comparison expression or switch statement should not cause a string allocation; the Go compiler optimizes these into direct byte-slice comparisons.

func (*ObjectState) Next 

func (obj *ObjectState) Next() bool

Next checks whether an object property is available and returns true if so. It returns false if the Reader has reached the end of the object, or if any previous Reader operation failed, or if the object was empty or null.

If Next returns true, you can then get the property name with Name, and use Reader methods such as Bool or String to read the property value. If you do not care about the value, simply calling Next again without calling a Reader method will discard the value, just as if you had called SkipValue on the reader.

See ObjectState for example code.

func (ObjectState) WithRequiredProperties 

func (obj ObjectState) WithRequiredProperties(requiredProps []string) ObjectState

WithRequiredProperties adds a requirement that the specified JSON property name(s) must appear in the JSON object at some point before it ends.

This method returns a new, modified ObjectState. It should be called before the first time you call Next. For instance: 

requiredProps := []string{"key", "name"}
for obj := reader.Object().WithRequiredProperties(requiredProps); obj.Next(); {
    switch string(obj.Name()) { ... }
}

When the end of the object is reached (and Next() returns false), if one of the required properties has not yet been seen, and no other error has occurred, the Reader's error state will be set to a RequiredPropertyError.

For efficiency, it is best to preallocate the list of property names globally rather than creating it inline.

Example 
requiredProps := []string{"key", "name"}
r := NewReader([]byte(`{"name": "x"}`))
var key, name string
for obj := r.Object().WithRequiredProperties(requiredProps); obj.Next(); {
	switch string(obj.Name()) {
	case "key":
		key = r.String()
	case "name":
		name = r.String()
	}
}
if err := r.Error(); err != nil {
	if rpe, ok := err.(RequiredPropertyError); ok {
		fmt.Println("missing property:", rpe.Name)
	} else {
		fmt.Println("unexpected error:", err)
	}
} else {
	fmt.Println(key, name)
}

Output:

missing property: key

type Readable 

type Readable interface {
	// ReadFromJSONReader attempts to read the object's state from a Reader.
	//
	// This method does not need to return an error value because Reader remembers when it
	// has encountered an error.
	ReadFromJSONReader(*Reader)
}

Readable is an interface for types that can read their data from a Reader.

type Reader 

type Reader struct {
	// contains filtered or unexported fields
}

Reader is a high-level API for reading JSON data sequentially.

It is designed to make writing custom unmarshallers for application types as convenient as possible. The general usage pattern is as follows:

- Values are parsed in the order that they appear.

- In general, the caller should know what data type is expected. Since it is common for properties to be nullable, the methods for reading scalar types have variants for allowing a null instead of the specified type. If the type is completely unknown, use Any.

- For reading array or object structures, the Array and Object methods return a struct that keeps track of additional reader state while that structure is being parsed.

- If any method encounters an error (due to either malformed JSON, or well-formed JSON that did not match the caller's data type expectations), the Reader permanently enters a failed state and remembers that error; all subsequent method calls will return the same error and no more parsing will happen. This means that the caller does not necessarily have to check the error return value of any individual method, although it can.

func NewReader 

func NewReader(data []byte) Reader

NewReader creates a Reader that consumes the specified JSON input data.

This function returns the struct by value (Reader, not *Reader). This avoids the overhead of a heap allocation since, in typical usage, the Reader will not escape the scope in which it was declared and can remain on the stack.

Example 
r := NewReader([]byte(`"a \"good\" string"`))
s := r.String()
if err := r.Error(); err != nil {
	fmt.Println("error:", err.Error())
} else {
	fmt.Println(s)
}

Output:

a "good" string

func (*Reader) AddError 

func (r *Reader) AddError(err error)

AddError sets the Reader's error value and puts it into a failed state. If the parameter is nil or the Reader was already in a failed state, it does nothing.

Example 
r := NewReader([]byte(`[1,2,3,4,5]`))
values := []int{}
for arr := r.Array(); arr.Next(); {
	n := r.Int()
	values = append(values, n)
	if n > 1 {
		r.AddError(fmt.Errorf("got an error after %d", n))
	}
}
err := r.Error()
fmt.Println(values, err)

Output:

[1 2] got an error after 2

func (*Reader) Any 

func (r *Reader) Any() AnyValue

Any reads a single value of any type, if it is a scalar value or a null, or prepares to read the value if it is an array or object.

The returned AnyValue's Kind field indicates the value type. If it is BoolValue, NumberValue, or StringValue, check the corresponding Bool, Number, or String property. If it is ArrayValue or ObjectValue, the AnyValue's Array or Object field has been initialized with an ArrayState or ObjectState just as if you had called the Reader's Array or Object method.

If there is a parsing error, the return value is the same as for a null and the Reader enters a failed state, which you can detect with Error().

Example 
printValue := func(input string) {
	r := NewReader([]byte(input))
	value := r.Any()
	switch value.Kind {
	case NullValue:
		fmt.Println("a null")
	case BoolValue:
		fmt.Println("a bool:", value.Bool)
	case NumberValue:
		fmt.Println("a number:", value.Number)
	case StringValue:
		fmt.Println("a string:", value.String)
	case ArrayValue:
		n := 0
		for value.Array.Next() {
			n++ // for this example, we're not looking at the actual element value
		}
		fmt.Println("an array with", n, "elements")
	case ObjectValue:
		n := 0
		for value.Object.Next() {
			n++ // for this example, we're not looking at the actual element value
		}
	}
}
printValue(`123`)
printValue(`["a","b"]`)

Output:

a number: 123
an array with 2 elements

func (*Reader) Array 

func (r *Reader) Array() ArrayState

Array attempts to begin reading a JSON array value. If successful, the return value will be an ArrayState containing the necessary state for iterating through the array elements.

The ArrayState is used only for the iteration state; to read the value of each array element, you will still use the Reader's methods.

If there is a parsing error, or the next value is not an array, the returned ArrayState is a stub whose Next() method always returns false, and the Reader enters a failed state, which you can detect with Error().

See ArrayState for example code.

Example 
r := NewReader([]byte(`[1,2]`))
values := []int{}
for arr := r.Array(); arr.Next(); {
	values = append(values, r.Int())
}
fmt.Println("values:", values)

Output:

values: [1 2]

func (*Reader) ArrayOrNull 

func (r *Reader) ArrayOrNull() ArrayState

ArrayOrNull attempts to either begin reading an JSON array value, or read a null. In the case of an array, the return value will be an ArrayState containing the necessary state for iterating through the array elements; the ArrayState's IsDefined() method will return true. In the case of a null, the returned ArrayState will be a stub whose Next() and IsDefined() methods always returns false.

The ArrayState is used only for the iteration state; to read the value of each array element, you will still use the Reader's methods.

If there is a parsing error, or the next value is neither an array nor a null, the return value is the same as for a null but the Reader enters a failed state, which you can detect with Error().

See ArrayState for example code.

Example 
printArray := func(input string) {
	r := NewReader([]byte(input))
	values := []int{}
	arr := r.Array()
	for arr.Next() {
		values = append(values, r.Int())
	}
	fmt.Println(input, "->", values, "... IsDefined =", arr.IsDefined())
}
printArray("null")
printArray("[1,2]")

Output:

null -> [] ... IsDefined = false
[1,2] -> [1 2] ... IsDefined = true

func (*Reader) Bool 

func (r *Reader) Bool() bool

Bool attempts to read a boolean value.

If there is a parsing error, or the next value is not a boolean, the return value is false and the Reader enters a failed state, which you can detect with Error().

Example 
r := NewReader([]byte(`true`))
var value bool = r.Bool()
if err := r.Error(); err != nil {
	fmt.Println("error:", err)
} else {
	fmt.Println("value:", value)
}

Output:

value: true

func (*Reader) BoolOrNull 

func (r *Reader) BoolOrNull() (value bool, nonNull bool)

BoolOrNull attempts to read either a boolean value or a null. In the case of a boolean, the return values are (value, true); for a null, they are (false, false).

If there is a parsing error, or the next value is neither a boolean nor a null, the return values are (false, false) and the Reader enters a failed state, which you can detect with Error().

Example 
r1 := NewReader([]byte(`null`))
if value1, nonNull := r1.BoolOrNull(); nonNull {
	fmt.Println("value1:", value1)
}
r2 := NewReader([]byte(`false`))
if value2, nonNull := r2.BoolOrNull(); nonNull {
	fmt.Println("value2:", value2)
}

Output:

value2: false

func (*Reader) Error 

func (r *Reader) Error() error

Error returns the first error that the Reader encountered, if the Reader is in a failed state, or nil if it is still in a good state.

func (*Reader) Float64 

func (r *Reader) Float64() float64

Float64 attempts to read a numeric value and returns it as a float64.

If there is a parsing error, or the next value is not a number, the return value is zero and the Reader enters a failed state, which you can detect with Error(). Non-numeric types are never converted to numbers.

Example 
r := NewReader([]byte(`1234.5`))
var value float64 = r.Float64()
if err := r.Error(); err != nil {
	fmt.Println("error:", err)
} else {
	fmt.Println("value:", value)
}

Output:

value: 1234.5

func (*Reader) Float64OrNull 

func (r *Reader) Float64OrNull() (float64, bool)

Float64OrNull attempts to read either a numeric value or a null. In the case of a number, the return values are (value, true); for a null, they are (0, false).

If there is a parsing error, or the next value is neither a number nor a null, the return values are (0, false) and the Reader enters a failed state, which you can detect with Error().

Example 
r1 := NewReader([]byte(`null`))
if value1, nonNull := r1.Float64OrNull(); nonNull {
	fmt.Println("value1:", value1)
}
r2 := NewReader([]byte(`0`))
if value2, nonNull := r2.Float64OrNull(); nonNull {
	fmt.Println("value2:", value2)
}

Output:

value2: 0

func (*Reader) Int 

func (r *Reader) Int() int

Int attempts to read a numeric value and returns it as an int.

If there is a parsing error, or the next value is not a number, the return value is zero and the Reader enters a failed state, which you can detect with Error(). Non-numeric types are never converted to numbers.

Example 
r := NewReader([]byte(`123`))
var value int = r.Int()
if err := r.Error(); err != nil {
	fmt.Println("error:", err)
} else {
	fmt.Println("value:", value)
}

Output:

value: 123

func (*Reader) IntOrNull 

func (r *Reader) IntOrNull() (int, bool)

IntOrNull attempts to read either an integer numeric value or a null. In the case of a number, the return values are (value, true); for a null, they are (0, false).

If there is a parsing error, or the next value is neither a number nor a null, the return values are (0, false) and the Reader enters a failed state, which you can detect with Error().

Example 
r1 := NewReader([]byte(`null`))
if value1, nonNull := r1.IntOrNull(); nonNull {
	fmt.Println("value1:", value1)
}
r2 := NewReader([]byte(`0`))
if value2, nonNull := r2.IntOrNull(); nonNull {
	fmt.Println("value2:", value2)
}

Output:

value2: 0

func (*Reader) Null 

func (r *Reader) Null() error

Null attempts to read a null value, returning an error if the next token is not a null.

Example 
r := NewReader([]byte(`null`))
if err := r.Null(); err != nil {
	fmt.Println("error:", err)

} else {
	fmt.Println("got a null")
}

Output:

got a null

func (*Reader) Object 

func (r *Reader) Object() ObjectState

Object attempts to begin reading a JSON object value. If successful, the return value will be an ObjectState containing the necessary state for iterating through the object properties.

The ObjectState is used only for the iteration state; to read the value of each property, you will still use the Reader's methods.

If there is a parsing error, or the next value is not an object, the returned ObjectState is a stub whose Next() method always returns false, and the Reader enters a failed state, which you can detect with Error().

See ObjectState for example code.

Example 
r := NewReader([]byte(`{"a":1,"b":2}`))
items := []string{}
for obj := r.Object(); obj.Next(); {
	name := obj.Name()
	value := r.Int()
	items = append(items, fmt.Sprintf("%s=%d", name, value))
}
fmt.Println("items:", items)

Output:

items: [a=1 b=2]

func (*Reader) ObjectOrNull 

func (r *Reader) ObjectOrNull() ObjectState

ObjectOrNull attempts to either begin reading an JSON object value, or read a null. In the case of an object, the return value will be an ObjectState containing the necessary state for iterating through the object properties; the ObjectState's IsDefined() method will return true. In the case of a null, the returned ObjectState will be a stub whose Next() and IsDefined() methods always returns false.

The ObjectState is used only for the iteration state; to read the value of each property, you will still use the Reader's methods.

If there is a parsing error, or the next value is neither an object nor a null, the return value is the same as for a null but the Reader enters a failed state, which you can detect with Error().

See ObjectState for example code.

Example 
printObject := func(input string) {
	r := NewReader([]byte(input))
	items := []string{}
	obj := r.Object()
	for obj.Next() {
		name := obj.Name()
		value := r.Int()
		items = append(items, fmt.Sprintf("%s=%d", name, value))
	}
	fmt.Println(input, "->", items, "... IsDefined =", obj.IsDefined())
}
printObject("null")
printObject(`{"a":1,"b":2}`)

Output:

null -> [] ... IsDefined = false
{"a":1,"b":2} -> [a=1 b=2] ... IsDefined = true

func (*Reader) ReplaceError 

func (r *Reader) ReplaceError(err error)

ReplaceError sets the Reader's error value and puts it into a failed state, replacing any previously reported error. If the parameter is nil, it does nothing (a failed state cannot be changed to a non-failed state).

func (*Reader) RequireEOF 

func (r *Reader) RequireEOF() error

RequireEOF returns nil if all of the input has been consumed (not counting whitespace), or an error if not.

Example 
r := NewReader([]byte(`100,"extra"`))
n := r.Int()
err := r.RequireEOF()
fmt.Println(n, err)

Output:

100 unexpected data after end of JSON value at position 3

func (*Reader) SkipValue 

func (r *Reader) SkipValue() error

SkipValue consumes and discards the next JSON value of any type. For an array or object value, it recurses to also consume and discard all array elements or object properties.

func (*Reader) String 

func (r *Reader) String() string

String attempts to read a string value.

If there is a parsing error, or the next value is not a string, the return value is "" and the Reader enters a failed state, which you can detect with Error(). Types other than string are never converted to strings.

Example 
r := NewReader([]byte(`"a \"good\" string"`))
var value string = r.String()
if err := r.Error(); err != nil {
	fmt.Println("error:", err)
} else {
	fmt.Println("value:", value)
}

Output:

value: a "good" string

func (*Reader) StringOrNull 

func (r *Reader) StringOrNull() (string, bool)

StringOrNull attempts to read either a string value or a null. In the case of a string, the return values are (value, true); for a null, they are ("", false).

If there is a parsing error, or the next value is neither a string nor a null, the return values are ("", false) and the Reader enters a failed state, which you can detect with Error().

Example 
r1 := NewReader([]byte(`null`))
if value1, nonNull := r1.StringOrNull(); nonNull {
	fmt.Println("value1:", "\""+value1+"\"")
}
r2 := NewReader([]byte(`""`))
if value2, nonNull := r2.StringOrNull(); nonNull {
	fmt.Println("value2:", "\""+value2+"\"")
}

Output:

value2: ""

type RequiredPropertyError 

type RequiredPropertyError struct {
	// Name is the name of a required object property that was not found.
	Name string

	// Offset is the approximate character index within the input where the error occurred
	// (at or near the end of the JSON object).
	Offset int
}

RequiredPropertyError is returned by Reader if a JSON object did not contain a property that was designated as required (by using ObjectState.WithRequiredProperties).

func (RequiredPropertyError) Error 

func (e RequiredPropertyError) Error() string

Error returns a description of the error.

type SyntaxError 

type SyntaxError struct {
	// Message is a descriptive message.
	Message string

	// Offset is the approximate character index within the input where the error occurred.
	Offset int

	// Value, if not empty, is the token that caused the error.
	Value string
}

SyntaxError is returned by Reader if the input is not well-formed JSON.

func (SyntaxError) Error 

func (e SyntaxError) Error() string

Error returns a description of the error.

type TypeError 

type TypeError struct {
	// Expected is the type of JSON value that the caller requested.
	Expected ValueKind

	// Actual is the type of JSON value that was found.
	Actual ValueKind

	// Nullable is true if the caller indicated that a null value was acceptable in this context.
	Nullable bool

	// Offset is the approximate character index within the input where the error occurred.
	Offset int
}

TypeError is returned by Reader if the type of JSON value that was read did not match what the caller requested.

func (TypeError) Error 

func (e TypeError) Error() string

Error returns a description of the error.

type ValueKind 

type ValueKind int

ValueKind defines the allowable value types for Reader.Any. 

const (
	// NullValue means the value is a null.
	NullValue ValueKind = iota

	// BoolValue means the value is a boolean.
	BoolValue ValueKind = iota

	// NumberToken means the value is a number.
	NumberValue ValueKind = iota

	// StringValue means the value is a string.
	StringValue ValueKind = iota

	// ArrayValue means the value is an array.
	ArrayValue ValueKind = iota

	// ObjectValue means the value is an object.
	ObjectValue ValueKind = iota
)

func (ValueKind) String 

func (k ValueKind) String() string

String returns a description of the ValueKind.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.