jsonparser

package module
v0.0.0-...-6b8dae9 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 15, 2017 License: MIT Imports: 7 Imported by: 0

README

License

JsonValue dynamic parser

JsonValue is a lightweight json parses which excells at efficient parsing of dynamic or nested json data. And it does this up to 10 times faster than standard encoding/json package (depending on payload size and usage) with minimal memory footprint. See benchmarks below.

The main reason to use JsonValue is if you do not know the exact sturcture of your data beforehand (as required by encoding/json) or you want efficientaccess to data at certain paths, without needing the whole json tree to be parsed and copied to go structs.

The JsonParser library is build as a minimalistic wrapper on top of the jsonparser library (https://github.com/buger/jsonparser). Many thanks to buger for providing this awesome, blazingly fast json parser! The JsonValue library aims to make it easier and less error prone to parse complicated json structures. Instead of dealing with byte arrays it allows you to parse your json in a simpler OO style. We were also able to fix and improve some functionality which could not be changed before because of backward-compatibity.

Example

For the given JSON our goal is to extract the user's full name, number of github followers and avatar.

import "github.com/stverhae/jsonvalue"

...

data := []byte(`{
  "person": {
    "name": {
      "first": "Leonid",
      "last": "Bugaev",
      "fullName": "Leonid Bugaev"
    },
    "github": {
      "handle": "buger",
      "followers": 109
    },
    "avatars": [
      { "url": "https://avatars1.githubusercontent.com/u/14009?v=3&s=460", "type": "thumbnail" }
    ]
  },
  "company": {
    "name": "Acme"
  }
}`)
//As you work with the library you will always deal with JsonValues, these wrap your actual data and can be used to dive deeper in the json tree and retrieve usable go types (strings, numbers, arrays, maps, etc.

//always start by callig Parse of your json data, this returns a JsonValue
json := jsonvalue.Parse(data)

// You can specify key path by providing arguments to Get function
//this returns a *JsonValue object containing the data you requested
jsonName := json.Get("person", "name", "fullName")

//you can easily check the type of data you got and retrieve it
if jsonName.IsString() {
    fmt.Println(jsonName.GetString())
} else {
    //complain when name is not a string
    fmt.Printf("Help I got an %v!", jsonName.Type)
}

// If you already know what kind of data to expect you can immediately use `GetSting`, `GetInt`, `GetFloat`, `GetBool`, etc. with the correct path
json.GetInt("person", "github", "followers")

// When you give the path to an object, you get a JsonValue (as always) which you can use to further parse the json object
company := json.Get("company") // `company` => `{"name": "Acme"}`
company.GetString("name")
//this is identical to
json.GetString("company", "name")
//also identical to (you get the idea..)
name := json.Get("company", "name").GetString()

// if at any point the parsing fails because your json is malformed or you provided an invalid path the JsonValue will contain an error
//use `JsonValue.Err()` to check this. `JsonValue` implements `Error()` so it is a valid go `error`
test := json.Get("company", "doesnotexist").Get("nono")
if test.Err() != nil {
    fmt.Println("Parsing failed: ", test) //will print the error
}
//if you use a `GetXYZ` function on a JsonValue with a parse error, it will simply return that error.
var size int64
if value, _, err := json.GetInt("company", "size"); err == nil {
  size = value
}

// You can use `ArrayEach` helper to iterate items in a jsonArray [item1, item2 .... itemN]
json.Get("person", "avatars").ArrayEach(func(avatar *JsonValue) {
	fmt.Println(avatar.GetString("url"))
})

// Or use can access fields by index!
json.GetString("person", "avatars", "[0]", "url")


// Or parse the whole array and convert it to a go array (of JsonValues)
avatars := json.Get("person", "avatars")
if avatars.IsArray() {
    for _, avatar := range avatars.ToArray() {
        fmt.Println(avatar.GetString("url"))
    }
}

// You can use `ObjectEach` helper to iterate objects { "key1":object1, "key2":object2, .... "keyN":objectN }
json.Get("person", "name").ObjectEach(func(key string, value *JsonValue) {
        fmt.Printf("Key: '%s' Value: '%s' Type: %s\n", key, value.String(), value.Type)
})

// Or use `ToMap` to concvert the whole object to a go map
for key, value := range json.Get("person", "name").ToMap {
        fmt.Printf("Key: '%s' Value: '%s' Type: %s\n", key, value.String(), value.Type)
})

// The most efficient way to extract multiple keys is `AllKeys`
paths := [][]string{
  {"person", "name", "fullName"},
  {"person", "avatars", "[0]", "url"},
  {"company", "url"},
}
for _, value := range json.AllKeys(paths...) {
    fmt.Println(value.String())
})

// For more information see docs below

Documentation

Index

Constants

View Source
const (
	NotExist = ValueType(iota)
	String
	Number
	Object
	Array
	Boolean
	Null
	Unknown
)

Variables

View Source
var (
	KeyPathNotFoundError       = errors.New("Key path not found")
	UnknownValueTypeError      = errors.New("Unknown value type")
	MalformedJsonError         = errors.New("Malformed JSON error")
	MalformedStringError       = errors.New("Value is string, but can't find closing '\"' symbol")
	MalformedArrayError        = errors.New("Value is array, but can't find closing ']' symbol")
	MalformedObjectError       = errors.New("Value looks like object, but can't find closing '}' symbol")
	MalformedValueError        = errors.New("Value looks like Number/Boolean/None, but can't find its end: ',' or '}' symbol")
	MalformedStringEscapeError = errors.New("Encountered an invalid escape sequence in a string")
)

Errors

Functions

func ArrayEach

func ArrayEach(data []byte, cb func(value []byte, dataType ValueType, offset int, err error), keys ...string) (offset int, err error)

ArrayEach is used when iterating arrays, accepts a callback function with the same return arguments as `Get`.

func EachKey

func EachKey(data []byte, cb func(int, []byte, ValueType, error), paths ...[]string) int

func GetBoolean

func GetBoolean(data []byte, keys ...string) (val bool, err error)

GetBoolean returns the value retrieved by `Get`, cast to a bool if possible. The offset is the same as in `Get`. If key data type do not match, it will return error.

func GetFloat

func GetFloat(data []byte, keys ...string) (val float64, err error)

GetFloat returns the value retrieved by `Get`, cast to a float64 if possible. The offset is the same as in `Get`. If key data type do not match, it will return an error.

func GetInt

func GetInt(data []byte, keys ...string) (val int64, err error)

GetInt returns the value retrieved by `Get`, cast to a int64 if possible. If key data type do not match, it will return an error.

func GetString

func GetString(data []byte, keys ...string) (val string, err error)

GetString returns the value retrieved by `Get`, cast to a string if possible, trying to properly handle escape and utf8 symbols If key data type do not match, it will return an error.

func GetUnsafeString

func GetUnsafeString(data []byte, keys ...string) (val string, err error)

GetUnsafeString returns the value retrieved by `Get`, use creates string without memory allocation by mapping string to slice memory. It does not handle escape symbols.

func ObjectEach

func ObjectEach(data []byte, callback func(key []byte, value []byte, dataType ValueType, offset int) error, keys ...string) (err error)

ObjectEach iterates over the key-value pairs of a JSON object, invoking a given callback for each such entry

func ParseBoolean

func ParseBoolean(b []byte) (bool, error)

ParseBoolean parses a Boolean ValueType into a Go bool (not particularly useful, but here for completeness)

func ParseFloat

func ParseFloat(b []byte) (float64, error)

ParseNumber parses a Number ValueType into a Go float64

func ParseInt

func ParseInt(b []byte) (int64, error)

ParseInt parses a Number ValueType into a Go int64

func ParseString

func ParseString(b []byte) (string, error)

ParseString parses a String ValueType into a Go string (the main parsing work is unescaping the JSON string)

func Unescape

func Unescape(in, out []byte) ([]byte, error)

unescape unescapes the string contained in 'in' and returns it as a slice. If 'in' contains no escaped characters:

Returns 'in'.

Else, if 'out' is of sufficient capacity (guaranteed if cap(out) >= len(in)):

'out' is used to build the unescaped string and is returned with no extra allocation

Else:

A new slice is allocated and returned.

Types

type JsonValue

type JsonValue struct {
	Type ValueType
	// contains filtered or unexported fields
}

func ParseJson

func ParseJson(data []byte, keys ...string) *JsonValue

func (*JsonValue) AllKeys

func (jv *JsonValue) AllKeys(paths ...[]string) (res []*JsonValue, err error)

func (*JsonValue) ArrayEach

func (jv *JsonValue) ArrayEach(cb func(value *JsonValue)) error

func (*JsonValue) ArrayEachWithError

func (jv *JsonValue) ArrayEachWithError(cb func(value *JsonValue) error) error

func (*JsonValue) ArrayEachWithIndex

func (jv *JsonValue) ArrayEachWithIndex(cb func(idx int, value *JsonValue)) error

func (*JsonValue) EachKey

func (jv *JsonValue) EachKey(cb func(idx int, value *JsonValue), paths ...[]string) error

func (*JsonValue) Err

func (jv *JsonValue) Err() error

func (*JsonValue) Error

func (jv *JsonValue) Error() string

func (*JsonValue) Get

func (jv *JsonValue) Get(keys ...string) *JsonValue

func (*JsonValue) GetBool

func (jv *JsonValue) GetBool(keys ...string) (bool, error)

func (*JsonValue) GetBoolArray

func (jv *JsonValue) GetBoolArray(keys ...string) ([]bool, error)

func (*JsonValue) GetFloat

func (jv *JsonValue) GetFloat(keys ...string) (float64, error)

func (*JsonValue) GetFloatArray

func (jv *JsonValue) GetFloatArray(keys ...string) ([]float64, error)

func (*JsonValue) GetInt

func (jv *JsonValue) GetInt(keys ...string) (int64, error)

func (*JsonValue) GetIntArray

func (jv *JsonValue) GetIntArray(keys ...string) ([]int64, error)

func (*JsonValue) GetString

func (jv *JsonValue) GetString(keys ...string) (string, error)

func (*JsonValue) GetStringArray

func (jv *JsonValue) GetStringArray(keys ...string) (res []string, err error)

func (*JsonValue) GetStringUnsafe

func (jv *JsonValue) GetStringUnsafe(keys ...string) (string, error)

func (*JsonValue) Index

func (jv *JsonValue) Index(indices ...int) *JsonValue

func (*JsonValue) IsArray

func (jv *JsonValue) IsArray() bool

func (*JsonValue) IsBoolean

func (jv *JsonValue) IsBoolean() bool

func (*JsonValue) IsFloat

func (jv *JsonValue) IsFloat() bool

func (*JsonValue) IsInt

func (jv *JsonValue) IsInt() bool

func (*JsonValue) IsNumber

func (jv *JsonValue) IsNumber() bool

func (*JsonValue) IsObject

func (jv *JsonValue) IsObject() bool

func (*JsonValue) IsString

func (jv *JsonValue) IsString() bool

func (*JsonValue) ObjectEach

func (jv *JsonValue) ObjectEach(cb func(key string, value *JsonValue)) error

func (*JsonValue) RawBytes

func (jv *JsonValue) RawBytes() []byte

func (*JsonValue) String

func (jv *JsonValue) String() string

func (*JsonValue) ToArray

func (jv *JsonValue) ToArray() ([]*JsonValue, error)

func (*JsonValue) ToMap

func (jv *JsonValue) ToMap() (res map[string]*JsonValue, err error)

type ValueType

type ValueType int

Data types available in valid JSON data.

func Get

func Get(data []byte, keys ...string) (value []byte, dataType ValueType, offset int, err error)

Get - Receives data structure, and key path to extract value from.

Returns: `value` - Pointer to original data structure containing key value, or just empty slice if nothing found or error `dataType` - Can be: `NotExist`, `String`, `Number`, `Object`, `Array`, `Boolean` or `Null` `offset` - Offset from provided data structure where key value ends. Used mostly internally, for example for `ArrayEach` helper. `err` - If key not found or any other parsing issue it should return error. If key not found it also sets `dataType` to `NotExist`

Accept multiple keys to specify path to JSON value (in case of quering nested structures). If no keys provided it will try to extract closest JSON value (simple ones or object/array), useful for reading streams or arrays, see `ArrayEach` implementation.

Get calls the internal get function, but will strip quotes from strings returned. (breaks abstraction, but kept for compatibility)

func (ValueType) String

func (vt ValueType) String() string

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL