jscan

package module
v1.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 28, 2023 License: BSD-3-Clause Imports: 9 Imported by: 0

README

Coverage Status GoReportCard Go Reference

jscan

jscan provides a high-performance zero-allocation JSON iterator for Go. It's not compatible with encoding/json and doesn't provide the usual Marshal/Unmarshal capabilities, instead it focuses on fast and efficient scanning over JSON strings with on-the-fly validation and error reporting.

jscan is tested against https://github.com/nst/JSONTestSuite, a comprehensive test suite for RFC 8259 compliant JSON parsers.

Example

https://go.dev/play/p/v-VeiMO2fsJ

package main

import (
	"fmt"

	"github.com/romshark/jscan"
)

func main() {
	j := `{
		"s": "value",
		"t": true,
		"f": false,
		"0": null,
		"n": -9.123e3,
		"o0": {},
		"a0": [],
		"o": {
			"k": "\"v\"",
			"a": [
				true,
				null,
				"item",
				-67.02e9,
				["foo"]
			]
		},
		"a3": [
			0,
			{
				"a3.a3":8
			}
		]
	}`

	err := jscan.Scan(jscan.Options{
		CachePath:  true,
		EscapePath: true,
	}, j, func(i *jscan.Iterator) (err bool) {
		fmt.Printf("| value:\n")
		fmt.Printf("|  level:      %d\n", i.Level)
		if k := i.Key(); k != "" {
			fmt.Printf("|  key:        %q\n", i.Key())
		}
		fmt.Printf("|  valueType:  %s\n", i.ValueType)
		if v := i.Value(); v != "" {
			fmt.Printf("|  value:      %q\n", i.Value())
		}
		fmt.Printf("|  arrayIndex: %d\n", i.ArrayIndex)
		fmt.Printf("|  path:       '%s'\n", i.Path())
		return false // No Error, resume scanning
	})

	if err.IsErr() {
		fmt.Printf("ERR: %s\n", err)
		return
	}

}

Benchmark Results

The following results were recorded on an Apple M1 Max MBP running macOS 13.2.1

goos: darwin
goarch: arm64
package version
pkg.go.dev/encoding/json go1.20.2
github.com/go-faster/jx v1.0.0
github.com/go-faster/jx v1.0.0
github.com/json-iterator/go v1.1.12
github.com/sinhashubham95/jsonic v1.1.0
github.com/tidwall/gjson v1.14.4
github.com/valyala/fastjson v1.6.4

Calculating statistics for a tiny JSON document ({"x":0}):

implementation ns/op B/op allocs/op
jscan 46.39 ns/op 0 B/op 0 allocs/op
encoding-json 46.39 ns/op 0 B/op 0 allocs/op
jsoniter 85.98 ns/op 160 B/op 2 allocs/op
gofaster-jx 54.21 ns/op 0 B/op 0 allocs/op
valyala-fastjson 50.17 ns/op 0 B/op 0 allocs/op

Calculating statistics for a small JSON document (335 bytes):

implementation ns/op B/op allocs/op
jscan 486.3 ns/op 0 B/op 0 allocs/op
jsoniter 768.7 ns/op 224 B/op 12 allocs/op
gofaster-jx 561.2 ns/op 0 B/op 0 allocs/op
valyala-fastjson 553.0 ns/op 0 B/op 0 allocs/op

Calculating statistics for a large JSON document (26.1 MB):

implementation ns/op B/op allocs/op
jscan 24468140 ns/op 6 B/op 0 allocs/op
jsoniter 54457668 ns/op 32851588 B/op 1108519 allocs/op
gofaster-jx 28043963 ns/op 8 B/op 0 allocs/op
valyala-fastjson 28241526 ns/op 7 B/op 0 allocs/op

Calculating statistics for an object that contains a key and a string value consisting entirely of escape sequences (~3KB):

implementation ns/op B/op allocs/op
jscan 1891 ns/op 0 B/op 0 allocs/op
jsoniter 7862 ns/op 2208 B/op 16 allocs/op
gofaster-jx 6677 ns/op 504 B/op 6 allocs/op
valyala-fastjson 11292 ns/op 0 B/op 0 allocs/op

Array of 1024 integers:

implementation ns/op B/op allocs/op
jscan 20105 ns/op 0 B/op 0 allocs/op
jsoniter 38419 ns/op 16528 B/op 1025 allocs/op
gofaster-jx 29924 ns/op 0 B/op 0 allocs/op
valyala-fastjson 21763 ns/op 0 B/op 0 allocs/op

Array of 1024 floating point numbers:

implementation ns/op B/op allocs/op
jscan 19196 ns/op 0 B/op 0 allocs/op
jsoniter 43730 ns/op 16528 B/op 1025 allocs/op
gofaster-jx 37036 ns/op 0 B/op 0 allocs/op
valyala-fastjson 21320 ns/op 6 B/op 0 allocs/op

Array of 1024 strings:

implementation ns/op B/op allocs/op
jscan 267332 ns/op 0 B/op 0 allocs/op
jsoniter 585148 ns/op 670313 B/op 1019 allocs/op
gofaster-jx 166966 ns/op 0 B/op 0 allocs/op
valyala-fastjson 61301 ns/op 50 B/op 0 allocs/op

Array of 1024 nullable booleans:

implementation ns/op B/op allocs/op
jscan 11592 ns/op 0 B/op 0 allocs/op
jsoniter 21736 ns/op 144 B/op 1 allocs/op
gofaster-jx 33209 ns/op 0 B/op 0 allocs/op
valyala-fastjson 10913 ns/op 0 B/op 0 allocs/op

Get by path:

implementation ns/op B/op allocs/op
jscan 229.7 ns/op 16 B/op 2 allocs/op
jsoniter 815.8 ns/op 496 B/op 19 allocs/op
tidwallgjson 148.9 ns/op 16 B/op 2 allocs/op
valyalafastjson 108.9 ns/op 0 B/op 0 allocs/op
sinhashubham95jsonic 194.1 ns/op 96 B/op 1 allocs/op

Validation:

Tiny

implementation ns/op B/op allocs/op
jscan 35.13 ns/op 0 B/op 0 allocs/op
encoding-json 42.93 ns/op 0 B/op 0 allocs/op
jsoniter 45.14 ns/op 0 B/op 0 allocs/op
gofaster-jx 39.19 ns/op 0 B/op 0 allocs/op
tidwallgjson 15.85 ns/op 0 B/op 0 allocs/op
valyala-fastjson 19.25 ns/op 0 B/op 0 allocs/op

Small

implementation ns/op B/op allocs/op
jscan 391.5 ns/op 0 B/op 0 allocs/op
encoding-json 903.8 ns/op 0 B/op 0 allocs/op
jsoniter 717.3 ns/op 56 B/op 7 allocs/op
gofaster-jx 392.2 ns/op 0 B/op 0 allocs/op
tidwallgjson 336.0 ns/op 0 B/op 0 allocs/op
valyala-fastjson 372.0 ns/op 0 B/op 0 allocs/op

Large

implementation ns/op B/op allocs/op
jscan 21481452 ns/op 7 B/op 0 allocs/op
encoding-json 68749373 ns/op 29 B/op 0 allocs/op
jsoniter 44151891 ns/op 13583525 B/op 644362 allocs/op
gofaster-jx 20625554 ns/op 24 B/op 0 allocs/op
tidwallgjson 27378477 ns/op 0 B/op 0 allocs/op
valyala-fastjson 25695158 ns/op 0 B/op 0 allocs/op

Validating an object that contains a key and a string value consisting entirely of escape sequences (~3KB):

implementation ns/op B/op allocs/op
jscan 1879 ns/op 0 B/op 0 allocs/op
encoding-json 9313 ns/op 0 B/op 0 allocs/op
jsoniter 7858 ns/op 2064 B/op 15 allocs/op
gofaster-jx 5897 ns/op 0 B/op 0 allocs/op
tidwallgjson 2989 ns/op 0 B/op 0 allocs/op
valyala-fastjson 9062 ns/op 0 B/op 0 allocs/op

Unwinding Stack

implementation ns/op B/op allocs/op
jscan 2614 ns/op 0 B/op 0 allocs/op
encoding-json 5133 ns/op 24 B/op 1 allocs/op
jsoniter 65817 ns/op 33149 B/op 1033 allocs/op
gofaster-jx 398530 ns/op 65687 B/op 1026 allocs/op
tidwallgjson 14033 ns/op 0 B/op 0 allocs/op
valyala-fastjson 4876865 ns/op 52431864 B/op 4134 allocs/op

Array of 1024 integers

implementation ns/op B/op allocs/op
jscan 17170 ns/op 0 B/op 0 allocs/op
encoding-json 34104 ns/op 0 B/op 0 allocs/op
jsoniter 21317 ns/op 0 B/op 0 allocs/op
gofaster-jx 17843 ns/op 0 B/op 0 allocs/op
tidwallgjson 13525 ns/op 0 B/op 0 allocs/op
valyala-fastjson 14741 ns/op 0 B/op 0 allocs/op

Array of 1024 floats

implementation ns/op B/op allocs/op
jscan 12925 ns/op 0 B/op 0 allocs/op
encoding-json 36390 ns/op 0 B/op 0 allocs/op
jsoniter 66119 ns/op 8755 B/op 547 allocs/op
gofaster-jx 20111 ns/op 0 B/op 0 allocs/op
tidwallgjson 12176 ns/op 0 B/op 0 allocs/op
valyala-fastjson 17346 ns/op 0 B/op 0 allocs/op

Array of 1024 nullable booleans

implementation ns/op B/op allocs/op
jscan 4968 ns/op 0 B/op 0 allocs/op
encoding-json 20239 ns/op 0 B/op 0 allocs/op
jsoniter 16538 ns/op 0 B/op 0 allocs/op
gofaster-jx 20987 ns/op 0 B/op 0 allocs/op
tidwallgjson 5081 ns/op 0 B/op 0 allocs/op
valyala-fastjson 4891 ns/op 0 B/op 0 allocs/op

Array of 1024 strings

implementation ns/op B/op allocs/op
jscan 264281 ns/op 0 B/op 0 allocs/op
encoding-json 1391270 ns/op 0 B/op 0 allocs/op
jsoniter 507315 ns/op 0 B/op 0 allocs/op
gofaster-jx 152812 ns/op 0 B/op 0 allocs/op
tidwallgjson 501928 ns/op 0 B/op 0 allocs/op
valyala-fastjson 253798 ns/op 0 B/op 0 allocs/op

Documentation

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func Valid added in v1.0.1 

func Valid(s string) bool

Valid returns true if s is valid JSON, otherwise returns false.

func ValidBytes added in v1.0.1 

func ValidBytes(s []byte) bool

ValidBytes returns true if s is valid JSON, otherwise returns false.

Types

type Error 

type Error struct {
	Src   string
	Index int
	Code  ErrorCode
}

func Get 

func Get(s, path string, escapePath bool, fn func(*Iterator)) Error

Get calls fn for the value at the given path. The value path is defined by keys separated by a dot and index access operators for arrays. If no value is found for the given path then fn isn't called and no error is returned. If escapePath then all dots and square brackets are expected to be escaped.

WARNING: Fields exported by *Iterator provided in fn must not be mutated! Do not use or alias *Iterator after fn returns!

Example 
package main

import (
	"fmt"

	"github.com/romshark/jscan"
)

func main() {
	j := `[false,[[2, {"[escaped]":[{"test-key":"string value"}]}]]]`

	if err := jscan.Get(
		j, `[1][0][1].\[escaped\][0].test-key`,
		true, func(i *jscan.Iterator) {
			fmt.Println(i.Value())
		},
	); err.IsErr() {
		fmt.Printf("ERR: %s\n", err)
		return
	}

}

Output:

string value

func Scan 

func Scan(
	o Options,
	s string,
	fn func(*Iterator) (err bool),
) Error

Scan calls fn for every scanned value including objects and arrays. Scan returns true if there was an error or if fn returned true, otherwise it returns false. If cachePath == true then paths are generated and cached on the fly reducing their performance penalty.

WARNING: Fields exported by *Iterator provided in fn must not be mutated! Do not use or alias *Iterator after fn returns!

Example 
package main

import (
	"fmt"

	"github.com/romshark/jscan"
)

func main() {
	j := `{
		"s": "value",
		"t": true,
		"f": false,
		"0": null,
		"n": -9.123e3,
		"o0": {},
		"a0": [],
		"o": {
			"k": "\"v\"",
			"a": [
				true,
				null,
				"item",
				-67.02e9,
				["foo"]
			]
		},
		"a3": [
			0,
			{
				"a3.a3":8
			}
		]
	}`

	err := jscan.Scan(jscan.Options{
		CachePath:  true,
		EscapePath: true,
	}, j, func(i *jscan.Iterator) (err bool) {
		fmt.Printf("| value:\n")
		fmt.Printf("|  level:      %d\n", i.Level)
		if k := i.Key(); k != "" {
			fmt.Printf("|  key:        %q\n", i.Key())
		}
		fmt.Printf("|  valueType:  %s\n", i.ValueType)
		if v := i.Value(); v != "" {
			fmt.Printf("|  value:      %q\n", i.Value())
		}
		fmt.Printf("|  arrayIndex: %d\n", i.ArrayIndex)
		fmt.Printf("|  path:       '%s'\n", i.Path())
		return false // No Error, resume scanning
	})

	if err.IsErr() {
		fmt.Printf("ERR: %s\n", err)
		return
	}

}

Output:

| value:
|  level:      0
|  valueType:  object
|  arrayIndex: -1
|  path:       ''
| value:
|  level:      1
|  key:        "s"
|  valueType:  string
|  value:      "value"
|  arrayIndex: -1
|  path:       's'
| value:
|  level:      1
|  key:        "t"
|  valueType:  true
|  value:      "true"
|  arrayIndex: -1
|  path:       't'
| value:
|  level:      1
|  key:        "f"
|  valueType:  false
|  value:      "false"
|  arrayIndex: -1
|  path:       'f'
| value:
|  level:      1
|  key:        "0"
|  valueType:  null
|  value:      "null"
|  arrayIndex: -1
|  path:       '0'
| value:
|  level:      1
|  key:        "n"
|  valueType:  number
|  value:      "-9.123e3"
|  arrayIndex: -1
|  path:       'n'
| value:
|  level:      1
|  key:        "o0"
|  valueType:  object
|  arrayIndex: -1
|  path:       'o0'
| value:
|  level:      1
|  key:        "a0"
|  valueType:  array
|  arrayIndex: -1
|  path:       'a0'
| value:
|  level:      1
|  key:        "o"
|  valueType:  object
|  arrayIndex: -1
|  path:       'o'
| value:
|  level:      2
|  key:        "k"
|  valueType:  string
|  value:      "\\\"v\\\""
|  arrayIndex: -1
|  path:       'o.k'
| value:
|  level:      2
|  key:        "a"
|  valueType:  array
|  arrayIndex: -1
|  path:       'o.a'
| value:
|  level:      3
|  valueType:  true
|  value:      "true"
|  arrayIndex: 0
|  path:       'o.a[0]'
| value:
|  level:      3
|  valueType:  null
|  value:      "null"
|  arrayIndex: 1
|  path:       'o.a[1]'
| value:
|  level:      3
|  valueType:  string
|  value:      "item"
|  arrayIndex: 2
|  path:       'o.a[2]'
| value:
|  level:      3
|  valueType:  number
|  value:      "-67.02e9"
|  arrayIndex: 3
|  path:       'o.a[3]'
| value:
|  level:      3
|  valueType:  array
|  arrayIndex: 4
|  path:       'o.a[4]'
| value:
|  level:      4
|  valueType:  string
|  value:      "foo"
|  arrayIndex: 0
|  path:       'o.a[4][0]'
| value:
|  level:      1
|  key:        "a3"
|  valueType:  array
|  arrayIndex: -1
|  path:       'a3'
| value:
|  level:      2
|  valueType:  number
|  value:      "0"
|  arrayIndex: 0
|  path:       'a3[0]'
| value:
|  level:      2
|  valueType:  object
|  arrayIndex: 1
|  path:       'a3[1]'
| value:
|  level:      3
|  key:        "a3.a3"
|  valueType:  number
|  value:      "8"
|  arrayIndex: -1
|  path:       'a3[1].a3\.a3'
Example (Error_handling) 
package main

import (
	"fmt"

	"github.com/romshark/jscan"
)

func main() {
	j := `"something...`

	err := jscan.Scan(jscan.Options{}, j, func(i *jscan.Iterator) (err bool) {
		fmt.Println("This shall never be executed")
		return false // No Error, resume scanning
	})

	if err.IsErr() {
		fmt.Printf("ERR: %s\n", err)
		return
	}

}

Output:

ERR: error at index 13: unexpected EOF

func Validate added in v1.0.1 

func Validate(s string) Error

Validate returns an error if s is invalid JSON.

func (Error) Error 

func (e Error) Error() string

func (Error) IsErr 

func (e Error) IsErr() bool

IsErr returns true if there is an error, otherwise returns false.

type ErrorBytes added in v1.0.1 

type ErrorBytes struct {
	Src   []byte
	Index int
	Code  ErrorCode
}

func GetBytes added in v1.0.1 

func GetBytes(
	s, path []byte,
	escapePath bool,
	fn func(*IteratorBytes),
) ErrorBytes

Get calls fn for the value at the given path. The value path is defined by keys separated by a dot and index access operators for arrays. If no value is found for the given path then fn isn't called and no error is returned. If escapePath then all dots and square brackets are expected to be escaped.

WARNING: Fields exported by *IteratorBytes in fn must not be mutated! Do not use or alias *IteratorBytes after fn returns!

func ScanBytes added in v1.0.1 

func ScanBytes(
	o Options,
	s []byte,
	fn func(*IteratorBytes) (err bool),
) ErrorBytes

ScanBytes calls fn for every scanned value including objects and arrays. Scan returns true if there was an error or if fn returned true, otherwise it returns false. If cachePath == true then paths are generated and cached on the fly reducing their performance penalty.

WARNING: Fields exported by *IteratorBytes in fn must not be mutated! Do not use or alias *IteratorBytes after fn returns!

func ValidateBytes added in v1.0.1 

func ValidateBytes(s []byte) ErrorBytes

ValidateBytes returns an error if s is invalid JSON.

func (ErrorBytes) Error added in v1.0.1 

func (e ErrorBytes) Error() string

func (ErrorBytes) IsErr added in v1.0.1 

func (e ErrorBytes) IsErr() bool

IsErr returns true if there is an error, otherwise returns false.

type ErrorCode 

type ErrorCode int8

ErrorCode defines the error type. 

const (
	ErrorCodeInvalidEscapeSeq ErrorCode
	ErrorCodeIllegalControlChar
	ErrorCodeUnexpectedEOF
	ErrorCodeUnexpectedToken
	ErrorCodeMalformedNumber
	ErrorCodeCallback
)

All error codes

type Iterator 

type Iterator struct {
	ValueType                       ValueType
	Level                           int
	KeyStart, KeyEnd, KeyLenEscaped int
	ValueStart, ValueEnd            int
	ArrayIndex                      int
	// contains filtered or unexported fields
}

Iterator provides access to the recently scanned value.

func (*Iterator) Key 

func (i *Iterator) Key() string

Key returns the field key if any.

func (*Iterator) Path 

func (i *Iterator) Path() (s string)

Path returns the stringified path.

func (*Iterator) ScanPath 

func (i *Iterator) ScanPath(fn func(keyStart, keyEnd, arrIndex int))

ScanPath calls fn for every element in the path of the value. If keyStart is != -1 then the element is a field value, otherwise arrIndex indicates the index of the item in the underlying array.

func (*Iterator) Value 

func (i *Iterator) Value() string

Value returns the value if any.

func (*Iterator) ViewPath 

func (i *Iterator) ViewPath(fn func(p []byte))

ViewPath calls fn and provides the stringified path.

WARNING: do not use or alias p after fn returns! Only viewing or copying are considered safe! Use (*Iterator).Path instead for a safer and more convenient API.

type IteratorBytes added in v1.0.1 

type IteratorBytes struct {
	ValueType                       ValueType
	Level                           int
	KeyStart, KeyEnd, KeyLenEscaped int
	ValueStart, ValueEnd            int
	ArrayIndex                      int
	// contains filtered or unexported fields
}

IteratorBytes provides access to the recently scanned value.

func (*IteratorBytes) Key added in v1.0.1 

func (i *IteratorBytes) Key() []byte

Key returns the field key if any.

func (*IteratorBytes) Path added in v1.0.1 

func (i *IteratorBytes) Path() (s []byte)

Path returns the stringified path.

func (*IteratorBytes) ScanPath added in v1.0.1 

func (i *IteratorBytes) ScanPath(fn func(keyStart, keyEnd, arrIndex int))

ScanPath calls fn for every element in the path of the value. If keyStart is != -1 then the element is a field value, otherwise arrIndex indicates the index of the item in the underlying array.

func (*IteratorBytes) Value added in v1.0.1 

func (i *IteratorBytes) Value() []byte

Value returns the value if any.

func (*IteratorBytes) ViewPath added in v1.0.1 

func (i *IteratorBytes) ViewPath(fn func(p []byte))

ViewPath calls fn and provides the stringified path.

WARNING: do not use or alias p after fn returns! Only viewing or copying are considered safe! Use (*IteratorBytes).Path instead for a safer and more convenient API.

type Options 

type Options struct {
	CachePath  bool
	EscapePath bool
}

type ValueType 

type ValueType int

ValueType defines a JSON value type 

const (
	ValueTypeObject ValueType
	ValueTypeArray
	ValueTypeNull
	ValueTypeFalse
	ValueTypeTrue
	ValueTypeString
	ValueTypeNumber
)

JSON value types

func (ValueType) String 

func (t ValueType) String() string

Source Files

View all Source files

Directories

Path Synopsis
internal
jsonnum
keyescape
stack
strfind

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.