README
¶
jscan
jscan provides high-performance zero-allocation JSON iterator and validator for Go. This module doesn't provide Marshal/Unmarshal capabilities, instead it focuses on highly efficient iteration over JSON data with on-the-fly validation.
jscan is tested against https://github.com/nst/JSONTestSuite, a comprehensive test suite for RFC 8259 compliant JSON parsers.
See jscan-benchmark for benchmark results 🏎️ 🏁.
Example
https://go.dev/play/p/moP3l9EkebF
package main
import (
"fmt"
"github.com/romshark/jscan/v2"
)
func main() {
j := `{
"s": "value",
"t": true,
"f": false,
"0": null,
"n": -9.123e3,
"o0": {},
"a0": [],
"o": {
"k": "\"v\"",
"a": [
true,
null,
"item",
-67.02e9,
["foo"]
]
},
"a3": [
0,
{
"a3.a3":8
}
]
}`
err := jscan.Scan(j, func(i *jscan.Iterator[string]) (err bool) {
fmt.Printf("%q:\n", i.Pointer())
fmt.Printf("├─ valueType: %s\n", i.ValueType().String())
if k := i.Key(); k != "" {
fmt.Printf("├─ key: %q\n", k[1:len(k)-1])
}
if ai := i.ArrayIndex(); ai != -1 {
fmt.Printf("├─ arrayIndex: %d\n", ai)
}
if v := i.Value(); v != "" {
fmt.Printf("├─ value: %q\n", v)
}
fmt.Printf("└─ level: %d\n", i.Level())
return false // Resume scanning
})
if err.IsErr() {
fmt.Printf("ERR: %s\n", err)
return
}
}
Documentation
¶
Index ¶
- Constants
- func Valid[S ~string | ~[]byte](s S) bool
- type Error
- func Scan[S ~string | ~[]byte](s S, fn func(*Iterator[S]) (err bool)) (err Error[S])
- func ScanOne[S ~string | ~[]byte](s S, fn func(*Iterator[S]) (err bool)) (trailing S, err Error[S])
- func Validate[S ~string | ~[]byte](s S) Error[S]
- func ValidateOne[S ~string | ~[]byte](s S) (trailing S, err Error[S])
- type ErrorCode
- type Iterator
- func (i *Iterator[S]) ArrayIndex() int
- func (i *Iterator[S]) Key() (key S)
- func (i *Iterator[S]) KeyIndex() int
- func (i *Iterator[S]) KeyIndexEnd() int
- func (i *Iterator[S]) Level() int
- func (i *Iterator[S]) Pointer() (s S)
- func (i *Iterator[S]) ScanStack(fn func(keyIndex, keyEnd, arrayIndex int))
- func (i *Iterator[S]) Value() (value S)
- func (i *Iterator[S]) ValueIndex() int
- func (i *Iterator[S]) ValueIndexEnd() int
- func (i *Iterator[S]) ValueType() ValueType
- func (i *Iterator[S]) ViewPointer(fn func(p []byte))
- type Parser
- type Validator
- type ValueType
Examples ¶
Constants ¶
const ( DefaultStackSizeIterator = 64 DefaultStackSizeValidator = 128 )
Default stack sizes
Variables ¶
This section is empty.
Functions ¶
Types ¶
type Error ¶
type Error[S ~string | ~[]byte] struct { // Src refers to the original source. Src S // Index points to the error start index in the source. Index int // Code indicates the type of the error. Code ErrorCode }
Error is a syntax error encountered during validation or iteration. The only exception is ErrorCodeCallback which indicates a callback explicitly breaking by returning true instead of a syntax error. (Error).IsErr() returning false is equivalent to err == nil.
func Scan ¶
Scan calls fn for every encountered value including objects and arrays. When an object or array is encountered fn will also be called for each of its member and element values.
Unlike (*Parser).Scan this function will take an iterator instance from a global iterator pool and can therefore be less efficient. Consider reusing a Parser instance instead.
TIP: Explicitly cast s to string or []byte to use the global iterator pools and avoid an unecessary iterator allocation such as when dealing with json.RawMessage and similar types derived from string or []byte.
m := json.RawMessage(`1`) jscan.Scan([]byte(m), // Cast m to []byte to avoid allocation!
WARNING: Don't use or alias *Iterator[S] after fn returns!
Example ¶
j := `{
"s": "value",
"t": true,
"f": false,
"0": null,
"n": -9.123e3,
"o0": {},
"a0": [],
"o": {
"k": "\"v\"",
"a": [
true,
null,
"item",
-67.02e9,
["foo"]
]
},
"a3": [
0,
{
"a3.a3":8
}
]
}`
err := jscan.Scan(j, func(i *jscan.Iterator[string]) (err bool) {
fmt.Printf("%q:\n", i.Pointer())
fmt.Printf("├─ valueType: %s\n", i.ValueType().String())
if k := i.Key(); k != "" {
fmt.Printf("├─ key: %q\n", k[1:len(k)-1])
}
if ai := i.ArrayIndex(); ai != -1 {
fmt.Printf("├─ arrayIndex: %d\n", ai)
}
if v := i.Value(); v != "" {
fmt.Printf("├─ value: %q\n", v)
}
fmt.Printf("└─ level: %d\n", i.Level())
return false // No Error, resume scanning
})
if err.IsErr() {
fmt.Printf("ERR: %s\n", err)
return
}
Output: "": ├─ valueType: object └─ level: 0 "/s": ├─ valueType: string ├─ key: "s" ├─ value: "\"value\"" └─ level: 1 "/t": ├─ valueType: true ├─ key: "t" ├─ value: "true" └─ level: 1 "/f": ├─ valueType: false ├─ key: "f" ├─ value: "false" └─ level: 1 "/0": ├─ valueType: null ├─ key: "0" ├─ value: "null" └─ level: 1 "/n": ├─ valueType: number ├─ key: "n" ├─ value: "-9.123e3" └─ level: 1 "/o0": ├─ valueType: object ├─ key: "o0" └─ level: 1 "/a0": ├─ valueType: array ├─ key: "a0" └─ level: 1 "/o": ├─ valueType: object ├─ key: "o" └─ level: 1 "/o/k": ├─ valueType: string ├─ key: "k" ├─ value: "\"\\\"v\\\"\"" └─ level: 2 "/o/a": ├─ valueType: array ├─ key: "a" └─ level: 2 "/o/a/0": ├─ valueType: true ├─ arrayIndex: 0 ├─ value: "true" └─ level: 3 "/o/a/1": ├─ valueType: null ├─ arrayIndex: 1 ├─ value: "null" └─ level: 3 "/o/a/2": ├─ valueType: string ├─ arrayIndex: 2 ├─ value: "\"item\"" └─ level: 3 "/o/a/3": ├─ valueType: number ├─ arrayIndex: 3 ├─ value: "-67.02e9" └─ level: 3 "/o/a/4": ├─ valueType: array ├─ arrayIndex: 4 └─ level: 3 "/o/a/4/0": ├─ valueType: string ├─ arrayIndex: 0 ├─ value: "\"foo\"" └─ level: 4 "/a3": ├─ valueType: array ├─ key: "a3" └─ level: 1 "/a3/0": ├─ valueType: number ├─ arrayIndex: 0 ├─ value: "0" └─ level: 2 "/a3/1": ├─ valueType: object ├─ arrayIndex: 1 └─ level: 2 "/a3/1/a3.a3": ├─ valueType: number ├─ key: "a3.a3" ├─ value: "8" └─ level: 3
Example (Decode2DIntArray) ¶
j := `[[1,2,34,567],[8901,2147483647,-1,42]]`
s := [][]int{}
currentIndex := 0
err := jscan.Scan(j, func(i *jscan.Iterator[string]) (err bool) {
switch i.Level() {
case 0: // Root array
return i.ValueType() != jscan.ValueTypeArray
case 1: // Sub-array
if i.ValueType() != jscan.ValueTypeArray {
return true
}
currentIndex = len(s)
s = append(s, []int{})
return false
}
if i.ValueType() != jscan.ValueTypeNumber {
// Unexpected array element type
return true
}
vi, errp := strconv.ParseInt(i.Value(), 10, 32)
if errp != nil {
// Not a valid 32-bit signed integer
return true
}
s[currentIndex] = append(s[currentIndex], int(vi))
return false
})
if err.IsErr() {
fmt.Println(err.Error())
return
}
fmt.Println(s)
Output: [[1 2 34 567] [8901 2147483647 -1 42]]
Example (Error_handling) ¶
j := `"something...`
err := jscan.Scan(j, func(i *jscan.Iterator[string]) (err bool) {
fmt.Println("This shall never be executed")
return false // No Error, resume scanning
})
if err.IsErr() {
fmt.Printf("ERR: %s\n", err)
return
}
Output: ERR: error at index 13: unexpected EOF
func ScanOne ¶
func ScanOne[S ~string | ~[]byte]( s S, fn func(*Iterator[S]) (err bool), ) (trailing S, err Error[S])
ScanOne calls fn for every encountered value including objects and arrays. When an object or array is encountered fn will also be called for each of its member and element values.
Unlike Scan, ScanOne doesn't return ErrorCodeUnexpectedToken when it encounters anything other than EOF after reading a valid JSON value. Returns an error if any and trailing as substring of s with the scanned value cut. In case of an error trailing will be a substring of s cut up until the index where the error was encountered.
Unlike (*Parser).ScanOne this function will take an iterator instance from a global iterator pool and can therefore be less efficient. Consider reusing a Parser instance instead.
TIP: Explicitly cast s to string or []byte to use the global iterator pools and avoid an unecessary iterator allocation such as when dealing with json.RawMessage and similar types derived from string or []byte.
m := json.RawMessage(`1`) jscan.ScanOne([]byte(m), // Cast m to []byte to avoid allocation!
WARNING: Don't use or alias *Iterator[S] after fn returns!
func Validate ¶
Validate returns an error if s is invalid JSON.
Unlike (*Validator).Validate this function will take a validator instance from a global pool and can therefore be less efficient. Consider reusing a Validator instance instead.
TIP: Explicitly cast s to string or []byte to use the global validator pools and avoid an unecessary validator allocation such as when dealing with json.RawMessage and similar types derived from string or []byte.
m := json.RawMessage(`1`) jscan.Validate([]byte(m), // Cast m to []byte to avoid allocation!
func ValidateOne ¶
ValidateOne scans one JSON value from s and returns an error if it's invalid and trailing as substring of s with the scanned value cut. In case of an error trailing will be a substring of s cut up until the index where the error was encountered.
Unlike (*Validator).ValidateOne this function will take a validator instance from a global pool and can therefore be less efficient. Consider reusing a Validator instance instead.
TIP: Explicitly cast s to string or []byte to use the global validator pools and avoid an unecessary validator allocation such as when dealing with json.RawMessage and similar types derived from string or []byte.
m := json.RawMessage(`1`) jscan.ValidateOne([]byte(m), // Cast m to []byte to avoid allocation!
Example ¶
s := `-120.4` +
`"string"` +
`{"key":"value"}` +
`[0,1]` +
`true` +
`false` +
`null`
for offset, x := 0, s; x != ""; offset = len(s) - len(x) {
var err jscan.Error[string]
if x, err = jscan.ValidateOne(x); err.IsErr() {
panic(fmt.Errorf("unexpected error: %w", err))
}
fmt.Println(s[offset : len(s)-len(x)])
}
Output: -120.4 "string" {"key":"value"} [0,1] true false null
type ErrorCode ¶
type ErrorCode int8
ErrorCode defines the error type.
const ( // ErrorCodeInvalidEscape indicates the encounter of an invalid escape sequence. ErrorCodeInvalidEscape ErrorCode // ErrorCodeIllegalControlChar indicates the encounter of // an illegal control character in the source. ErrorCodeIllegalControlChar // ErrorCodeUnexpectedEOF indicates the encounter an unexpected end of file. ErrorCodeUnexpectedEOF // ErrorCodeUnexpectedToken indicates the encounter of an unexpected token. ErrorCodeUnexpectedToken // ErrorCodeMalformedNumber indicates the encounter of a malformed number. ErrorCodeMalformedNumber // ErrorCodeCallback indicates return of true from the callback function. ErrorCodeCallback )
type Iterator ¶
Iterator provides access to the recently encountered value.
func (*Iterator[S]) ArrayIndex ¶
ArrayIndex returns either the index of the element value in the array or -1 if the value isn't inside an array.
func (*Iterator[S]) Key ¶
func (i *Iterator[S]) Key() (key S)
Key returns either the object member key or "" when the value isn't a member of an object and hence doesn't have a key.
func (*Iterator[S]) KeyIndex ¶
KeyIndex returns either the start index of the member key string in the source or -1 when the value isn't a member of an object and hence doesn't have a key.
func (*Iterator[S]) KeyIndexEnd ¶
KeyIndexEnd returns either the end index of the member key string in the source or -1 when the value isn't a member of an object and hence doesn't have a key.
func (*Iterator[S]) Level ¶
Level returns the depth level of the current value.
For example in the following JSON: `[1,2,3]` the array is situated at level 0 while the integers inside are situated at level 1.
func (*Iterator[S]) Pointer ¶
func (i *Iterator[S]) Pointer() (s S)
Pointer returns the JSON pointer in RFC-6901 format.
func (*Iterator[S]) ScanStack ¶
ScanStack calls fn for every element in the stack. If keyIndex is != -1 then the element is a member value, otherwise arrayIndex indicates the index of the element in the underlying array.
func (*Iterator[S]) ValueIndex ¶
ValueIndex returns the start index of the value in the source.
func (*Iterator[S]) ValueIndexEnd ¶
ValueIndexEnd returns the end index of the value in the source if any. Object and array values have a -1 end index because their end is unknown during traversal.
func (*Iterator[S]) ViewPointer ¶
ViewPointer calls fn and provides the buffer holding the JSON pointer in RFC-6901 format. Consider using (*Iterator[S]).Pointer() instead for safety and convenience.
WARNING: do not use or alias p after fn returns, only reading and copying p are considered safe!
type Parser ¶
Parser wraps an iterator in a reusable instance. Reusing a parser instance is more efficient than global functions that rely on a global iterator pool.
func NewParser ¶
NewParser creates a new reusable parser instance. A higher preallocStackFrames value implies greater memory usage but also reduces the chance of dynamic memory allocations if the JSON depth surpasses the stack size. preallocStackFrames of 32 is equivalent to ~1KiB of memory usage on 64-bit systems (1 frame = ~32 bytes). Use DefaultStackSizeIterator when not sure.
func (*Parser[S]) Scan ¶
Scan calls fn for every encountered value including objects and arrays. When an object or array is encountered fn will also be called for each of its member and element values.
WARNING: Don't use or alias *Iterator[S] after fn returns!
func (*Parser[S]) ScanOne ¶
ScanOne calls fn for every encountered value including objects and arrays. When an object or array is encountered fn will also be called for each of its member and element values.
Unlike Scan, ScanOne doesn't return ErrorCodeUnexpectedToken when it encounters anything other than EOF after reading a valid JSON value. Returns an error if any and trailing as substring of s with the scanned value cut. In case of an error trailing will be a substring of s cut up until the index where the error was encountered.
WARNING: Don't use or alias *Iterator[S] after fn returns!
type Validator ¶
Validator is a reusable validator instance. The validator is more efficient than the parser at JSON validation. A validator instance can be more efficient than global Valid, Validate and ValidateOne function calls due to potential stack frame allocation avoidance.
func NewValidator ¶
NewValidator creates a new reusable validator instance. A higher preallocStackFrames value implies greater memory usage but also reduces the chance of dynamic memory allocations if the JSON depth surpasses the stack size. preallocStackFrames of 1024 is equivalent to ~1KiB of memory usage (1 frame = 1 byte). Use DefaultStackSizeValidator when not sure.
func (*Validator[S]) Valid ¶
Valid returns true if s is a valid JSON value, otherwise returns false.
func (*Validator[S]) Validate ¶
Validate returns an error if s is invalid JSON, otherwise returns a zero value of Error[S].
func (*Validator[S]) ValidateOne ¶
ValidateOne scans one JSON value from s and returns an error if it's invalid and trailing as substring of s with the scanned value cut. In case of an error trailing will be a substring of s cut up until the index where the error was encountered.
Source Files
Directories