ksql

package module
v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 29, 2023 License: Apache-2.0, MIT Imports: 15 Imported by: 0

README

ksql

Project status GoDoc License

Is a JSON data expression lexer, parser, cli and library.

Requirements
  • Go 1.19+
How to install CLI
~ go install github.com/go-playground/ksql/cmd/ksql
Usage
package main

import (
	"fmt"

	"github.com/go-playground/ksql"
)

func main() {
	expression := []byte(`.properties.employees > 20`)
	input := []byte(`{"name":"MyCompany", "properties":{"employees": 50}`)
	ex, err := ksql.Parse(expression)
	if err != nil {
		panic(err)
	}

	result, err := ex.Calculate(input)
	if err != nil {
		panic(err)
	}
	fmt.Printf("%v\n", result)
}
CLI Usage
~ ksql '(.field1 + 1) /2' '{"field1": 1}'
or
echo '{"field1": 1}' | ksql '(.field1 + 1) /2'
Expressions

Expressions support most mathematical and string expressions see below for details:

Syntax & Rules
Token Example Syntax Rules
Equals == supports both == and =.
Add + N/A
Subtract - N/A
Multiply * N/A
Divide / N/A
Gt > N/A
Gte >= N/A
Lt < N/A
Lte <= N/A
OpenParen ( N/A
CloseParen ) N/A
OpenBracket [ N/A
CloseBracket ] N/A
Comma , N/A
QuotedString "sample text" Must start and end with an unescaped " character
Number 123.45 Must start and end with a space or '+' or '-' when hard coded value in expression and supports 0-9 +- e characters for numbers and exponent notation.
BooleanTrue true Accepts true as a boolean only.
BooleanFalse false Accepts false as a boolean only.
SelectorPath .selector_path Starts with a . and ends with whitespace blank space. This crate currently uses gjson and so the full gjson syntax for identifiers is supported.
And && N/A
Not ! Must be before Boolean identifier or expression or be followed by an operation
Or || N/A
Contains CONTAINS Ends with whitespace blank space.
ContainsAny CONTAINS_ANY Ends with whitespace blank space.
ContainsAll CONTAINS_ALL Ends with whitespace blank space.
In IN Ends with whitespace blank space.
Between BETWEEN Starts & ends with whitespace blank space. example 1 BETWEEN 0 10
StartsWith STARTSWITH Ends with whitespace blank space.
EndsWith ENDSWITH Ends with whitespace blank space.
NULL NULL N/A
Coerce COERCE Coerces one data type into another using in combination with 'Identifier'. Syntax is COERCE <expression> _identifer_.
Identifier _identifier_ Starts and end with an _ used with 'COERCE' to cast data types, see table below with supported values. You can combine multiple coercions if separated by a COMMA.
Colon : N/A
COERCE Types
Type Description
_datetime_ This attempts to convert the type into a DateTime.
_lowercase_ This converts the text into lowercase.
_uppercase_ This converts the text into uppercase.
_title_ This converts the text into title case, when the first letter is capitalized but the rest lower cased.
_string_ This converts the value into a string and supports the Value's String, Number, Bool, DateTime with nanosecond precision.
_number_ This converts the value into an f64 number and supports the Value's Null, String, Number, Bool and DateTime.
_substr_[n:n] This allows taking a substring of a string value. this returns Null if no match at specified indices exits.
License
Licensed under either of Apache License, Version 2.0 or MIT license at your option.
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in Proteus by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

Documentation

Index

Constants

View Source 
const (
	SelectorPath = iota
	QuotedString
	Number
	BooleanTrue
	BooleanFalse
	Null
	Equals
	Add
	Subtract
	Multiply
	Divide
	Gt
	Gte
	Lt
	Lte
	And
	Or
	Not
	Contains
	ContainsAny
	ContainsAll
	In
	Between
	StartsWith
	EndsWith
	OpenBracket
	CloseBracket
	Comma
	OpenParen
	CloseParen
	Coerce
	Identifier
	Colon
)

Variables

View Source 
var (
	// Coercions is a `map` of all coercions guarded by a Mutex for use allowing registration,
	// removal or even replacing of existing coercions.
	Coercions = syncext.NewRWMutex2(map[string]func(p *Parser, constEligible bool, expression Expression) (stillConstEligible bool, e Expression, err error){
		"_datetime_": func(_ *Parser, constEligible bool, expression Expression) (stillConstEligible bool, e Expression, err error) {
			expression = coerceDateTime{value: expression}
			if constEligible {
				value, err := expression.Calculate([]byte{})
				if err != nil {
					return false, nil, err
				}
				return constEligible, coercedConstant{value: value}, nil
			} else {
				return false, expression, nil
			}
		},
		"_lowercase_": func(_ *Parser, constEligible bool, expression Expression) (stillConstEligible bool, e Expression, err error) {
			expression = coerceLowercase{value: expression}
			if constEligible {
				value, err := expression.Calculate([]byte{})
				if err != nil {
					return false, nil, err
				}
				return constEligible, coercedConstant{value: value}, nil
			} else {
				return false, expression, nil
			}
		},
		"_string_": func(_ *Parser, constEligible bool, expression Expression) (stillConstEligible bool, e Expression, err error) {
			expression = coerceString{value: expression}
			if constEligible {
				value, err := expression.Calculate([]byte{})
				if err != nil {
					return false, nil, err
				}
				return constEligible, coercedConstant{value: value}, nil
			} else {
				return false, expression, nil
			}
		},
		"_number_": func(_ *Parser, constEligible bool, expression Expression) (stillConstEligible bool, e Expression, err error) {
			expression = coerceNumber{value: expression}
			if constEligible {
				value, err := expression.Calculate([]byte{})
				if err != nil {
					return false, nil, err
				}
				return constEligible, coercedConstant{value: value}, nil
			} else {
				return false, expression, nil
			}
		},
		"_uppercase_": func(_ *Parser, constEligible bool, expression Expression) (stillConstEligible bool, e Expression, err error) {
			expression = coerceUppercase{value: expression}
			if constEligible {
				value, err := expression.Calculate([]byte{})
				if err != nil {
					return false, nil, err
				}
				return constEligible, coercedConstant{value: value}, nil
			} else {
				return false, expression, nil
			}
		},
		"_title_": func(_ *Parser, constEligible bool, expression Expression) (stillConstEligible bool, e Expression, err error) {
			expression = coerceTitle{value: expression}
			if constEligible {
				value, err := expression.Calculate([]byte{})
				if err != nil {
					return false, nil, err
				}
				return constEligible, coercedConstant{value: value}, nil
			} else {
				return false, expression, nil
			}
		},
		"_substr_": func(p *Parser, constEligible bool, expression Expression) (stillConstEligible bool, e Expression, err error) {

			leftBracket := p.Tokenizer.Next()
			if leftBracket.IsNone() {
				return false, nil, ErrCustom{S: "Expected [ after _substr_"}
			} else if leftBracket.Unwrap().IsErr() {
				return false, nil, ErrInvalidCoerce{Err: leftBracket.Unwrap().Err()}
			} else if leftBracket.Unwrap().Unwrap().Kind != OpenBracket {
				return false, nil, ErrCustom{S: "Expected [ after _substr_"}
			}

			// number or colon
			var startIndex optionext.Option[int]
			token := p.Tokenizer.Next()
			if token.IsNone() {
				return false, nil, ErrCustom{S: "Expected number or colon after _substr_["}
			} else if token.Unwrap().IsErr() {
				return false, nil, ErrInvalidCoerce{Err: token.Unwrap().Err()}
			} else {
				token := token.Unwrap().Unwrap()
				start := int(token.Start)
				switch token.Kind {
				case Colon:
				case Number:
					i64, err := strconv.ParseInt(string(p.Exp[start:start+int(token.Len)]), 10, 64)
					if err != nil {
						return false, nil, err
					}
					startIndex = optionext.Some(int(i64))
				default:
					return false, nil, ErrCustom{S: fmt.Sprintf("Expected number after _substr_[ but got %s", string(p.Exp[start:start+int(token.Len)]))}
				}
			}

			if startIndex.IsSome() {
				colon := p.Tokenizer.Next()
				if colon.IsNone() {
					return false, nil, ErrCustom{S: "Expected : after _substr_[n"}
				} else if colon.Unwrap().IsErr() {
					return false, nil, ErrInvalidCoerce{Err: colon.Unwrap().Err()}
				} else if colon.Unwrap().Unwrap().Kind != Colon {
					return false, nil, ErrCustom{S: "Expected : after _substr_[n"}
				}
			}

			// number or end bracket
			var endIndex optionext.Option[int]
			token = p.Tokenizer.Next()
			if token.IsNone() {
				return false, nil, ErrCustom{S: "Expected number or ] after _substr_["}
			} else if token.Unwrap().IsErr() {
				return false, nil, ErrInvalidCoerce{Err: token.Unwrap().Err()}
			} else {
				token := token.Unwrap().Unwrap()
				start := int(token.Start)
				switch token.Kind {
				case CloseBracket:
				case Number:
					i64, err := strconv.ParseInt(string(p.Exp[start:start+int(token.Len)]), 10, 64)
					if err != nil {
						return false, nil, err
					}
					endIndex = optionext.Some(int(i64))
				default:
					return false, nil, ErrCustom{S: fmt.Sprintf("Expected number after _substr_[n: but got %s", string(p.Exp[start:start+int(token.Len)]))}
				}
			}

			if endIndex.IsSome() {
				rightBracket := p.Tokenizer.Next()
				if rightBracket.IsNone() {
					return false, nil, ErrCustom{S: "Expected ] after _substr_[n:n"}
				} else if rightBracket.Unwrap().IsErr() {
					return false, nil, ErrInvalidCoerce{Err: rightBracket.Unwrap().Err()}
				} else if rightBracket.Unwrap().Unwrap().Kind != CloseBracket {
					return false, nil, ErrCustom{S: "Expected ] after _substr_[n:n"}
				}
			}

			switch {
			case startIndex.IsSome() && endIndex.IsSome() && startIndex.Unwrap() > endIndex.Unwrap():
				return false, nil, ErrCustom{S: fmt.Sprintf("Start index %d cannot be greater than end index %d", startIndex.Unwrap(), endIndex.Unwrap())}
			case startIndex.IsNone() && endIndex.IsNone():
				return false, nil, ErrCustom{S: "Start and end index for substr cannot both be None"}
			}

			expression = coerceSubstr{
				value: expression,
				start: startIndex,
				end:   endIndex,
			}
			if constEligible {
				value, err := expression.Calculate([]byte{})
				if err != nil {
					return false, nil, err
				}
				return constEligible, coercedConstant{value: value}, nil
			} else {
				return false, expression, nil
			}
		},
	})
)

Functions

This section is empty.

Types

type ErrCustom added in v0.9.0 

type ErrCustom struct {
	S string
}

ErrCustom represents a custom error

func (ErrCustom) Error added in v0.9.0 

func (e ErrCustom) Error() string

type ErrInvalidBool 

type ErrInvalidBool struct {
	// contains filtered or unexported fields
}

ErrInvalidBool represents an invalid boolean

func (ErrInvalidBool) Error 

func (e ErrInvalidBool) Error() string

type ErrInvalidCoerce added in v0.9.0 

type ErrInvalidCoerce struct {
	Err error
}

ErrInvalidCoerce represents an invalid Coerce error

func (ErrInvalidCoerce) Error added in v0.9.0 

func (e ErrInvalidCoerce) Error() string

type ErrInvalidIdentifier 

type ErrInvalidIdentifier struct {
	// contains filtered or unexported fields
}

ErrInvalidIdentifier represents an invalid identifier

func (ErrInvalidIdentifier) Error 

func (e ErrInvalidIdentifier) Error() string

type ErrInvalidKeyword 

type ErrInvalidKeyword struct {
	// contains filtered or unexported fields
}

ErrInvalidKeyword represents an invalid keyword keyword

func (ErrInvalidKeyword) Error 

func (e ErrInvalidKeyword) Error() string

type ErrInvalidNumber 

type ErrInvalidNumber struct {
	// contains filtered or unexported fields
}

ErrInvalidNumber represents an invalid number

func (ErrInvalidNumber) Error 

func (e ErrInvalidNumber) Error() string

type ErrInvalidSelectorPath added in v0.2.0 

type ErrInvalidSelectorPath struct {
	// contains filtered or unexported fields
}

ErrInvalidSelectorPath represents an invalid selector string

func (ErrInvalidSelectorPath) Error added in v0.2.0 

func (e ErrInvalidSelectorPath) Error() string

type ErrUnsupportedCharacter 

type ErrUnsupportedCharacter struct {
	// contains filtered or unexported fields
}

ErrUnsupportedCharacter represents an unsupported character is expression being lexed.

func (ErrUnsupportedCharacter) Error 

func (e ErrUnsupportedCharacter) Error() string

type ErrUnsupportedCoerce added in v0.2.0 

type ErrUnsupportedCoerce struct {
	// contains filtered or unexported fields
}

ErrUnsupportedCoerce represents a comparison of incompatible types type casts

func (ErrUnsupportedCoerce) Error added in v0.2.0 

func (e ErrUnsupportedCoerce) Error() string

type ErrUnsupportedTypeComparison 

type ErrUnsupportedTypeComparison struct {
	// contains filtered or unexported fields
}

ErrUnsupportedTypeComparison represents a comparison of incompatible types

func (ErrUnsupportedTypeComparison) Error 

func (e ErrUnsupportedTypeComparison) Error() string

type ErrUnterminatedString 

type ErrUnterminatedString struct {
	// contains filtered or unexported fields
}

ErrUnterminatedString represents an unterminated string

func (ErrUnterminatedString) Error 

func (e ErrUnterminatedString) Error() string

type Expression 

type Expression interface {

	// Calculate will execute the parsed expression and apply it against the supplied data.
	//
	// # Errors
	//
	// Will return `Err` if the expression cannot be applied to the supplied data due to invalid
	// data type comparisons.
	Calculate(src []byte) (any, error)
}

Expression Represents a stateless parsed expression that can be applied to JSON data.

func Parse 

func Parse(expression []byte) (Expression, error)

Parse lex's' the provided expression and returns an Expression to be used/applied to data.

type LexerResult 

type LexerResult struct {
	// contains filtered or unexported fields
}

LexerResult represents a token lexed result

type Parser added in v0.9.0 

type Parser struct {
	Exp       []byte
	Tokenizer itertools.PeekableIterator[resultext.Result[Token, error]]
}

Parser parses and returns a supplied expression

type Token 

type Token struct {
	Start uint32
	Len   uint16
	Kind  TokenKind
}

Token represents a lexed token

type TokenKind 

type TokenKind uint8

TokenKind is the type of token lexed.

type Tokenizer 

type Tokenizer struct {
	// contains filtered or unexported fields
}

Tokenizer is a lexer for the KSQL expression syntax.

func NewTokenizer added in v0.2.0 

func NewTokenizer(src []byte) *Tokenizer

NewTokenizer creates a new Tokenizer for use

func (*Tokenizer) Next added in v0.2.0 

func (t *Tokenizer) Next() optionext.Option[resultext.Result[Token, error]]

Source Files

View all Source files

Directories

Path Synopsis
_examples
custom-coercion
simple
cmd
ksql

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.