kqlfilter

package module
v0.3.4 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 16, 2023 License: MIT Imports: 9 Imported by: 1

README

KQL filter package

GoDoc

This package contains Kibana Query Language parser.

go get "github.com/mycujoo/go-stdlib/pkg/kqlfilter"

Usage

You can use either Parse or ParseAST to parse a KQL filter.

Parse will return a Filter struct, which is simple to use, but does not support all KQL features.

package main

import (
    "fmt"

    "github.com/mycujoo/go-stdlib/pkg/kqlfilter"
)

func main() {
    filter, err := kqlfilter.Parse("foo:bar", false)
    if err != nil {
        panic(err)
    }

    fmt.Println(filter)
}

ParseAST will return an AST struct, which is more complex to use, but supports all KQL features. It returns an AST struct, which is a tree of Nodes.

package main

import (
    "fmt"

    "github.com/mycujoo/go-stdlib/pkg/kqlfilter"
)

func main() {
    ast, err := kqlfilter.ParseAST("foo:bar")
    if err != nil {
        panic(err)
    }

    fmt.Println(ast)
}

Documentation

Index

Constants

View Source
const (
	FilterToSpannerFieldColumnTypeString = iota
	FilterToSpannerFieldColumnTypeInt64
	FilterToSpannerFieldColumnTypeFloat64
	FilterToSpannerFieldColumnTypeBool
	FilterToSpannerFieldColumnTypeTimestamp
)
View Source
const (
	FilterToSquirrelSqlFieldColumnTypeString = iota
	FilterToSquirrelSqlFieldColumnTypeInt
	FilterToSquirrelSqlFieldColumnTypeFloat
	FilterToSquirrelSqlFieldColumnTypeBool
	FilterToSquirrelSqlFieldColumnTypeTimestamp
)

Variables

This section is empty.

Functions

This section is empty.

Types

type AndNode

type AndNode struct {
	NodeType
	Pos

	Nodes []Node // The clauses nodes in lexical order.
	// contains filtered or unexported fields
}

AndNode holds multiple sub queries.

func (*AndNode) String

func (q *AndNode) String() string

type Clause

type Clause struct {
	Field string
	// One of the following: `=`, `<`, `<=`, `>`, `>=`, `IN`
	Operator string
	// List of values for the clause.
	// For `IN` operator, this is a list of values to match against.
	// For other operators, this is a list of one string.
	Values []string
}

func (*Clause) ToSquirrelSql

func (c *Clause) ToSquirrelSql(stmt sq.SelectBuilder, config FilterToSquirrelSqlFieldConfig) (sq.SelectBuilder, error)

type Filter

type Filter struct {
	Clauses []Clause
}

func Parse

func Parse(input string, enableRangeOperator bool) (Filter, error)

Parse parses a filter string into a Filter struct. The filter string must not contain any boolean operators, parentheses or nested queries. The filter string must contain only simple clauses of the form "field:value", where all clauses are AND'ed. Optionally, range operators can be enabled, e.g. for expressions involving date ranges. If you need to parse a more complex filter string, use ParseAST instead.

func (Filter) ToSpannerSQL

func (f Filter) ToSpannerSQL(fieldConfigs map[string]FilterToSpannerFieldConfig) ([]string, map[string]any, error)

ToSpannerSQL turns a Filter into a partial StandardSQL statement. It takes a map of fields that are allowed to be queried via this filter (as a user should not be able to query all db columns via a filter). It returns a partial SQL statement that can be added to a WHERE clause, along with associated params. An example follows.

Given a Filter that looks like this:

[(Field: "userId", Operator: "=", Values: []string{"12345"}), (Field: "email", Operator: "=", Values: []string{"john@example.*"})]

and fieldConfigs that looks like this:

{
	"userId": (ColumnName: "user_id", ColumnType: FilterToSpannerFieldColumnTypeInt64,  AllowPartialMatch: false),
	"email":  (ColumnName: "email",   ColumnType: FilterToSpannerFieldColumnTypeString, AllowPartialMatch: true)
}

This returns a slice of SQL conditions that can be appended to an existing WHERE clause (make sure to AND these first):

["user_id=@KQL0", "email LIKE @KQL1"]

and params:

{
	"@KQL0": int64(12345),
	"@KQL1": "john@example.%"
}

For multi-value fields, the returned SQL conditions will use the IN operator:

[(Field: "team_id", Operator: "IN", Values: []string{"T1", "T2"})]

{
	"team_id": (ColumnName: "user_id", ColumnType: FilterToSpannerFieldColumnTypeString, AllowMultipleValues: true),
}

SQL would be:

["team_id IN (@KQL0,@KQL1)"]

and params:

{
	"@KQL0": "T1",
	"@KQL1": "T2"
}

Note: The Clause Operator is contextually used/ignored. It only works with INT64, FLOAT64 and TIMESTAMP types currently.

func (Filter) ToSquirrelSql

func (f Filter) ToSquirrelSql(stmt sq.SelectBuilder, fieldConfigs map[string]FilterToSquirrelSqlFieldConfig) (sq.SelectBuilder, error)

type FilterToSpannerFieldColumnType

type FilterToSpannerFieldColumnType int

func (FilterToSpannerFieldColumnType) String

type FilterToSpannerFieldConfig

type FilterToSpannerFieldConfig struct {
	// SQL table column name. Can be omitted if the column name is equal to the key in the fieldConfigs map.
	ColumnName string
	// SQL column type. Defaults to FilterToSpannerFieldColumnTypeString.
	ColumnType FilterToSpannerFieldColumnType
	// Allow prefix matching when a wildcard (`*`) is present at the end of a string.
	// Only applicable for FilterToSpannerFieldColumnTypeString. Defaults to false.
	AllowPrefixMatch bool
	// Allow multiple values for this field. Defaults to false.
	AllowMultipleValues bool
	// A function that takes a string value as provided by the user and converts it to `any` result that matches how it is
	// stored in the database. This should return an error when the user is providing a value that is illegal for this
	// particular field. Defaults to using the provided value as-is.
	MapValue func(string) (any, error)
}

type FilterToSquirrelSqlFieldColumnType

type FilterToSquirrelSqlFieldColumnType int

type FilterToSquirrelSqlFieldConfig

type FilterToSquirrelSqlFieldConfig struct {
	// SQL table column name. Can be omitted if the column name is equal to the key in the fieldConfigs map.
	ColumnName string
	// SQL column type. Defaults to FilterToSquirrelSqlFieldColumnTypeString.
	ColumnType FilterToSquirrelSqlFieldColumnType
	// Allow prefix matching when a wildcard (`*`) is present at the end of a string.
	// Only applicable for FilterToSpannerFieldColumnTypeString. Defaults to false.
	AllowPrefixMatch bool
	// Allow multiple values for this field. Defaults to false.
	AllowMultipleValues bool
	// A function that takes a string value as provided by the user and converts it to string result that matches how it
	// should be as users' input. This should return an error when the user is providing a value that is illegal or unexpected
	// for this particular field. Defaults to using the provided value as-is.
	MapValue func(string) (any, error)
	// A function that handle parsing the sql statement by itself.
	// If set, all other fields in the config will be ignored
	CustomBuilder func(stmt sq.SelectBuilder, operator string, values []string) (sq.SelectBuilder, error)
}

type IsNode

type IsNode struct {
	NodeType
	Pos

	Identifier string
	Value      Node // The clauses nodes in lexical order.
	// contains filtered or unexported fields
}

IsNode holds equality check.

func (*IsNode) String

func (q *IsNode) String() string

type LiteralNode

type LiteralNode struct {
	NodeType
	Pos

	Value string
	// contains filtered or unexported fields
}

LiteralNode holds literal value.

func (*LiteralNode) String

func (q *LiteralNode) String() string

type NestedNode

type NestedNode struct {
	NodeType
	Pos

	Expr Node // The clauses nodes in lexical order.
	// contains filtered or unexported fields
}

NestedNode holds nested sub query.

func (*NestedNode) String

func (q *NestedNode) String() string

type Node

type Node interface {
	Type() NodeType
	String() string
	Position() Pos // byte position of start of node in full original input string
	// contains filtered or unexported methods
}

A Node is an element in the parse tree.

func ParseAST

func ParseAST(input string, options ...ParserOption) (n Node, err error)

ParseAST parses a filter string into an AST. The filter string must be a valid Kibana query language filter string.

type NodeMapper added in v0.3.4

type NodeMapper struct {
	TransformIdentifierFunc func(string) string
	TransformValueFunc      func(string) string
}

func NewNodeMapper added in v0.3.4

func NewNodeMapper() NodeMapper

func (NodeMapper) Map added in v0.3.4

func (m NodeMapper) Map(ast Node) error

type NodeType

type NodeType int

NodeType identifies the type of parse tree node.

const (
	NodeOr NodeType = iota // Plain text.
	NodeAnd
	NodeNot
	NodeIs
	NodeRange
	NodeNested
	NodeLiteral
)

func (NodeType) Type

func (t NodeType) Type() NodeType

Type returns itself and provides an easy default implementation for embedding in a Node. Embedded in all non-trivial Nodes.

type NotNode

type NotNode struct {
	NodeType
	Pos

	Expr Node // Negated node.
	// contains filtered or unexported fields
}

NotNode holds a negated sub query.

func (*NotNode) String

func (q *NotNode) String() string

type OrNode

type OrNode struct {
	NodeType
	Pos

	Nodes []Node // The clauses nodes in lexical order.
	// contains filtered or unexported fields
}

OrNode holds multiple sub queries.

func (*OrNode) String

func (q *OrNode) String() string

type ParserOption

type ParserOption func(*parser)

ParserOption is a function that configures a parser.

func DisableComplexExpressions

func DisableComplexExpressions() ParserOption

DisableComplexExpressions disables complex expressions.

func WithMaxComplexity

func WithMaxComplexity(complexity int) ParserOption

WithMaxComplexity sets limit to maximum number of individual clauses separated by boolean operators.

func WithMaxDepth

func WithMaxDepth(depth int) ParserOption

WithMaxDepth sets limit to maximum number of nesting.

type Pos

type Pos int

Pos represents a byte position in the original input text from which this template was parsed.

func (Pos) Position

func (p Pos) Position() Pos

type RangeNode

type RangeNode struct {
	NodeType
	Pos

	Identifier string
	Operator   RangeOperator
	Value      Node // The clauses nodes in lexical order.
	// contains filtered or unexported fields
}

RangeNode holds range check.

func (*RangeNode) String

func (q *RangeNode) String() string

type RangeOperator

type RangeOperator int
const (
	RangeOperatorGt RangeOperator = iota
	RangeOperatorGte
	RangeOperatorLt
	RangeOperatorLte
)

func (RangeOperator) String

func (o RangeOperator) String() string

Directories

Path Synopsis
elastic module

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL