README
¶
DCParser - SQL Parser optimized based on PingCAP/parser
Introduction
DCParser is a single project built for compatibility with TiDB for PostgreSQL. It is based on PingCAP/parser, but it is difficult to merge into the original parser repository because there are many changes. So a new project is created to provide parsing abilities for TiDB for PostgreSQL.
Development
The current development is driven by the implementation of TiDB for PostgreSQL. The implements of Schema structure, system functions, system views and other functions of TiDB for PostgreSQL depend on the parser modification. Currently, the DCParser can fully meet the functional requirements of the TiDB for PostgreSQL. If TiDB for PostgreSQL needs new system functions, system views and other functions, you also need to modify them in DCParser.
Quick start
DCParser project is the sql parser of TiDB for PostgreSQL. The most common way to use it is to import it into the TiDB for PostgreSQL project and play its role by parsing sql statements.
DCParser is written in Go language. SO to use DCParser, you need to ensure that the Go environment is installed.
In the TiDB for PostgreSQL project, it has been declared that DCParser needs to be pulled. You can use the vendor command to get the project before compiling the project.
In the GOPATH of the development environment, there will be the source files of the DCParser project that TiDB for PostgreSQL depends on. Modify the parser.y file, and then use the following command to compile the new parser.go file.
goyacc -o parser.go parser.y
Contributing
We greatly welcome and appreciate developers to contribute to DCParser to realize other features of TiDB for PostgreSQL.
License
The license is pending.
Documentation
¶
Index ¶
- Variables
- func DigestHash(sql string) (result string)deprecated
- func DigestNormalized(normalized string) (result string)
- func Normalize(sql string) (result string)
- func NormalizeDigest(sql string) (normalized, digest string)
- func ParseErrorWith(errstr string, lineno int) error
- func ParseHint(input string, sqlMode mysql.SQLMode, initPos Pos) ([]*ast.TableOptimizerHint, []error)
- func TrimComment(txt string) string
- type Parser
- type Pos
- type Scanner
- func (s *Scanner) AppendError(err error)
- func (s *Scanner) EnableWindowFunc(val bool)
- func (s *Scanner) Errorf(format string, a ...interface{}) (err error)
- func (s *Scanner) Errors() (warns []error, errs []error)
- func (s *Scanner) GetSQLMode() mysql.SQLMode
- func (s *Scanner) InheritScanner(sql string) *Scanner
- func (s *Scanner) Lex(v *yySymType) int
- func (s *Scanner) SetSQLMode(mode mysql.SQLMode)
Constants ¶
This section is empty.
Variables ¶
var ( ErrWarnOptimizerHintUnsupportedHint = terror.ClassParser.NewStd(mysql.ErrWarnOptimizerHintUnsupportedHint) ErrWarnOptimizerHintInvalidToken = terror.ClassParser.NewStd(mysql.ErrWarnOptimizerHintInvalidToken) ErrWarnMemoryQuotaOverflow = terror.ClassParser.NewStd(mysql.ErrWarnMemoryQuotaOverflow) ErrWarnOptimizerHintParseError = terror.ClassParser.NewStd(mysql.ErrWarnOptimizerHintParseError) ErrWarnOptimizerHintInvalidInteger = terror.ClassParser.NewStd(mysql.ErrWarnOptimizerHintInvalidInteger) )
var ( // ErrSyntax returns for sql syntax error. ErrSyntax = terror.ClassParser.NewStd(mysql.ErrSyntax) // ErrParse returns for sql parse error. ErrParse = terror.ClassParser.NewStd(mysql.ErrParse) // ErrUnknownCharacterSet returns for no character set found error. ErrUnknownCharacterSet = terror.ClassParser.NewStd(mysql.ErrUnknownCharacterSet) // ErrInvalidYearColumnLength returns for illegal column length for year type. ErrInvalidYearColumnLength = terror.ClassParser.NewStd(mysql.ErrInvalidYearColumnLength) // ErrWrongArguments returns for illegal argument. ErrWrongArguments = terror.ClassParser.NewStd(mysql.ErrWrongArguments) // ErrWrongFieldTerminators returns for illegal field terminators. ErrWrongFieldTerminators = terror.ClassParser.NewStd(mysql.ErrWrongFieldTerminators) // ErrTooBigDisplayWidth returns for data display width exceed limit . ErrTooBigDisplayWidth = terror.ClassParser.NewStd(mysql.ErrTooBigDisplaywidth) // ErrTooBigPrecision returns for data precision exceed limit. ErrTooBigPrecision = terror.ClassParser.NewStd(mysql.ErrTooBigPrecision) // ErrUnknownAlterLock returns for no alter lock type found error. ErrUnknownAlterLock = terror.ClassParser.NewStd(mysql.ErrUnknownAlterLock) // ErrUnknownAlterAlgorithm returns for no alter algorithm found error. ErrUnknownAlterAlgorithm = terror.ClassParser.NewStd(mysql.ErrUnknownAlterAlgorithm) // ErrWrongValue returns for wrong value ErrWrongValue = terror.ClassParser.NewStd(mysql.ErrWrongValue) // ErrWarnDeprecatedSyntaxNoReplacement return when the syntax was deprecated and there is no replacement. ErrWarnDeprecatedSyntaxNoReplacement = terror.ClassParser.NewStd(mysql.ErrWarnDeprecatedSyntaxNoReplacement) // ErrWarnDeprecatedIntegerDisplayWidth share the same code 1681, and it will be returned when length is specified in integer. ErrWarnDeprecatedIntegerDisplayWidth = terror.ClassParser.NewStdErr(mysql.ErrWarnDeprecatedSyntaxNoReplacement, mysql.Message("Integer display width is deprecated and will be removed in a future release.", nil)) // SpecFieldPattern special result field pattern SpecFieldPattern = regexp.MustCompile(`(\/\*!(M?[0-9]{5,6})?|\*\/)`) )
var SpecialCommentsController = specialCommentsCtrl{ // contains filtered or unexported fields }
SpecialCommentsController controls whether special comments like `/*T![xxx] yyy */` can be parsed as `yyy`. To add such rules, please use SpecialCommentsController.Register(). For example:
SpecialCommentsController.Register("30100");
Now the parser will treat
select a, /*T![30100] mysterious_keyword */ from t;
and
select a, mysterious_keyword from t;
equally. Similar special comments without registration are ignored by parser.
Functions ¶
func DigestHash
deprecated
DigestHash generates the digest of statements. it will generate a hash on normalized form of statement text which removes general property of a statement but keeps specific property.
for example: both DigestHash('select 1') and DigestHash('select 2') => e1c71d1661ae46e09b7aaec1c390957f0d6260410df4e4bc71b9c8d681021471
Deprecated: It is logically consistent with NormalizeDigest.
func DigestNormalized ¶
DigestNormalized generates the digest of a normalized sql. it will generate a hash on a normalized sql. Normalize + DigestNormalized equals to NormalizeDigest.
for example: DigestNormalized('select ?') DigestNormalized should be called with a normalized SQL string (like 'select ?') generated by function Normalize. do not call with SQL which is not normalized, DigestNormalized('select 1') and DigestNormalized('select 2') is not the same
func Normalize ¶
Normalize generates the normalized statements. it will get normalized form of statement text which removes general property of a statement but keeps specific property.
for example: Normalize('select 1 from b where a = 1') => 'select ? from b where a = ?'
func NormalizeDigest ¶
NormalizeDigest combines Normalize and DigestNormalized into one method.
func ParseErrorWith ¶
ParseErrorWith returns "You have a syntax error near..." error message compatible with mysql.
func ParseHint ¶
func ParseHint(input string, sqlMode mysql.SQLMode, initPos Pos) ([]*ast.TableOptimizerHint, []error)
ParseHint parses an optimizer hint (the interior of `/*+ ... */`).
func TrimComment ¶
TrimComment trim comment for special comment code of MySQL.
Types ¶
type Parser ¶
type Parser struct {
// contains filtered or unexported fields
}
Parser represents a parser instance. Some temporary objects are stored in it to reduce object allocation during Parse function.
func (*Parser) EnableWindowFunc ¶
EnableWindowFunc controls whether the parser to parse syntax related with window function.
func (*Parser) Parse ¶
func (parser *Parser) Parse(sql, charset, collation string) (stmt []ast.StmtNode, warns []error, err error)
Parse parses a query string to raw ast.StmtNode. If charset or collation is "", default charset and collation will be used.
func (*Parser) ParseOneStmt ¶
ParseOneStmt parses a query and returns an ast.StmtNode. The query must have one statement, otherwise ErrSyntax is returned.
func (*Parser) SetSQLMode ¶
SetSQLMode sets the SQL mode for parser.
type Scanner ¶
type Scanner struct {
// contains filtered or unexported fields
}
Scanner implements the yyLexer interface.
func (*Scanner) AppendError ¶
AppendError sets error into scanner. Scanner satisfies yyLexer interface which need this function.
func (*Scanner) EnableWindowFunc ¶
EnableWindowFunc controls whether the scanner recognize the keywords of window function.
func (*Scanner) Errorf ¶
Errorf tells scanner something is wrong. Scanner satisfies yyLexer interface which need this function.
func (*Scanner) GetSQLMode ¶
GetSQLMode return the SQL mode of scanner.
func (*Scanner) InheritScanner ¶
InheritScanner returns a new scanner object which inherits configurations from the parent scanner.
func (*Scanner) Lex ¶
Lex returns a token and store the token value in v. Scanner satisfies yyLexer interface. 0 and invalid are special token id this function would return: return 0 tells parser that scanner meets EOF, return invalid tells parser that scanner meets illegal character.
func (*Scanner) SetSQLMode ¶
SetSQLMode sets the SQL mode for scanner.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package ast is the abstract syntax tree parsed from a SQL statement by parser.
|
Package ast is the abstract syntax tree parsed from a SQL statement by parser. |
|
Goyacc is a version of yacc generating Go parsers.
|
Goyacc is a version of yacc generating Go parsers. |