Documentation
¶
Overview ¶
Package syms defines the symbols and their properties that are accumulated from a parsed file, and are then used for e.g., completion lookup, etc.
We looked at several different standards for formats, types, etc:
LSP: https://microsoft.github.io/language-server-protocol/specification useful to enable GoPi to act as an LSP server at some point. additional symbol kinds: https://github.com/Microsoft/language-server-protocol/issues/344
See also: github.com/sourcegraph/sourcegraph and specifically: /cmd/frontend/graphqlbackend/search_symbols.go it seems to use https://github.com/universal-ctags/ctags for the raw data..
Other relevant guidance comes from the go compiler system which is used extensively in github.com/mdemsky/gocode for example. In particular: go/types/scope.go type.go, and package.go contain the relevant data structures for how information is organized for compiled go packages, which have all this data cached and avail to be imported via the go/importer which returns a go/types/Package which in turn contains Scope's which in turn contain Objects that define the elements of the compiled language.
Index ¶
- Variables
- func CacheFilename(lang fi.Known, filename string) (string, error)
- func GoPiCacheDir(lang fi.Known) (string, error)
- func GoRelPath(filename string) (string, error)
- func InitCatMap()
- func InitSub2CatMap()
- func InitSubCatMap()
- func SaveSymCache(sy *Symbol, lang fi.Known, filename string) error
- func SaveSymDoc(sy *Symbol, lang fi.Known, filename string) error
- type Kinds
- func (tk Kinds) Cat() Kinds
- func (i Kinds) Desc() string
- func (tk Kinds) InCat(other Kinds) bool
- func (tk Kinds) InSub2Cat(other Kinds) bool
- func (tk Kinds) InSubCat(other Kinds) bool
- func (i Kinds) Int64() int64
- func (tk Kinds) IsCat() bool
- func (tk Kinds) IsPrimitiveNonPtr() bool
- func (tk Kinds) IsPtr() bool
- func (tk Kinds) IsSub2Cat() bool
- func (tk Kinds) IsSubCat() bool
- func (i Kinds) IsValid() bool
- func (i Kinds) MarshalText() ([]byte, error)
- func (i *Kinds) SetInt64(in int64)
- func (i *Kinds) SetString(s string) error
- func (i Kinds) String() string
- func (tk Kinds) Sub2Cat() Kinds
- func (tk Kinds) SubCat() Kinds
- func (i *Kinds) UnmarshalText(text []byte) error
- func (i Kinds) Values() []enums.Enum
- type SymMap
- func (sm *SymMap) Add(sy *Symbol)
- func (sm *SymMap) AddNew(name string, kind token.Tokens, fname string, reg lex.Reg) *Symbol
- func (sm *SymMap) Alloc()
- func (sm *SymMap) CopyFrom(src SymMap, srcIsNewer bool)
- func (sm *SymMap) FindContainsRegion(fpath string, pos lex.Pos, extraLns int, kind token.Tokens, matches *SymMap)
- func (sm *SymMap) FindKind(kind token.Tokens, matches *SymMap)
- func (sm *SymMap) FindKindScoped(kind token.Tokens, matches *SymMap)
- func (sm *SymMap) FindName(nm string) (*Symbol, bool)
- func (sm *SymMap) FindNamePrefix(seed string, matches *SymMap)
- func (sm *SymMap) FindNamePrefixRecursive(seed string, matches *SymMap)
- func (sm *SymMap) FindNamePrefixScoped(seed string, matches *SymMap)
- func (sm *SymMap) FindNameScoped(nm string) (*Symbol, bool)
- func (sm *SymMap) First() *Symbol
- func (sm *SymMap) KindNames(sorted bool) []string
- func (sm *SymMap) Names(sorted bool) []string
- func (sm *SymMap) OpenJSON(filename string) error
- func (sm *SymMap) Reset()
- func (sm *SymMap) SaveJSON(filename string) error
- func (sm *SymMap) Slice(sorted bool) []*Symbol
- func (sm *SymMap) WriteDoc(out io.Writer, depth int)
- type SymNames
- type SymStack
- type Symbol
- func (sy *Symbol) AddChild(child *Symbol) bool
- func (sy *Symbol) AddScopesMap(sm SymMap, add bool)
- func (sy *Symbol) AddScopesStack(ss SymStack) bool
- func (sy *Symbol) AllocScopes()
- func (sy *Symbol) Clone() *Symbol
- func (sy *Symbol) CopyFromScope(src *Symbol)
- func (sy *Symbol) CopyFromSrc(cp *Symbol)
- func (sy *Symbol) FindAnyChildren(seed string, scope1, scope2 SymMap, kids *SymMap) bool
- func (sy *Symbol) HasChildren() bool
- func (sy *Symbol) IsTemp() bool
- func (sy *Symbol) Label() string
- func (sy *Symbol) NonPtrTypeName() string
- func (sy *Symbol) OpenJSON(filename string) error
- func (sy *Symbol) SaveJSON(filename string) error
- func (sy *Symbol) String() string
- func (sy *Symbol) WriteDoc(out io.Writer, depth int)
- type Type
- func (ty *Type) AddScopesStack(ss SymStack)
- func (ty *Type) AllocScopes()
- func (ty *Type) ArgString() string
- func (ty *Type) Clone() *Type
- func (ty *Type) CopyFromSrc(cp *Type)
- func (ty *Type) NonPtrType() string
- func (ty *Type) ReturnString() string
- func (ty *Type) String() string
- func (ty *Type) WriteDoc(out io.Writer, depth int)
- type TypeEl
- type TypeEls
- type TypeKindSize
- type TypeMap
- func (tm *TypeMap) Add(ty *Type)
- func (tm *TypeMap) Alloc()
- func (tm *TypeMap) Clone() TypeMap
- func (tm *TypeMap) CopyFrom(src TypeMap, srcIsNewer bool)
- func (tm *TypeMap) KindNames(sorted bool) []string
- func (tm *TypeMap) Names(sorted bool) []string
- func (tm *TypeMap) PrintUnknowns()
- func (tm *TypeMap) WriteDoc(out io.Writer, depth int)
Constants ¶
This section is empty.
Variables ¶
var CatMap map[Kinds]Kinds
CatMap is the map into the category level for each kind
Categories
var ReflectKindMap = map[reflect.Kind]Kinds{ reflect.Invalid: Unknown, reflect.Int: Int, reflect.Int8: Int8, reflect.Int16: Int16, reflect.Int32: Int32, reflect.Int64: Int64, reflect.Uint: Uint, reflect.Uint8: Uint8, reflect.Uint16: Uint16, reflect.Uint32: Uint32, reflect.Uint64: Uint64, reflect.Uintptr: Uintptr, reflect.Float32: Float32, reflect.Float64: Float64, reflect.Complex64: Complex64, reflect.Complex128: Complex128, reflect.Bool: Bool, reflect.Array: Array, reflect.Chan: Chan, reflect.Func: Func, reflect.Interface: Interface, reflect.Map: Map, reflect.Ptr: Ptr, reflect.Slice: List, reflect.String: String, reflect.Struct: Struct, reflect.UnsafePointer: Ptr, }
var Sub2CatMap map[Kinds]Kinds
Sub2CatMap is the map into the sub2-category level for each kind
var Sub2Cats = []Kinds{ Unknown, Primitive, Numeric, Integer, Fixed, Float, Bool, Composite, Tuple, Array, List, Matrix, Tensor, Map, Set, Struct, Chan, Function, Func, Method, Interface, KindsN, }
Sub2-Categories
var SubCatMap map[Kinds]Kinds
SubCatMap is the map into the sub-category level for each kind
var SubCats = []Kinds{ Unknown, Primitive, Numeric, Bool, Composite, Tuple, Array, List, Matrix, Tensor, Map, Set, Struct, Chan, Function, Func, Method, Interface, KindsN, }
Sub-Categories
Functions ¶
func CacheFilename ¶
CacheFilename returns the filename to use for cache file for given filename
func GoPiCacheDir ¶
GoPiCacheDir returns the GoPi cache directory for given language, and ensures that it exists
func SaveSymCache ¶
SaveSymCache saves cache of symbols starting with given symbol (typically a package, module, library), which is at given filename
Types ¶
type Kinds ¶
type Kinds int32 //enums:enum
Kinds is a complete set of basic type categories and sub(sub..) categories -- these describe builtin types -- user-defined types must be some combination / version of these builtin types.
See: https://en.wikipedia.org/wiki/List_of_data_structures
const ( // Unknown is the nil kind -- kinds should be known in general.. Unknown Kinds = iota // Category: Primitive, in the strict sense of low-level, atomic, small, fixed size Primitive // SubCat: Numeric Numeric // Sub2Cat: Integer Integer // Sub3Cat: Signed -- track this using props in types, not using Sub3 level Signed Int Int8 Int16 Int32 Int64 // Sub3Cat: Unsigned Unsigned Uint Uint8 Uint16 Uint32 Uint64 Uintptr // generic raw pointer data value -- see also Ptr, Ref for more semantic cases // Sub3Cat: Ptr, Ref etc -- in Numeric, Integer even though in some languages // pointer arithmetic might not be allowed, for some cases, etc Ptr // pointer -- element is what we point to (kind of a composite type) Ref // reference -- element is what we refer to UnsafePtr // for case where these are distinguished from Ptr (Go) -- similar to Uintptr // Sub2Cat: Fixed point -- could be under integer, but.. Fixed Fixed26_6 Fixed16_6 Fixed0_32 // Sub2Cat: Floating point Float Float16 Float32 Float64 // Sub3Cat: Complex -- under floating point Complex Complex64 Complex128 // SubCat: Bool Bool // Category: Composite -- types composed of above primitive types Composite // SubCat: Tuple -- a fixed length 1d collection of elements that can be of any type // Type.Els required for each element Tuple Range // a special kind of tuple for Python ranges // SubCat: Array -- a fixed length 1d collection of same-type elements // Type.Els has one element for type Array // SubCat: List -- a variable-length 1d collection of same-type elements // This is Slice for Go // Type.Els has one element for type List String // List of some type of char rep -- Type.Els is type, as all Lists // SubCat: Matrix -- a twod collection of same-type elements // has two Size values, one for each dimension Matrix // SubCat: Tensor -- an n-dimensional collection of same-type elements // first element of Size is number of dimensions, rest are dimensions Tensor // SubCat: Map -- an associative array / hash map / dictionary // Type.Els first el is key, second is type Map // SubCat: Set -- typically a degenerate form of hash map with no value Set FrozenSet // python's frozen set of fixed values // SubCat: Struct -- like a tuple but with specific semantics in most languages // Type.Els are the fields, and if there is an inheritance relationship these // are put first with relevant identifiers -- in Go these are unnamed fields Struct Class Object // Chan: a channel (Go Specific) Chan // Category: Function -- types that are functions // Type.Els are the params and return values in order, with Size[0] being number // of params and Size[1] number of returns Function // SubCat: Func -- a standalone function Func // SubCat: Method -- a function with a specific receiver (e.g., on a Class in C++, // or on any type in Go). // First Type.Els is receiver param -- included in Size[0] Method // SubCat: Interface -- an abstract definition of a set of methods (in Go) // Type.Els are the Methods with the receiver type missing or Unknown Interface )
The list of Kinds
const KindsN Kinds = 51
KindsN is the highest valid value for type Kinds, plus one.
func KindsValues ¶
func KindsValues() []Kinds
KindsValues returns all possible values for the type Kinds.
func (Kinds) IsPrimitiveNonPtr ¶
func (Kinds) MarshalText ¶
MarshalText implements the encoding.TextMarshaler interface.
func (*Kinds) SetString ¶
SetString sets the Kinds value from its string representation, and returns an error if the string is invalid.
func (Kinds) Sub2Cat ¶
Sub2Cat returns the sub2-category that a given kind lives in, using Sub2CatMap
func (*Kinds) UnmarshalText ¶
UnmarshalText implements the encoding.TextUnmarshaler interface.
type SymMap ¶
SymMap is a map between symbol names and their full information. A given project will have a top-level SymMap and perhaps local maps for individual files, etc. Namespaces / packages can be created and elements added to them to create appropriate scoping structure etc. Note that we have to use pointers for symbols b/c otherwise it is very expensive to re-assign values all the time -- https://github.com/golang/go/issues/3117
func (*SymMap) CopyFrom ¶
CopyFrom copies all the symbols from given source map into this one, including merging everything from common elements. Symbols with Type resolved are retained when there are duplicates. srcIsNewer means that the src map has the newer information to grab for updating the symbol region info during the merge.
func (*SymMap) FindContainsRegion ¶
func (sm *SymMap) FindContainsRegion(fpath string, pos lex.Pos, extraLns int, kind token.Tokens, matches *SymMap)
FindContainsRegion looks for given symbol kind that contains the given source file path (must be filepath.Abs file path) and position. Returns all instances found. Uses cat / subcat based token matching -- if you specify a category-level or subcategory level token, it will match everything in that group. if you specify kind = token.None then all tokens that contain region will be returned. extraLns are extra lines added to the symbol region for purposes of matching.
func (*SymMap) FindKind ¶
FindKind looks for given symbol kind within this map and any children on the map Returns all instances found. Uses cat / subcat based token matching -- if you specify a category-level or subcategory level token, it will match everything in that group
func (*SymMap) FindKindScoped ¶
FindKindScoped looks for given symbol kind within this map and any children on the map that are of subcategory token.NameScope (i.e., namespace, module, package, library). Returns all instances found. Uses cat / subcat based token matching -- if you specify a category-level or subcategory level token, it will match everything in that group
func (*SymMap) FindName ¶
FindName looks for given symbol name within this map and any children on the map
func (*SymMap) FindNamePrefix ¶
FindNamePrefix looks for given symbol name prefix within this map adds to given matches map (which can be nil), for more efficient recursive use
func (*SymMap) FindNamePrefixRecursive ¶
FindNamePrefixRecursive looks for given symbol name prefix within this map and any children on the map. adds to given matches map (which can be nil), for more efficient recursive use
func (*SymMap) FindNamePrefixScoped ¶
FindNamePrefixScoped looks for given symbol name prefix within this map and any children on the map that are of subcategory token.NameScope (i.e., namespace, module, package, library) adds to given matches map (which can be nil), for more efficient recursive use
func (*SymMap) FindNameScoped ¶
FindNameScoped looks for given symbol name within this map and any children on the map that are of subcategory token.NameScope (i.e., namespace, module, package, library)
func (*SymMap) First ¶
First returns the first symbol in the map -- only sensible when there is just one such element
func (*SymMap) KindNames ¶
KindNames returns a slice of the kind:names in this map, optionally sorted
type SymNames ¶
SymNames provides a map-list of symbol names, indexed by their token kinds. Used primarily for specifying Scopes
type SymStack ¶
type SymStack []*Symbol
SymStack is a simple stack (slice) of symbols
func (*SymStack) FindNameScoped ¶
FindNameScoped searches top-down in the stack for something with the given name in symbols that are of subcategory token.NameScope (i.e., namespace, module, package, library)
type Symbol ¶
type Symbol struct {
// name of the symbol
Name string
// additional detail and specification of the symbol -- e.g. if a function, the signature of the function
Detail string
// lexical kind of symbol, using token.Tokens list
Kind token.Tokens
// Type name for this symbol -- if it is a type, this is its corresponding type representation -- if it is a variable then this is its type
Type string
// index for ordering children within a given scope, e.g., fields in a struct / class
Index int
// full filename / URI of source
Filename string
// region in source encompassing this item -- if = RegZero then this is a temp symbol and children are not added to it
Region lex.Reg
// region that should be selected when activated, etc
SelectReg lex.Reg
// relevant scoping / parent symbols, e.g., namespace, package, module, class, function, etc..
Scopes SymNames
// children of this symbol -- this includes e.g., methods and fields of classes / structs / types, and all elements within packages, etc
Children SymMap
// types defined within the scope of this symbol
Types TypeMap
// Ast node that created this symbol -- only valid during parsing
Ast ki.Ki `json:"-" xml:"-"`
}
Symbol contains the information for everything about a given symbol that is created by parsing, and can be looked up. It corresponds to the LSP DocumentSymbol structure, and the Go Object type.
func NewSymbol ¶
NewSymbol returns a new symbol with the basic info filled in -- SelectReg defaults to Region
func OpenSymCache ¶
OpenSymCache opens cache of symbols into given symbol (typically a package, module, library), which is at given filename -- returns time stamp when cache was last saved
func (*Symbol) AddChild ¶
AddChild adds a child symbol, if this parent symbol is not temporary returns true if item name was added and NOT already on the map, and false if it was already or parent is temp. Always adds new symbol in any case. If parent symbol is of the NameType subcategory, then index of child is set to the size of this child map before adding.
func (*Symbol) AddScopesMap ¶
AddScopesMap adds a given scope element(s) from map to this Symbol. if add is true, add this symbol to those scopes if they are not temporary.
func (*Symbol) AddScopesStack ¶
AddScopesStack adds a given scope element(s) from stack to this Symbol. Adds this symbol as a child to the top of the scopes if it is not temporary -- returns true if so added.
func (*Symbol) AllocScopes ¶
func (sy *Symbol) AllocScopes()
AllocScopes allocates scopes map if nil
func (*Symbol) Clone ¶
Clone returns a clone copy of this symbol. Does NOT copy the Children or Types -- caller can decide about that.
func (*Symbol) CopyFromScope ¶
CopyFromScope copies the Children and Types from given other symbol for scopes (e.g., Go package), to merge with existing.
func (*Symbol) CopyFromSrc ¶
CopyFromSrc copies all the source-related fields from other symbol (no Type, Types, or Children). Ast is only copied if non-nil.
func (*Symbol) FindAnyChildren ¶
FindAnyChildren finds children of this symbol using either direct children if those are present, or the type name if present -- used for completion routines. Adds to kids map. scope1, scope2 are used for looking up type name. If seed is non-empty it is used as a prefix for filtering children names. Returns false if no children were found.
func (*Symbol) HasChildren ¶
HasChildren returns true if this symbol has children
func (*Symbol) IsTemp ¶
IsTemp returns true if this is temporary symbol that is used for scoping but is not otherwise permanently added to list of symbols. Indicated by Zero Region.
func (*Symbol) NonPtrTypeName ¶
NonPtrTypeName returns the name of the type without any leading * or &
type Type ¶
type Type struct {
// name of the type -- can be the name of a field or the role for a type element
Name string
// kind of type -- overall nature of the type
Kind Kinds
// documentation about this type, extracted from code
Desc string
// set to true after type has been initialized during post-parse processing
Inited bool `inactive:"-"`
// elements of this type -- ordering and meaning varies depending on the Kind of type -- for Primitive types this is the parent type, for Composite types it describes the key elements of the type: Tuple = each element's type; Array = type of elements; Struct = each field, etc (see docs for each in Kinds)
Els TypeEls
// methods defined for this type
Meths TypeMap
// for primitive types, this is the number of bytes, for composite types, it is the number of elements, which can be multi-dimensional (e.g., for functions, number of params is (including receiver param for methods) and return vals is )
Size []int
// full filename / URI of source where type is defined (may be empty for auto types)
Filename string
// region in source encompassing this type
Region lex.Reg
// relevant scoping / parent symbols, e.g., namespace, package, module, class, function, etc..
Scopes SymNames
// additional type properties, such as const, virtual, static -- these are just recorded textually and not systematized to keep things open-ended -- many of the most important properties can be inferred from the Kind property
Props ki.Props
// Ast node that corresponds to this type -- only valid during parsing
Ast ki.Ki `json:"-" xml:"-"`
}
Type contains all the information about types. Types can be builtin or composed of builtin types. Each type can have one or more elements, e.g., fields for a struct or class, multiple values for a go function, or the two types for a map (key, value), etc..
func (*Type) AddScopesStack ¶
AddScopesStack adds a given scope element(s) from stack to this Type.
func (*Type) Clone ¶
Clone returns a deep copy of this type, cloning / copying all sub-elements except the Ast, and Inited
func (*Type) CopyFromSrc ¶
CopyFromSrc copies source-level data from given other type
func (*Type) NonPtrType ¶
NonPtrType returns the non-pointer name of this type, if it is a pointer type otherwise just returns Name
func (*Type) ReturnString ¶
ReturnString() returns string of return vals of function if it is a function type
type TypeEl ¶
type TypeEl struct {
// element name -- e.g., field name for struct, or functional name for other types
Name string
// type name -- looked up on relevant lists -- includes scoping / package / namespace name as appropriate
Type string
}
TypeEl is a type element -- has a name (local to the type, e.g., field name) and a type name that can be looked up in a master list of types
type TypeEls ¶
type TypeEls []TypeEl
TypeEls are the type elements for types
func (*TypeEls) StringRange ¶
StringRange() returns a string rep of range of items
type TypeKindSize ¶
TypeKindSize is used for initialization of builtin typemaps
type TypeMap ¶
TypeMap is a map of types for quick looking up by name
func (*TypeMap) Clone ¶
Clone returns deep copy of this type map -- types are Clone() copies. returns nil if this map is empty
func (*TypeMap) CopyFrom ¶
CopyFrom copies all the types from given source map into this one. Types that have Kind != Unknown are retained over unknown ones. srcIsNewer means that the src type is newer and should be used for other data like source
func (*TypeMap) KindNames ¶
KindNames returns a slice of the kind:names in this map, optionally sorted
func (*TypeMap) PrintUnknowns ¶
func (tm *TypeMap) PrintUnknowns()
PrintUnknowns prints all the types that have a Kind = Unknown indicates an error in type resolution
Source Files