README
¶
Search Index
Search Term Normalization
The user input search term undergoes normalization:
- Trim space
- Lowercase
- Remove invalid UTF-8 characters
- Detect and remove quotes in the form '" (activates exact search mode)
Wildcards are not supported.
Generic Text Normalization
- Trim space
- Lowercase
- Remove invalid UTF-8 characters
Documentation
¶
Index ¶
- func CamelCaseSplit(src string) (entries []string)
- type SearchIndexRecord
- type SearchIndexStore
- func (index *SearchIndexStore) IndexHash(publicKey *btcec.PublicKey, blockchainVersion, blockNumber uint64, ...) (err error)
- func (index *SearchIndexStore) IndexNewBlock(publicKey *btcec.PublicKey, blockchainVersion, blockNumber uint64, raw []byte)
- func (index *SearchIndexStore) IndexNewBlockDecoded(publicKey *btcec.PublicKey, blockchainVersion, blockNumber uint64, ...)
- func (index *SearchIndexStore) LookupHash(selector SearchSelector, resultMap map[uuid.UUID]*SearchIndexRecord) (err error)
- func (index *SearchIndexStore) Search(term string) (results []SearchIndexRecord)
- func (index *SearchIndexStore) SearchNodeIDBasedOnHash(hash []byte) (NodeIDs [][]byte, err error)
- func (index *SearchIndexStore) UnindexBlockchain(publicKey *btcec.PublicKey)
- func (index *SearchIndexStore) UnindexHash(fileID uuid.UUID, hash []byte) (err error)
- type SearchSelector
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func CamelCaseSplit ¶
Split splits the camelcase word and returns a list of words. It also supports digits. Both lower camel case and upper camel case are supported. For more info please check: http://en.wikipedia.org/wiki/CamelCase
Examples
"" => [""] "lowercase" => ["lowercase"] "Class" => ["Class"] "MyClass" => ["My", "Class"] "MyC" => ["My", "C"] "HTML" => ["HTML"] "PDFLoader" => ["PDF", "Loader"] "AString" => ["A", "String"] "SimpleXMLParser" => ["Simple", "XML", "Parser"] "vimRPCPlugin" => ["vim", "RPC", "Plugin"] "GL11Version" => ["GL", "11", "Version"] "99Bottles" => ["99", "Bottles"] "May5" => ["May", "5"] "BFG9000" => ["BFG", "9000"] "BöseÜberraschung" => ["Böse", "Überraschung"] "Two spaces" => ["Two", " ", "spaces"] "BadUTF8\xe2\xe2\xa1" => ["BadUTF8\xe2\xe2\xa1"]
Splitting rules
- If string is not valid UTF-8, return it without splitting as -> removed in this fork. single item array.
- Assign all unicode characters into one of 4 sets: lower case letters, upper case letters, numbers, and all other characters.
- Iterate through characters of string, introducing splits between adjacent characters that belong to different sets.
- Iterate through array of split strings, and if a given string is upper case: if subsequent string is lower case: move last character of upper case string to beginning of lower case string
Types ¶
type SearchIndexRecord ¶
type SearchIndexRecord struct {
// List of selectors that found the result. Multiple keywords may find the same file.
Selectors []SearchSelector
// result data
FileID uuid.UUID
PublicKey *btcec.PublicKey
BlockchainVersion uint64
BlockNumber uint64
}
SearchIndexRecord identifies a hash to a given file
type SearchIndexStore ¶
type SearchIndexStore struct {
Database store.Store // The database storing the blockchain.
sync.RWMutex
}
This database stores hashes of keywords for file search.
func InitSearchIndexStore ¶
func InitSearchIndexStore(DatabaseDirectory string) (searchIndex *SearchIndexStore, err error)
func (*SearchIndexStore) IndexHash ¶
func (index *SearchIndexStore) IndexHash(publicKey *btcec.PublicKey, blockchainVersion, blockNumber uint64, fileID uuid.UUID, hash []byte) (err error)
IndexHash indexes a new hash
func (*SearchIndexStore) IndexNewBlock ¶
func (index *SearchIndexStore) IndexNewBlock(publicKey *btcec.PublicKey, blockchainVersion, blockNumber uint64, raw []byte)
func (*SearchIndexStore) IndexNewBlockDecoded ¶
func (index *SearchIndexStore) IndexNewBlockDecoded(publicKey *btcec.PublicKey, blockchainVersion, blockNumber uint64, recordsDecoded []interface{})
Indexes a new decoded block. Currently it only indexes file records.
func (*SearchIndexStore) LookupHash ¶
func (index *SearchIndexStore) LookupHash(selector SearchSelector, resultMap map[uuid.UUID]*SearchIndexRecord) (err error)
LookupHash returns all index records stored for the hash.
func (*SearchIndexStore) Search ¶
func (index *SearchIndexStore) Search(term string) (results []SearchIndexRecord)
func (*SearchIndexStore) SearchNodeIDBasedOnHash ¶
func (index *SearchIndexStore) SearchNodeIDBasedOnHash(hash []byte) (NodeIDs [][]byte, err error)
SearchNodeIDBasedOnHash Provides a list of NodeIDs based on the hash provided This is used to find out which nodes are hosting which files based on the hash provided
func (*SearchIndexStore) UnindexBlockchain ¶
func (index *SearchIndexStore) UnindexBlockchain(publicKey *btcec.PublicKey)
UnindexBlockchain deletes all index for a given blockchain. This is intentionally not done on a version/block level, because it could easily lead to orphans.
func (*SearchIndexStore) UnindexHash ¶
func (index *SearchIndexStore) UnindexHash(fileID uuid.UUID, hash []byte) (err error)
UnindexHash deletes a index record. If there are no more files associated with the hash, the entire hash record is deleted.
type SearchSelector ¶
type SearchSelector struct {
Word string // Normalized version of the word
Hash []byte // Hash of the word
ExactSearch bool // Indicates this is an exact search term, for example a full filename.
}
A search selector is a term that discovers a file.
Source Files