Directories ¶
Path | Synopsis |
---|---|
Package commands contains commands as they would be called by the end user.
|
Package commands contains commands as they would be called by the end user. |
Package components groups together the main operational components of the crawler.
|
Package components groups together the main operational components of the crawler. |
crawler
Package crawler is grouped around the Crawler component, crawling and indexing content from an AnnotatedResource.
|
Package crawler is grouped around the Crawler component, crawling and indexing content from an AnnotatedResource. |
extractor
Package extractor is grouped around the Extractor component, extracting metadata from an AnnotatedResource.
|
Package extractor is grouped around the Extractor component, extracting metadata from an AnnotatedResource. |
index
Package index is grouped around the Index component, representing an index which stores and retrieves document properties.
|
Package index is grouped around the Index component, representing an index which stores and retrieves document properties. |
protocol
Package protocol is grouped around the Protocol component, the interface with one or multiple protocols.
|
Package protocol is grouped around the Protocol component, the interface with one or multiple protocols. |
queue
Package queue provides queueing semantics.
|
Package queue provides queueing semantics. |
sniffer
Package sniffer contains sniffer components which can be wired into a libp2p dht node by proxying the datastore.
|
Package sniffer contains sniffer components which can be wired into a libp2p dht node by proxying the datastore. |
Package config provides central and canonical representation, reading, parsing and validation of configuration for components.
|
Package config provides central and canonical representation, reading, parsing and validation of configuration for components. |
Package instr contains common instrumentation tooling.
|
Package instr contains common instrumentation tooling. |
Package types consists of common datatypes used by other components for internal representation, not for exporting or indexing.
|
Package types consists of common datatypes used by other components for internal representation, not for exporting or indexing. |
Package utils contains common utility functions used in other components.
|
Package utils contains common utility functions used in other components. |
Click to show internal directories.
Click to hide internal directories.