Documentation ¶
Overview ¶
Package crawler is grouped around the Crawler component, crawling and indexing content from an AnnotatedResource.
Index ¶
Constants ¶
This section is empty.
Variables ¶
View Source
var ( // ErrDirectoryTooLarge is returned by Ls() when a directory is larger `Config.MaxDirSize`. ErrDirectoryTooLarge = t.WrappedError{Err: t.ErrInvalidResource, Msg: "directory too large"} )
Functions ¶
This section is empty.
Types ¶
type Config ¶
type Config struct { DirEntryBufferSize uint // Size of buffer for processing directory entry channels. MinUpdateAge time.Duration // The minimum age for items to be updated. StatTimeout time.Duration // Timeout for Stat() calls. DirEntryTimeout time.Duration // Timeout *between* directory entries. MaxDirSize uint // Maximum number of directory entries }
Config contains configuration for a Crawler.
func DefaultConfig ¶
func DefaultConfig() *Config
DefaultConfig generates a default configuration for a Crawler.
type Crawler ¶
type Crawler struct { *instr.Instrumentation // contains filtered or unexported fields }
Crawler allows crawling of resources.
Click to show internal directories.
Click to hide internal directories.