Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Config ¶
type Config struct {
WokerSetting *util.ConcurrencyConfig
HttpClient *http.Client
}
func NewDefaultConfig ¶
func NewDefaultConfig() *Config
Setup crawler config based on default values defined in utl
type CrawlerIManager ¶
type CrawlerIManager interface {
Process()
GetLinks() []string
GetErrors() []error
// ✋🏻 For Testing Only!
ExtractTestCall(resp *http.Response, url string)
}
Interface bound to the crawler object, exposes public functions
func NewCrawler ¶
func NewCrawler( initlUrl string, config Config, ) (CrawlerIManager, error)
Source Files
Click to show internal directories.
Click to hide internal directories.