Documentation ¶
Overview ¶
Package crawler is a crawler that runs periodically for each user and updates the database if it finds new results. It can also warn the user by email when there is new results.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
Types ¶
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
Client implements a rpc client for the crawler.
type ResultGetter ¶
ResultGetter is an interface for something that fetches results.
type ResultGetterClient ¶
ResultGetterClient interface for sending a request to get results.
type Scheduler ¶
type Scheduler struct {
// contains filtered or unexported fields
}
Scheduler handles scheduling crawler runs for every user.
func NewScheduler ¶
func NewScheduler(config *SchedulerConfig) *Scheduler
NewScheduler creates a new scuduler object.
func (*Scheduler) QueueAsync ¶
QueueAsync tells the scheduler do a run for a user async
type SchedulerConfig ¶
type SchedulerConfig struct { ResultGetters []ResultGetter UserStore user.Store CrawlerConfigStore crawlerconfig.Store UserResultsStore results.Store Sender tools.Sender }
SchedulerConfig initializes the scheduler.
type User ¶
type User struct { ID string Classes []api.Class Nip string Code string Name string Email string DoneCh chan bool }
User contains info about the user of a ResultGetter run.
type Webservice ¶
type Webservice struct {
// contains filtered or unexported fields
}
Webservice is the exported type for rpc.