Documentation
¶
Index ¶
Constants ¶
View Source
const ( // NewURLExchange is the exchange used when an URL is schedule for crawling NewURLExchange = "url.new" // TimeoutURLExchange is the exchange used when a crawling fail because of timeout TimeoutURLExchange = "url.timeout" // NewResourceExchange is the exchange used when a new resource has been crawled NewResourceExchange = "resource.new" // ConfigExchange is the exchange used to dispatch new configuration ConfigExchange = "config" )
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Event ¶
type Event interface {
// Exchange returns the exchange where event should be push
Exchange() string
}
Event represent a event
type NewResourceEvent ¶
type NewResourceEvent struct {
URL string `json:"url"`
Body string `json:"body"`
Headers map[string]string `json:"headers"`
Time time.Time `json:"time"`
}
NewResourceEvent represent a crawled resource
func (*NewResourceEvent) Exchange ¶
func (msg *NewResourceEvent) Exchange() string
Exchange returns the exchange where event should be push
type NewURLEvent ¶
type NewURLEvent struct {
URL string `json:"url"`
}
NewURLEvent represent an URL to crawl
func (*NewURLEvent) Exchange ¶
func (msg *NewURLEvent) Exchange() string
Exchange returns the exchange where event should be push
type Publisher ¶
type Publisher interface {
PublishEvent(event Event) error
PublishJSON(exchange string, msg RawMessage) error
Close() error
}
Publisher is something that push an event
func NewPublisher ¶
NewPublisher create a new Publisher instance
type RawMessage ¶ added in v0.8.0
RawMessage is a raw message as viewed by the messaging system
type Subscriber ¶
type Subscriber interface {
Publisher
// Read RawMessage and deserialize it into proper Event
Read(msg *RawMessage, event Event) error
// Subscribe to named exchange with unique consuming guaranty
Subscribe(exchange, queue string, handler Handler) error
// SubscribeAll subscribe to given exchange but ensure everyone on the exchange receive the messages
SubscribeAll(exchange string, handler Handler) error
}
Subscriber is something that read msg from an event queue
func NewSubscriber ¶
func NewSubscriber(amqpURI string, prefetch int) (Subscriber, error)
NewSubscriber create a new subscriber and connect it to given server
type TimeoutURLEvent ¶ added in v0.8.0
type TimeoutURLEvent struct {
URL string `json:"url"`
}
TimeoutURLEvent represent a failed crawling because of timeout
func (*TimeoutURLEvent) Exchange ¶ added in v0.8.0
func (msg *TimeoutURLEvent) Exchange() string
Exchange returns the exchange where event should be push
Source Files
Click to show internal directories.
Click to hide internal directories.