Documentation ¶
Index ¶
Constants ¶
View Source
const ( // NewURLExchange is the exchange used when an URL is schedule for crawling NewURLExchange = "url.new" // TimeoutURLExchange is the exchange used when a crawling fail because of timeout TimeoutURLExchange = "url.timeout" // NewResourceExchange is the exchange used when a new resource has been crawled NewResourceExchange = "resource.new" // ConfigExchange is the exchange used to dispatch new configuration ConfigExchange = "config" )
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Event ¶
type Event interface { // Exchange returns the exchange where event should be push Exchange() string }
Event represent a event
type NewResourceEvent ¶
type NewResourceEvent struct { URL string `json:"url"` Body string `json:"body"` Headers map[string]string `json:"headers"` Time time.Time `json:"time"` }
NewResourceEvent represent a crawled resource
func (*NewResourceEvent) Exchange ¶
func (msg *NewResourceEvent) Exchange() string
Exchange returns the exchange where event should be push
type NewURLEvent ¶
type NewURLEvent struct {
URL string `json:"url"`
}
NewURLEvent represent an URL to crawl
func (*NewURLEvent) Exchange ¶
func (msg *NewURLEvent) Exchange() string
Exchange returns the exchange where event should be push
type Publisher ¶
type Publisher interface { PublishEvent(event Event) error PublishJSON(exchange string, msg RawMessage) error Close() error }
Publisher is something that push an event
func NewPublisher ¶
NewPublisher create a new Publisher instance
type RawMessage ¶
RawMessage is a raw message as viewed by the messaging system
type Subscriber ¶
type Subscriber interface { Publisher // Read RawMessage and deserialize it into proper Event Read(msg *RawMessage, event Event) error // Subscribe to named exchange with unique consuming guaranty Subscribe(exchange, queue string, handler Handler) error // SubscribeAll subscribe to given exchange but ensure everyone on the exchange receive the messages SubscribeAll(exchange string, handler Handler) error }
Subscriber is something that read msg from an event queue
func NewSubscriber ¶
func NewSubscriber(amqpURI string, prefetch int) (Subscriber, error)
NewSubscriber create a new subscriber and connect it to given server
type TimeoutURLEvent ¶
type TimeoutURLEvent struct {
URL string `json:"url"`
}
TimeoutURLEvent represent a failed crawling because of timeout
func (*TimeoutURLEvent) Exchange ¶
func (msg *TimeoutURLEvent) Exchange() string
Exchange returns the exchange where event should be push
Click to show internal directories.
Click to hide internal directories.