Package extensions implements various helper addons for Colly
RandomUserAgent generates a random browser user agent on every request
Referrer sets valid Referrer HTTP header to requests. Warning: this extension works only if you use Request.Visit from callbacks instead of Collector.Visit.
URLLengthFilter filters out requests with URLs longer than URLLengthLimit