Documentation ¶
Index ¶
Constants ¶
View Source
const CRAWLERS int = 2
View Source
const Domain string = "youporn.com"
View Source
const Host string = SubDomain + "." + Domain
View Source
const Root string = Scheme + "://" + Host
View Source
const Scheme string = "http"
View Source
const SubDomain string = "www"
View Source
const UA string = "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36"
Variables ¶
View Source
var YPIDRegex *regexp.Regexp
Functions ¶
func ExtractYPIDFromURL ¶
func IntegrateAndCrawl ¶
func IntegrateAndCrawl(dump string)
IntegrateAndCrawl looks for changes in the dump file and once the content has been integrated start monitoring the website every hour
Types ¶
This section is empty.
Click to show internal directories.
Click to hide internal directories.