extraction

func CheckHourlyMax ¶

func CheckHourlyMax(article *m.Article, currentTime time.Time, currentVisits int)

Given an article, a currentTime, and a currentVisits variable, check the last value in the article.Visits array (lastInterval). Does an in-memory replace of the values if necessary

If currentTime.Hour() == lastInterval.Hour()

if currentVisits > lastInterval.Max
		lastInterval.Max = currentVisits

If currentTime.Hour() > lastInterval.Hour()

append onto article.Visits array using this hour

If currentTime.Hour() < lastInterval.Hour()

ignore

func ExtractArticleURLsFromSearchResults ¶

func ExtractArticleURLsFromSearchResults(term string, page int) ([]string, error)

func ExtractDataFromDocument ¶

func ExtractDataFromDocument(doc *gq.Document, url string, includeTitle bool, fromJson bool) *m.ExtractedBody

func ExtractDataFromHTMLAtURL ¶

func ExtractDataFromHTMLAtURL(url string, includeTitle bool) *m.ExtractedBody

func ExtractDataFromHTMLString ¶

func ExtractDataFromHTMLString(html string, url string, includeTitle bool) *m.ExtractedBody

func IsSameDay ¶

func IsSameDay(t1 time.Time, t2 time.Time) bool

Are t1 and t2 on the same day?

func ParseAuthor ¶

func ParseAuthor(author string) []string

func ParseAuthors ¶

func ParseAuthors(authors []string) []string

func RoundHourDown ¶

func RoundHourDown(t time.Time) time.Time