chapter5

func Crawl ¶

func Crawl(site, dir string) (int, error)

Crawl a webpage and downloads pages in that domain and saves results in destination dir.

Exercise 5.13: Modify crawl to make local copies of the pages it finds, creating directories as necessary. Don’t make copies of pages that come from a different domain. For example, if the original page comes from golang.org, save all files from there, but exclude ones from vimeo.com. Added concurrency as part of chapter 8

func E51FindLinks ¶

func E51FindLinks(href []string, n *html.Node) []string

E51FindLinks Exercise 5.1 make links using traversal non-loop recursive

func Expand ¶

func Expand(s string, f func(string) string) string

Expand exercise 5.9 replaces $var with f(var) and returns output

func InitCli ¶

func InitCli()

InitCli for command: the-gpl parse -site=http://...

  eg: the-gpl parse -type=links     -site=https://www.yahoo.com
		   the-gpl parse -type=outline -site=https://www.yahoo.com
		   the-gpl parse -type=images -site=https://www.yahoo.com
		   the-gpl parse -type=scripts -site=https://www.yahoo.com
		   the-gpl parse -type=scripts -site=https://www.yahoo.com
		   the-gpl parse -type=css -site=https://www.yahoo.com
		   the-gpl parse -type=pretty -site=https://www.yahoo.com
		   the-gpl parse -type=crawl -site=https://www.yahoo.com -dir=dest-dir

func Join ¶

func Join(sep string, words ...string) string

Join words by adding separator among them.

Exercise 5.15: Write a variadic version of strings.Join

func MaxInt ¶

func MaxInt(numbers ...int) int

MaxInt returns maximum of integers, if no int specified return MaxInt

Exercise 5.15: Write variadic functions max and min, analogous to sum. What should these
functions do when called with no arguments?

func MaxIntOf ¶

func MaxIntOf(n1 int, numbers ...int) int

MaxIntOf returns maximum of integers with at least 1 argument

Exercise 5.15: Write variants that require at least one argument.

func MinInt ¶

func MinInt(numbers ...int) int

MinInt finds minimum of numbers in a series, return minimum int number if nothing to compare

func MinIntOf ¶

func MinIntOf(n1 int, numbers ...int) int

MinIntOf finds minimum of integers with at least one int as argument

func ParseCss ¶

func ParseCss(url string) ([]string, error)

ParseCss returns all full CSS unique href links of a website pointed to by url.

func ParseImages ¶

func ParseImages(url string) ([]string, error)

ParseImages returns all full images unique href links of a website pointed to by url.

func ParseLinks ¶

func ParseLinks(url string) ([]string, error)

ParseLinks returns all full path unique href links of a website pointed to by url.

func ParseOutline ¶

func ParseOutline(url string) ([]string, error)

ParseOutline returns outline of a website pointed to by url.

The outline is array of strings

func ParseOutlineCount ¶

func ParseOutlineCount(url string) (map[string]int, error)

ParseOutlineCount returns count of tags website pointed to by url.

The outline is array of strings

func ParseScripts ¶

func ParseScripts(url string) ([]string, error)

ParseScripts returns all full scripts unique href links of a website pointed to by url.

func ParseText ¶

func ParseText(url string) ([]string, error)

ParseText returns text nodes data except script and style elements

func PrettyHTML ¶

func PrettyHTML(url string) ([]string, error)

PrettyHTML prints HTML of an url in pretty fashion

func TopologicalSort ¶

func TopologicalSort(m map[string][]string) []string

TopologicalSort sorts a map into a topological sorted string of dependencies

func TopologicalSortMap ¶

func TopologicalSortMap(m map[string]map[string]bool) []string

TopologicalSortMap topological-sorts using map.

Exercise 5.10: Rewrite TopologicalSort to use maps instead of slices and eliminate the initial sort.
Verify that the results, though nondeterministic, are valid topological orderings.