crawler

package
v0.0.0-...-439617f Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 2, 2023 License: Apache-2.0 Imports: 8 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func CountElements

func CountElements(countEl map[string]int, n *html.Node) map[string]int

CountElements count elements

func CountWordsAndImages

func CountWordsAndImages(n *html.Node) (words, images int)

CountWordsAndImages count words and images

func EndElement

func EndElement(n *html.Node)

EndElement print formated end html element

func Fetch

func Fetch(url string) (filename string, n int64, err error)

Fetch save HTTP response to the file

func ForEachNode

func ForEachNode(doc *html.Node, pre, post func(doc *html.Node))

ForEachNode call function pre(x) and post(x) for every x in the tree with root n. Both functions are optional

func GetURL

func GetURL(url string) (doc *html.Node, err error)

GetURL try parse html body from URI string

func StartElement

func StartElement(n *html.Node)

StartElement print formated start html element

func TextFromHTML

func TextFromHTML(res []string, n *html.Node) []string

TextFromHTML get text from html

func Visit

func Visit(links []string, n *html.Node) []string

Visit appends to links each link found in n and returns the result.

Types

This section is empty.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL