web

package
v0.1.6 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 5, 2023 License: BSD-3-Clause Imports: 8 Imported by: 0

Documentation

Overview

Package web interacts with web pages.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func GetText added in v0.1.3

func GetText(n *html.Node, addSpaces bool) string

GetText concatenates all text content in and under n.

func SetUserAgent

func SetUserAgent(ua string)

SetUserAgent sets a value for the "User-Agent" header to be sent in all future HTTP requests.

Types

type Page

type Page struct {
	Root *html.Node
}

Page represents a parsed HTML page.

func FetchPage

func FetchPage(ctx context.Context, url string) (*Page, error)

FetchPage fetches and parses the HTML page at the supplied URL.

func (*Page) Query

func (p *Page) Query(query string) QueryResult

Query calls QueryNode using p.Root.

func (*Page) QueryAll added in v0.1.3

func (p *Page) QueryAll(query string) QueryAllResult

QueryAll returns all nodes matched by the supplied CSS selector. Unlike Query/QueryNode, an error is not returned if no nodes are matched.

type QueryAllResult added in v0.1.3

type QueryAllResult struct {
	Nodes []*html.Node
	Err   error
}

QueryAllResult contains the result of a call to QueryAll.

func (QueryAllResult) Text added in v0.1.3

func (res QueryAllResult) Text(addSpaces bool) ([]string, error)

Text returns the contents of all text nodes under res.Nodes.

type QueryResult

type QueryResult struct {
	Node *html.Node
	Err  error
}

QueryResult contains the result of a call to Query or QueryNode.

func QueryNode added in v0.1.3

func QueryNode(root *html.Node, query string) QueryResult

QueryNode returns the first node matched by the supplied CSS selector. The returned result has a non-nil Err field if no node was matched.

func (QueryResult) Attr

func (res QueryResult) Attr(attr string) (string, error)

Attr returns the first occurrence of the named attribute. An error is returned if the attribute isn't present.

func (QueryResult) Text

func (res QueryResult) Text(addSpaces bool) (string, error)

Text recursively concatenates the contents of all child text nodes. TODO: Add more control over formatting, e.g. trimming.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL