htmlquery

package module

v1.0.0 Latest Latest Go to latest Published: Feb 4, 2019 License: MIT Imports: 7 Imported by: 0

Details

Valid go.mod file

The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.
Redistributable license

Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.
Tagged version

Modules with tagged versions give importers more predictable builds.
Stable version

When a project reaches major version v1 it is considered stable.
Learn more about best practices

Repository

github.com/Kangaroux/htmlquery

README ¶

htmlquery

Overview

htmlquery is an XPath query package for HTML, lets you extract data or evaluate from HTML documents by an XPath expression.

Changelogs

2019-02-04

#7 Removed deprecated FindEach() and FindEachWithBreak() methods.

2018-12-28

Avoid adding duplicate elements to list for Find() method. #6

Installation

$ go get github.com/antchfx/htmlquery

Getting Started

Load HTML document from URL.

doc, err := htmlquery.LoadURL("http://example.com/")

Load HTML document from string.

s := `<html>....</html>`
doc, err := htmlquery.Parse(strings.NewReader(s))

Find all A elements.

list := htmlquery.Find(doc, "//a")

Find all A elements that have `href` attribute.

list := range htmlquery.Find(doc, "//a[@href]")

Find all A elements and only get `href` attribute self.

list := range htmlquery.Find(doc, "//a/@href")

Find the third A element.

a := htmlquery.FindOne(doc, "//a[3]")

Evaluate the number of all IMG element.

expr, _ := xpath.Compile("count(//img)")
v := expr.Evaluate(htmlquery.CreateXPathNavigator(doc)).(float64)
fmt.Printf("total count is %f", v)

Quick Tutorial

func main() {
	doc, err := htmlquery.LoadURL("https://www.bing.com/search?q=golang")
	if err != nil {
		panic(err)
	}
	// Find all news item.
	for i, n := range htmlquery.Find(doc, "//ol/li") {
		a := htmlquery.FindOne(n, "//a")
		fmt.Printf("%d %s(%s)\n", i, htmlquery.InnerText(a), htmlquery.SelectAttr(a, "href"))
	}
}

List of supported XPath query packages

Name	Description
htmlquery	XPath query package for the HTML document
xmlquery	XPath query package for the XML document
jsonquery	XPath query package for the JSON document

Questions

Please let me know if you have any questions.

Documentation ¶

Overview ¶

Package htmlquery provides extract data from HTML documents using XPath expression.

Index ¶

func Find(top *html.Node, expr string) []*html.Node
func FindOne(top *html.Node, expr string) *html.Node
func InnerText(n *html.Node) string
func LoadURL(url string) (*html.Node, error)
func OutputHTML(n *html.Node, self bool) string
func Parse(r io.Reader) (*html.Node, error)
func SelectAttr(n *html.Node, name string) (val string)
type NodeNavigator
- func CreateXPathNavigator(top *html.Node) *NodeNavigator

Constants ¶

This section is empty.

Variables ¶

This section is empty.

Functions ¶

func Find ¶

func Find(top *html.Node, expr string) []*html.Node

Find searches the html.Node that matches by the specified XPath expr.

func FindOne ¶

func FindOne(top *html.Node, expr string) *html.Node

FindOne searches the html.Node that matches by the specified XPath expr, and returns first element of matched html.Node.

func InnerText ¶

func InnerText(n *html.Node) string

InnerText returns the text between the start and end tags of the object.

func LoadURL ¶

func LoadURL(url string) (*html.Node, error)

LoadURL loads the HTML document from the specified URL.

func OutputHTML ¶

func OutputHTML(n *html.Node, self bool) string

OutputHTML returns the text including tags name.

func Parse ¶

func Parse(r io.Reader) (*html.Node, error)

Parse returns the parse tree for the HTML from the given Reader.

func SelectAttr ¶

func SelectAttr(n *html.Node, name string) (val string)

SelectAttr returns the attribute value with the specified name.

Types ¶

type NodeNavigator ¶

type NodeNavigator struct {
	// contains filtered or unexported fields
}

func CreateXPathNavigator ¶

func CreateXPathNavigator(top *html.Node) *NodeNavigator

CreateXPathNavigator creates a new xpath.NodeNavigator for the specified html.Node.

func (*NodeNavigator) Copy ¶

func (h *NodeNavigator) Copy() xpath.NodeNavigator

func (*NodeNavigator) Current ¶

func (h *NodeNavigator) Current() *html.Node

func (*NodeNavigator) LocalName ¶

func (h *NodeNavigator) LocalName() string

func (*NodeNavigator) MoveTo ¶

func (h *NodeNavigator) MoveTo(other xpath.NodeNavigator) bool

func (*NodeNavigator) MoveToChild ¶

func (h *NodeNavigator) MoveToChild() bool

func (*NodeNavigator) MoveToFirst ¶

func (h *NodeNavigator) MoveToFirst() bool

func (*NodeNavigator) MoveToNext ¶

func (h *NodeNavigator) MoveToNext() bool

func (*NodeNavigator) MoveToNextAttribute ¶

func (h *NodeNavigator) MoveToNextAttribute() bool

func (*NodeNavigator) MoveToParent ¶

func (h *NodeNavigator) MoveToParent() bool

func (*NodeNavigator) MoveToPrevious ¶

func (h *NodeNavigator) MoveToPrevious() bool

func (*NodeNavigator) MoveToRoot ¶

func (h *NodeNavigator) MoveToRoot()

func (*NodeNavigator) NodeType ¶

func (h *NodeNavigator) NodeType() xpath.NodeType

func (*NodeNavigator) Prefix ¶

func (*NodeNavigator) Prefix() string

func (*NodeNavigator) String ¶

func (h *NodeNavigator) String() string

func (*NodeNavigator) Value ¶

func (h *NodeNavigator) Value() string

Source Files ¶

View all Source files

query.go

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL

README ¶

htmlquery

Overview

Changelogs

Installation

Getting Started

Load HTML document from URL.

Load HTML document from string.

Find all A elements.

Find all A elements that have href attribute.

Find all A elements and only get href attribute self.

Find the third A element.

Evaluate the number of all IMG element.

Quick Tutorial

List of supported XPath query packages

Questions

Documentation ¶

Overview ¶

Index ¶

Constants ¶

Variables ¶

Functions ¶

func Find ¶

func FindOne ¶

func InnerText ¶

func LoadURL ¶

func OutputHTML ¶

func Parse ¶

func SelectAttr ¶

Types ¶

type NodeNavigator ¶

func CreateXPathNavigator ¶

func (*NodeNavigator) Copy ¶

func (*NodeNavigator) Current ¶

func (*NodeNavigator) LocalName ¶

func (*NodeNavigator) MoveTo ¶

func (*NodeNavigator) MoveToChild ¶

func (*NodeNavigator) MoveToFirst ¶

func (*NodeNavigator) MoveToNext ¶

func (*NodeNavigator) MoveToNextAttribute ¶

func (*NodeNavigator) MoveToParent ¶

func (*NodeNavigator) MoveToPrevious ¶

func (*NodeNavigator) MoveToRoot ¶

func (*NodeNavigator) NodeType ¶

func (*NodeNavigator) Prefix ¶

func (*NodeNavigator) String ¶

func (*NodeNavigator) Value ¶

Source Files ¶

Find all A elements that have `href` attribute.

Find all A elements and only get `href` attribute self.