scraper

package module
v0.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 30, 2023 License: BSD-2-Clause Imports: 10 Imported by: 0

README

Scraper

Install dependencies

Running with chrome
Docker
docker-compose up
Mac

There is a known issue that doesn't let the docker run on an arm based Mac. As such, I recommend to run the .docker/entrypoint.sh without virtualization instead of using the docker.

For example:

CHROME_CLI="/Applications/Google Chrome.app/Contents/MacOS/Google Chrome" CHROME_PORT=9222 /bin/bash .docker/chrome_entrypoint.sh

Usage

TODO: setup an example of usage

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Attr

func Attr(n *html.Node, name string) string

func AttrContains

func AttrContains(nodes []*html.Node, attrName string, value string) []*html.Node

func FetchURL

func FetchURL(url string, options map[string]interface{}) ([]byte, error)

func HtmlToTree

func HtmlToTree(raw []byte) (*html.Node, error)

func Query

func Query(n *html.Node, query string) []*html.Node

func TreeToHTML

func TreeToHTML(n *html.Node) []byte

Types

This section is empty.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL