utils

package
v2.6.4+incompatible Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 23, 2018 License: GPL-3.0 Imports: 4 Imported by: 1

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func BaseURL

func BaseURL(rawUrl string) (*url.URL, error)

Returns an URL using only the host and scheme of the other

func URLsToStrings

func URLsToStrings(urls ...*url.URL) []string

Converts an slice of URLs to an array of strings

Types

type GoqueryClient

type GoqueryClient struct {
	http.Client
}

Goquery client used to scrape data from web pages. This struct is necessary because the default timeout into http.DefaultClient is 0. Because of that, we use a custom instance of http.Client.

func NewDefaultGoqueryClient

func NewDefaultGoqueryClient() *GoqueryClient

func (*GoqueryClient) NewDocument

func (gq *GoqueryClient) NewDocument(url string) (*goquery.Document, error)

Generates a new Goquery document from the given url using a custom http.Client, if possible.

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL