goscraper

package module
v0.0.0-...-2cdaa8e Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 28, 2018 License: MIT Imports: 7 Imported by: 1

README

goscraper

Web Scraping library in GO

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Field

type Field struct {
	// XPath selector
	Selector string
	// Callback is used to post process selected field
	Callback func(n *xml.Node) (interface{}, error)
}

Field defines how data should be selected and process

type Scrape

type Scrape struct {
	// definitions of fields that will be scraped
	Fields map[string]Field
	// HTTP headers used in query
	Headers map[string]string
}

Scrape type defines what to scrape and headers used in HTTP request, and allows to process data

func (*Scrape) ProcessURL

func (s *Scrape) ProcessURL(url string) (map[string]interface{}, error)

ProcessURL method is used to fetch website contents and process it's data

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL