goscraper

package module

v0.0.0-...-fb723c5 Latest Latest Go to latest Published: Dec 1, 2018 License: MIT Imports: 9 Imported by: 1

Details

Valid go.mod file

The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.
Redistributable license

Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.
Tagged version

Modules with tagged versions give importers more predictable builds.
Stable version

When a project reaches major version v1 it is considered stable.
Learn more about best practices

Repository

github.com/farmx/goscraper

Links

Open Source Insights

README ¶

goscraper

Golang package to quickly return the webpage title

Usage

func main() {
	s, err := goscraper.Scrape("https://www.w3.org/", 5)
	if err != nil {
		fmt.Println(err)
		return
	}
	fmt.Printf("Title : %s\n", s.Preview.Title)
}

output:

Title : World Wide Web Consortium (W3C)

License

Goscraper is licensed under the MIT License.

Documentation ¶

Index ¶

Variables
type Document
- func Scrape(uri string, maxRedirect int) (*Document, error)
type DocumentPreview
type Scraper
- func (scraper *Scraper) Scrape() (*Document, error)

Constants ¶

This section is empty.

Variables ¶

View Source

var (
	EscapedFragment string = "_escaped_fragment_="
)

Functions ¶

This section is empty.

Types ¶

type Document ¶

type Document struct {
	Body    bytes.Buffer
	Preview DocumentPreview
}

func Scrape ¶

func Scrape(uri string, maxRedirect int) (*Document, error)

type DocumentPreview ¶

type DocumentPreview struct {
	Title string
	Link  string
}

type Scraper ¶

type Scraper struct {
	Url                *url.URL
	EscapedFragmentUrl *url.URL
	MaxRedirect        int
}

func (*Scraper) Scrape ¶

func (scraper *Scraper) Scrape() (*Document, error)

Source Files ¶

View all Source files

goscraper.go

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL