goscraper

package module
v1.0.4 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 29, 2022 License: MIT Imports: 10 Imported by: 0

README

goscraper

Package to quickly return a preview of a webpage, you can get easily its title, description & images

Usage

func main() {
	s, err := goscraper.Scrape("https://www.w3.org/", 5)
	if err != nil {
		fmt.Println(err)
		return
	}
	fmt.Printf("Icon : %s\n", s.Preview.Icon)
	fmt.Printf("Name : %s\n", s.Preview.Name)
	fmt.Printf("Title : %s\n", s.Preview.Title)
	fmt.Printf("Description : %s\n", s.Preview.Description)
	fmt.Printf("Image: %s\n", s.Preview.Images[0])
	fmt.Printf("Url : %s\n", s.Preview.Link)
}

License

Goscraper is licensed under the MIT License.

Documentation

Index

Constants

This section is empty.

Variables

View Source
var (
	EscapedFragment = "_escaped_fragment_="
)

Functions

This section is empty.

Types

type Document

type Document struct {
	Body    bytes.Buffer
	Preview DocumentPreview
}

func Scrape

func Scrape(uri string, maxRedirect int) (*Document, error)

type DocumentPreview

type DocumentPreview struct {
	Icon          string
	Name          string
	Title         string
	Description   string
	Images        []string
	Link          string
	PublishedTime string
}

type Scraper

type Scraper struct {
	Url                *url.URL
	EscapedFragmentUrl *url.URL
	MaxRedirect        int
}

func (*Scraper) Scrape

func (scraper *Scraper) Scrape() (*Document, error)

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL