scrapper

package module

v0.0.0-...-795c19c Latest Latest Go to latest Published: Jan 19, 2021 License: AGPL-3.0 Imports: 4 Imported by: 0

Details

Valid go.mod file

The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.
Redistributable license

Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.
Tagged version

Modules with tagged versions give importers more predictable builds.
Stable version

When a project reaches major version v1 it is considered stable.
Learn more about best practices

Repository

github.com/JohnnyVM/go-scrapper

README ¶

go-scrapper

Mostly a simple scrapper easy to integrate

TODO

Documentation ¶

Index ¶

func MakeAbs(u string, host string) string
func MakeUrlsAbs(urls []string, origin string) []string
func SimpleScrapper(url string) ([]string, error)

Constants ¶

This section is empty.

Variables ¶

This section is empty.

Functions ¶

func MakeUrlsAbs ¶

func MakeUrlsAbs(urls []string, origin string) []string

MakeUrlsAbs transform relative urls to absolute

func SimpleScrapper ¶

func SimpleScrapper(url string) ([]string, error)

SimpleScrapper basic function, get url the url from pages and return Notas: Esto es un scrapper en su forma más general y válido para los casos donde se puede asumir que javascript/PHP no tiene que ver en la carga de la página, por eso se ignora el fragment en páginas dinámicas no se puede eliminar el fragmento de la url ya que se usa para modificar el contenido

Types ¶

This section is empty.

Source Files ¶

View all Source files

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL

README ¶

go-scrapper

TODO

Documentation ¶

Index ¶

Constants ¶

Variables ¶

Functions ¶

func MakeAbs ¶

func MakeUrlsAbs ¶

func SimpleScrapper ¶

Types ¶

Source Files ¶