crawl

package

v0.0.0-...-98f7181 Latest Latest Go to latest Published: Oct 19, 2017 License: MIT Imports: 8 Imported by: 0

Details

Valid go.mod file

The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.
Redistributable license

Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.
Tagged version

Modules with tagged versions give importers more predictable builds.
Stable version

When a project reaches major version v1 it is considered stable.
Learn more about best practices

Repository

github.com/l-vitaly/golang-test-task

Links

Open Source Insights

Documentation ¶

Index ¶

Variables
type Crawl
- func New() Crawl
type Element
type Meta
type Middleware
- func LoggingMiddleware(logger log.Logger) Middleware
type Result

Constants ¶

This section is empty.

Variables ¶

View Source

var (
	// ErrTooManyRedirects too many redirects
	ErrTooManyRedirects = errors.New("too many redirects")
	// ErrMissingLocation missing location
	ErrMissingLocation = errors.New("missing location")
)

Functions ¶

This section is empty.

Types ¶

type Crawl ¶

type Crawl interface {
	Do(url string) (Result, error)
}

Crawl scrape page interface.

func New ¶

func New() Crawl

New create crawl instance.

type Element ¶

type Element struct {
	TagName string `json:"tag_name"`
	Count   int    `json:"count"`
}

Element statistics on the tags of the loaded page.

type Meta ¶

type Meta struct {
	Status        int    `json:"status"`
	ContentType   string `json:"content_type,omitempty"`
	ContentLength int    `json:"content_length,omitempty"`
}

Meta meta information of the requested page.

type Middleware ¶

type Middleware func(Crawl) Crawl

Middleware describes a service middleware.

func LoggingMiddleware ¶

func LoggingMiddleware(logger log.Logger) Middleware

LoggingMiddleware takes a logger as a dependency and returns a LoggingMiddleware.

type Result ¶

type Result struct {
	URL      string    `json:"url"`
	Meta     Meta      `json:"meta"`
	Elements []Element `json:"elements"`
	Err      error     `json:"-"`
}

Result parsing result.

Source Files ¶

View all Source files

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL