xrobotstag

package module

v0.0.0-...-4cd7d88 Latest Latest Go to latest Published: Mar 24, 2017 License: MIT Imports: 5 Imported by: 0

Details

Valid go.mod file

The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.
Redistributable license

Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.
Tagged version

Modules with tagged versions give importers more predictable builds.
Stable version

When a project reaches major version v1 it is considered stable.
Learn more about best practices

Repository

github.com/nielsole/xrobotstag

Links

Open Source Insights

README ¶

X-Robots-Tag

This is a parser for X-Robots tags both in the header and within the html. Currently this is untested and has never run in production!

Usage

HTTP header

robotsTag := RobotsTag{}
RobotsTagFromHeaders(&resp.Header, "mybotname", &robotsTag)

Meta Tag

robotsTag := RobotsTag{}
RobotsTagFromHtmlTag(htmlNode, "mybotname", robotsTag)

Documentation ¶

Index ¶

func RobotsTagFromHtmlTag(token *html.Node, botname string, parsedTag *RobotsTag) error
type RobotsTag
- func RobotsTagFromHeaders(header *http.Header, botname string) *RobotsTag

Constants ¶

This section is empty.

Variables ¶

This section is empty.

Functions ¶

func RobotsTagFromHtmlTag ¶

func RobotsTagFromHtmlTag(token *html.Node, botname string, parsedTag *RobotsTag) error

Types ¶

type RobotsTag ¶

type RobotsTag struct {
	All      bool
	Noindex  bool
	Nofollow bool
	// None is equivalent to noindex and nofollow. Omitted
	Noarchive        bool
	Nosnippet        bool
	Noodp            bool
	Notranslate      bool
	Noimageindex     bool
	UnavailableAfter *time.Time
}

func RobotsTagFromHeaders ¶

func RobotsTagFromHeaders(header *http.Header, botname string) *RobotsTag

Source Files ¶

View all Source files

parser.go

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL