crawlerflagger

package
v0.0.0-...-99d6a0d Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 25, 2019 License: BSD-3-Clause Imports: 1 Imported by: 0

Documentation

Overview

Package to identify crawlers based on their User Agent.

BenchmarkExactMatch/case0-8             10000000               182 ns/op
BenchmarkExactMatch/case101-8           10000000               128 ns/op
BenchmarkExactMatch/case200-8           10000000               137 ns/op
BenchmarkExactMatch/case300-8           10000000               124 ns/op
BenchmarkExactMatch/case400-8           20000000               113 ns/op
BenchmarkExactMatch/miss-8             200000000                 8.03 ns/op
BenchmarkRegExpMatch/case0-8             5000000               292 ns/op
BenchmarkRegExpMatch/case101-8            200000              7335 ns/op
BenchmarkRegExpMatch/case200-8            100000             12866 ns/op
BenchmarkRegExpMatch/case300-8            100000             21898 ns/op
BenchmarkRegExpMatch/case400-8             50000             26515 ns/op
BenchmarkRegExpMatch/miss-8                50000             23963 ns/op

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Crawler

type Crawler struct {
	Pattern      string
	Instances    []string
	URL          *string
	Description  *string
	AdditionDate *string
	DependsOn    []string
}

func ExactMatch

func ExactMatch(ua string) *Crawler

ExactMatch allows you to identify if and which kind of crawler a User Agent belongs to The data are taken from the https://github.com/monperrus/crawler-user-agents/ project

func RegexpMatch

func RegexpMatch(ua string) *Crawler

RegexpMatch allows you to identify if and which kind of crawler a User Agent belongs to The data are taken from the https://github.com/monperrus/crawler-user-agents/ project

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL