checker

package
v0.0.0-...-6e90328 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Feb 15, 2023 License: MIT Imports: 7 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source
var (
	// HostsOfBatchHTTPSScorer contains 10 host with https scheme which are often crawled.
	HostsOfBatchHTTPSScorer = []string{

		"https://www.liepin.com",
		"https://www.lagou.com",
		"https://www.zhipin.com",

		"https://nj.lianjia.com",
		"https://nanjing.anjuke.com",
		"https://nj.58.com",

		"https://www.ctrip.com",
		"https://www.qunar.com",

		"https://music.163.com",
		"https://maoyan.com",
	}
)

Functions

This section is empty.

Types

type BatchHTTPSScorer

type BatchHTTPSScorer struct {
	// contains filtered or unexported fields
}

BatchHTTPSScorer tryRequest visiting a batch of HTTPS websites and grade the proxy by response time.

func (*BatchHTTPSScorer) Score

func (s *BatchHTTPSScorer) Score(pxy *proxy.Proxy) int8

Score tryRequest to use proxy visit each host, and modifies the corresponding proxy score based on the return value .

type Scorer

type Scorer interface {
	// Score calculates the proxy's score.
	Score(pxy *proxy.Proxy) int8
}

Scorer is the interface used to score a proxy.

func NewBatchHTTPSScorer

func NewBatchHTTPSScorer(hosts []string) Scorer

NewBatchHTTPSScorer returns a new scorer, the hosts can't be empty or bigger than maximum score. The timeout is calculated by the length of hosts. If response time smaller than timeout/2, score increments by (timeout/2 - RT) else score decrements by (RT - timeout/2). So, if all host tryRequest failed, the proxy score will be reduced to 0

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL