httpclient

package
v0.0.0-...-279ef49 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 27, 2024 License: GPL-3.0 Imports: 26 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	MIMEJSON              = "application/json"
	MIMEHTML              = "text/html"
	MIMEXML               = "application/xml"
	MIMETextXML           = "text/xml"
	MIMEPlain             = "text/plain"
	MIMEPOSTForm          = "application/x-www-form-urlencoded"
	MIMEMultipartPOSTForm = "multipart/form-data"
	MIMEXPROTOBUF         = "application/x-protobuf"
	MIMEXMSGPACK          = "application/x-msgpack"
	MIMEMSGPACK           = "application/msgpack"
	MIMEYAML              = "application/x-yaml"
)

Content-Type https://www.iana.org/assignments/media-types/media-types.xhtml

Variables

This section is empty.

Functions

func AddCookie 

func AddCookie(cookie *http.Cookie)

func AddCookies 

func AddCookies(cookies []*http.Cookie)

func Cookies 

func Cookies(url string) ([]*http.Cookie, error)

get cookies from client cookie jar

func DelCookie 

func DelCookie(cookie *http.Cookie)

func GeneratorWithConfig 

func GeneratorWithConfig(config *Config) func(*url.URL, proxy.Dialer) (proxy.Dialer, error)

GeneratorWithConfig is like NewWithConfig, but is suitable for passing to proxy.RegisterDialerType while maintaining configuration options.

This is to enable registration of an http(s) proxy with options, e.g.: 

proxy.RegisterDialerType("https", connectproxy.GeneratorWithConfig(
        &connectproxy.Config{DialTimeout: 5 * time.Minute},
))

func Get 

func Get(url string) (statusCode int, resp []byte, err error)

func NewConnectproxy 

func NewConnectproxy(u *url.URL, forward proxy.Dialer) (proxy.Dialer, error)

NewConnectproxy returns a proxy.Dialer given a URL specification and an underlying proxy.Dialer for it to make network requests. NewConnectproxy may be passed to proxy.RegisterDialerType for the schemes "http" and "https". The convenience function RegisterDialerFromURL simplifies this.

func NewWithConfig 

func NewWithConfig(u *url.URL, forward proxy.Dialer, config *Config) (proxy.Dialer, error)

NewWithConfig is like NewConnectproxy, but allows control over various options.

func Post 

func Post(url, contentType string, body interface{}) (statusCode int, resp []byte, err error)

func RegisterDialerFromURL 

func RegisterDialerFromURL(registerHTTP, registerHTTPS bool)

RegisterDialerFromURL is a convenience wrapper around proxy.RegisterDialerType, which registers the given URL as a for the schemes "http" and/or "https", as controlled by registerHTTP and registerHTTPS. If both registerHTTP and registerHTTPS are false, RegisterDialerFromURL is a no-op.

Types

type Client 

type Client struct {
	// contains filtered or unexported fields
}

func NewClient 

func NewClient(opts ...ClientOption) *Client

func NewClientHttp2 

func NewClientHttp2(opts ...ClientOption) *Client

func NewDefaultClient 

func NewDefaultClient(opts ...ClientOption) *Client

func NewWithClient 

func NewWithClient(hc *http.Client, opts ...ClientOption) *Client

func SetCheckProxy 

func SetCheckProxy(checkProxy func(response *Response) bool) *Client

func SetCheckRedirect 

func SetCheckRedirect(checkRedirect func(req *http.Request, via []*http.Request) error) *Client

func SetIdleConnTimeout 

func SetIdleConnTimeout(timeout time.Duration) *Client

func SetJar 

func SetJar(jar http.CookieJar) *Client

func SetMaxConnsPerHost 

func SetMaxConnsPerHost(n int) *Client

func SetMaxIdleConns 

func SetMaxIdleConns(n int) *Client

func SetMaxIdleConnsPerHost 

func SetMaxIdleConnsPerHost(n int) *Client

func SetMetadata 

func SetMetadata(mds ...map[string]string) *Client

func SetProxy 

func SetProxy(scheme, ip, port, username, password string) *Client

SetProxy http 127.0.0.1 8888 http 127.0.0.1 8888 username password socks5 127.0.0.1 8888 username password

func SetProxySelector 

func SetProxySelector(selector ProxySelector) *Client

func SetProxyURL 

func SetProxyURL(proxyURL string) *Client

SetProxyURL Proxy：http://127.0.0.1:8888 Proxy：http://username:password@127.0.0.1:8888

func SetTimeout 

func SetTimeout(timeout time.Duration) *Client

func SetTransport 

func SetTransport(rt http.RoundTripper) *Client

func (*Client) AddCookie 

func (c *Client) AddCookie(cookie *http.Cookie) *Client

func (*Client) AddCookies 

func (c *Client) AddCookies(cookies []*http.Cookie) *Client

func (*Client) Cookies 

func (c *Client) Cookies(url string) ([]*http.Cookie, error)

func (*Client) DelCookie 

func (c *Client) DelCookie(cookie *http.Cookie) *Client

func (*Client) Get 

func (c *Client) Get(url string) (statusCode int, resp []byte, err error)

func (*Client) GetCookieJar 

func (c *Client) GetCookieJar() http.CookieJar

func (*Client) GetMetaDataByKey 

func (c *Client) GetMetaDataByKey(key string) string

GetMetaDataByKey 通过key获取metadata

func (*Client) KeepParamAddOrder 

func (c *Client) KeepParamAddOrder(keepParamAddOrder bool) *Client

func (*Client) NewRequest 

func (c *Client) NewRequest(method, url string) *Request

func (*Client) NewRequestWithContext 

func (c *Client) NewRequestWithContext(ctx context.Context, method, url string) *Request

func (*Client) Post 

func (c *Client) Post(url, contentType string, body interface{}) (statusCode int, resp []byte, err error)

func (*Client) SetCookie 

func (c *Client) SetCookie(cookie *http.Cookie) *Client

SetCookie 避免重复的cookie

func (*Client) SetCookieJar 

func (c *Client) SetCookieJar(cookieJar http.CookieJar)

func (*Client) SetCookies 

func (c *Client) SetCookies(cookies []*http.Cookie) *Client

func (*Client) SetJsonEscapeHTML 

func (c *Client) SetJsonEscapeHTML(jsonEscapeHTML bool) *Client

func (*Client) SetJsonIndent 

func (c *Client) SetJsonIndent(prefix, indent string) *Client

func (*Client) SetProxy 

func (c *Client) SetProxy(ps ProxySelector) *Client

SetProxy 设置代理时保证用底层共用一个client

func (*Client) SetTimeout 

func (c *Client) SetTimeout(timeout time.Duration) *Client

func (*Client) SetTransport 

func (c *Client) SetTransport(transport http.RoundTripper) *Client

type ClientOption 

type ClientOption func(*Client)

func WithCheckProxy 

func WithCheckProxy(checkProxy func(response *Response) bool) ClientOption

func WithCheckRedirect 

func WithCheckRedirect(checkRedirect func(req *http.Request, via []*http.Request) error) ClientOption

func WithIdleConnTimeout 

func WithIdleConnTimeout(timeout time.Duration) ClientOption

func WithJar 

func WithJar(jar http.CookieJar) ClientOption

func WithMaxConnsPerHost 

func WithMaxConnsPerHost(n int) ClientOption

func WithMaxIdleConns 

func WithMaxIdleConns(n int) ClientOption

func WithMaxIdleConnsPerHost 

func WithMaxIdleConnsPerHost(n int) ClientOption

func WithMetadata 

func WithMetadata(mds ...map[string]string) ClientOption

func WithProxySelector 

func WithProxySelector(selector ProxySelector) ClientOption

func WithTimeout 

func WithTimeout(timeout time.Duration) ClientOption

func WithTransport 

func WithTransport(rt http.RoundTripper) ClientOption

type Config 

type Config struct {
	// ServerName is the name to use in the TLS connection to (not through)
	// the proxy server if different from the host in the URL.
	// Specifically, this is used in the ServerName field of the
	// *tls.Config used in connections to TLS-speaking proxy servers.
	ServerName string

	// For proxy servers supporting TLS connections (to, not through),
	// skip TLS certificate validation.
	InsecureSkipVerify bool // Passed directly to tls.Dial

	// Header sets the headers in the initial HTTP CONNECT request.  See
	// the documentation for http.Request for more information.
	Header http.Header

	// DialTimeout is an optional timeout for connections through (not to)
	// the proxy server.
	DialTimeout time.Duration
}

Config allows various parameters to be configured. It is used with NewWithConfig. The config passed to NewWithConfig may be changed between requests. If it is, the changes will affect all current and future invocations of the returned proxy.Dialer's Dial method.

type Debug 

type Debug struct {
	TlsConnCount     map[string]int64 // 此host建立过多少个tls链接
	TlsHandShakeTime map[string]int64 // 与host建立tls链接的tls握手时延, 毫秒
	// contains filtered or unexported fields
}

func (*Debug) GetTlsConnCount 

func (d *Debug) GetTlsConnCount(host string) int64

func (*Debug) GetTlsHandShakeTime 

func (d *Debug) GetTlsHandShakeTime(host string) int64

func (*Debug) IncrTlsConnCount 

func (d *Debug) IncrTlsConnCount(host string)

func (*Debug) SetTlsHandShakeTime 

func (d *Debug) SetTlsHandShakeTime(host string, t int64)

type DecryptFunc 

type DecryptFunc = func(string) (string, error)

type ErrorConnectionTimeout 

type ErrorConnectionTimeout error

ErrorConnectionTimeout is returned if the connection through the proxy server was not able to be made before the configured timeout expired.

type ErrorUnsupportedScheme 

type ErrorUnsupportedScheme error

ErrorUnsupportedScheme is returned if a scheme other than "http" or "https" is used.

type HostnameProxySelector 

type HostnameProxySelector struct {
	// contains filtered or unexported fields
}

HostnameProxySelector 特定 Hostname 使用特定代理

func NewHostNameProxySelector 

func NewHostNameProxySelector(proxyInvalid func(ctx context.Context)) *HostnameProxySelector

func (*HostnameProxySelector) ProxyFunc 

func (s *HostnameProxySelector) ProxyFunc(req *http.Request) (*url.URL, error)

ProxyFunc 实现ProxySelector接口

func (*HostnameProxySelector) ProxyInvalid 

func (s *HostnameProxySelector) ProxyInvalid(ctx context.Context)

func (*HostnameProxySelector) SetProxyURL 

func (s *HostnameProxySelector) SetProxyURL(proxy *url.URL, urls ...string) error

type Ja3Slice 

type Ja3Slice []uint16

func (*Ja3Slice) ToString 

func (ja3Slice *Ja3Slice) ToString() string

type ProxySelector 

type ProxySelector interface {
	ProxyFunc(req *http.Request) (*url.URL, error)
	ProxyInvalid(ctx context.Context)
}

type ProxyURLSelector 

type ProxyURLSelector struct {
	// contains filtered or unexported fields
}

func NewProxyURLSelector 

func NewProxyURLSelector(proxy *url.URL, proxyInvalid func(ctx context.Context)) (*ProxyURLSelector, error)

func (*ProxyURLSelector) ProxyFunc 

func (s *ProxyURLSelector) ProxyFunc(req *http.Request) (*url.URL, error)

func (*ProxyURLSelector) ProxyInvalid 

func (s *ProxyURLSelector) ProxyInvalid(ctx context.Context)

type Request 

type Request struct {
	// contains filtered or unexported fields
}

func NewRequest 

func NewRequest(method, url string) *Request

func NewRequestWithContext 

func NewRequestWithContext(ctx context.Context, method, url string) *Request

func (*Request) AddCookie 

func (r *Request) AddCookie(cookie *http.Cookie) *Request

AddCookie 添加cookie client CookieJar 默认不启用，如需使用 cookie ,先设置client 启用 CookieJar 只允许通过 client 设置，避免并发创建

func (*Request) AddCookies 

func (r *Request) AddCookies(cookies []*http.Cookie) *Request

AddCookies 添加cookie client CookieJar 默认不启用，如需使用 cookie ,先设置client 启用 CookieJar 只允许通过 client 设置，避免并发创建

func (*Request) AddHead 

func (r *Request) AddHead(key, value string) *Request

AddHead 添加head 自动规范化

func (*Request) AddHeads 

func (r *Request) AddHeads(headers http.Header) *Request

AddHeads 添加head 自动规范化

func (*Request) Byte 

func (r *Request) Byte() (statusCode int, resp []byte, err error)

func (*Request) Context 

func (r *Request) Context() context.Context

Context 获取请求的Context

func (*Request) CustomRequest 

func (r *Request) CustomRequest(f func(request *Request)) *Request

CustomRequest 自定义Request

func (*Request) Do 

func (r *Request) Do() (*Response, error)

Do TODO: retry

func (*Request) GetUrl 

func (r *Request) GetUrl() string

func (*Request) KeepQueryParamOrder 

func (r *Request) KeepQueryParamOrder(keepParamAddOrder bool) *Request

KeepQueryParamOrder 保持查询参数添加顺序

func (*Request) SetBody 

func (r *Request) SetBody(contentType string, body interface{}) *Request

SetBody 设置body

func (*Request) SetCheckProxy 

func (r *Request) SetCheckProxy(checkProxy func(response *Response) bool) *Request

SetCheckProxy 设置代理检查函数

func (*Request) SetCheckRedirect 

func (r *Request) SetCheckRedirect(checkRedirect func(req *http.Request, via []*http.Request) error) *Request

SetCheckRedirect 设置该请求的重定向函数

func (*Request) SetClient 

func (r *Request) SetClient(cli *Client) *Request

func (*Request) SetFormData 

func (r *Request) SetFormData(key, value string) *Request

SetFormData 添加请求参数

func (*Request) SetHead 

func (r *Request) SetHead(key, value string) *Request

SetHead 设置head 自动规范化

func (*Request) SetHeads 

func (r *Request) SetHeads(headers http.Header) *Request

SetHeads 设置head 自动规范化

func (*Request) SetJsonEscapeHTML 

func (r *Request) SetJsonEscapeHTML() *Request

SetJsonEscapeHTML 设置该请求json编码时是否转义HTML字符

func (*Request) SetJsonIndent 

func (r *Request) SetJsonIndent(prefix, indent string) *Request

SetJsonIndent 设置该请求json编码时的缩进格式 都为空不进行缩进

func (*Request) SetQueryParam 

func (r *Request) SetQueryParam(key, value string) *Request

SetQueryParam 添加URL path参数

func (*Request) SetQueryParams 

func (r *Request) SetQueryParams(params map[string]string) *Request

SetQueryParam 添加URL path参数

func (*Request) SetRawFormData 

func (r *Request) SetRawFormData(formData map[string]string) *Request

SetRawFormData 添加请求参数

func (*Request) SetRawHead 

func (r *Request) SetRawHead(key, value string) *Request

SetRawHead 设置head 不自动规范化

func (*Request) SetRawHeads 

func (r *Request) SetRawHeads(heads map[string]string) *Request

SetRawHeads 设置head 不自动规范化

func (*Request) String 

func (r *Request) String() (statusCode int, resp string, err error)

func (*Request) StringWithDecrypt 

func (r *Request) StringWithDecrypt(decrypt DecryptFunc) (statusCode int, resp string, err error)

func (*Request) Unmarshal 

func (r *Request) Unmarshal(val interface{}) (err error)

func (*Request) UnmarshalWithDecrypt 

func (r *Request) UnmarshalWithDecrypt(decrypt DecryptFunc, val interface{}) (err error)

func (*Request) WithContext 

func (r *Request) WithContext(ctx context.Context) *Request

WithContext 设置请求的Context

type Response 

type Response struct {
	// contains filtered or unexported fields
}

Response 请求结果

func (*Response) Body 

func (r *Response) Body() (body []byte, err error)

Body 返回请求结果的body 超时时间包括body的读取 请求结束后要尽快读取

func (*Response) Cookies 

func (r *Response) Cookies() []*http.Cookie

Cookies 返回请求结果的Cookie

func (*Response) FromJSON 

func (r *Response) FromJSON(v interface{}) error

FromJSON 解析请求结果到 v

func (*Response) Headers 

func (r *Response) Headers() http.Header

Headers 返回请求结果的heads

func (*Response) Location 

func (r *Response) Location() (*stdurl.URL, error)

Location 返回重定向地址

func (*Response) Response 

func (r *Response) Response() *http.Response

func (*Response) StatusCode 

func (r *Response) StatusCode() int

StatusCode 返回状态码

func (*Response) ToFile 

func (r *Response) ToFile(filename string) error

ToFile 保存请求结果到文件

type TlsConnOption 

type TlsConnOption func(*Transport)

func WithHandShakeTimeout 

func WithHandShakeTimeout(handShakeTimeout time.Duration) TlsConnOption

func WithTcpDialTimeout 

func WithTcpDialTimeout(dialTimeout time.Duration) TlsConnOption

func WithTlsConnOptClientHelloID 

func WithTlsConnOptClientHelloID(id tls.ClientHelloID) TlsConnOption

func WithTlsConnOptClientHelloSpec 

func WithTlsConnOptClientHelloSpec(spec *tls.ClientHelloSpec) TlsConnOption

自定义tls指纹： HelloCustom + ClientHelloSpec

func WithTlsConnOptProxyAddr 

func WithTlsConnOptProxyAddr(proxyAddr string) TlsConnOption

type Transport 

type Transport struct {
	ProxyAddr string // "socks5://tst:ge5@127.0.0.1:8889" or "http://127.0.0.1:8888"

	ClientHelloSpec *tls.ClientHelloSpec // 仅当clientHelloID为HelloCustom时有用
	ClientHelloID   tls.ClientHelloID

	*Debug // 用于调试
	// contains filtered or unexported fields
}

包装http1和http2的transport 1. 通过底层的transport, 支持链接池管理 2. tls握手后自动选择http1.1或http2 3. 支持多线程访问

func NewTransport 

func NewTransport(opts ...TlsConnOption) *Transport

func (*Transport) JA3 

func (transpoort *Transport) JA3() (string, string)

获取ja3指纹

func (*Transport) RoundTrip 

func (t *Transport) RoundTrip(req *http.Request) (*http.Response, error)

func (*Transport) SetH1Transport 

func (t *Transport) SetH1Transport(h1T *http.Transport)

设置http1.1的transport模板，否则使用默认

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.