goproxy: github.com/elazarl/goproxy Index | Files | Directories

package goproxy

import "github.com/elazarl/goproxy"

Taken from $GOROOT/src/pkg/net/http/chunked needed to write https responses to client.

Package goproxy provides a customizable HTTP proxy, supporting hijacking HTTPS connection.

The intent of the proxy, is to be usable with reasonable amount of traffic yet, customizable and programable.

The proxy itself is simply an `net/http` handler.

Typical usage is

proxy := goproxy.NewProxyHttpServer()
proxy.OnRequest(..conditions..).Do(..requesthandler..)
proxy.OnRequest(..conditions..).DoFunc(..requesthandlerFunction..)
proxy.OnResponse(..conditions..).Do(..responesHandler..)
proxy.OnResponse(..conditions..).DoFunc(..responesHandlerFunction..)
http.ListenAndServe(":8080", proxy)

Adding a header to each request

proxy.OnRequest().DoFunc(func(r *http.Request,ctx *goproxy.ProxyCtx) (*http.Request, *http.Response){
	r.Header.Set("X-GoProxy","1")
	return r, nil
})

Note that the function is called before the proxy sends the request to the server

For printing the content type of all incoming responses

proxy.OnResponse().DoFunc(func(r *http.Response, ctx *goproxy.ProxyCtx)*http.Response{
	println(ctx.Req.Host,"->",r.Header.Get("Content-Type"))
	return r
})

note that we used the ProxyCtx context variable here. It contains the request and the response (Req and Resp, Resp is nil if unavailable) of this specific client interaction with the proxy.

To print the content type of all responses from a certain url, we'll add a ReqCondition to the OnResponse function:

proxy.OnResponse(goproxy.UrlIs("golang.org/pkg")).DoFunc(func(r *http.Response, ctx *goproxy.ProxyCtx)*http.Response{
	println(ctx.Req.Host,"->",r.Header.Get("Content-Type"))
	return r
})

We can write the condition ourselves, conditions can be set on request and on response

var random = ReqConditionFunc(func(r *http.Request) bool {
	return rand.Intn(1) == 0
})
var hasGoProxyHeader = RespConditionFunc(func(resp *http.Response,req *http.Request)bool {
	return resp.Header.Get("X-GoProxy") != ""
})

Caution! If you give a RespCondition to the OnRequest function, you'll get a run time panic! It doesn't make sense to read the response, if you still haven't got it!

Finally, we have convenience function to throw a quick response

proxy.OnResponse(hasGoProxyHeader).DoFunc(func(r*http.Response,ctx *goproxy.ProxyCtx)*http.Response {
	r.Body.Close()
	return goproxy.NewResponse(ctx.Req, goproxy.ContentTypeText, http.StatusForbidden, "Can't see response with X-GoProxy header!")
})

we close the body of the original repsonse, and return a new 403 response with a short message.

Example use cases:

1. https://github.com/elazarl/goproxy/tree/master/examples/goproxy-avgsize

To measure the average size of an Html served in your site. One can ask all the QA team to access the website by a proxy, and the proxy will measure the average size of all text/html responses from your host.

2. [not yet implemented]

All requests to your web servers should be directed through the proxy, when the proxy will detect html pieces sent as a response to AJAX request, it'll send a warning email.

3. https://github.com/elazarl/goproxy/blob/master/examples/goproxy-httpdump/

Generate a real traffic to your website by real users using through proxy. Record the traffic, and try it again for more real load testing.

4. https://github.com/elazarl/goproxy/tree/master/examples/goproxy-no-reddit-at-worktime

Will allow browsing to reddit.com between 8:00am and 17:00pm

5. https://github.com/elazarl/goproxy/tree/master/examples/goproxy-jquery-version

Will warn if multiple versions of jquery are used in the same domain.

6. https://github.com/elazarl/goproxy/blob/master/examples/goproxy-upside-down-ternet/

Modifies image files in an HTTP response via goproxy's image extension found in ext/.

Index

Package Files

actions.go certs.go chunked.go counterecryptor.go ctx.go dispatcher.go doc.go https.go logger.go proxy.go responses.go signer.go websocket.go

Constants

const (
    ConnectAccept = iota
    ConnectReject
    ConnectMitm
    ConnectHijack
    ConnectHTTPMitm
    ConnectProxyAuthHijack
)
const (
    ContentTypeText = "text/plain"
    ContentTypeHtml = "text/html"
)

Variables

var (
    OkConnect       = &ConnectAction{Action: ConnectAccept, TLSConfig: TLSConfigFromCA(&GoproxyCa)}
    MitmConnect     = &ConnectAction{Action: ConnectMitm, TLSConfig: TLSConfigFromCA(&GoproxyCa)}
    HTTPMitmConnect = &ConnectAction{Action: ConnectHTTPMitm, TLSConfig: TLSConfigFromCA(&GoproxyCa)}
    RejectConnect   = &ConnectAction{Action: ConnectReject, TLSConfig: TLSConfigFromCA(&GoproxyCa)}
)
var CA_CERT = []byte("" /* 2127 byte string literal not displayed */)
var CA_KEY = []byte("" /* 3244 byte string literal not displayed */)
var GoproxyCa, _ = tls.X509KeyPair(CA_CERT, CA_KEY)

func NewResponse Uses

func NewResponse(r *http.Request, contentType string, status int, body string) *http.Response

Will generate a valid http response to the given request the response will have the given contentType, and http status. Typical usage, refuse to process requests to local addresses:

proxy.OnRequest(IsLocalHost()).DoFunc(func(r *http.Request, ctx *goproxy.ProxyCtx) (*http.Request,*http.Response) {
	return nil,NewResponse(r,goproxy.ContentTypeHtml,http.StatusUnauthorized,
		`<!doctype html><html><head><title>Can't use proxy for local addresses</title></head><body/></html>`)
})

func TLSConfigFromCA Uses

func TLSConfigFromCA(ca *tls.Certificate) func(host string, ctx *ProxyCtx) (*tls.Config, error)

func TextResponse Uses

func TextResponse(r *http.Request, text string) *http.Response

Alias for NewResponse(r,ContentTypeText,http.StatusAccepted,text)

type CertStorage Uses

type CertStorage interface {
    Fetch(hostname string, gen func() (*tls.Certificate, error)) (*tls.Certificate, error)
}

type ConnectAction Uses

type ConnectAction struct {
    Action    ConnectActionLiteral
    Hijack    func(req *http.Request, client net.Conn, ctx *ProxyCtx)
    TLSConfig func(host string, ctx *ProxyCtx) (*tls.Config, error)
}

type ConnectActionLiteral Uses

type ConnectActionLiteral int

type CounterEncryptorRand Uses

type CounterEncryptorRand struct {
    // contains filtered or unexported fields
}

func NewCounterEncryptorRandFromKey Uses

func NewCounterEncryptorRandFromKey(key interface{}, seed []byte) (r CounterEncryptorRand, err error)

func (*CounterEncryptorRand) Read Uses

func (c *CounterEncryptorRand) Read(b []byte) (n int, err error)

func (*CounterEncryptorRand) Seed Uses

func (c *CounterEncryptorRand) Seed(b []byte)

type FuncHttpsHandler Uses

type FuncHttpsHandler func(host string, ctx *ProxyCtx) (*ConnectAction, string)

A wrapper that would convert a function to a HttpsHandler interface type

var AlwaysMitm FuncHttpsHandler = func(host string, ctx *ProxyCtx) (*ConnectAction, string) {
    return MitmConnect, host
}

AlwaysMitm is a HttpsHandler that always eavesdrop https connections, for example to eavesdrop all https connections to www.google.com, we can use

proxy.OnRequest(goproxy.ReqHostIs("www.google.com")).HandleConnect(goproxy.AlwaysMitm)
var AlwaysReject FuncHttpsHandler = func(host string, ctx *ProxyCtx) (*ConnectAction, string) {
    return RejectConnect, host
}

AlwaysReject is a HttpsHandler that drops any CONNECT request, for example, this code will disallow connections to hosts on any other port than 443

proxy.OnRequest(goproxy.Not(goproxy.ReqHostMatches(regexp.MustCompile(":443$"))).
	HandleConnect(goproxy.AlwaysReject)

func (FuncHttpsHandler) HandleConnect Uses

func (f FuncHttpsHandler) HandleConnect(host string, ctx *ProxyCtx) (*ConnectAction, string)

FuncHttpsHandler should implement the RespHandler interface

type FuncReqHandler Uses

type FuncReqHandler func(req *http.Request, ctx *ProxyCtx) (*http.Request, *http.Response)

A wrapper that would convert a function to a ReqHandler interface type

func (FuncReqHandler) Handle Uses

func (f FuncReqHandler) Handle(req *http.Request, ctx *ProxyCtx) (*http.Request, *http.Response)

FuncReqHandler.Handle(req,ctx) <=> FuncReqHandler(req,ctx)

type FuncRespHandler Uses

type FuncRespHandler func(resp *http.Response, ctx *ProxyCtx) *http.Response

A wrapper that would convert a function to a RespHandler interface type

func (FuncRespHandler) Handle Uses

func (f FuncRespHandler) Handle(resp *http.Response, ctx *ProxyCtx) *http.Response

FuncRespHandler.Handle(req,ctx) <=> FuncRespHandler(req,ctx)

type HttpsHandler Uses

type HttpsHandler interface {
    HandleConnect(req string, ctx *ProxyCtx) (*ConnectAction, string)
}

When a client send a CONNECT request to a host, the request is filtered through all the HttpsHandlers the proxy has, and if one returns true, the connection is sniffed using Man in the Middle attack. That is, the proxy will create a TLS connection with the client, another TLS connection with the destination the client wished to connect to, and would send back and forth all messages from the server to the client and vice versa. The request and responses sent in this Man In the Middle channel are filtered through the usual flow (request and response filtered through the ReqHandlers and RespHandlers)

type Logger Uses

type Logger interface {
    Printf(format string, v ...interface{})
}

type ProxyConds Uses

type ProxyConds struct {
    // contains filtered or unexported fields
}

ProxyConds is used to aggregate RespConditions for a ProxyHttpServer. Upon calling ProxyConds.Do, it will register a RespHandler that would handle the HTTP response from remote server if all conditions on the HTTP response are met.

func (*ProxyConds) Do Uses

func (pcond *ProxyConds) Do(h RespHandler)

ProxyConds.Do will register the RespHandler on the proxy, h.Handle(resp,ctx) will be called on every request that matches the conditions aggregated in pcond.

func (*ProxyConds) DoFunc Uses

func (pcond *ProxyConds) DoFunc(f func(resp *http.Response, ctx *ProxyCtx) *http.Response)

ProxyConds.DoFunc is equivalent to proxy.OnResponse().Do(FuncRespHandler(f))

type ProxyCtx Uses

type ProxyCtx struct {
    // Will contain the client request from the proxy
    Req *http.Request
    // Will contain the remote server's response (if available. nil if the request wasn't send yet)
    Resp         *http.Response
    RoundTripper RoundTripper
    // will contain the recent error that occurred while trying to send receive or parse traffic
    Error error
    // A handle for the user to keep data in the context, from the call of ReqHandler to the
    // call of RespHandler
    UserData interface{}
    // Will connect a request to a response
    Session int64
    // contains filtered or unexported fields
}

ProxyCtx is the Proxy context, contains useful information about every request. It is passed to every user function. Also used as a logger.

func (*ProxyCtx) Charset Uses

func (ctx *ProxyCtx) Charset() string

Will try to infer the character set of the request from the headers. Returns the empty string if we don't know which character set it used. Currently it will look for charset=<charset> in the Content-Type header of the request.

func (*ProxyCtx) Logf Uses

func (ctx *ProxyCtx) Logf(msg string, argv ...interface{})

Logf prints a message to the proxy's log. Should be used in a ProxyHttpServer's filter This message will be printed only if the Verbose field of the ProxyHttpServer is set to true

proxy.OnRequest().DoFunc(func(r *http.Request,ctx *goproxy.ProxyCtx) (*http.Request, *http.Response){
	nr := atomic.AddInt32(&counter,1)
	ctx.Printf("So far %d requests",nr)
	return r, nil
})

func (*ProxyCtx) RoundTrip Uses

func (ctx *ProxyCtx) RoundTrip(req *http.Request) (*http.Response, error)

func (*ProxyCtx) Warnf Uses

func (ctx *ProxyCtx) Warnf(msg string, argv ...interface{})

Warnf prints a message to the proxy's log. Should be used in a ProxyHttpServer's filter This message will always be printed.

proxy.OnRequest().DoFunc(func(r *http.Request,ctx *goproxy.ProxyCtx) (*http.Request, *http.Response){
	f,err := os.OpenFile(cachedContent)
	if err != nil {
		ctx.Warnf("error open file %v: %v",cachedContent,err)
		return r, nil
	}
	return r, nil
})

type ProxyHttpServer Uses

type ProxyHttpServer struct {

    // KeepDestinationHeaders indicates the proxy should retain any headers present in the http.Response before proxying
    KeepDestinationHeaders bool
    // setting Verbose to true will log information on each request sent to the proxy
    Verbose         bool
    Logger          Logger
    NonproxyHandler http.Handler

    Tr  *http.Transport
    // ConnectDial will be used to create TCP connections for CONNECT requests
    // if nil Tr.Dial will be used
    ConnectDial func(network string, addr string) (net.Conn, error)
    CertStore   CertStorage
    // contains filtered or unexported fields
}

The basic proxy type. Implements http.Handler.

func NewProxyHttpServer Uses

func NewProxyHttpServer() *ProxyHttpServer

NewProxyHttpServer creates and returns a proxy server, logging to stderr by default

func (*ProxyHttpServer) NewConnectDialToProxy Uses

func (proxy *ProxyHttpServer) NewConnectDialToProxy(https_proxy string) func(network, addr string) (net.Conn, error)

func (*ProxyHttpServer) NewConnectDialToProxyWithHandler Uses

func (proxy *ProxyHttpServer) NewConnectDialToProxyWithHandler(https_proxy string, connectReqHandler func(req *http.Request)) func(network, addr string) (net.Conn, error)

func (*ProxyHttpServer) OnRequest Uses

func (proxy *ProxyHttpServer) OnRequest(conds ...ReqCondition) *ReqProxyConds

ProxyHttpServer.OnRequest Will return a temporary ReqProxyConds struct, aggregating the given condtions. You will use the ReqProxyConds struct to register a ReqHandler, that would filter the request, only if all the given ReqCondition matched. Typical usage:

proxy.OnRequest(UrlIs("example.com/foo"),UrlMatches(regexp.MustParse(`.*\.exampl.\com\./.*`)).Do(...)

func (*ProxyHttpServer) OnResponse Uses

func (proxy *ProxyHttpServer) OnResponse(conds ...RespCondition) *ProxyConds

OnResponse is used when adding a response-filter to the HTTP proxy, usual pattern is

proxy.OnResponse(cond1,cond2).Do(handler) // handler.Handle(resp,ctx) will be used
			// if cond1.HandleResp(resp) && cond2.HandleResp(resp)

func (*ProxyHttpServer) ServeHTTP Uses

func (proxy *ProxyHttpServer) ServeHTTP(w http.ResponseWriter, r *http.Request)

Standard net/http function. Shouldn't be used directly, http.Serve will use it.

type ReqCondition Uses

type ReqCondition interface {
    RespCondition
    HandleReq(req *http.Request, ctx *ProxyCtx) bool
}

ReqCondition.HandleReq will decide whether or not to use the ReqHandler on an HTTP request before sending it to the remote server

func SrcIpIs Uses

func SrcIpIs(ips ...string) ReqCondition

SrcIpIs returns a ReqCondition testing whether the source IP of the request is one of the given strings

type ReqConditionFunc Uses

type ReqConditionFunc func(req *http.Request, ctx *ProxyCtx) bool

ReqConditionFunc.HandleReq(req,ctx) <=> ReqConditionFunc(req,ctx)

var IsLocalHost ReqConditionFunc = func(req *http.Request, ctx *ProxyCtx) bool {
    return req.URL.Host == "::1" ||
        req.URL.Host == "0:0:0:0:0:0:0:1" ||
        localHostIpv4.MatchString(req.URL.Host) ||
        req.URL.Host == "localhost"
}

IsLocalHost checks whether the destination host is explicitly local host (buggy, there can be IPv6 addresses it doesn't catch)

func DstHostIs Uses

func DstHostIs(host string) ReqConditionFunc

DstHostIs returns a ReqCondition testing wether the host in the request url is the given string

func Not Uses

func Not(r ReqCondition) ReqConditionFunc

Not returns a ReqCondition negating the given ReqCondition

func ReqHostIs Uses

func ReqHostIs(hosts ...string) ReqConditionFunc

ReqHostIs returns a ReqCondition, testing whether the host to which the request is directed to equal to one of the given strings

func ReqHostMatches Uses

func ReqHostMatches(regexps ...*regexp.Regexp) ReqConditionFunc

ReqHostMatches returns a ReqCondition, testing whether the host to which the request was directed to matches any of the given regular expressions.

func UrlHasPrefix Uses

func UrlHasPrefix(prefix string) ReqConditionFunc

UrlHasPrefix returns a ReqCondition checking wether the destination URL the proxy client has requested has the given prefix, with or without the host. For example UrlHasPrefix("host/x") will match requests of the form 'GET host/x', and will match requests to url 'http://host/x'

func UrlIs Uses

func UrlIs(urls ...string) ReqConditionFunc

UrlIs returns a ReqCondition, testing whether or not the request URL is one of the given strings with or without the host prefix. UrlIs("google.com/","foo") will match requests 'GET /' to 'google.com', requests `'GET google.com/' to any host, and requests of the form 'GET foo'.

func UrlMatches Uses

func UrlMatches(re *regexp.Regexp) ReqConditionFunc

UrlMatches returns a ReqCondition testing whether the destination URL of the request matches the given regexp, with or without prefix

func (ReqConditionFunc) HandleReq Uses

func (c ReqConditionFunc) HandleReq(req *http.Request, ctx *ProxyCtx) bool

func (ReqConditionFunc) HandleResp Uses

func (c ReqConditionFunc) HandleResp(resp *http.Response, ctx *ProxyCtx) bool

ReqConditionFunc cannot test responses. It only satisfies RespCondition interface so that to be usable as RespCondition.

type ReqHandler Uses

type ReqHandler interface {
    Handle(req *http.Request, ctx *ProxyCtx) (*http.Request, *http.Response)
}

ReqHandler will "tamper" with the request coming to the proxy server If Handle returns req,nil the proxy will send the returned request to the destination server. If it returns nil,resp the proxy will skip sending any requests, and will simply return the response `resp` to the client.

type ReqProxyConds Uses

type ReqProxyConds struct {
    // contains filtered or unexported fields
}

ReqProxyConds aggregate ReqConditions for a ProxyHttpServer. Upon calling Do, it will register a ReqHandler that would handle the request if all conditions on the HTTP request are met.

func (*ReqProxyConds) Do Uses

func (pcond *ReqProxyConds) Do(h ReqHandler)

ReqProxyConds.Do will register the ReqHandler on the proxy, the ReqHandler will handle the HTTP request if all the conditions aggregated in the ReqProxyConds are met. Typical usage:

proxy.OnRequest().Do(handler) // will call handler.Handle(req,ctx) on every request to the proxy
proxy.OnRequest(cond1,cond2).Do(handler)
// given request to the proxy, will test if cond1.HandleReq(req,ctx) && cond2.HandleReq(req,ctx) are true
// if they are, will call handler.Handle(req,ctx)

func (*ReqProxyConds) DoFunc Uses

func (pcond *ReqProxyConds) DoFunc(f func(req *http.Request, ctx *ProxyCtx) (*http.Request, *http.Response))

DoFunc is equivalent to proxy.OnRequest().Do(FuncReqHandler(f))

func (*ReqProxyConds) HandleConnect Uses

func (pcond *ReqProxyConds) HandleConnect(h HttpsHandler)

HandleConnect is used when proxy receives an HTTP CONNECT request, it'll then use the HttpsHandler to determine what should it do with this request. The handler returns a ConnectAction struct, the Action field in the ConnectAction struct returned will determine what to do with this request. ConnectAccept will simply accept the request forwarding all bytes from the client to the remote host, ConnectReject will close the connection with the client, and ConnectMitm, will assume the underlying connection is an HTTPS connection, and will use Man in the Middle attack to eavesdrop the connection. All regular handler will be active on this eavesdropped connection. The ConnectAction struct contains possible tlsConfig that will be used for eavesdropping. If nil, the proxy will use the default tls configuration.

proxy.OnRequest().HandleConnect(goproxy.AlwaysReject) // rejects all CONNECT requests

func (*ReqProxyConds) HandleConnectFunc Uses

func (pcond *ReqProxyConds) HandleConnectFunc(f func(host string, ctx *ProxyCtx) (*ConnectAction, string))

HandleConnectFunc is equivalent to HandleConnect, for example, accepting CONNECT request if they contain a password in header

io.WriteString(h,password)
passHash := h.Sum(nil)
proxy.OnRequest().HandleConnectFunc(func(host string, ctx *ProxyCtx) (*ConnectAction, string) {
	c := sha1.New()
	io.WriteString(c,ctx.Req.Header.Get("X-GoProxy-Auth"))
	if c.Sum(nil) == passHash {
		return OkConnect, host
	}
	return RejectConnect, host
})

func (*ReqProxyConds) HijackConnect Uses

func (pcond *ReqProxyConds) HijackConnect(f func(req *http.Request, client net.Conn, ctx *ProxyCtx))

type RespCondition Uses

type RespCondition interface {
    HandleResp(resp *http.Response, ctx *ProxyCtx) bool
}

RespCondition.HandleReq will decide whether or not to use the RespHandler on an HTTP response before sending it to the proxy client. Note that resp might be nil, in case there was an error sending the request.

func ContentTypeIs Uses

func ContentTypeIs(typ string, types ...string) RespCondition

ContentTypeIs returns a RespCondition testing whether the HTTP response has Content-Type header equal to one of the given strings.

type RespConditionFunc Uses

type RespConditionFunc func(resp *http.Response, ctx *ProxyCtx) bool

RespConditionFunc.HandleResp(resp,ctx) <=> RespConditionFunc(resp,ctx)

func (RespConditionFunc) HandleResp Uses

func (c RespConditionFunc) HandleResp(resp *http.Response, ctx *ProxyCtx) bool

type RespHandler Uses

type RespHandler interface {
    Handle(resp *http.Response, ctx *ProxyCtx) *http.Response
}

after the proxy have sent the request to the destination server, it will "filter" the response through the RespHandlers it has. The proxy server will send to the client the response returned by the RespHandler. In case of error, resp will be nil, and ctx.RoundTrip.Error will contain the error

func HandleBytes Uses

func HandleBytes(f func(b []byte, ctx *ProxyCtx) []byte) RespHandler

HandleBytes will return a RespHandler that read the entire body of the request to a byte array in memory, would run the user supplied f function on the byte arra, and will replace the body of the original response with the resulting byte array.

type RoundTripper Uses

type RoundTripper interface {
    RoundTrip(req *http.Request, ctx *ProxyCtx) (*http.Response, error)
}

type RoundTripperFunc Uses

type RoundTripperFunc func(req *http.Request, ctx *ProxyCtx) (*http.Response, error)

func (RoundTripperFunc) RoundTrip Uses

func (f RoundTripperFunc) RoundTrip(req *http.Request, ctx *ProxyCtx) (*http.Response, error)

Directories

PathSynopsis
examples/cascadeproxy
examples/goproxy-basic
examples/goproxy-customca
examples/goproxy-eavesdropper
examples/goproxy-httpdump
examples/goproxy-jquery-version
examples/goproxy-no-reddit-at-worktime
examples/goproxy-sokeepalive
examples/goproxy-sslstrip
examples/goproxy-stats
examples/goproxy-transparent
examples/goproxy-upside-down-ternet
examples/goproxy-websockets
examples/goproxy-yui-minifyThis example would minify standalone Javascript files (identified by their content type) using the command line utility YUI compressor http://yui.github.io/yuicompressor/ Example usage:
ext/auth
ext/htmlextension to goproxy that will allow you to easily filter web browser related content.
ext/image
regretable
transport

Package goproxy imports 32 packages (graph) and is imported by 668 packages. Updated 2019-10-11. Refresh now. Tools for package owners.