README
¶
gocurl
Simplified version of curl written in Go.
- Supports a limited subset of curl options.
- Supports some flags that curl does not. Read more about the new stuff.
- Why in the world you need another curl?
- How to install gocurl?
- How to use gocurl?
- New stuff
- All command-line arguments
Why in the world you need another curl?
Curl is certainly awesome, but sometimes I need to have better control over what's happening on the inside and to be able to debug it. It seemed easier to me to implement the necessary parts of curl in Go.
Also, I'd like to be able to extend it with what fits my specific needs. Unfortunately, curl is a bit too huge for that now.
How to install gocurl?
- Using homebrew:
brew install ameshkov/tap/gocurl - From source:
go install github.com/ameshkov/gocurl@latest - You can use a Docker image:
docker run --rm ghcr.io/ameshkov/gocurl --help - You can get a binary from the releases page.
How to use gocurl?
Use it the same way you use original curl.
gocurl https://httpbin.agrd.workers.dev/getmake aGETrequest.gocurl -d "test" -v https://httpbin.agrd.workers.dev/postmake aPOSTrequest withtestdata.gocurl -I https://httpbin.agrd.workers.dev/headmake aHEADrequest.gocurl -I --insecure https://expired.badssl.com/do not verify TLS certificate.gocurl -I --http1.1 https://httpbin.agrd.workers.dev/headforce use HTTP/1.1.gocurl -I --http2 https://httpbin.agrd.workers.dev/headforce use HTTP/2.gocurl -I --http3 https://httpbin.agrd.workers.dev/headforce use HTTP/3.gocurl -x socks5://user:pass@host:port https://httpbin.agrd.workers.dev/getuse a proxy server.gocurl -I --tlsv1.3 https://tls-v1-2.badssl.com:1012/force use TLS v1.3.gocurl -I --connect-to="httpbin.agrd.workers.dev:443:172.67.152.85:443" https://httpbin.agrd.workers.dev/headconnect to the specified IP addresses.gocurl -I --resolve="httpbin.agrd.workers.dev:443:172.67.152.85" https://httpbin.agrd.workers.dev/headresolve the hostname to the specified IP address. Note, that unlikecurl,gocurlignores port in this option.
New stuff
Also, you can use some new stuff that is not supported by curl.
gocurl --json-output https://httpbin.agrd.workers.dev/getwrite output in machine-readable format (JSON).gocurl --tls-split-hello=5:50 https://httpbin.agrd.workers.dev/getsplit TLS ClientHello in two parts and make a 50ms delay after sending the first part.gocurl -v --ech https://crypto.cloudflare.com/cdn-cgi/traceenables support for ECH (Encrypted Client Hello) for the request. More on this below.gocurl --dns-servers="tls://dns.google" https://httpbin.agrd.workers.dev/getuses custom DNS-over-TLS server to resolve hostnames. More on this below.gocurl --experiment=pq https://pq.cloudflareresearch.com/enables post-quantum cryptography support for the request. More on this below.gocurl wss://httpbin.agrd.workers.dev/wssends a WS upgrade request.gocurl -d "test message" wss://httpbin.agrd.workers.dev/wsestablishes a WS connection, sends the first message through it and reads the response.
Encrypted ClientHello
ECH or Encrypted Client Hello is a new standard that allows completely
encrypting TLS Client Hello. Currently, the RFC is in the draft stage,
but it is already supported by some big names like Cloudflare.
gocurl supports ECH and provides several options to use it.
The simple option is just add --ech flag and see what happens:
gocurl -v --ech https://crypto.cloudflare.com/cdn-cgi/trace
In this case, gocurl will try to discover the ECH configuration from DNS
records and then use them to establish the connection.
Instead of that, you may choose to supply your own configuration in the same base64-encoded format as used by the SVCB record:
# Send a type=https query and find ech record there.
% dig -t https crypto.cloudflare.com. +short
1 . alpn="http/1.1,h2" ipv4hint=162.159.137.85,162.159.138.85 ech=AEX+DQBBvgAgACARWS42g5NmDZo5pIpTWSwHzTwzdRKPdUW732QbyUeyDQAEAAEAAQASY2xvdWRmbGFyZS1lY2guY29tAAA= ipv6hint=2606:4700:7::a29f:8955,2606:4700:7::a29f:8a55
# You can now pass it to gocurl.
gocurl -v \
--echconfig="AEX+DQBBvgAgACARWS42g5NmDZo5pIpTWSwHzTwzdRKPdUW732QbyUeyDQAEAAEAAQASY2xvdWRmbGFyZS1lY2guY29tAAA=" \
https://crypto.cloudflare.com/cdn-cgi/trace
Interesting thing about ECH is that it may connect even if you use an expired configuration (see HelloRetryRequest in the RFC). It depends on both the server and the client implementation and does not work with Cloudflare at the moment.
Here's what happens under the hood:
gocurlresolvescrypto.cloudflare.comIP address and connects to it.- It sends TLS ClientHello (outer) with encrypted inner ClientHello to that IP
address. The ServerName field in the outer ClientHello is set to the one that
is encoded in the ECH configuration (in this example it will be
cloudflare-ech.com), and in the inner encrypted ClientHello it will be set tocrypto.cloudflare.com.
You may want to configure a specific "client-facing" server instead and the way
to do that is to use --connect-to. Let's send a request to cloudflare.com
and use crypto.cloudflare.com as a client-facing server for that.
gocurl -v \
--connect-to="cloudflare.com:443:crypto.cloudflare.com:443" \
--echconfig="AEX+DQBBvgAgACARWS42g5NmDZo5pIpTWSwHzTwzdRKPdUW732QbyUeyDQAEAAEAAQASY2xvdWRmbGFyZS1lY2guY29tAAA=" \
https://cloudflare.com/cdn-cgi/trace
For this command to work you may need to replace
--echconfigwith the current one discovered using DNS as was explained before.
Here's what happens now:
gocurlconnects tocrypto.cloudflare.com(client-facing relay).- Sends a TLS ClientHello with
cloudflare-ech.comin the Server Name extension. - Establishes a TLS connection with
cloudflare.comusing the inner encrypted ClientHello.
Custom DNS servers
gocurl allows using custom DNS servers to resolve hostnames when making
requests. This can be achieved by using --dns-servers command-line argument.
curl with c-ares also supports this command-line argument, but gocurl
adds encrypted DNS support on top of it, it supports all popular DNS encryption
protocols: DNS-over-QUIC, DNS-over-HTTPS, DNS-over-TLS and DNSCrypt.
You can specify multiple DNS servers, in this case gocurl will attempt to use
them one by one until it receives a response or until all of them fail:
gocurl \
--dns-servers="tls://dns.adguard-dns.com,tls://dns.google" \
https://example.org/
-
DNS-over-QUIC
gocurl --dns-servers="quic://dns.adguard-dns.com" https://example.org/ -
DNS-over-HTTPS
gocurl --dns-servers="https://dns.adguard-dns.com/dns-query" https://example.org/ -
DNS-over-TLS
gocurl --dns-servers="tls://dns.adguard-dns.com" https://example.org/ -
DNSCrypt
gocurl \ --dns-servers sdns://AQIAAAAAAAAAFDE3Ni4xMDMuMTMwLjEzMDo1NDQzINErR_JS3PLCu_iZEIbq95zkSV2LFsigxDIuUso_OQhzIjIuZG5zY3J5cHQuZGVmYXVsdC5uczEuYWRndWFyZC5jb20 \ https://example.org/
WebSocket support
gocurl provides some initial support for WebSocket protocol. It may be
extended in the future, see the corresponding Github issue.
gocurl wss://httpbin.agrd.workers.dev/wssends a WS upgrade request.gocurl -d "test message" wss://httpbin.agrd.workers.dev/wsestablishes a WS connection, sends the first message through it and reads the response.
Experimental flags
Experimental flags are added to gocurl whenever there's a feature that may be
completely changed or removed in the future. Experiments can be enabled using
the --experiment=<name[:value]> argument where name is the experiment name
and value is an optional string value (the need for it depends on the actual
experiment).
Post-quantum (PQ) cryptography has been designed to be secure against the
threat of quantum computers. You can learn more about it from Cloudflare's
blog post. gocurl supports it via the --experiment=pq flag.
Note, that it is not available for --http3 at the moment.
gocurl --experiment=pq https://pq.cloudflareresearch.com/
All command-line arguments
Usage:
gocurl [OPTIONS]
Application Options:
--url=<URL> URL the request will be made to.
Can be specified without any flags.
-X, --request=<method> HTTP method. GET by default.
-d, --data=<data> Sends the specified data to the
HTTP server using content type
application/x-www-form-urlencoded.
-H, --header= Extra header to include in the
request. Can be specified multiple
times.
-x, --proxy=[protocol://username:password@]host[:port] Use the specified proxy. The proxy
string can be specified with a
protocol:// prefix.
--connect-to=<HOST1:PORT1:HOST2:PORT2> For a request to the given
HOST1:PORT1 pair, connect to
HOST2:PORT2 instead. Can be
specified multiple times.
-I, --head Fetch the headers only.
-k, --insecure Disables TLS verification of the
connection.
--tlsv1.3 Forces gocurl to use TLS v1.3 or
newer.
--tlsv1.2 Forces gocurl to use TLS v1.2 or
newer.
--tls-max=<VERSION> (TLS) VERSION defines maximum
supported TLS version. Can be 1.2
or 1.3. The minimum acceptable
version is set by tlsv1.2 or
tlsv1.3.
--ciphers=<space-separated list of ciphers> Specifies which ciphers to use in
the connection, see
https://go.dev/src/crypto/tls/ciph-
er_suites.go for the full list of
available ciphers.
--http1.1 Forces gocurl to use HTTP v1.1.
--http2 Forces gocurl to use HTTP v2.
--http3 Forces gocurl to use HTTP v3.
--ech Enables ECH support for the
request.
--echconfig=<base64-encoded data> ECH configuration to use for this
request. Implicitly enables --ech
when specified.
-4, --ipv4 This option tells gocurl to use
IPv4 addresses only when resolving
host names.
-6, --ipv6 This option tells gocurl to use
IPv6 addresses only when resolving
host names.
--dns-servers=<DNSADDR1,DNSADDR2> DNS servers to use when making the
request. Supports encrypted DNS:
tls://, https://, quic://, sdns://
--resolve=<[+]host:port:addr[,addr]...> Provide a custom address for a
specific host. port is ignored by
gocurl. '*' can be used instead of
the host name. Can be specified
multiple times.
--tls-split-hello=<CHUNKSIZE:DELAY> An option that allows splitting
TLS ClientHello in two parts in
order to avoid common DPI systems
detecting TLS. CHUNKSIZE is the
size of the first bytes before
ClientHello is split, DELAY is
delay in milliseconds before
sending the second part.
--json-output Makes gocurl write
machine-readable output in JSON
format.
-o, --output=<file> Defines where to write the
received data. If not set, gocurl
will write everything to stdout.
--experiment=<name[:value]> Allows enabling experimental
options. See the documentation for
available options. Can be
specified multiple times.
-v, --verbose Verbose output (optional).
Help Options:
-h, --help Show this help message
Documentation
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
internal
|
|
|
client
Package client is responsible for creating HTTP client and request.
|
Package client is responsible for creating HTTP client and request. |
|
client/cfcrypto
Package cfcrypto is a package that uses Cloudflare's TLS fork to provide features missing in crypto/tls.
|
Package cfcrypto is a package that uses Cloudflare's TLS fork to provide features missing in crypto/tls. |
|
client/connectto
Package connectto implements the --connect-to command-line argument logic that allows "redirecting" connections to hosts.
|
Package connectto implements the --connect-to command-line argument logic that allows "redirecting" connections to hosts. |
|
client/dialer
Package dialer introduces base interfaces for the structs that are responsible for connecting to a host or modifying the connection logic.
|
Package dialer introduces base interfaces for the structs that are responsible for connecting to a host or modifying the connection logic. |
|
client/proxy
Package proxy implements the proxy logic (--proxy argument of the command-line tool).
|
Package proxy implements the proxy logic (--proxy argument of the command-line tool). |
|
client/splittls
Package splittls implements the --tls-split-hello logic and allows splitting TLS ClientHello.
|
Package splittls implements the --tls-split-hello logic and allows splitting TLS ClientHello. |
|
client/websocket
Package websocket is responsible for handling WebSocket connections.
|
Package websocket is responsible for handling WebSocket connections. |
|
cmd
Package cmd is the entry point of the tool.
|
Package cmd is the entry point of the tool. |
|
config
Package config is responsible for parsing and validating cmd arguments.
|
Package config is responsible for parsing and validating cmd arguments. |
|
output
Package output is responsible for writing logs and received response data.
|
Package output is responsible for writing logs and received response data. |
|
resolve
Package resolve is responsible for everything DNS-related in gocurl.
|
Package resolve is responsible for everything DNS-related in gocurl. |
|
version
Package version exports some getters for the project's version values.
|
Package version exports some getters for the project's version values. |