go-recon

External recon toolkit

Coded with 💙 by D3Ext

Introduction • Tools • Installation • Usage • Contributing

README in Spanish

Introduction

This project started as various Golang scripts to automatically perform tedious processes and to perform external recon, between another bunch of things. Over time I polished the tools and finally decided to take it seriously, in this way I would also learn to use Golang channels and concurrency, so tools are fast and configurable.

This toolkit provides tools for different purposes while performing external recon. Most functions are also available and can be used through the official package API for your own tools. Feel free to contribute by reporting issues or discussing ideas.

See Wiki for further info

Tools

• gr-subdomains: Enumerate subdomains of a domain using 8 different providers (passively)
• gr-urls: Find URLs (endpoints) of a domain from different sources (Wayback, AlienVault)
• gr-probe: Probe active subdomains and URLs (http and https) fastly, custom concurrency and more
• gr-403: Try to bypass pages that return 403 status code (multiple techniques)
• gr-openredirects: Fuzz for potential open redirects on given URLs using a payload/list of payloads
• gr-dns: Retrieve DNS info from domains
• gr-aws: Enumerate S3 buckets for given domain using permutations, verify bucket lists and much more
• gr-waf: Identify which WAF is running on a domain
• gr-filter: Remove useless URLs from list, apply filters, create custom filter patterns
• gr-replace: Replace given keyword or parameter value with provided value from URLs of a list
• gr-secrets: Search for API keys and leaked secrets in HTML and JS pages
• gr-crawl: Fastly crawl urls for gathering URLs and JS endpoints, with custom depth and other options
• gr-whois: Perform WHOIS query against domains

Features

• Speed and concurrency
• Easy usage and configurable via CLI arguments
• Tools can be combined between them
• Multiple output formats (STDOUT, TXT, JSON, CSV)
• Input as URL, domains or STDIN
• Direct access to official package API
• Tested on Linux

Installation

Compile and install from source code via Github:

git clone https://github.com/D3Ext/go-recon
cd go-recon
make
sudo make install

The binaries will be compiled and installed on PATH, so you just will have to execute it from CLI

$ gr-subdomains

Extra

To install a set of custom filters/patterns and a Bash autocompletion script, you could execute the following command:

make extra

Then if you press TAB twice when using gr-subdomains or gr-filter, you will see the available providers and filters.

Usage

All tools have similar usage and CLI parameters

Example help panel

  __ _  ___        _ __ ___  ___ ___  _ __
 / _` |/ _ \ _____| '__/ _ \/ __/ _ \| '_ \
| (_| | (_) |_____| | |  __/ (_| (_) | | | |
 \__, |\___/      |_|  \___|\___\___/|_| |_|
  __/ |     by D3Ext
 |___/      v0.1

Usage of gr-subdomains:
  INPUT:
    -d, -domain string      domain to find its subdomains (i.e. example.com)
    -l, -list string        file containing a list of domains to find their subdomains (one domain per line)

  OUTPUT:
    -o, -output string          file to write subdomains into
    -oj, -output-json string    file to write subdomains into (JSON format)

  PROVIDERS:
    -all                      use all available providers to discover subdomains (slower than default)
    -p, -providers string[]   providers to use for subdomain discovery (separated by comma)
    -lp, -list-providers      list available providers

  CONFIG:
    -proxy string         proxy to send requests through (i.e. http://127.0.0.1:8080)
    -t, -timeout int      milliseconds to wait before each request timeout (default=5000)
    -c, -color            print colors on output
    -q, -quiet            print neither banner nor logging, only print output

  DEBUG:
    -version      show go-recon version
    -h, -help     print help panel

Examples:
    gr-subdomains -d example.com -o subdomains.txt -c
    gr-subdomains -l domains.txt -p crt,hackertarget -t 8000
    cat domain.txt | gr-subdomains -all
    cat domain.txt | gr-subdomains -p anubis -oj subdomains.json -c

See here for ideas and real examples about how to use go-recon for external reconnaisance

Demo

API

Install official go-recon Golang package like this:

go get github.com/D3Ext/go-recon/pkg/go-recon

If you want to use go-recon in your own Golang code see here

TODO

• Parameter to control used providers
• CSV output
• More tools and features
• Dockerfile
• Changelog
• HTML results reports
• More optimization
• Compare results with other tools such as subfinder, gau, httprobe...

References

Inspired and motivated by some awesome tools like this:

https://github.com/lc/gau
https://github.com/lc/subjs
https://github.com/tomnomnom/httprobe
https://github.com/projectdiscovery/subfinder
https://github.com/tomnomnom/waybackurls
https://github.com/projectdiscovery/nuclei
https://github.com/tomnomnom/qsreplace
https://github.com/hakluke/hakrawler
https://github.com/gocolly/colly/
https://github.com/d3mondev/puredns
https://github.com/blacklanternsecurity/bbot
https://github.com/s0md3v/uro
https://github.com/nytr0gen/deduplicate
https://github.com/smaranchand/bucky
https://github.com/projectdiscovery/interactsh
https://github.com/swisskyrepo/PayloadsAllTheThings
https://github.com/1ndianl33t/Gf-Patterns
https://github.com/r3curs1v3-pr0xy/sub404
https://github.com/devanshbatham/ParamSpider
https://github.com/m4ll0k/SecretFinder
https://github.com/MrEmpy/mantra
https://github.com/iamj0ker/bypass-403
https://github.com/edoardottt/favirecon
https://github.com/hueristiq/xs3scann3r

Contributing

See CONTRIBUTING.md

Changelog

See CHANGELOG.md

License

This project is under MIT license

Copyright © 2023, D3Ext

Support