README
¶
webanalyze
This is a fork of Wappalyzer in Go. This tool is designed to be performant and allows to test huge lists of hosts.
👾 Added piping capabilities where the output is a one line csv for each host
Installation and usage
Precompiled releases
Precompiled releases can be downloaded directly here.
Build
If you want to build for yourself:
$ go install -v github.com/TheeEclipse/webanalyze/cmd/webanalyze@latest
$ webanalyze -update # loads new technologies.json file from wappalyzer project
Or rather just wget the technologies file
$ webanalyze -h
Usage of webanalyze:
-apps string
app definition file. (default "technologies.json")
-crawl int
links to follow from the root page (default 0)
-host string
single host to test
-hosts string
filename with hosts, one host per line.
-output string
output format (stdout|csv|json) (default "stdout")
-search
searches all urls with same base domain (i.e. example.com and sub.example.com) (default true)
-silent
avoid printing header (default false)
-update
update apps file
-worker int
number of worker (default 4)
The -update flags downloads a current version of technologies.json from the wappalyzer repository to the current folder.
See cmd/webanalyze/main.go for an example on how to use this as a library.
Example
$ root@Normal-Use-Instance:~# webanalyze -host robinverton.de -crawl 1 -silent
http://robinverton.de (0.5s): React, (JavaScript frameworks) HSTS, (Security) Netlify, (PaaS, CDN)
$ root@Normal-Use-Instance:~# webanalyze -host robinverton.de -crawl 1 -silent | anew 2.txt
http://robinverton.de (0.5s): HSTS, (Security) Netlify, (PaaS, CDN) React, (JavaScript frameworks)
root@Normal-Use-Instance:~#
$ root@Normal-Use-Instance:~# webanalyze -host robinverton.de -crawl 1
:: webanalyze : v0.3.9
:: workers : 4
:: technologies : technologies.json
:: crawl count : 1
:: search subdomains : true
:: follow redirects : false
http://robinverton.de (0.5s): React, (JavaScript frameworks) HSTS, (Security) Netlify, (PaaS, CDN)
root@Normal-Use-Instance:~#
Documentation
¶
Index ¶
Constants ¶
const VERSION = "0.3.9"
const WappazlyerRoot = "https://raw.githubusercontent.com/AliasIO/wappalyzer/master/src"
Variables ¶
This section is empty.
Functions ¶
func DownloadFile ¶
DownloadFile pulls the latest technologies.json file from the Wappalyzer github
Types ¶
type App ¶
type App struct {
Cats StringArray `json:"cats"`
CatNames []string `json:"category_names"`
Cookies map[string]string `json:"cookies"`
Headers map[string]string `json:"headers"`
Meta map[string]StringArray `json:"meta"`
HTML StringArray `json:"html"`
Script StringArray `json:"scripts"`
URL StringArray `json:"url"`
Website string `json:"website"`
Implies StringArray `json:"implies"`
HTMLRegex []AppRegexp `json:"-"`
ScriptRegex []AppRegexp `json:"-"`
URLRegex []AppRegexp `json:"-"`
HeaderRegex []AppRegexp `json:"-"`
MetaRegex []AppRegexp `json:"-"`
CookieRegex []AppRegexp `json:"-"`
}
App type encapsulates all the data about an App from technologies.json
type AppsDefinition ¶
type AppsDefinition struct {
Apps map[string]App `json:"technologies"`
Cats map[string]Category `json:"categories"`
}
AppsDefinition type encapsulates the json encoding of the whole technologies.json file
type Category ¶
type Category struct {
Name string `json:"name"`
}
Category names defined by wappalyzer
type Job ¶
type Job struct {
URL string
Body []byte
Headers http.Header //map[string][]string
Cookies []*http.Cookie
Crawl int
SearchSubdomain bool
// contains filtered or unexported fields
}
Job may consist only of a URL, in which case webanalyse will proceed to download from that URL, or it may consist of the Body and Headers of a request to a URL and the URL itself, in which case these fields will be trusted and used for analysis without further network traffic. If a Job is constructed using the OfflineJob constructor then a flag will be set to prevent downloading regardless of the contents (or absence) of the Body or Headers fields.
func NewOfflineJob ¶
NewOfflineJob constructs a job out of the constituents of a webanalyzer analysis; a URL, a body, and response headers. This constructor also sets a flag to explicitly prevent fetching from the URL even if the body and headers are nil or empty. Use this for...offline jobs.
func NewOnlineJob ¶
func NewOnlineJob(url, body string, headers map[string][]string, crawlCount int, searchSubdomain bool, redirect bool) *Job
NewOnlineJob constructs a job that may either have a URL only, or a URL, Body and Headers. If it contains at least a URL and Body, then webanalyzer will not re-download the data, but if a Body is absent then downloading will be attempted.
type Match ¶
type Match struct {
App `json:"app"`
AppName string `json:"app_name"`
Matches [][]string `json:"matches"`
Version string `json:"version"`
}
Match type encapsulates the App information from a match on a document
type Result ¶
type Result struct {
Host string `json:"host"`
Matches []Match `json:"matches"`
Duration time.Duration `json:"duration"`
Error error `json:"error"`
}
Result type encapsulates the result information from a given host
type StringArray ¶
type StringArray []string
StringArray type is a wrapper for []string for use in unmarshalling the technologies.json
func (*StringArray) UnmarshalJSON ¶
func (t *StringArray) UnmarshalJSON(data []byte) error
UnmarshalJSON is a custom unmarshaler for handling bogus technologies.json types from wappalyzer
type WebAnalyzer ¶
type WebAnalyzer struct {
// contains filtered or unexported fields
}
WebAnalyzer types holds an analyzation job
func NewWebAnalyzer ¶
NewWebAnalyzer initializes webanalyzer by passing a reader of the app definition and an schedulerChan, which allows the scanner to add scan jobs on its own
func (*WebAnalyzer) CategoryById ¶
func (wa *WebAnalyzer) CategoryById(cid string) string
Source Files
Directories