README
¶
webanalyze
This is a port of Wappalyzer in Go. This tool is designed to be performant and allows to test huge lists of hosts.
Installation and usage
$ go get -u github.com/rverton/webanalyze/...
$ webanalyze -update # loads new apps.json file from wappalyzer project
$ webanalyze -h
Usage of webanalyze:
-apps string
app definition file. (default "apps.json")
-host string
single host to test
-hosts string
filename with hosts, one host per line.
-output string
output format (stdout|csv|json) (default "stdout")
-update
update apps file
-worker int
number of worker (default 4)
The -update flags downloads a current version of apps.json from the wappalyzer repository to the current folder.
Display
Run cmd/webanalyze/index.html (on sth. like SimpleHTTPServer) to display results in a searchable dashboard.
Development / Usage as a lib
See cmd/webanalyze/main.go for an example.
Example
$ webanalyze -host https://stackshare.io
2019/01/05 23:41:45 Scanning with 4 workers.
2019/01/05 23:41:46 [+] https://stackshare.io (1.025640074s):
2019/01/05 23:41:46 - jQuery, (JavaScript Libraries)
2019/01/05 23:41:46 - Cowboy, (Web Frameworks, Web Servers)
2019/01/05 23:41:46 - Erlang, (Programming Languages)
2019/01/05 23:41:46 - Ruby on Rails, (Web Frameworks)
2019/01/05 23:41:46 - Ruby, (Programming Languages)
$ webanalyze -host https://stackshare.io -output csv
2019/01/05 23:45:04 Scanning with 4 workers.
Host,Category,App,Version
https://stackshare.io,"Web Frameworks,Web Servers",Cowboy,
https://stackshare.io,Programming Languages,Erlang,
https://stackshare.io,Web Frameworks,Ruby on Rails,
https://stackshare.io,Programming Languages,Ruby,
https://stackshare.io,JavaScript Libraries,jQuery,
Documentation
¶
Index ¶
Constants ¶
const WappalyzerURL = "https://raw.githubusercontent.com/AliasIO/Wappalyzer/master/src/apps.json"
WappalyzerURL is the link to the latest apps.json file in the Wappalyzer repo
Variables ¶
This section is empty.
Functions ¶
func DownloadFile ¶
DownloadFile pulls the latest apps.json file from the Wappalyzer github
Types ¶
type App ¶
type App struct {
Cats StringArray `json:"cats"`
CatNames []string `json:"category_names"`
Cookies map[string]string `json:"cookies"`
Headers map[string]string `json:"headers"`
Meta map[string]string `json:"meta"`
HTML StringArray `json:"html"`
Script StringArray `json:"script"`
URL StringArray `json:"url"`
Website string `json:"website"`
Implies StringArray `json:"implies"`
HTMLRegex []AppRegexp `json:"-"`
ScriptRegex []AppRegexp `json:"-"`
URLRegex []AppRegexp `json:"-"`
HeaderRegex []AppRegexp `json:"-"`
MetaRegex []AppRegexp `json:"-"`
CookieRegex []AppRegexp `json:"-"`
}
App type encapsulates all the data about an App from apps.json
type AppsDefinition ¶
type AppsDefinition struct {
Apps map[string]App `json:"apps"`
Cats map[string]Category `json:"categories"`
}
AppsDefinition type encapsulates the json encoding of the whole apps.json file
var AppDefs *AppsDefinition
AppDefs provides access to the unmarshalled apps.json file
type Category ¶
type Category struct {
Name string `json:"name"`
}
Category names defined by wappalyzer
type Job ¶
type Job struct {
URL string
Body []byte
Headers http.Header //map[string][]string
Cookies []*http.Cookie
// contains filtered or unexported fields
}
Job may consist only of a URL, in which case webanalyse will proceed to download from that URL, or it may consist of the Body and Headers of a request to a URL and the URL itself, in which case these fields will be trusted and used for analysis without further network traffic. If a Job is constructed using the OfflineJob constructor then a flag will be set to prevent downloading regardless of the contents (or absence) of the Body or Headers fields.
func NewOfflineJob ¶
NewOfflineJob constructs a job out of the constituents of a webanalyzer analysis; a URL, a body, and response headers. This constructor also sets a flag to explicitly prevent fetching from the URL even if the body and headers are nil or empty. Use this for...offline jobs.
func NewOnlineJob ¶
NewOnlineJob constructs a job that may either have a URL only, or a URL, Body and Headers. If it contains at least a URL and Body, then webanalyzer will not re-download the data, but if a Body is absent then downloading will be attempted.
type Match ¶
type Match struct {
App `json:"app"`
AppName string `json:"app_name"`
Matches [][]string `json:"matches"`
Version string `json:"version"`
}
Match type encapsulates the App information from a match on a document
type StringArray ¶
type StringArray []string
StringArray type is a wrapper for []string for use in unmarshalling the apps.json
func (*StringArray) UnmarshalJSON ¶
func (t *StringArray) UnmarshalJSON(data []byte) error
UnmarshalJSON is a custom unmarshaler for handling bogus apps.json types from wappalyzer
Source Files
Directories