README
¶
yarascan
A set of composable tools for incremental filesystem scanning using YARA rules.
The tools can be combined with the likes of find or rdup to provide full or incremental scanning and reporting. The suite includes a centralized server with a database.
Installation
To install from source you can use the following command:
$ go install git.autistici.org/ai3/tools/yarascan/cmd/yarascan
which will install the yarascan binary in $GOPATH/bin.
Packages
The yarascan tool can be packaged either as a Debian package (it requires at least buster to build), or as a Docker image. This is due to the libyara3 dependency.
Usage
The yarascan utility supports a few different commands. The most important are scan and report, which are meant to be combined together in a pipeline, e.g.:
$ rdup -F '%n\n' /dev/null /home \
| yarascan scan --rules=... \
| yarascan report --report-url=...
scan
The scan subcommand loads YARA rules and applies them to a set of
files. It expects a list of files on standard input, one per line. It
will not follow symbolic links or descend into directories. It can be
easily combined with the find or rdup utilities. For rdup, the
-F '%n\\n' option is required to make it output just the file names.
The --rules option can be used to specify the location of the YARA rules to load. Rules will be compiled when the program starts. The engine will execute on a number of concurrent threads specified by the --num-workers option, by default equal to the number of available cores.
The output of the scan command consists of JSON-encoded detection objects, one for each of the matches returned by the YARA engine. Every detection is serialized to JSON on a separate line, a "streaming" form of JSON record encoding which is suitable to be read by the yarascan report command.
Incremental usage
It's possible to incrementally scan a filesystem using rdup with the -m option, eventually combining it with the --retry-file option to the report command (see below) to improve reliability:
$ rdup -F '%n\n' -m /var/lib/backups/list /home \
| yarascan scan --rules=... \
| yarascan report --retry-file=/var/lib/backups/retry --report-url=...
report
The report subcommand reads YARA detections and reports them over HTTP to a centralized database server via its submission API.
If the --retry-file option is specified, the tool will maintain in the given file a list of the failed records, to be retried on the next run. This way it's possible to implement something close to reliable delivery, even in face of long-term failures of the database servers.
server
The server command runs a small HTTP(S) server offering an API to the centralized database of detections. Multiple hosts can feed their data to this centralized service. Authentication is handled by the TLS layer (ref.). The only database backend available is SQLite.
Documentation
¶
Overview ¶
Most code lifted from https://github.com/botherder/kraken.
Index ¶
Constants ¶
This section is empty.
Variables ¶
var Hostname string
Hostname (current machine).
Functions ¶
This section is empty.
Types ¶
type Detection ¶
type Detection struct {
Host string `json:"host"`
Path string `json:"path"`
Signature string `json:"signature"`
Timestamp time.Time `json:"timestamp"`
}
Detection entry.
func NewDetection ¶
NewDetection builds a new Detection object for the current host.
type FindDetectionsBySiteRequest ¶
type FindDetectionsBySiteRequest struct {
Site string `json:"site"`
}
FindDetectionsBySiteRequest is the request type for /api/search/by_site.
type FindDetectionsBySiteResponse ¶
type FindDetectionsBySiteResponse struct {
Detections []*PathDetection `json:"detections"`
}
FindDetectionsBySiteResponse is the result type for FindDetectionsBySiteRequest.
type FindUnresolvedDetectionPathsRequest ¶
type FindUnresolvedDetectionPathsRequest struct {
Host string `json:"host"`
}
type FindUnresolvedDetectionPathsResponse ¶
type FindUnresolvedDetectionPathsResponse struct {
Paths []string `json:"paths"`
}
type PathDetection ¶
type PathDetection struct {
Host string `json:"host"`
Path string `json:"path"`
Site string `json:"site"`
Signatures []string `json:"signatures"`
FirstStamp time.Time `json:"first_stamp"`
LastStamp time.Time `json:"last_stamp"`
}
PathDetection summarizes the detections by path, aggregating all non-resolved detections over time.
type ResolveEntry ¶
type ResolveRequest ¶
type ResolveRequest struct {
Host string `json:"host"`
Entries []*ResolveEntry `json:"entries"`
}
type Scanner ¶
type Scanner struct {
// contains filtered or unexported fields
}
Scanner can apply YARA rules to files.
func NewScanner ¶
NewScanner loads YARA rules from rulesPath and returns a new Scanner.
type SubmissionRequest ¶
type SubmissionRequest struct {
Detections []*Detection `json:"detections"`
}
SubmissionRequest is the request type for /api/submission.
Source Files
Directories