filedriller

package module

v0.0.0-...-9028c0b Latest Latest Go to latest Published: Jan 3, 2024 License: GPL-3.0 Imports: 27 Imported by: 0

Details

Valid go.mod file

The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.
Redistributable license

Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.
Tagged version

Modules with tagged versions give importers more predictable builds.
Stable version

When a project reaches major version v1 it is considered stable.
Learn more about best practices

Repository

github.com/dla-marbach/filedriller

Links

Open Source Insights

README ¶

filedriller

IMPORTANT NOTE: filedriller is deprecated and will be EOL'd. Please check out the successor FileTrove https://github.com/steffenfritz/FileTrove and https://filetrove.fritz.wtf

filedriller walks a directory tree and identifies all regular files by type with siegfried. Furthermore it creates UUIDv4s, hash sums (md5, sha1, sha256, sha512 or blake2b-512) and filedriller can check if the file is in the NSRL.

The NSRL check expects a Redis server that serves NSRL SHA-1 hashes. You can use my docker image

iPres paper 2021: University of Vienna

Status

v1.0-BETA

For issues see the issue tab.

Installation

Binary release

Download the file for your platform and execute it on the command line. The executables are named friller.

Note: If the build badge above is green and says passing, it is a good idea to install from source.

or

From source

 go get github.com/steffenfritz/filedriller/cmd/friller

then

Download signature file
```
friller -download
```

Optional NSRL:

 - docker pull ampoffcom/nslredis:122022

 - docker images

 - docker run -p 6379:6379 $IMAGEID

When you pass the -redisserv flag, friller sends a SHA-1 hash to the specified server.

Usage Examples

Fetch the pronom.sig file
```
 friller --download
```
Without Redis / NSRL
```
 friller --in SOMEDIRECTORY
```

With Redis / NSRL

 friller --in SOMEDIRECTORY --redisserv localhost

With alternate output file

 friller --in SOMEDIRECTORY -output foo.csv

asciinema recording: https://asciinema.org/a/ZPAW3ovkYNR4flK5C5wmi2GAA

Output

The output is written to a CSV file. Schema of the file:

Filename, SizeInByte, Registry, PUID, Name, Version, MIME, ByteMatch, IdentificationNote, 
Hash Name, UUID, AccessTime, ModTime, ChangeTime, BirthTime inNSRL, Entropy

Flags

Usage of ./friller:

--download, -d

	Download siegfried's signature file

--entropy, -e

	Calculate the entropy of files. Limited to file sizes up to 1GB

--errlog, -w

    Error log file (default "errorlogs.txt")

--hash, -h string

	The hash algorithm to use: md5, sha1, sha256, sha512, blake2b-512 (default "sha256")

--in, -i string

	Root directory to work on

--log, -l string

     Log file (default "logs.txt")

--output, -o string

	Output file (default "info.csv")

--redisport, -s string

	Redis port number for a NSRL database (default "6379")

--redisserv, -p string

	Redis server address for a NSRL database

--version, -v

	Print version and build info

Documentation ¶

Index ¶

Constants
Variables
func BenchmarkCreateFileList(b *testing.B)
func BenchmarkIdentifyFiles(b *testing.B)
func CreateErrorLogger(errlogFile string)
func CreateFileList(rootDir string) ([]string, []string)
func CreateLogger(logFile string)
func CreateUUID() string
func DownloadPronom() error
func Hashit(inFile string, hashalg string) []byte
func IdentifyFSInfo(entryList []string)
func IdentifyFiles(fileList []string, hashDigest string, nsrlEnabled bool, conn redis.Conn, ...) []string
func RedisConnect(r RedisConf) redis.Conn
func RedisGet(conn redis.Conn, hashSum string) string
func WriteCSV(oFile *string, hashAlg *string, resultList []string) error
func WriteLogfile(Version string, Build string, SigFile string, hashAlg string, nsrlEnabled bool, ...)
type Config
type RedisConf

Constants ¶

View Source

const (
	// MaxFileSize is the max size file thatr should be processed. This defaults to 1 GB.
	MaxFileSize = 1073741824
	// MaxEntropyChunk is the max byte size of a chunk read
	MaxEntropyChunk = 256000
)

View Source

const Pronomurl string = "https://github.com/dla-marbach/filedriller/raw/main/third_party/pronom.sig"

Pronomurl holds the location for my copy of pronom signature file taken from Siegfried

Variables ¶

View Source

var (
	// WarningLogger writes warnings to a log file
	WarningLogger *log.Logger
	// InfoLogger writes info to a log file
	InfoLogger *log.Logger
	// ErrorLogger writes warnings to a log file
	ErrorLogger *log.Logger
)

Functions ¶

func BenchmarkCreateFileList ¶

func BenchmarkCreateFileList(b *testing.B)

BenchmarkCreateFileList benchmarks the creation of the file list that woll be processed

func BenchmarkIdentifyFiles ¶

func BenchmarkIdentifyFiles(b *testing.B)

BenchmarkIdentifyFiles benchmarks the process of identifying one file

func CreateErrorLogger ¶

func CreateErrorLogger(errlogFile string)

CreateErrorLogger creates a custom logger for errors and warnings

func CreateFileList ¶

func CreateFileList(rootDir string) ([]string, []string)

CreateFileList creates a list of file paths and a directory listing

func CreateLogger ¶

func CreateLogger(logFile string)

CreateLogger creates a custom logger

func CreateUUID ¶

func CreateUUID() string

CreateUUID returns a UUID v4 as a string

func DownloadPronom ¶

func DownloadPronom() error

DownloadPronom downloads a pronom signature file

func Hashit ¶

func Hashit(inFile string, hashalg string) []byte

Hashit hashes a file using the provided hash algorithm

func IdentifyFSInfo ¶

func IdentifyFSInfo(entryList []string)

IdentifyDirs reads metadata from the filesystem

func IdentifyFiles ¶

func IdentifyFiles(fileList []string, hashDigest string, nsrlEnabled bool, conn redis.Conn, entroEnabled bool) []string

IdentifyFiles creates metadata with siegfried and hashsum

func RedisConnect ¶

func RedisConnect(r RedisConf) redis.Conn

RedisConnect creates a connection to a Redis server

func RedisGet ¶

func RedisGet(conn redis.Conn, hashSum string) string

RedisGet returns the boolean answering if a hash is in the NSRL

func WriteCSV ¶

func WriteCSV(oFile *string, hashAlg *string, resultList []string) error

WriteCSV writes each entry in the result list to a single line of a csv file

func WriteLogfile ¶

func WriteLogfile(Version string, Build string, SigFile string, hashAlg string, nsrlEnabled bool, entro bool, fileList []string, resultList []string)

WriteLogfile creates a summary log file after the identification run

Types ¶

type Config ¶

type Config struct {
	RootDir     string
	HashAlg     string
	RedisServer string
	RedisPort   string
	SFile       bool
	OFile       string
	IFile       string
	Entro       bool
}

Config maps all flags to a struct. This is used in griller, the filedriller GUI

type RedisConf ¶

type RedisConf struct {
	Server *string
	Port   *string
}

RedisConf holds the config of a redis server

Source Files ¶

View all Source files

Directories ¶

Path	Synopsis
cmd
friller

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL