The highest tagged major version is
Documentation
¶
Index ¶
- type DB
- func (d *DB) BuildBlacklistedCollection()
- func (d *DB) BuildConnectionsCollection()
- func (d *DB) BuildHostnamesCollection()
- func (d *DB) BuildHostsCollection()
- func (d *DB) BuildHttpCollection()
- func (d *DB) BuildScanningCollection()
- func (d *DB) BuildTBDCollection()
- func (d *DB) BuildUniqueConnectionsCollection()
- func (d *DB) BuildUrlsCollection()
- func (d *DB) BuildUserAgentCollection()
- type DBMetaInfo
- type MetaDBHandle
- func (m *MetaDBHandle) AddNewDB(name string) error
- func (m *MetaDBHandle) GetDatabases() []string
- func (m *MetaDBHandle) GetFiles() []*PFile
- func (m *MetaDBHandle) GetUnAnalysedDatabases() []string
- func (m *MetaDBHandle) MarkCompleted(f *PFile) error
- func (m *MetaDBHandle) MarkDBCompleted(name string) error
- func (m *MetaDBHandle) UpdateFiles(files []*PFile) []*PFile
- type PFile
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type DB ¶
type DB struct {
// contains filtered or unexported fields
}
DB is the workhorse container for messing with the database
func (*DB) BuildBlacklistedCollection ¶
func (d *DB) BuildBlacklistedCollection()
///////////////////////////////////////////////////////////////////////////// ////////////////// LAYER 2 COLLECTION BUILDING FUNCTIONS //////////////////// /////////////////////////////////////////////////////////////////////////////
- Name: BuildBlacklistedCollection
- Purpose: Builds the blacklisted collection
- Build Type:
- Source:
- comments:
func (*DB) BuildConnectionsCollection ¶
func (d *DB) BuildConnectionsCollection()
BuildConnectionsCollection builds the 'conn' collection. Sourced from the bro parser.
func (*DB) BuildHostnamesCollection ¶
func (d *DB) BuildHostnamesCollection()
* Name: BuildHostnamesCollection * Purpose: Builds the hostnames collection * Build Type: aggregation * Source: urls collection * comments: Relies on the url collection being built
func (*DB) BuildHostsCollection ¶
func (d *DB) BuildHostsCollection()
BuildHostsCollection builds the 'host' collection for this timeframe. Note that this is a different host collection that the one found in HostsIntelDB. This host collection references only hosts found in this time frame, info from the HostsIntelDB collection can be found by following the 'intelid' field after it is populated by the cymru and blacklist modules. Runs via mongodb aggregation. Sourced from the 'conn' table. TODO: Confirm that this section of code is not faster than an aggregation from the 'uconn' table which should have less repeated data.
func (*DB) BuildHttpCollection ¶
func (d *DB) BuildHttpCollection()
BuildHttpCollection builds the 'http' collection. Sourced from the bro parser.
func (*DB) BuildScanningCollection ¶
func (d *DB) BuildScanningCollection()
* Name: BuildScanningCollection * Purpose: Builds the scanning collection * Build Type: aggregation * Source: connections table * comments:
func (*DB) BuildTBDCollection ¶
func (d *DB) BuildTBDCollection()
* Name: BuildTBDCollection * Purpose: Builds the TBD collection * Build Type: * Source: * comments:
func (*DB) BuildUniqueConnectionsCollection ¶
func (d *DB) BuildUniqueConnectionsCollection()
BuildUniqeConnectionsCollection builds the 'uconn' collection. Runs via mongodb aggreggation. Sourced from the 'conn' collection.
func (*DB) BuildUrlsCollection ¶
func (d *DB) BuildUrlsCollection()
* Name: BuildUrlsCollection * Purpose: Builds the urls collection * Build Type: map reduce -> aggregation * Source: http table * comments:
func (*DB) BuildUserAgentCollection ¶
func (d *DB) BuildUserAgentCollection()
* Name: BuildUserAgentCollection * Purpose: Builds the useragent collection * Build Type: aggregation * Source: http table * comments:
type DBMetaInfo ¶
type DBMetaInfo struct {
ID bson.ObjectId `bson:"_id,omitempty"` // Ident
Name string `bson:"name"` // Top level name of the database
Analysed bool `bson:"analyzed"` // Has this database been analyzed
}
DBMetaInfo defines some information about the database
type MetaDBHandle ¶
type MetaDBHandle struct {
DB string // Database path
Session *mgo.Session // Session to the database
// contains filtered or unexported fields
}
MetaDBHandle exports control for the meta database
func NewMetaDBHandle ¶
func NewMetaDBHandle(cfg *config.Resources) *MetaDBHandle
NewMetaDBHandle takes in a configuration and returns a MetaDBHandle controller
func (*MetaDBHandle) AddNewDB ¶
func (m *MetaDBHandle) AddNewDB(name string) error
AddNewDB adds a new database tot he DBMetaInfo table
func (*MetaDBHandle) GetDatabases ¶
func (m *MetaDBHandle) GetDatabases() []string
GetDatabases returns a list of databases being tracked in metadb or an empty array on failure
func (*MetaDBHandle) GetFiles ¶
func (m *MetaDBHandle) GetFiles() []*PFile
GetFiles gets a list of all PFile objects in the database if successful return a list of files from the database, in the case of failure return a zero length list of files and generat a log message.
func (*MetaDBHandle) GetUnAnalysedDatabases ¶
func (m *MetaDBHandle) GetUnAnalysedDatabases() []string
GetUnAnalyzedDatabases builds a list of database names which have yet to be analyzed and returns
func (*MetaDBHandle) MarkCompleted ¶
func (m *MetaDBHandle) MarkCompleted(f *PFile) error
markComplete will mark a file as having been completed in the database
func (*MetaDBHandle) MarkDBCompleted ¶
func (m *MetaDBHandle) MarkDBCompleted(name string) error
MarkDBCompleted marks a database as having been analyzed
func (*MetaDBHandle) UpdateFiles ¶
func (m *MetaDBHandle) UpdateFiles(files []*PFile) []*PFile
updateFiles updates the files table with all of the new files from a recent walk of the dir structure at the end of the update we return a new GetFiles array so that the parser knows which files to get to parsing.
type PFile ¶
type PFile struct {
ID bson.ObjectId `bson:"_id,omitempty"`
Path string `bson:"filepath"`
Hash string `bson:"hash"`
Length int64 `bson:"length"`
Parsed int64 `bson:"time_complete"`
Mod time.Time `bson:"modified"`
DataBase string `bson:"database"`
}
PFile retains everything we need to know about a given file
Source Files
Directories