Documentation
¶
Index ¶
- Constants
- func ConcatFiles(logger *log.Logger, inputFiles []string, outputFile string, ...) (e error)
- func ConcatFilesParallelByDate(logType string, inputFiles []string, outputFile, outputDir string, ...)
- func ConcatToStdout(logger *log.Logger, inputFiles []string, deleteInputAfterRead bool, ...) (e error)
- func GenRuntimeConfig(globalConfig *viper.Viper, cmd *cobra.Command)
- func InitBars(dayCount int, taskCount int, logger *log.Logger) (pool *pb.Pool, dayBar *pb.ProgressBar, taskBar *pb.ProgressBar)
- func ParseLogs(cmd *cobra.Command, ...)
- func ParseSharedArgs(cmd *cobra.Command, timeRange string, logDir string, outputDir string, ...) (startTime time.Time, endTime time.Time, resolvedOutDir string, ...)
- func ReadGlobalConfig() (globalConfig *viper.Viper)
- func TryCreateDir(dir string, empty bool) (err error)
- func WaitForConfirm(cmd *cobra.Command) (start bool)
- type Config
- type DataSource
Constants ¶
const ( TimeFormatShort = "2006/01/02:15" TimeFormatLong = "2006/01/02:15:04:05" TimeFormatLongNum = "20060102:15:04:05.000" TimeFormatHuman = "2006/01/02 15:04:05" TimeFormatDate = "2006/01/02" TimeFormatDateNum = "2006_01_02_" )
Variables ¶
This section is empty.
Functions ¶
func ConcatFiles ¶
func ConcatFiles(logger *log.Logger, inputFiles []string, outputFile string, deleteInputAfterRead bool, ignoreMissing bool) (e error)
takes a list of files, sorts them and concats them into a single file. if deleteInputAfterRead, also deletes the input after use.
func ConcatFilesParallelByDate ¶
func ConcatFilesParallelByDate(logType string, inputFiles []string, outputFile, outputDir string, logger *log.Logger, curDate time.Time, wgDate *sync.WaitGroup, wgAll *sync.WaitGroup, bar *pb.ProgressBar)
Waits until the given sync group is done. When it finishes, concats all files together of that particular date, and then lets the global sync group know it has finished.
func ConcatToStdout ¶
func ConcatToStdout(logger *log.Logger, inputFiles []string, deleteInputAfterRead bool, ignoreMissing bool) (e error)
takes a list of files and writes them to STDOUT
func InitBars ¶
func InitBars(dayCount int, taskCount int, logger *log.Logger) (pool *pb.Pool, dayBar *pb.ProgressBar, taskBar *pb.ProgressBar)
set up task, bar interface.
func ParseLogs ¶
func ParseLogs(cmd *cobra.Command, logHandler func(string, string, time.Time, *sync.WaitGroup, *pb.ProgressBar), logger *log.Logger, startTime time.Time, endTime time.Time, logType string, resolvedLogDir string, resolvedOutDir string, threads int, singleFile bool, writeStdout bool)
takes a log type, time range, zeek log directory, thread information, and output directory info. it then parses logs based on the logHandler and then outputs the files to the given directory, all parallelized.
func ParseSharedArgs ¶
func ParseSharedArgs(cmd *cobra.Command, timeRange string, logDir string, outputDir string, logTypeArg string) (startTime time.Time, endTime time.Time, resolvedOutDir string, resolvedLogDir string, logType string)
parses and verifies arguments that are global to the root command.
func ReadGlobalConfig ¶
takes a global config from /etc/nagini or ~/.config/nagini, reads in vars that are present, and passes them as a viper config.
func TryCreateDir ¶
tries to create a directory at the given path. the parent directory must already exist. if the directory already exists, will check to make sure write permissions
- additionally, if the empty flag is set, then it will enforce that the directory is empty.
func WaitForConfirm ¶
ask the user to continue or exit. Returns true if continue, false if not.
Types ¶
type Config ¶
type Config struct {
DataSources []DataSource `yaml:"data_sources"` // data_sources
}
The High-Level Config
func ParseConfig ¶
Read the YAML config file from the specified path by string input, and then populate a struct based on present fields. Returns the struct parsed and if there was an error in parsing.
type DataSource ¶
type DataSource struct {
Name string // name
Threads int // threads
// one of: use specified log-path OR specify
ManualPath string `yaml:"manual_path"` // manual_path
Type string `yaml:"log_type"` //log_type
}
The DataSource struct represents fields for an individual data source found in the config YAML file. It represents an individual log pull set, which will be stored in {ProjectName}/{Name}, unless ManualPath is specified. It will use Threads as the number of threads on the system to pull data with.
Source Files