Documentation
¶
Index ¶
- func DefaultDetectorTypesImplementing[T any]() map[detectorspb.DetectorType]struct{}
- func DefaultDetectors() []detectors.Detector
- func FragmentFirstLineAndLink(chunk *sources.Chunk) (int64, *int64, string)
- func FragmentLineOffset(chunk *sources.Chunk, result *detectors.Result) (int64, bool)
- func SetResultLineNumber(chunk *sources.Chunk, result *detectors.Result, fragStart int64, mdLine *int64) bool
- func SupportsLineNumbers(sourceType sourcespb.SourceType) bool
- func UpdateLink(ctx context.Context, metadata *source_metadatapb.MetaData, link string, ...) error
- type Engine
- func (e *Engine) ChunksChan() <-chan *sources.Chunk
- func (e *Engine) DetectorAvgTime() map[string][]time.Duration
- func (e *Engine) Finish(ctx context.Context) error
- func (e *Engine) GetDetectorsMetrics() map[string]time.Duration
- func (e *Engine) GetMetrics() Metrics
- func (e *Engine) HasFoundResults() bool
- func (e *Engine) ResultsChan() chan detectors.ResultWithMetadata
- func (e *Engine) ScanChunk(chunk *sources.Chunk)
- func (e *Engine) ScanCircleCI(ctx context.Context, token string) error
- func (e *Engine) ScanDocker(ctx context.Context, conn *anypb.Any) error
- func (e *Engine) ScanFileSystem(ctx context.Context, c sources.FilesystemConfig) error
- func (e *Engine) ScanGCS(ctx context.Context, c sources.GCSConfig) error
- func (e *Engine) ScanGit(ctx context.Context, c sources.GitConfig) error
- func (e *Engine) ScanGitHub(ctx context.Context, c sources.GithubConfig) error
- func (e *Engine) ScanGitLab(ctx context.Context, c sources.GitlabConfig) error
- func (e *Engine) ScanS3(ctx context.Context, c sources.S3Config) error
- func (e *Engine) ScanSyslog(ctx context.Context, c sources.SyslogConfig) error
- type EngineOption
- func WithConcurrency(concurrency uint8) EngineOption
- func WithDecoders(decoders ...decoders.Decoder) EngineOption
- func WithDetectors(verify bool, d ...detectors.Detector) EngineOption
- func WithFilterDetectors(filterFunc func(detectors.Detector) bool) EngineOption
- func WithFilterUnverified(filter bool) EngineOption
- func WithOnlyVerified(onlyVerified bool) EngineOption
- func WithPrintAvgDetectorTime(printAvgDetectorTime bool) EngineOption
- func WithPrinter(printer Printer) EngineOption
- type Metrics
- type Printer
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func DefaultDetectorTypesImplementing ¶
func DefaultDetectorTypesImplementing[T any]() map[detectorspb.DetectorType]struct{}
func DefaultDetectors ¶
func FragmentFirstLineAndLink ¶
FragmentFirstLineAndLink extracts the first line number and the link from the chunk metadata. It returns:
- The first line number of the fragment.
- A pointer to the line number, facilitating direct updates.
- The link associated with the fragment. This link may be updated in the chunk metadata if there's a change in the line number.
func FragmentLineOffset ¶
FragmentLineOffset sets the line number for a provided source chunk with a given detector result.
func SetResultLineNumber ¶
func SetResultLineNumber(chunk *sources.Chunk, result *detectors.Result, fragStart int64, mdLine *int64) bool
SetResultLineNumber sets the line number in the provided result.
func SupportsLineNumbers ¶
func SupportsLineNumbers(sourceType sourcespb.SourceType) bool
SupportsLineNumbers determines if a line number can be found for a source type.
func UpdateLink ¶
func UpdateLink(ctx context.Context, metadata *source_metadatapb.MetaData, link string, line int64) error
UpdateLink updates the link of the provided source metadata.
Types ¶
type Engine ¶
func Start ¶
func Start(ctx context.Context, options ...EngineOption) (*Engine, error)
Start the engine with options.
func (*Engine) ChunksChan ¶
func (*Engine) DetectorAvgTime ¶
DetectorAvgTime returns the average time taken by each detector.
func (*Engine) Finish ¶
Finish waits for running sources to complete and workers to finish scanning chunks before closing their respective channels. Once Finish is called, no more sources may be scanned by the engine.
func (*Engine) GetDetectorsMetrics ¶
GetDetectorsMetrics returns a copy of the average time taken by each detector.
func (*Engine) GetMetrics ¶
GetMetrics returns a copy of Metrics. It's safe for concurrent use, and the caller can't modify the original data.
func (*Engine) HasFoundResults ¶
HasFoundResults returns true if any results are found.
func (*Engine) ResultsChan ¶
func (e *Engine) ResultsChan() chan detectors.ResultWithMetadata
func (*Engine) ScanChunk ¶
ScanChunk injects a chunk into the output stream of chunks to be scanned. This method should rarely be used. TODO: Remove when dependencies no longer rely on this functionality.
func (*Engine) ScanCircleCI ¶
ScanCircleCI scans CircleCI logs.
func (*Engine) ScanDocker ¶
ScanDocker scans a given docker connection.
func (*Engine) ScanFileSystem ¶
ScanFileSystem scans a given file system.
func (*Engine) ScanGitHub ¶
ScanGitHub scans Github with the provided options.
func (*Engine) ScanGitLab ¶
ScanGitLab scans GitLab with the provided configuration.
func (*Engine) ScanSyslog ¶
ScanSyslog is a source that scans syslog files.
type EngineOption ¶
type EngineOption func(*Engine)
func WithConcurrency ¶
func WithConcurrency(concurrency uint8) EngineOption
func WithDecoders ¶
func WithDecoders(decoders ...decoders.Decoder) EngineOption
func WithDetectors ¶
func WithDetectors(verify bool, d ...detectors.Detector) EngineOption
func WithFilterDetectors ¶
func WithFilterDetectors(filterFunc func(detectors.Detector) bool) EngineOption
WithFilterDetectors applies a filter to the configured list of detectors. If the filterFunc returns true, the detector will be included for scanning. This option applies to the existing list of detectors configured, so the order this option appears matters. All filtering happens before scanning.
func WithFilterUnverified ¶
func WithFilterUnverified(filter bool) EngineOption
WithFilterUnverified sets the filterUnverified flag on the engine. If set to true, the engine will only return the first unverified result for a chunk for a detector.
func WithOnlyVerified ¶
func WithOnlyVerified(onlyVerified bool) EngineOption
WithOnlyVerified sets the onlyVerified flag on the engine. If set to true, the engine will only print verified results.
func WithPrintAvgDetectorTime ¶
func WithPrintAvgDetectorTime(printAvgDetectorTime bool) EngineOption
WithPrintAvgDetectorTime sets the printAvgDetectorTime flag on the engine. If set to true, the engine will print the average time taken by each detector. This option allows us to measure the time taken for each detector ONLY if the engine is configured to print the results. Calculating the average time taken by each detector is an expensive operation and should be avoided unless specified by the user.
func WithPrinter ¶
func WithPrinter(printer Printer) EngineOption
WithPrinter sets the Printer on the engine.
type Metrics ¶
type Metrics struct {
BytesScanned uint64
ChunksScanned uint64
VerifiedSecretsFound uint64
UnverifiedSecretsFound uint64
AvgDetectorTime map[string]time.Duration
ScanDuration time.Duration
// contains filtered or unexported fields
}
Metrics for the scan engine for external consumption.
Source Files