Documentation ¶
Index ¶
- Variables
- func Crc32HashKeyGenerator(path string) (string, error)
- func FullCrc32HashKeyGenerator(path string) (string, error)
- func FullSha256HashKeyGenerator(path string) (string, error)
- func GetResults(c Cfg) ([]string, error)
- func Sha256HashKeyGenerator(path string) (string, error)
- func StreamResults(c Cfg, dupesChan chan []string) error
- type Cfg
- type Filters
- type FiltersList
- type KeyGeneratorFunc
- type Paths
Constants ¶
This section is empty.
Variables ¶
var ( // Used to skip a file during key generation. // // These kind of errors are ignored and not returned to the caller // of dupescout.GetResults() or dupescout.StreamResults(). ErrSkipFile = fmt.Errorf("skip file") )
Functions ¶
func Crc32HashKeyGenerator ¶
Crc32HashKeyGenerator is the default if no KeyGenerator is specified.
Generates a crc32 hash of the first 16KB of the file contents as the key, which should be enough to achieve a good balance of uniqueness, collision resistance, and performance for most files.
func FullCrc32HashKeyGenerator ¶
Generates a crc32 hash of the entire file contents as the key, which is a lot slower than HashKeyGenerator but should be more accurate.
func FullSha256HashKeyGenerator ¶
Generates a sha256 hash of the entire file contents as the key
func GetResults ¶
Runs the duplicate search and returns a slice of all duplicate paths.
func Sha256HashKeyGenerator ¶
Generates a sha256 hash of the first 16KB of the file contents as the key
func StreamResults ¶
Runs the duplicate search and streams the duplicate paths to the provided channel as they are found.
Types ¶
type Cfg ¶
type Cfg struct { KeyGenerator KeyGeneratorFunc // Function to generate a key based on the file path. Paths // List of paths to search in for duplicates. Filters // Filters to apply when searching for duplicates. Workers int // Number of workers to use when searching for duplicates. }
type Filters ¶
type Filters struct { ExtInclude FiltersList // List of file extensions to include. ExtExclude FiltersList // List of file extensions to exclude. DirsExclude FiltersList // List of directories or subdirectories to exclude. SkipSubdirs bool // Skip subdirectories. HiddenInclude bool // Include hidden files and directories. }
type FiltersList ¶
type FiltersList []string
Satisfies the flag.Value interface, string values can be provided as a csv or space separated list.
`flag.Var(&cfg.DirsExclude "exclude-dirs", "exclude directories or subdirectories")
func (*FiltersList) Set ¶
func (fl *FiltersList) Set(val string) error
func (*FiltersList) String ¶
func (fl *FiltersList) String() string
type KeyGeneratorFunc ¶
KeyGenerator generates a key for a given file path, which then is mapped to a list of file paths that share the same key (duplicates).
The provided KeyGeneratorFuncs hash the file contents to generate the key, but the logic can be anything as long as it's deterministic. For example, you could generate a key based on the file name, size, etc.