Documentation
¶
Index ¶
- func CountLines(path string) (int, error)
- func EnsureDirExists(dirPath string) (string, error)
- func EnsureDirPathExists(dirPath string) error
- func FileExists(path string) bool
- func GetHourlyDirs(prefixDir string, start, end time.Time) []string
- func Must(s string, err error) string
- func MustOpen(name string) *os.File
- func MustReaddir(dir *os.File) []os.FileInfo
- func ReadAllFilesRecursive(dir string, excludeDirs []string) (map[string][]byte, error)
- func ReadAndSha1File(path string) (contents, hash string, err error)
- func TwoLevelRadixPath(pathSegments ...string) string
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func CountLines ¶
CountLines opens the given path and counts the number of lines in the file. Returns -1 with a non-nil error if an error is encountered.
func EnsureDirExists ¶
EnsureDirExists checks whether the given path to a directory exits and creates it if necessary. Returns the absolute path that corresponds to the input path and an error indicating a problem.
func EnsureDirPathExists ¶
EnsureDirPathExists checks whether the directories of the given file path exist and creates them if necessary. Returns an error if there was a problem creating the path.
func FileExists ¶
FileExists returns true if the given path exists and false otherwise. If there is an error it will return false and log the error message.
func GetHourlyDirs ¶
GetHourlyDirs generates paths with the given prefix for all the paths of the form 'prefix/<year>/<month>/<day>/<hour>'. All path segments are padded with zeros where necessary, e.g. prefix/2018/01/02/09. All hours are from 0-23. This is intended to enumerate the directories that should be polled when ingesting data. startTS and endTS are seconds since the Unix epoch. Both time stamps are inclusive. The generated directories are based on UTC being the locale. If the prefix is empty string, there will be no slash before the year.
func Must ¶
Must checks whether err in the provided pair (s, err) is nil. If so it returns s otherwise it cause the program to stop with the error message.
func MustOpen ¶
MustOpen opens the file of the given name, returning an *os.File on success, or causing the program to stop with the error message.
func MustReaddir ¶
MustReaddir returns a slice of os.FileInfo for every file in the given dir. This is equivalent to calling dir.Readdir(-1), except this call will stop the program on an error
func ReadAllFilesRecursive ¶
ReadAllFilesRecursive recursively reads all files in the given dir and returns a map of filename to contents.
func ReadAndSha1File ¶
ReadAndSha1File opens a file, reads the contents, hashes them using the sha1 hashing algorithm and returns the contents and hash. If the file doesn't exist, the err will be nil and both contents and hash will be empty string.
func TwoLevelRadixPath ¶
TwoLevelRadixPath expands a path (defined by one or more path segments) by adding two additional directories based on the filename in the last segment. i.e. TwoLevelRadixPath("/some", "path", "to", "abcdefgh.txt") will return "/some/path/to/ab/cd/abcdefgh.txt". If the filename does not have at least four characters before a period the return values is equivalent to filepath.Join(pathSegments...).
Types ¶
This section is empty.
Source Files