README
¶
fst - File System Testing aids
The suggested package name pronounciation is "fist".
This is the version 1 branch of the
fstpackage. The upcoming version 2 simplifies the API significantly. It is, however, backward incompatible and will require some client code modifications. The version 2 is in beta until about the end of Martch 2019 - see the v2 branch. Please, file an issue with a feedback or a bug.
Purpose
At times it is desireable to test a program behavior which creates or modifies files and directories. Such tests may be quite involved especially if checking permissions or timestamps. A proper cleanup is also considered a nuisance. The whole effort becomes extra burdesome as such filesystem manipulation has to be tested itself - so one ends up with tests of tests.
The fst library is a tested set of functions aiming to alleviate the burden. It makes creating and comparing filesystem trees of regular files and directories for testing puposes simple.
Highlights
The three most used functtions in the fst library are TempCloneChdir, TempCreateChdir, and TreeDiff. For details on these and other functions, see the examples and documentation at the https://godoc.org/go.didenko.com/fst page.
TempCloneChdir
It is used to clone an existing directory with all it's content, permissions, and timestamps. Consider this example:
old, cleanup, err := TempCloneChdir("mock")
if err != nil {
t.Fatal(err)
}
defer cleanup()
If an error was returned, then no temporary directory was left behind (after a reasonable cleanup effort). Otherwise, the mock directory's content will be cloned into a new temporary directory and the calling process will change into it. The old variable in the example will contain the original directory where the running process was at the time of the TempCloneChdir call.
The cleanup function has the code to change back to the original directory and then delete the temporary directory.
As the TempCloneChdir relies on the TreeCopy function, it will attempt to recreate both permissions and timestamps from the source directory. Keep in mind, that popular version control systems like Git and Mercurial do not preserve original files' timestamps. If your tests rely on timestamped files or directories then TreeCreate or it's derivative TempCreateChdir functions are your friends.
TempCreateChdir
The TempCreateChdir function provides an API-like way to create and populate a temporary directory tree for testing. It takes an io.Reader, from which is expects to receive lines with tab-separated fields describing the directories and files to be populated. Here is an example:
tree := `
2017-11-12T13:14:15Z 0750 settings/
2017-11-12T13:14:15Z 0640 settings/theme1.toml key = val1
2017-11-12T13:14:15Z 0640 settings/theme2.toml key = val2
`
treeR := strings.NewReader(tree)
old, cleanup, err = TempCreateChdir(treeR)
if err != nil {
t.Fatal(err)
}
defer cleanup()
If there is no error, TempCreateChdir will create the settings directory with files theme1.toml and theme2.toml, with the specified key/value pairs as content in a new directory.
The TempCreateChdir function removes the temporary directory it creates as a part of the cleanup logic. It also does a best-effort attempt to remove the directory is an error occurred during it's operation.
TreeDiff
The TreeDiff function produces a human-readable output of differences between two directory trees for diagnostic purposes. The resulting slice of strings is empty if no differences are found.
Criteria for comparing filesystem objects varies based on a task, so TreeDiff takes a list of comparator functions. The most common ones are provided with the fst package. Users are free to provide their own additional comparators which satisfy the FileRank signature.
A quick example of a common TreeDiff use:
diffs, err := TreeDiff(
"dir1", "dir2",
ByName, ByDir, BySize, ByContent(t))
if err != nil {
t.Fatal(err)
}
if diffs != nil {
t.Logf("Differences between dir1 and dir2:\n%v\n", diffs)
}
Note, that while the BySize comparator is redundant in presense of the ByContent comparator, in most cases the cheaper size comparison will avoid a more expensive content comparison. The comparator order is significant, because once an earlier comparator returns true, the later comparators are not run.
It is easy to provide overly restrictive permissions using the tree cloning and tree creation functions. When unable to access needed information, TreeDiff will return a related error. While specifics may vary it is often safest to set user read and execute permissions for directories and user read permission for files.
Limitations
Functions in fst expect a reasonably shallow and small directory structures to deal with, as that is what usually happens in testing. During build-up, tear-down, and comparisons it creates collections of filesystem object names in memory. It is not necessarily efficient but allows for more graceful permissions handling.
If you are concerned that you will hold a few copies of full filenames' lists during the execution, then this library may be a poor match to your needs.
Documentation
¶
Overview ¶
Package fst is a collection of functions to help testing filesyste objects modifications. It focuses on creating and cleaning up a baseline filesyetem state.
The following are addressed use cases:
1. Create a directory hierarchy via an API
2. Create a directory hierarchy via a copy of a template
3. Write a provided test mock data to files
4. Contain all test activity in a temporatry directory
5. Compare two directories recursively
Index ¶
- func ByDir(left, right *FileInfoPath) bool
- func ByName(left, right *FileInfoPath) bool
- func ByPerm(left, right *FileInfoPath) bool
- func BySize(left, right *FileInfoPath) bool
- func ByTime(left, right *FileInfoPath) bool
- func FileDelAll(root, name string) error
- func Less(left, right *FileInfoPath, cmps ...FileRank) bool
- func TempCloneChdir(src string) (string, func(), error)
- func TempCloneDir(src string) (string, func(), error)
- func TempCreateChdir(config io.Reader) (string, func(), error)
- func TempInitChdir() (string, func(), error)
- func TempInitDir() (string, func(), error)
- func TreeCopy(src, dst string) error
- func TreeCreate(config io.Reader) error
- func TreeDiff(a string, b string, comps ...FileRank) ([]string, error)
- type FileInfoPath
- type FileRank
Examples ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func ByDir ¶
func ByDir(left, right *FileInfoPath) bool
ByDir differentiates directorries vs. files and puts directories earlier in a sort order
func ByName ¶
func ByName(left, right *FileInfoPath) bool
ByName is basic for comparing directories and should be provided as a first comparator in most cases
func ByPerm ¶
func ByPerm(left, right *FileInfoPath) bool
ByPerm compares bits 0-8 of Unix-like file permissions
func BySize ¶
func BySize(left, right *FileInfoPath) bool
BySize compares sizes of files, given that both of the files are regular files as opposed to not directories, etc.
func ByTime ¶
func ByTime(left, right *FileInfoPath) bool
ByTime compares files' last modification times with up to 10µs precision to accommodate filesyustem quirks
func FileDelAll ¶
FileDelAll recursevely removes file `name` from the `root` directory. It is useful to get truly empty directories after cloning checked out almost empty directories containing a stake file like `.gitkeep`
func Less ¶
func Less(left, right *FileInfoPath, cmps ...FileRank) bool
Less applies provided comparators to the pair of *FileInfoPath structs.
func TempCloneChdir ¶
TempCloneChdir clones a temporary directory in the same fashion as TempCloneDir. It also changes into the newly cloned temporary directory and adds returning back to the old working directory to the returned cleanup function. The returned values are:
1. a string containing the previous working directory
2. a cleanup function to change back to the old working directory and to delete the temporary directory
3. an error
func TempCloneDir ¶
TempCloneDir function creates a copy of an existing directory with it's content - regular files only - for holding temporary test files.
The returned values are:
1. a string containing the created temporary directory path
2. a cleanup function to change back to the old working directory and to delete the temporary directory
3. an error
If there was an error while cloning the temporary directory, then the returned directory name is empty, cleanup funcion is nil, and the temp folder is expected to be already removed.
The clone attempts to maintain the basic original Unix permissions (9-bit only, from the rxwrwxrwx set). If, however, the user does not have read permission for a file, or read+execute permission for a directory, then the clone process will naturally fail.
Example ¶
root, cleanup, err := TempCloneDir("./mock")
if err != nil {
log.Fatal(err)
}
defer cleanup()
fmt.Printf("Here goes the code using the temporary directory at %s\n", root)
Output:
func TempCreateChdir ¶
TempCreateChdir is a combination of `TempInitChdir` and `TreeCreate` functions. It creates a termporary directory, changes into it, populates it fron the provided `config` as `TreeCreate` would, and returns the old directory name and the cleanup function.
Example ¶
dirMark := func(fi os.FileInfo) string {
if fi.IsDir() {
return "/"
}
return ""
}
dirs := `
2001-01-01T01:01:01Z 0750 a/
2001-01-01T01:01:01Z 0750 b/
2001-01-01T01:01:01Z 0700 c.txt "This is a two line\nfile with\ta tab\n"
2001-01-01T01:01:01Z 0700 d.txt No need to quote a single line without tabs
2002-01-01T01:01:01Z 0700 "has\ttab/"
2002-01-01T01:01:01Z 0700 "has\ttab/e.mb" "# Markdown...\n\n... also ***possible***\n"
2002-01-01T01:01:01Z 0700 "\u263asmiles\u263a/"
`
reader := strings.NewReader(dirs)
_, cleanup, err := TempCreateChdir(reader)
if err != nil {
log.Fatal(err)
}
defer cleanup()
files, err := ioutil.ReadDir(".")
if err != nil {
log.Printf("%v\n", err)
return
}
fmt.Printf(
"%v | %v | %s%s\n",
files[1].ModTime().UTC(),
files[1].Mode().Perm(),
files[1].Name(),
dirMark(files[1]),
)
fmt.Printf(
"%v | %v | %s%s\n",
files[2].ModTime().UTC(),
files[2].Mode().Perm(),
files[2].Name(),
dirMark(files[2]),
)
Output: 2001-01-01 01:01:01 +0000 UTC | -rwxr-x--- | b/ 2001-01-01 01:01:01 +0000 UTC | -rwx------ | c.txt
func TempInitChdir ¶
TempInitChdir creates a temporary directory in the same fashion as TempInitDir. It also changes into the newly created temporary directory and adds returning back to the old working directory to the returned cleanup function. The returned values are:
1. a string containing the previous working directory
2. a cleanup function to change back to the old working directory and to delete the temporary directory
3. an error
func TempInitDir ¶
TempInitDir function creates a directory for holding temporary files according to platform preferences and returns the directory name and a cleanup function.
The returned values are:
1. a string containing the created temporary directory path
2. a cleanup function to change back to the old working directory and to delete the temporary directory
3. an error
If there was an error while creating the temporary directory, then the returned directory name is empty, cleanup funcion is nil, and the temp folder is expected to be already removed.
Example ¶
root, cleanup, err := TempInitDir()
if err != nil {
log.Fatal(err)
}
defer cleanup()
fmt.Printf("Here goes the code using the temporary directory at %s\n", root)
Output:
func TreeCopy ¶
TreeCopy duplicates redular files and directories from inside the source directory into an existing destination directory.
func TreeCreate ¶
TreeCreate parses a suplied Reader for the tree information and follows the instructions to create files and directories.
The input has line records with three or four fields separated by one or more tabs. White space is trimmed on both ends of lines. Empty lines are skipped. The general line format is:
<1. time> <2. permissions> <3. name> <4. optional content>
Field 1: Time in RFC3339 format, as shown at https://golang.org/pkg/time/#RFC3339
Field 2: Octal (required) representation of FileMode, as at https://golang.org/pkg/os/#FileMode
Field 3: is the file or directory path to be created. If the first character of the path is a double-quote or a back-tick, then the path wil be passed through strconv.Unquote() function. It allows for using tab-containing or otherwise weird names. The quote or back-tick should be balanced at the end of the field.
If the path in Field 3 ends with a forward slash, then it is treated as a directory, otherwise - as a regular file.
Field 4: is optional content to be written into the file. It follows the same quotation rules as paths in Field 3. Directory entries ignore Field 4 if present.
It is up to the caller to deal with conflicting file and directory names in the input. TreeCreate processes the input line-by-line and will return with error at a first problem it runs into.
func TreeDiff ¶
TreeDiff produces a slice of human-readable notes about recursive differences between two directory trees on a filesystem. Only plan directories and plain files are compared in the tree. Specific comparisons are determined By the variadic slice of FileRank functions, like the ones in this package. A commonly used set of comparators is ByName, ByDir, BySize, and ByContent
Types ¶
type FileInfoPath ¶
FileInfoPath is a wrapper of os.FileInfo with an additional field to store the path to the file of interest
func MakeFipSlice ¶
func MakeFipSlice(files ...string) ([]*FileInfoPath, error)
MakeFipSlice creates a slice of *FileInfoPaths based on provided list of file names. It returns the first encountered error.
func NewFileInfoPath ¶
func NewFileInfoPath(path string) (*FileInfoPath, error)
NewFileInfoPath creates new FileInfoPath struct
func (*FileInfoPath) Path ¶
func (fip *FileInfoPath) Path() string
Path returns the stored full path in the FileInfoPath struct
type FileRank ¶
type FileRank func(left, right *FileInfoPath) bool
FileRank is the signature of functions which are provided to TreeDiff to compare two *FileInfoPath structs and related files.
When comparing filesystem objects, the algorithm used in the TreeDiff function expects the "less-than" logic from comparator functions. It means, that a comparator is expected to return "true" if and only if the "left" parameter is strictly less than the "right" parameter according to the comparator's criteria.
FileRank does not provide an error handling interface. If needed, it can be implemented via a closure. See the ByContent comparator generator for an example of it.
func ByContent ¶
ByContent returns a function which compares files' content without first comparing sizes. For example, file containing "aaa" will rank as lesser than the one containing "ab" even though it is opposite to their sizes. To consider sizes first, make sure to specify the BySize comparator earlier in the chain.
Source Files