luci: Index | Files

package fs

import ""

Package fs is file-system related utilities used internally by CIPD.


Package Files

doc.go errs.go file_attrs_posix.go files.go fs.go fs_posix.go path.go testing.go


const (
    // SiteServiceDir is a name of the directory inside an installation root
    // reserved for cipd stuff.
    SiteServiceDir = ".cipd"

func EnsureFile Uses

func EnsureFile(ctx context.Context, fs FileSystem, path string, content io.Reader) error

EnsureFile creates a file with the given content. It will create full directory path to the file if necessary.

func IsAccessDenied Uses

func IsAccessDenied(err error) bool

IsAccessDenied is true if err represents Windows' ERROR_ACCESS_DENIED.

Always false on Posix.

func IsCleanSlashPath Uses

func IsCleanSlashPath(p string) bool

IsCleanSlashPath returns true if path is a relative slash-separated path with no '..' or '.' entries and no '\\'. Basically "a/b/c/d".

func IsNotDir Uses

func IsNotDir(err error) bool

IsNotDir if true if err represents ENOTDIR or equivalent.

func IsNotEmpty Uses

func IsNotEmpty(err error) bool

IsNotEmpty is true if err represents ENOTEMPTY or equivalent.

func IsSubpath Uses

func IsSubpath(path, root string) bool

IsSubpath returns true if 'path' is 'root' or is inside a subdirectory of 'root'. Both 'path' and 'root' should be given as a native paths. If any of paths can't be converted to an absolute path returns false.

func TempDir Uses

func TempDir(dir string, prefix string, mode os.FileMode) (name string, err error)

TempDir is like ioutil.TempDir(dir, ""), but uses shorter path suffixes.

Path length is constraint resource of Windows.

Supposed to be used only in cases when the probability of a conflict is low (e.g. when 'dir' is some "private" directory, not global /tmp or something like that).

Additionally, this allows you to pass mode (which will respect the process umask). To get ioutils.TempDir behavior, pass 0700 for the mode.

type CreateFileOptions Uses

type CreateFileOptions struct {
    // Executable makes the file executable.
    Executable bool
    // Writable makes the file user-writable.
    Writable bool
    // ModTime, when non-zero, sets the mtime of the file.
    ModTime time.Time
    // WinAttrs is used on Windows.
    WinAttrs WinAttrs

CreateFileOptions provides arguments to Destination.CreateFile().

type Destination Uses

type Destination interface {
    // CreateFile opens a writer to extract some package file to.
    CreateFile(ctx context.Context, name string, opts CreateFileOptions) (io.WriteCloser, error)
    // CreateSymlink creates a symlink (with absolute or relative target).
    CreateSymlink(ctx context.Context, name string, target string) error

Destination knows how to create files and symlink when extracting a package.

All paths are slash separated and relative to the destination root.

'CreateFile' and 'CreateSymlink' can be called concurrently.

func ExistingDestination Uses

func ExistingDestination(dest string, fs FileSystem) Destination

ExistingDestination returns an object that knows how to create files in an existing directory in the file system.

Will use a provided FileSystem object to operate on files if given, otherwise uses a default one. If FileSystem is provided, dest must be in a subdirectory of the given FileSystem root.

Note that the returned object doesn't support transactional semantics, since transactionally writing a bunch of files into an existing directory is hard.

See NewDestination for writing files into a completely new directory. This method supports transactions.

type File Uses

type File interface {
    // Name returns slash separated file path relative to a package root
    // For example "dir/dir/file".
    Name() string

    // Size returns size of the file. 0 for symlinks.
    Size() uint64

    // Executable returns true if the file is executable.
    // Only used for Linux\Mac archives. false for symlinks.
    Executable() bool

    // Writable returns true if the file is user-writable.
    Writable() bool

    // ModTime returns modification time of the file. Zero value means no mtime is
    // recorded.
    ModTime() time.Time

    // Symlink returns true if the file is a symlink.
    Symlink() bool

    // SymlinkTarget return a path the symlink is pointing to.
    SymlinkTarget() (string, error)

    // WinAttrs returns the windows attributes, if any.
    WinAttrs() WinAttrs

    // Open opens the regular file for reading.
    // Returns error for symlink files.
    Open() (io.ReadCloser, error)

File defines a single file to be added or extracted from a package.

All paths are slash separated (including symlink targets).

func NewTestFile Uses

func NewTestFile(name string, data string, opts TestFileOpts) File

NewTestFile returns File implementation (Symlink == false) backed by a fake in-memory data. It is useful in unit tests.

func NewTestSymlink(name string, target string) File

NewTestSymlink returns File implementation (Symlink == true) backed by a fake in-memory data. It is useful in unit tests.

func NewWinTestFile Uses

func NewWinTestFile(name string, data string, attrs WinAttrs) File

NewWinTestFile returns a File implementation (Symlink == false, Executable == false) backed by a fake in-memory data with windows attributes. It is useful in unit tests.

func ScanFileSystem Uses

func ScanFileSystem(dir string, root string, exclude ScanFilter, scanOpts ScanOptions) ([]File, error)

ScanFileSystem returns all files in some file system directory in an alphabetical order. It returns only files, skipping directory entries (i.e. empty directories are completely invisible).

ScanFileSystem follows symbolic links which have a target in <root>/<SiteServiceDir> (i.e. the .cipd folder). Other symlinks will will be preserved as symlinks (see Symlink() method of File interface). Symlinks with absolute targets inside of the root will be converted to relative targets inside of the root.

It scans "dir" path, returning File objects that have paths relative to "root". It skips files and directories for which 'exclude(absolute path)' returns true. It also will always skip <root>/<SiteServiceDir>.

func WrapFile Uses

func WrapFile(abs string, root string, fileInfo *os.FileInfo, scanOpts ScanOptions) (File, error)

WrapFile constructs File object for some file in the file system, specified by its native absolute path 'abs' (subpath of 'root', also specified as a native absolute path). Returned File object has path relative to 'root'. If fileInfo is given, it will be used to grab file mode and size, otherwise os.Lstat will be used to get it. Recognizes symlinks.

type FileSystem Uses

type FileSystem interface {
    // Root returns absolute path to a directory FileSystem operates in.
    // All FS actions are restricted to this directory.
    Root() string

    // CaseSensitive returns true if the file system that has the root is
    // case-sensitive.
    CaseSensitive() (bool, error)

    // CwdRelToAbs converts a path relative to cwd to an absolute one.
    // If also verifies the path is under the root path of the FileSystem object.
    // If passed path is already absolute, just checks that it's under the root.
    CwdRelToAbs(path string) (string, error)

    // RootRelToAbs converts a path relative to Root() to an absolute one.
    // It verifies the path is under the root path of the FileSystem object.
    // If passed path is already absolute, just checks that it's under the root.
    RootRelToAbs(path string) (string, error)

    // OpenFile opens a file and returns its file handle.
    // Files opened with OpenFile can be safely manipulated by other
    // FileSystem functions.
    // This differs from os.Open notably on Windows, where OpenFile ensures that
    // files are open with FILE_SHARE_DELETE permisson to enable them to be
    // atomically renamed without contention.
    OpenFile(path string) (*os.File, error)

    // Stat returns a FileInfo describing the named file, following symlinks.
    Stat(ctx context.Context, path string) (os.FileInfo, error)

    // Lstat returns a FileInfo describing the named file, not following symlinks.
    Lstat(ctx context.Context, path string) (os.FileInfo, error)

    // EnsureDirectory creates a directory at given native path.
    // Follows symlinks. Does nothing it the path already exists and it is a
    // directory (or a symlink pointing to a directory). Replaces an existing file
    // with a directory.
    // It takes an absolute path or a path relative to the current working
    // directory and always returns absolute path.
    EnsureDirectory(ctx context.Context, path string) (string, error)

    // EnsureSymlink creates a symlink pointing to a target.
    // It will create full directory path to the symlink if necessary.
    EnsureSymlink(ctx context.Context, path string, target string) error

    // EnsureFile creates a file and calls the function to write file content.
    // It will create full directory path to the file if necessary.
    EnsureFile(ctx context.Context, path string, write func(*os.File) error) error

    // EnsureFileGone removes a file or an empty directory, logging the errors.
    // Missing file is not an error.
    // Fails if 'path' is a non-empty directory. Use EnsureDirectoryGone for this
    // case. Treats an empty directory as a file though (deletes it), since it is
    // difficult to distinguish the two without an extra syscall.
    EnsureFileGone(ctx context.Context, path string) error

    // EnsureDirectoryGone recursively removes a directory.
    EnsureDirectoryGone(ctx context.Context, path string) error

    // Renames oldpath to newpath as "atomically" as possible.
    // If newpath already exists (be it a file or a directory), removes it first.
    // If oldpath is a symlink, it's moved as is (e.g. as a symlink).
    // Not really atomic if newpath exists and it is a directory. There's a small
    // interval of time when 'oldpath' still exists, former 'newpath' is deleted,
    // but 'newpath' is not yet created. If someone creates 'newpath' during this
    // time, we retry the whole operation from scratch, and keep retrying like
    // that for 10 sec.
    Replace(ctx context.Context, oldpath, newpath string) error

    // CleanupTrash attempts to remove all files that ended up in the trash.
    // This is a best effort operation. Errors are logged (either at Debug or
    // Warning level, depending on severity of the trash state).
    CleanupTrash(ctx context.Context)

FileSystem abstracts operations that touch single file system subpath.

All functions operate in terms of native file paths. It exists mostly to hide differences between file system semantic on Windows and Linux\Mac.

func NewFileSystem Uses

func NewFileSystem(root, trash string) FileSystem

NewFileSystem returns default FileSystem implementation.

It operates with files under a given path. All methods accept absolute paths or paths relative to current working directory. FileSystem will ensure they are under 'root' directory.

It can also accept a path to a directory to put "trash" into: files that can't be removed because there are some processes keeping lock on them. This is useful on Windows when replacing running executables. The trash directory must be on the same disk as the root directory.

It 'trash' is empty string, the trash directory will be created under 'root'.

type ScanFilter Uses

type ScanFilter func(abs string) bool

ScanFilter is predicate used by ScanFileSystem to decide what files to skip.

type ScanOptions Uses

type ScanOptions struct {
    // PreserveModTime when true saves the file's modification time.
    PreserveModTime bool

    // PreserveWritable when true saves the file's writable bit for either user,
    // group or world (mask 0222), and converts it to user-only writable bit (mask
    // 0200).
    PreserveWritable bool

ScanOptions specify which file properties to preserve in the archive.

type TestFileOpts Uses

type TestFileOpts struct {
    Executable bool
    Writable   bool
    ModTime    time.Time

TestFileOpts holds options for NewTestFile method. Used in unittests.

type TransactionalDestination Uses

type TransactionalDestination interface {

    // Begin initiates a new write transaction.
    Begin(ctx context.Context) error
    // End finalizes package extraction (commit or rollback, based on success).
    End(ctx context.Context, success bool) error

TransactionalDestination is a destination that supports transactions.

It provides 'Begin' and 'End' methods: all calls to 'CreateFile' and 'CreateSymlink' should happen between them. No changes are really applied until End(true) is called. A call to End(false) discards any pending changes.

func NewDestination Uses

func NewDestination(dest string, fs FileSystem) TransactionalDestination

NewDestination returns a destination in the file system (a new directory) to extract a package to.

Will use a provided FileSystem object to operate on files if given, otherwise use a default one. If FileSystem is provided, dir must be in a subdirectory of the given FileSystem root.

type WinAttrs Uses

type WinAttrs uint32

WinAttrs represents the extra file attributes for windows.

const (
    WinAttrHidden WinAttrs = 0x2
    WinAttrSystem WinAttrs = 0x4

    WinAttrsAll WinAttrs = WinAttrHidden | WinAttrSystem

These are the valid WinAttrs values. They may be OR'd together to form a mask. These match the windows GetFileAttributes values.

func (WinAttrs) String Uses

func (w WinAttrs) String() string

Package fs imports 17 packages (graph) and is imported by 14 packages. Updated 2019-06-18. Refresh now. Tools for package owners.