Documentation ¶
Overview ¶
Package tree specifies a directory tree interface, for use by the synchronization algorithm. Many different kinds of backends could be created for this interface, like local filesystem, SFTP, a remote dtsync process over SSH, maybe even MTP.
Index ¶
- Variables
- func Copy(this, other Tree, source, targetParent FileInfo, progress chan int64) (info FileInfo, parentInfo FileInfo, err error)
- func CopyFile(outf Copier, inf io.Reader, source FileInfo, progress chan int64) (FileInfo, FileInfo, error)
- func Equal(file1, file2 Entry) (bool, error)
- func ErrChanged(path []string) *pathError
- func ErrChangedHash(path []string) *pathError
- func ErrFound(path []string) *pathError
- func ErrNoDirectory(path []string) *pathError
- func ErrNoRegular(path []string) *pathError
- func ErrNoSymlink(path []string) *pathError
- func ErrNotFound(path []string) *pathError
- func IsChanged(err error) bool
- func IsExist(err error) bool
- func IsNoDirectory(err error) bool
- func IsNoRegular(err error) bool
- func IsNoSymlink(err error) bool
- func IsNotExist(err error) bool
- func MatchFingerprint(info1, info2 FileInfo) bool
- func NewHash() hash.Hash
- func SortEntries(slice []Entry)
- func TreeTest(t Tester, fs1, fs2 TestTree)
- func Update(this, other Tree, source, target FileInfo, progress chan int64) (FileInfo, FileInfo, UpdateStats, error)
- type Copier
- type Entry
- type EntrySlice
- type ErrNotImplemented
- type FileInfo
- type FileInfoStruct
- func (fi *FileInfoStruct) HasMode() Mode
- func (fi *FileInfoStruct) Hash() Hash
- func (fi *FileInfoStruct) Inode() uint64
- func (fi *FileInfoStruct) ModTime() time.Time
- func (fi *FileInfoStruct) Mode() Mode
- func (fi *FileInfoStruct) Name() string
- func (fi *FileInfoStruct) RelativePath() []string
- func (fi *FileInfoStruct) Size() int64
- func (fi *FileInfoStruct) String() string
- func (fi *FileInfoStruct) Type() Type
- type FileTree
- type Filesystem
- type Hash
- type HashType
- type ListOptions
- type LocalFileTree
- type LocalFilesystem
- type LocalTree
- type Mode
- type PathError
- type RemoteTree
- type RsyncBasis
- type ScanOptions
- type ScanProgress
- type TestTree
- type Tester
- type Tree
- type Type
- type UpdateStats
Constants ¶
This section is empty.
Variables ¶
var ( ErrInvalidName = errors.New("tree: invalid file name") ErrInvalidFS = errors.New("tree: invalid filesystem string") ErrInvalidFileId = errors.New("tree: invalid file ID string") ErrCancelled = errors.New("tree: write cancelled") )
Error codes that can be used by any filesystem implementation.
Functions ¶
func Copy ¶
func Copy(this, other Tree, source, targetParent FileInfo, progress chan int64) (info FileInfo, parentInfo FileInfo, err error)
Copy copies the object indicated by the source to the target in the other tree. It returns the file info of the copied file (which theoretically may be different from the hash in the provided FileInfo) and it's parent. It may also return an error when the source metadata (e.g. modtime) does not match the actual file. Before the actual copy, the fingerprint of the source is compared with the stat results of the source.
func CopyFile ¶
func CopyFile(outf Copier, inf io.Reader, source FileInfo, progress chan int64) (FileInfo, FileInfo, error)
CopyFile acts like io.Copy, but it also hashes the data that passes through. The returned FileInfo has this hash.
func Equal ¶
Equal compares two entries, only returning true when this file and possible children (for directories) are exactly equal. Do not compare modification time.
func ErrChanged ¶
func ErrChanged(path []string) *pathError
ErrChanged is returned when a file's fingerprint changed between scan and sync.
func ErrChangedHash ¶
func ErrChangedHash(path []string) *pathError
ErrChangedHash is returned when a file's fingerprint changed when the hash turns out to be different during copy.
func ErrFound ¶
func ErrFound(path []string) *pathError
ErrFound is returned when a file is found when none was expected (e.g. on copy or when creating a directory).
func ErrNoDirectory ¶
func ErrNoDirectory(path []string) *pathError
ErrNoDirectory is returned when a directory was expected.
func ErrNoRegular ¶
func ErrNoRegular(path []string) *pathError
ErrNoRegular is returned when a regular file was expected.
func ErrNoSymlink ¶
func ErrNoSymlink(path []string) *pathError
ErrNoSymlink is returned when a symbolic link was expected.
func ErrNotFound ¶
func ErrNotFound(path []string) *pathError
ErrNotFound is returned when a file is not found (e.g. when trying to read it).
func IsChanged ¶
IsChanged returns true if (and only if) this is an error caused by a fingerprint that was different from the expected fingerprint.
func IsExist ¶
IsExist returns true if (and only if) this is a 'found' error (e.g. on directory creation). Very similar to os.IsExist.
func IsNoDirectory ¶
IsNoDirectory returns true if this is an error like ErrNoDirectory.
func IsNoRegular ¶
IsNoRegular returns true if this is an error like ErrNoRegular.
func IsNoSymlink ¶
IsNoSymlink returns true if this is an error like ErrNoSymlink.
func IsNotExist ¶
IsNotExist returns true if (and only if) this is a 'not found' error. Very similar to os.IsNotExist.
func MatchFingerprint ¶
MatchFingerprint returns whether two FileInfos would have the same fingerprint without actually creating the fingerprint.
func NewHash ¶
NewHash returns the hashing function used by interfaces implementing Entry and by Copy and Update.
Types ¶
type Copier ¶
type Copier interface { Write([]byte) (n int, err error) Finish() (info FileInfo, parentInfo FileInfo, err error) Cancel() error // must not do anything when finished }
Copier is returned by the Copy and Update methods. Calling Finish() closes the file, possibly moves it to the destination location, and returns the file's and the file parent's FileInfo. Calling Cancel() tries to undo writing the file. It also implements io.Writer.
type Entry ¶
type Entry interface { // Name returns the name of this file. Name() string // RelativePath returns the path relative to the root of this tree. RelativePath() []string // Type returns the file type (see the Type constants above) Type() Type // Mode returns the mode (permission) bits of this file. Mode() Mode // HasMode returns the permission bits the filesystem supports. HasMode() Mode // ModTime returns the last modification time. ModTime() time.Time // Size returns the filesize for regular files. For others, it's // implementation-dependent. Size() int64 // Id returns the inode and the filesystem for this file. If the inode isn't // available, return 0 for it. If the id is nonzero, the filesystem must be // non-nil as well. Id() (inode uint64, fs *LocalFilesystem) // Hash calculates the blake2b hash of the file and returns it. Hash() (Hash, error) // Info returns a FileInfo without a hash from this Entry. // It must be fast (it shouldn't do any I/O). Info() FileInfo // FullInfo returns a FileInfo with hash from this Entry. // The FileInfo comes from Info(), but it has the hash set for regular // files. FullInfo() (FileInfo, error) // Tree returns the tree this entry belongs to. Tree() Tree // Return a list of children, in alphabetic order. // Returns an error when this is not a directory. List(ListOptions) ([]Entry, error) }
Entry is one object tree, e.g. a file or directory. It can also be something else, like bookmarks in a browser, or an email message.
type EntrySlice ¶
type EntrySlice []Entry
EntrySlice is a sortable list of Entries, sorting in incrasing order by the name (Entry.Name()).
func (EntrySlice) Len ¶
func (es EntrySlice) Len() int
func (EntrySlice) Less ¶
func (es EntrySlice) Less(i, j int) bool
func (EntrySlice) Swap ¶
func (es EntrySlice) Swap(i, j int)
type ErrNotImplemented ¶
type ErrNotImplemented string
ErrNotImplemented is returned when a method is not implemented or a parameter is not of a required type.
func (ErrNotImplemented) Error ¶
func (feature ErrNotImplemented) Error() string
type FileInfo ¶
type FileInfo interface { // Name returns the name of this file. Name() string // RelativePath returns the path relative to the root of this tree. The last // element is always the same as the name. RelativePath() []string // Type returns the file type (see the Type constants above) Type() Type // Mode returns the mode bits (mainly permissions), see os.FileMode. Mode() Mode // HasMode returns the supported mode bits, 0 for the simplest filesystems // (e.g. FAT32). HasMode() Mode // ModTime returns the last modification time. ModTime() time.Time // Size returns the filesize for regular files. For others, it's // implementation-dependent. Size() int64 // Inode returns the inode number, if available (otherwise 0). Inode() uint64 // Hash returns the blake2b hash of the file, or nil if no hash is known // (e.g. for directories). Hash() Hash }
FileInfo is like os.FileInfo, but specific for the Tree interface. It is implemented by dtdiff.Entry.
type FileInfoStruct ¶
type FileInfoStruct struct {
// contains filtered or unexported fields
}
func (*FileInfoStruct) HasMode ¶
func (fi *FileInfoStruct) HasMode() Mode
func (*FileInfoStruct) Hash ¶
func (fi *FileInfoStruct) Hash() Hash
func (*FileInfoStruct) Inode ¶
func (fi *FileInfoStruct) Inode() uint64
func (*FileInfoStruct) ModTime ¶
func (fi *FileInfoStruct) ModTime() time.Time
func (*FileInfoStruct) Mode ¶
func (fi *FileInfoStruct) Mode() Mode
func (*FileInfoStruct) Name ¶
func (fi *FileInfoStruct) Name() string
func (*FileInfoStruct) RelativePath ¶
func (fi *FileInfoStruct) RelativePath() []string
func (*FileInfoStruct) Size ¶
func (fi *FileInfoStruct) Size() int64
func (*FileInfoStruct) String ¶
func (fi *FileInfoStruct) String() string
func (*FileInfoStruct) Type ¶
func (fi *FileInfoStruct) Type() Type
type FileTree ¶
type FileTree interface { Tree // Open a file for writing with the specified name in the parent, with the // metadata as specified in 'source'. Calling Finish() should create the // file atomically at the target location. For example, by writing to it at // a temporary location and moving it to the destination. // Implementations should check that the target doesn't yet exist. CreateFile(name string, parent, source FileInfo) (Copier, error) // Update this file. May do about the same as CreateFile. UpdateFile(file, source FileInfo) (Copier, error) // Create a symbolic link. CreateSymlink(name string, parent, source FileInfo, contents string) (FileInfo, FileInfo, error) // Update a symbolic link. UpdateSymlink(file, source FileInfo, contents string) (FileInfo, FileInfo, error) // Read the contents (target path) of a symbolic link. ReadSymlink(file FileInfo) (string, error) // CopySource returns a io.ReadCloser, but checks first whether the FileInfo // matches. CopySource(info FileInfo) (io.ReadCloser, error) }
FileTree is a filesystem (like) tree, with normal mkdir and open calls, and objects containing data blobs.
type Filesystem ¶
type Filesystem int64
type Hash ¶
Hash contains a hash type (hash/blake2b, or symlink target), and the contents of the hash (32-byte buffer or variable-width string).
type ListOptions ¶
type LocalFileTree ¶
type LocalFileTree interface { LocalTree FileTree // UpdateRsync returns a source file and a file to write, for use by the // rsync algorithm. UpdateRsync(info, source FileInfo) (RsyncBasis, Copier, error) // Get this file. This only exists to read the status file, not to implement // copying in the syncer! GetFile(name string) (io.ReadCloser, error) // PutFile is analogous to GetFile. PutFile(name string) (Copier, error) }
type LocalFilesystem ¶
type LocalTree ¶
type LocalTree interface { // Root returns the root entry of this tree. Root() Entry }
type RemoteTree ¶
type RemoteTree interface { Tree // RemoteScan issues a scan on the other end, and returns a reader for the // newly created .dtsync file. // // Implementations may either scan the whole tree and return once finished, // or start scanning while returning the data as far as they've scanned. // // The sendOptions are sent to the remote scanner (scan will start once // received), and recvOptions is a channel from which the options sent by // the remote can be read. RemoteScan(extraOptions *ScanOptions, sendOptions, recvOptions chan *ScanOptions, progress chan<- *ScanProgress, cancel chan struct{}) (io.ReadCloser, error) // SendStatus is used to overwrite the status file on the remote. Data (in // e.g. protobuf format) can be written to the Copier. SendStatus() (Copier, error) // Use the rsync algorithm (librsync) to send only small changes in files. // RsyncSrc sends a signature and receives the binary delta. RsyncSrc(file FileInfo, signature io.Reader) (delta io.ReadCloser, err error) // RsyncDst receives a signature and sends the binary delta. RsyncDst(file, source FileInfo) (signature io.Reader, delta Copier, err error) }
type ScanOptions ¶
type ScanOptions struct { Exclude []string Include []string Follow []string Perms Mode Replica string }
ScanOptions holds some options to send to the other replica.
func (*ScanOptions) Add ¶
func (o *ScanOptions) Add(options *ScanOptions)
type ScanProgress ¶
ScanProgress holds the current progress (total estimated number and current position) to send from the scanner to the UI.
func (*ScanProgress) After ¶
func (p1 *ScanProgress) After(p2 *ScanProgress) bool
func (*ScanProgress) Percent ¶
func (p *ScanProgress) Percent() float64
type TestTree ¶
type TestTree interface { FileTree // ReadInfo returns the FileInfo (with hash) for a particular path. ReadInfo(path []string) (FileInfo, error) // PutFileTest sets the file at the path to the specified contents. The // FileInfo returned does not have to contain the hash. PutFileTest(path []string, contents []byte) (FileInfo, error) }
type Tester ¶
type Tester interface { Error(...interface{}) Errorf(string, ...interface{}) Fatal(...interface{}) Fatalf(string, ...interface{}) FailNow() }
Tester is a helper interface, abstracting away *testing.T. This is useful as we don't have to import package "testing" this way, which would pollute the "flag" package with it's flags.
type Tree ¶
type Tree interface { // Close clears all resources allocated for this tree, if any. For example, // it closes a network connection. Close() error // mkdir: create a directory in this directory with the given name. The // modification time is unknown and will likely be the current time. CreateDir(name string, parent, source FileInfo) (FileInfo, error) // TODO update parent mtime // Remove removes the indicated file, but checks the fingerprint first. It // returns ErrChanged if the metadata does not match, or another error if // the remove failed. Remove(file FileInfo) (parentInfo FileInfo, err error) // Chmod updates the mode (permission) bits of a file. The newly returned // FileInfo contains the new mode. Chmod(target, source FileInfo) (FileInfo, error) }
Tree is an abstraction layer over various types of trees. It tries to be as generic as possible, making it possible to synchronize varying types of trees (filesystem, sftp, remote filesystem, mtp, etc.). It may even be possible to use it for very different trees, e.g. browser bookmark trees.
type Type ¶
type Type int
The type used for TYPE_* constants
type UpdateStats ¶
Directories ¶
Path | Synopsis |
---|---|
Package file implements the file tree interface (tree.Entry) for local filesystems.
|
Package file implements the file tree interface (tree.Entry) for local filesystems. |
Package memory implements the file tree interface (tree.Entry).
|
Package memory implements the file tree interface (tree.Entry). |
Package remote implements the client and server side of a scan over a networ.
|
Package remote implements the client and server side of a scan over a networ. |