rclone: github.com/ncw/rclone/vfs/vfscache Index | Files | Directories

package vfscache

import "github.com/ncw/rclone/vfs/vfscache"

Package vfscache deals with caching of files locally for the VFS layer


Package Files

cache.go item.go

type AddVirtualFn Uses

type AddVirtualFn func(remote string, size int64, isDir bool) error

AddVirtualFn if registered by the WithAddVirtual method, can be called to register the object or directory at remote as a virtual entry in directory listings.

This is used when reloading the Cache and uploading items need to go into the directory tree.

type Cache Uses

type Cache struct {
    // contains filtered or unexported fields

Cache opened files

func New Uses

func New(ctx context.Context, fremote fs.Fs, opt *vfscommon.Options, avFn AddVirtualFn) (*Cache, error)

New creates a new cache heirachy for fremote

This starts background goroutines which can be cancelled with the context passed in.

func (*Cache) AddVirtual Uses

func (c *Cache) AddVirtual(remote string, size int64, isDir bool) error

AddVirtual adds a virtual directory entry by calling the addVirtual callback if one has been registered.

func (*Cache) CleanUp Uses

func (c *Cache) CleanUp() error

CleanUp empties the cache of everything

func (*Cache) DirtyItem Uses

func (c *Cache) DirtyItem(name string) (item *Item)

DirtyItem the Item if it exists in the cache and is Dirty

name should be a remote path not an osPath

func (*Cache) Dump Uses

func (c *Cache) Dump() string

Dump the cache into a string for debugging purposes

func (*Cache) Exists Uses

func (c *Cache) Exists(name string) bool

Exists checks to see if the file exists in the cache or not.

This is done by bringing the item into the cache which will validate the backing file and metadata and then asking if the Item exists or not.

func (*Cache) InUse Uses

func (c *Cache) InUse(name string) bool

InUse returns whether the name is in use in the cache

name should be a remote path not an osPath

func (*Cache) Item Uses

func (c *Cache) Item(name string) (item *Item)

Item gets a cache item for name

To use it item.Open will need to be called

name should be a remote path not an osPath

func (*Cache) Remove Uses

func (c *Cache) Remove(name string) (wasWriting bool)

Remove should be called if name is deleted

This returns true if the file was in the transfer queue so may not have completedly uploaded yet.

func (*Cache) Rename Uses

func (c *Cache) Rename(name string, newName string, newObj fs.Object) (err error)

Rename the item in cache

func (*Cache) SetModTime Uses

func (c *Cache) SetModTime(name string, modTime time.Time)

SetModTime should be called to set the modification time of the cache file

func (*Cache) TotalInUse Uses

func (c *Cache) TotalInUse() (n int)

TotalInUse returns the number of items in the cache which are InUse

type Info Uses

type Info struct {
    ModTime     time.Time     // last time file was modified
    ATime       time.Time     // last time file was accessed
    Size        int64         // size of the file
    Rs          ranges.Ranges // which parts of the file are present
    Fingerprint string        // fingerprint of remote object
    Dirty       bool          // set if the backing file has been modified

Info is persisted to backing store

type Item Uses

type Item struct {
    // contains filtered or unexported fields

Item is stored in the item map

The Info field is written to the backing store to store status

func (*Item) Close Uses

func (item *Item) Close(storeFn StoreFn) (err error)

Close the cache file

func (*Item) Dirty Uses

func (item *Item) Dirty()

Dirty marks the item as changed and needing writeback

func (*Item) Exists Uses

func (item *Item) Exists() bool

Exists returns whether the backing file for the item exists or not

func (*Item) FindMissing Uses

func (item *Item) FindMissing(r ranges.Range) (outr ranges.Range)

FindMissing adjusts r returning a new ranges.Range which only contains the range which needs to be downloaded. This could be empty - check with IsEmpty. It also adjust this to make sure it is not larger than the file.

func (*Item) GetSize Uses

func (item *Item) GetSize() (size int64, err error)

GetSize gets the current size of the item

func (*Item) HasRange Uses

func (item *Item) HasRange(r ranges.Range) bool

HasRange returns true if the current ranges entirely include range

func (*Item) IsDirty Uses

func (item *Item) IsDirty() bool

IsDirty returns true if the item is dirty

func (*Item) Open Uses

func (item *Item) Open(o fs.Object) (err error)

Open the local file from the object passed in (which may be nil) which implies we are about to create the file

func (*Item) ReadAt Uses

func (item *Item) ReadAt(b []byte, off int64) (n int, err error)

ReadAt bytes from the file at off

func (*Item) Sync Uses

func (item *Item) Sync() (err error)

Sync commits the current contents of the file to stable storage. Typically, this means flushing the file system's in-memory copy of recently written data to disk.

func (*Item) Truncate Uses

func (item *Item) Truncate(size int64) (err error)

Truncate the item to the given size, creating it if necessary

If the new size is shorter than the existing size then the object will be shortened and marked as dirty.

If the new size is longer than the old size then the object will be extended and the extended data will be filled with zeros. The object will be marked as dirty in this case also.

func (*Item) WriteAt Uses

func (item *Item) WriteAt(b []byte, off int64) (n int, err error)

WriteAt bytes to the file at off

func (*Item) WriteAtNoOverwrite Uses

func (item *Item) WriteAtNoOverwrite(b []byte, off int64) (n int, skipped int, err error)

WriteAtNoOverwrite writes b to the file, but will not overwrite already present ranges.

This is used by the downloader to write bytes to the file

It returns n the total bytes processed and skipped the number of bytes which were processed but not actually written to the file.

type Items Uses

type Items []*Item

Items are a slice of *Item ordered by ATime

func (Items) Len Uses

func (v Items) Len() int

func (Items) Less Uses

func (v Items) Less(i, j int) bool

func (Items) Swap Uses

func (v Items) Swap(i, j int)

type StoreFn Uses

type StoreFn func(fs.Object)

StoreFn is called back with an object after it has been uploaded


writebackPackage writeback keeps track of the files which need to be written back to storage

Package vfscache imports 23 packages (graph). Updated 2020-08-03. Refresh now. Tools for package owners.