fsnode

package
v0.0.11 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 7, 2024 License: Apache-2.0 Imports: 10 Imported by: 0

Documentation

Overview

fsnode represents a filesystem as a directed graph (probably a tree for many implementations). Directories are nodes with out edges (children). Files are nodes without.

fsnode.T is designed for incremental iteration. Callers can step through the graph one link at a time (Parent.Child) or one level at a time (Parent.Children). In general, implementations should do incremental work for each step. See also: Cacheable.

Compared to fs.FS:

  • Leaf explicitly models an unopened file. fs users have to choose their own representation, like the pair (fs.FS, name string) or func Open(...).
  • Graph traversal (that is, directory listing) uses the one node type, rather than a separate one (like fs.DirEntry). Callers can access "all cheaply available FileInfo" during listing or can Open nodes if they want completeness at higher cost.
  • Parent offers one, explicit way of traversing the graph. fs.FS has optional ReadDirFS or callers can Open(".") and see if ReadDirFile is returned. (fs.ReadDir unifies these but also disallows pagination).
  • Only supports directories and files. fs.FS supports much more. TODO: Add symlinks?
  • fs.FS.Open naturally allows "jumping" several levels deep without step-by-step traversal. (See Parent.Child) for performance note.

Index

Constants

View Source
const CacheForever = time.Duration(-1)

Variables

This section is empty.

Functions

func CacheableFor

func CacheableFor(obj interface{}) time.Duration

CacheableFor returns the configured cache time if obj is Cacheable, otherwise returns default 0.

func IterateFull

func IterateFull(ctx context.Context, iter Iterator, dst []T) (int, error)

IterateFull reads the full len(dst) nodes from Iterator. If actual number read is less than len(dst), error is non-nil. Error is io.EOF for EOF. Unlike io.ReadFull, this doesn't return io.ErrUnexpectedEOF (unless iter does).

func Open

func Open(ctx context.Context, n Leaf) (fsctx.File, error)

Open opens the file of a leaf in (the commonly desired) read-only mode.

Types

type Cacheable

type Cacheable interface {
	// CacheableFor is the maximum allowed cache time.
	// Zero means don't cache. Negative means cache forever.
	// TODO: Make this a non-Duration type to avoid confusion with negatives?
	CacheableFor() time.Duration
}

Cacheable optionally lets users make use of caching. The cacheable data depends on which type Cacheable is defined on:

  • On any T, FileInfo.
  • On an fsctx.File, the FileInfo and contents.

Common T implementations are expected to be "views" of remote data sources not under our exclusive control (like local filesystem or S3). As such, callers should generally expect best-effort consistency, regardless of caching.

func NewCacheable

func NewCacheable(d time.Duration) Cacheable

type ChildrenGenerator

type ChildrenGenerator interface {
	GenerateChildren(context.Context) ([]T, error)
}

ChildrenGenerator generates child nodes.

func ConstChildren

func ConstChildren(children ...T) ChildrenGenerator

ConstChildren constructs a ChildrenGenerator that always returns the given children.

func FuncChildren

func FuncChildren(fn func(context.Context) ([]T, error)) ChildrenGenerator

FuncChildren constructs a ChildrenGenerator that simply invokes fn, for convenience.

type FileInfo

type FileInfo struct {
	// contains filtered or unexported fields
}

FileInfo implements os.FileInfo. Instances are immutable but convenient copy-and-set methods are provided for some fields. FileInfo implements (T).Info, so implementations of T can conveniently embed a FileInfo for simple cases, e.g. if the information is immutable.

func CopyFileInfo

func CopyFileInfo(info os.FileInfo) FileInfo

CopyFileInfo constructs FileInfo with the same public fields as info. It copies cacheability if available.

func NewDirInfo

func NewDirInfo(name string) FileInfo

NewDirInfo constructs FileInfo for a directory. Default ModePerm is 0555 (r-xr-xr-x). Other defaults are zero.

func NewRegInfo

func NewRegInfo(name string) FileInfo

NewRegInfo constructs FileInfo for a regular file. Default ModePerm is 0444 (r--r--r--). Other defaults are zero.

func NewSymlinkInfo

func NewSymlinkInfo(name string) FileInfo

NewSymlinkInfo constructs FileInfo for a symlink.

Create a symlink by using this FileInfo with a Leaf whose contents are the target path. The path may be relative or absolute.

func (FileInfo) CacheableFor

func (f FileInfo) CacheableFor() time.Duration

func (FileInfo) Equal

func (f FileInfo) Equal(g FileInfo) bool

func (FileInfo) Info

func (fi FileInfo) Info() os.FileInfo

Info implements (T).Info.

func (FileInfo) IsDir

func (f FileInfo) IsDir() bool

func (FileInfo) ModTime

func (f FileInfo) ModTime() time.Time

func (FileInfo) Mode

func (f FileInfo) Mode() os.FileMode

func (FileInfo) Name

func (f FileInfo) Name() string

func (FileInfo) Size

func (f FileInfo) Size() int64

func (FileInfo) Sys

func (f FileInfo) Sys() interface{}

func (FileInfo) WithCacheableFor

func (f FileInfo) WithCacheableFor(d time.Duration) FileInfo

func (FileInfo) WithModTime

func (f FileInfo) WithModTime(mod time.Time) FileInfo

func (FileInfo) WithModePerm

func (f FileInfo) WithModePerm(perm os.FileMode) FileInfo

func (FileInfo) WithModeType

func (f FileInfo) WithModeType(modeType os.FileMode) FileInfo

func (FileInfo) WithName

func (f FileInfo) WithName(name string) FileInfo

func (FileInfo) WithSize

func (f FileInfo) WithSize(size int64) FileInfo

func (FileInfo) WithSys

func (f FileInfo) WithSys(sys interface{}) FileInfo

type Iterator

type Iterator interface {
	// Next gets the next node. Must return (nil, io.EOF) at the end, not (non-nil, io.EOF).
	Next(context.Context) (T, error)
	// Close frees resources.
	Close(context.Context) error
}

Iterator yields child nodes iteratively.

Users must serialize their own method calls. No calls can be made after Close(). TODO: Do we need Stat here, maybe to update directory mode?

func MapIterator

func MapIterator(iter Iterator, fn func(context.Context, T) (T, error)) Iterator

MapIterator returns an Iterator that applies fn to each T yielded by iter.

func NewConcatIterator

func NewConcatIterator(iterators ...Iterator) Iterator

NewConcatIterator returns the elements of the given iterators in order, reading each until EOF. Manages calling Close on constituents (as it goes along and upon its own Close).

func NewIterator

func NewIterator(nodes ...T) Iterator

NewIterator returns an iterator that yields the given nodes.

func NewLazyIterator

func NewLazyIterator(make func(context.Context) ([]T, error)) Iterator

NewLazyIterator uses the given make function upon the first call to Next to make the nodes that it yields.

type Leaf

type Leaf interface {
	// T is implementation of common node operations. The FileInfo returned
	// by T.Info must be consistent with a regular file (mode and !IsDir).
	T
	// OpenFile opens the file. File.Stat()'s result must be consistent
	// with T.Info. flag holds the flag bits, specified the same as those
	// passed to os.OpenFile.See os.O_*.
	OpenFile(ctx context.Context, flag int) (fsctx.File, error)
}

Leaf is a T corresponding to a fsctx.File. It can be opened any number of times and must allow concurrent calls (it may lock internally if necessary).

func ConstLeaf

func ConstLeaf(info FileInfo, content []byte) Leaf

ConstLeaf constructs a leaf with constant contents. Caller must not modify content after call. Uses content's size (ignoring existing info.Size).

func FuncLeaf

func FuncLeaf(info FileInfo, open func(ctx context.Context, flag int) (fsctx.File, error)) Leaf

FuncLeaf constructs a Leaf from an open function. It's invoked every time; implementations should do their own caching if desired.

func ReaderAtLeaf

func ReaderAtLeaf(info FileInfo, r ioctx.ReaderAt) Leaf

ReaderAtLeaf constructs a Leaf whose file reads from r. Cacheability of both metadata and content is governed by info.

type Parent

type Parent interface {
	// T.Info must be consistent with directory (mode and IsDir).
	T
	// Child returns the named child. Returns nil, os.ErrNotExist if no such child exists.
	// name is not a path and must not contain '/'. It must satisfy fs.ValidPath, too.
	//
	// In some implementations, Child lookup may be relatively expensive and implementations
	// may want to reduce the cost of accessing a deeply-nested node. They may make all Child()
	// requests succeed immediately and then return path errors from listing or Leaf.Open
	// operations for the earlier path segment.
	Child(_ context.Context, name string) (T, error)
	// Children returns an iterator that can list all children.
	// Children takes no Context; it's expected to simply construct a view and return errors to
	// callers when they choose an Iterator operation.
	Children() Iterator
	// AddChildLeaf adds a child leaf to this parent, returning the new
	// leaf and an open file for the leaf's contents. The behavior of name
	// collisions may vary by implementation. It may be convenient to embed
	// ParentReadOnly if your Parent implementation is read-only.
	// TODO: Include mode?
	AddChildLeaf(_ context.Context, name string, flags uint32) (Leaf, fsctx.File, error)
	// AddChildParent adds a child parent to this parent, returning the new
	// parent. The behavior of name collisions may vary by implementation.
	// It may be convenient to embed ParentReadOnly if your Parent
	// implementation is read-only.
	AddChildParent(_ context.Context, name string) (Parent, error)
	// RemoveChild removes a child. It may be convenient to embed
	// ParentReadOnly if your Parent implementation is read-only.
	RemoveChild(_ context.Context, name string) error
}

Parent is a T that has zero or more child Ts.

func NewParent

func NewParent(info FileInfo, gen ChildrenGenerator) Parent

NewParent returns a Parent whose children are defined by gen. gen.GenerateChildren is called on every Parent query (including Child, which returns one result). Implementers should cache internally if necessary.

type ParentReadOnly

type ParentReadOnly struct{}

ParentReadOnly is a partial implementation of Parent interface functions that returns NotSupported errors for all write operations. It may be convenient to embed if your Parent implementation is read-only.

type MyParent struct {
	fsnode.ParentReadOnly
}

func (MyParent) ChildChild(context.Context, string) (T, error) { ... }
func (MyParent) Children() Iterator { ... }
// No need to implement write functions.

func (ParentReadOnly) AddChildLeaf

func (ParentReadOnly) AddChildParent

func (ParentReadOnly) AddChildParent(context.Context, string) (Parent, error)

func (ParentReadOnly) RemoveChild

func (ParentReadOnly) RemoveChild(context.Context, string) error

type T

type T interface {
	// Info provides immediately-available information. A subset of fields must be accurate:
	//   Name
	//   Mode&os.ModeType
	//   IsDir
	// The rest can be zero values if they're not immediately available.
	// Implementations may find FileInfo (in this package) convenient for embedding or use
	// in public API.
	//
	// Leaf.Open().Stat() gets complete information. That returned FileInfo must have the same
	// values for the fields listed above. The others can change if better information is
	// available.
	// TODO: Specify something about Info()'s return changing after a Stat call?
	Info() os.FileInfo
	// FSNodeT distinguishes T from os.FileInfo. It does nothing.
	FSNodeT()
}

T is a Parent or Leaf. A T that is not either of those is invalid.

func IterateAll

func IterateAll(ctx context.Context, iter Iterator) ([]T, error)

IterateAll reads iter until EOF. Returns nil error on success, not io.EOF (like io.ReadAll).

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL