README
¶
Package cloudeng.io/file
import cloudeng.io/file
go:build darwin
Functions
Func ContextWithFS
func ContextWithFS(ctx context.Context, container fs.ReadFileFS) context.Context
ContextWithFS returns a new context that contains the provided instance of fs.ReadFileFS stored with as a valye within it.
Func FSFromContext
func FSFromContext(ctx context.Context) (fs.ReadFileFS, bool)
FSFromContext returns the fs.ReadFileFS instance, if any, stored within the context.
Func FSOpen
func FSOpen(ctx context.Context, name string) (fs.File, error)
FSOpen will open name using the context's fs.ReadFileFS instance if one is present, otherwise it will use os.Open.
Func FSReadFile
func FSReadFile(ctx context.Context, name string) ([]byte, error)
FSreadAll will read name using the context's fs.ReadFileFS instance if one is present, otherwise it will use os.ReadFile.
Types
Type FS
type FS interface {
fs.FS
// Scheme returns the URI scheme that this FS supports. Scheme should
// be "file" for local file system access.
Scheme() string
// OpenCtx is like fs.Open but with a context.
OpenCtx(ctx context.Context, name string) (fs.File, error)
// Readlink returns the contents of a symbolic link.
Readlink(ctx context.Context, path string) (string, error)
// Stat will follow symlinks/redirects/aliases.
Stat(ctx context.Context, path string) (Info, error)
// Lstat will not follow symlinks/redirects/aliases.
Lstat(ctx context.Context, path string) (Info, error)
// Join is like filepath.Join for the filesystem supported by this filesystem.
Join(components ...string) string
// Base is like filepath.Base for the filesystem supported by this filesystem.
Base(path string) string
// IsPermissionError returns true if the specified error, as returned
// by the filesystem's implementation, is a result of a permissions error.
IsPermissionError(err error) bool
// IsNotExist returns true if the specified error, as returned by the
// filesystem's implementation, is a result of the object not existing.
IsNotExist(err error) bool
// XAttr returns extended attributes for the specified file.Info
// and file.
XAttr(ctx context.Context, path string, fi Info) (XAttr, error)
// SysXAttr returns a representation of the extended attributes using the
// native data type of the underlying file system. If existing is
// non-nil and is of that file-system specific type the contents of
// XAttr are merged into it.
SysXAttr(existing any, merge XAttr) any
}
FS extends fs.FS with Scheme and OpenCtx.
Functions
func LocalFS() FS
LocalFS returns an instance of file.FS that provides access to the local filesystem.
Type FSFactory
type FSFactory interface {
New(ctx context.Context, scheme string) (FS, error)
NewFromMatch(ctx context.Context, m cloudpath.Match) (FS, error)
}
FSFactory is implemented by types that can create a file.FS for a given URI scheme or for a cloudpath.Match. New is used for the common case where an FS can be created for an entire filesystem instance, whereas NewMatch is intended for the case where more granular approach is required. The implementations of FSFactory will typically store the authentication credentials required to create the FS when New or NewMatch is called. For AWS S3 for example, the information required to create an aws.Config will be stored in used when New or NewMatch are called. New will create an FS for S3 in general, whereas NewMatch can take more specific action such as creating an FS for a specific bucket or region with different credentials.
Type Info
type Info struct {
// contains filtered or unexported fields
}
Info implements fs.FileInfo to provide binary, gob and json encoding/decoding. The SysInfo field is not encoded/decoded and hence is only available for use within the process that Info was instantiated in.
Functions
func NewInfo(
name string,
size int64,
mode fs.FileMode,
modTime time.Time,
sysInfo any) Info
NewInfo creates a new instance of Info.
func NewInfoFromFileInfo(fi fs.FileInfo) Info
NewInfoFromFileInfo creates a new instance of Info from a fs.FileInfo.
Methods
func (fi *Info) AppendBinary(buf *bytes.Buffer) error
AppendBinary appends a binary encoded instance of Info to the supplied buffer.
func (fi *Info) DecodeBinary(data []byte) ([]byte, error)
DecodeBinary decodes the supplied data into the receiver and returns the remaining data.
func (fi Info) IsDir() bool
IsDir implements fs.FileInfo.
func (fi Info) MarshalBinary() ([]byte, error)
Implements encoding.BinaryMarshaler.
func (fi Info) MarshalJSON() ([]byte, error)
func (fi Info) ModTime() time.Time
ModTime implements fs.FileInfo.
func (fi Info) Mode() fs.FileMode
Mode implements fs.FileInfo.
func (fi Info) Name() string
Name implements fs.FileInfo.
func (fi *Info) SetSys(i any)
SetSys sets the SysInfo field. Note that the Sys field is never encoded/decoded.
func (fi Info) Size() int64
Size implements fs.FileInfo.
func (fi Info) Sys() any
Sys implements fs.FileInfo. Note that the Sys field is never encoded/decoded.
func (fi Info) Type() fs.FileMode
Type implements fs.Entry
func (fi *Info) UnmarshalBinary(data []byte) error
Implements encoding.BinaryUnmarshaler.
func (fi *Info) UnmarshalJSON(data []byte) error
Type InfoList
type InfoList []Info
InfoList represents a list of Info instances. It provides efficient encoding/decoding operations.
Functions
func DecodeBinaryInfoList(data []byte) (InfoList, []byte, error)
DecodeBinaryInfoList decodes the supplied data into an InfoList and returns the remaining data.
Methods
func (il InfoList) AppendBinary(buf *bytes.Buffer) error
AppendBinary appends a binary encoded instance of Info to the supplied byte slice.
func (il InfoList) AppendInfo(info Info) InfoList
Append appends an Info instance to the list and returns the updated list.
func (il InfoList) MarshalBinary() ([]byte, error)
MarshalBinary implements encoding.BinaryMarshaler.
func (il *InfoList) UnmarshalBinary(data []byte) (err error)
UnmarshalBinary implements encoding.BinaryUnmarshaler.
Type XAttr
type XAttr struct {
UID, GID int64 // -1 for non-posix filesystems that don't support numeric UID, GID
User, Group string // Used for systems that don't support numeric UID, GID
Device, FileID uint64
Blocks int64
Hardlinks uint64
}
XAttr represents extended information about a directory or file as obtained from the filesystem.
Methods
func (x XAttr) CompareGroup(o XAttr) bool
CompareGroup compares the GID fields if >=0 and the Group fields otherwise.
func (x XAttr) CompareUser(o XAttr) bool
CompareUser compares the UID fields if >=0 and the User fields otherwise.
Documentation
¶
Index ¶
- Variables
- func ContextWithFS(ctx context.Context, container ...fs.ReadFileFS) context.Context
- func FSFromContext(ctx context.Context) ([]fs.ReadFileFS, bool)
- func FSOpen(ctx context.Context, filename string) (fs.File, error)
- func FSReadFile(ctx context.Context, name string) ([]byte, error)
- type FS
- type Info
- func (fi *Info) AppendBinary(buf *bytes.Buffer) error
- func (fi *Info) DecodeBinary(data []byte) ([]byte, error)
- func (fi Info) IsDir() bool
- func (fi Info) MarshalBinary() ([]byte, error)
- func (fi Info) MarshalJSON() ([]byte, error)
- func (fi Info) ModTime() time.Time
- func (fi Info) Mode() fs.FileMode
- func (fi Info) Name() string
- func (fi *Info) SetSys(i any)
- func (fi Info) Size() int64
- func (fi Info) Sys() any
- func (fi Info) Type() fs.FileMode
- func (fi *Info) UnmarshalBinary(data []byte) error
- func (fi *Info) UnmarshalJSON(data []byte) error
- type InfoList
- type ObjectFS
- type XAttr
Constants ¶
This section is empty.
Variables ¶
var ErrNotImplemented = fmt.Errorf("not implemented")
ErrNotImplemented is returned by methods that are not implemented by a particular filesystem.
Functions ¶
func ContextWithFS ¶
ContextWithFS returns a new context that contains the provided instances of fs.ReadFileFS stored with as a value within it.
func FSFromContext ¶
func FSFromContext(ctx context.Context) ([]fs.ReadFileFS, bool)
FSFromContext returns the list of fs.ReadFileFS instancees, if any, stored within the context.
func FSOpen ¶
FSOpen will attempt to open filename using the context's set of fs.ReadFileFS instances (if any), in the order in which they were provided to ContextWithFS, returning the first successful result. If no fs.ReadFileFS instances are present in the context or none successfully open the file, then os.Open is used.
Types ¶
type FS ¶
type FS interface {
fs.FS
// Scheme returns the URI scheme that this FS supports. Scheme should
// be "file" for local file system access.
Scheme() string
// OpenCtx is like fs.Open but with a context.
OpenCtx(ctx context.Context, name string) (fs.File, error)
// Readlink returns the contents of a symbolic link.
Readlink(ctx context.Context, path string) (string, error)
// Stat will follow symlinks/redirects/aliases.
Stat(ctx context.Context, path string) (Info, error)
// Lstat will not follow symlinks/redirects/aliases.
Lstat(ctx context.Context, path string) (Info, error)
// Join is like filepath.Join for the filesystem supported by this filesystem.
Join(components ...string) string
// Base is like filepath.Base for the filesystem supported by this filesystem.
Base(path string) string
// IsPermissionError returns true if the specified error, as returned
// by the filesystem's implementation, is a result of a permissions error.
IsPermissionError(err error) bool
// IsNotExist returns true if the specified error, as returned by the
// filesystem's implementation, is a result of the object not existing.
IsNotExist(err error) bool
// XAttr returns extended attributes for the specified file.Info
// and file.
XAttr(ctx context.Context, path string, fi Info) (XAttr, error)
// SysXAttr returns a representation of the extended attributes using the
// native data type of the underlying file system. If existing is
// non-nil and is of that file-system specific type the contents of
// XAttr are merged into it.
SysXAttr(existing any, merge XAttr) any
}
FS extends fs.FS with Scheme and OpenCtx.
type Info ¶
type Info struct {
// contains filtered or unexported fields
}
Info implements fs.FileInfo to provide binary, gob and json encoding/decoding. The SysInfo field is not encoded/decoded and hence is only available for use within the process that Info was instantiated in.
func NewInfoFromFileInfo ¶
NewInfoFromFileInfo creates a new instance of Info from a fs.FileInfo.
func (*Info) AppendBinary ¶
AppendBinary appends a binary encoded instance of Info to the supplied buffer.
func (*Info) DecodeBinary ¶
DecodeBinary decodes the supplied data into the receiver and returns the remaining data.
func (Info) MarshalBinary ¶
Implements encoding.BinaryMarshaler.
func (Info) MarshalJSON ¶
func (*Info) SetSys ¶
SetSys sets the SysInfo field. Note that the Sys field is never encoded/decoded.
func (*Info) UnmarshalBinary ¶
Implements encoding.BinaryUnmarshaler.
func (*Info) UnmarshalJSON ¶
type InfoList ¶
type InfoList []Info
InfoList represents a list of Info instances. It provides efficient encoding/decoding operations.
func DecodeBinaryInfoList ¶
DecodeBinaryInfoList decodes the supplied data into an InfoList and returns the remaining data.
func (InfoList) AppendBinary ¶
AppendBinary appends a binary encoded instance of Info to the supplied byte slice.
func (InfoList) AppendInfo ¶
Append appends an Info instance to the list and returns the updated list.
func (InfoList) MarshalBinary ¶
MarshalBinary implements encoding.BinaryMarshaler.
func (*InfoList) UnmarshalBinary ¶
UnmarshalBinary implements encoding.BinaryUnmarshaler.
type ObjectFS ¶
type ObjectFS interface {
Get(ctx context.Context, path string) ([]byte, error)
Put(ctx context.Context, path string, perm fs.FileMode, data []byte) error
EnsurePrefix(ctx context.Context, path string, perm fs.FileMode) error
Delete(ctx context.Context, path string) error
// DeleteAll delets all objects with the specified prefix.
DeleteAll(ctx context.Context, prefix string) error
}
ObjectFS represents a writeable object store. It is intended to backed by cloud or local filesystems. The permissions may be ignored by some implementations.
type XAttr ¶
type XAttr struct {
UID, GID int64 // -1 for non-posix filesystems that don't support numeric UID, GID
User, Group string // Used for systems that don't support numeric UID, GID
Device, FileID uint64
Blocks int64
Hardlinks uint64
}
XAttr represents extended information about a directory or file as obtained from the filesystem.
func (XAttr) CompareGroup ¶
CompareGroup compares the GID fields if >=0 and the Group fields otherwise.
func (XAttr) CompareUser ¶
CompareUser compares the UID fields if >=0 and the User fields otherwise.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
Package checkpoint provides a mechanism for checkpointing the state of an ongoing operation.
|
Package checkpoint provides a mechanism for checkpointing the state of an ongoing operation. |
|
Package content provides support for working with different content types.
|
Package content provides support for working with different content types. |
|
processors
Package processor provides support for processing different content types.
|
Package processor provides support for processing different content types. |
|
Package crawl provides a framework for multilevel/recursive crawling files.
|
Package crawl provides a framework for multilevel/recursive crawling files. |
|
crawlcmd
Package crawlcmd provides support for building command line tools for crawling.
|
Package crawlcmd provides support for building command line tools for crawling. |
|
Package download provides a simple download mechanism that uses the fs.FS container API to implement the actual download.
|
Package download provides a simple download mechanism that uses the fs.FS container API to implement the actual download. |
|
Package filewalk provides support for concurrent traversal of file system directories and files.
|
Package filewalk provides support for concurrent traversal of file system directories and files. |
|
filewalktestutil
Package filewalktestutil provides utilities for testing code that uses filewalk.FS.
|
Package filewalktestutil provides utilities for testing code that uses filewalk.FS. |
|
go:build linux && amd64
|
go:build linux && amd64 |