README
¶
Generalized file system access for golang projects
filesystem mostly offers an abstract API which can be used to adapt actual file implementations to. filesystem provides a small common denominator of access routines for file systems which will be sufficient for most basic applications.
Actual implementations of file systems will be provided in separate projects and can be added in a plugin-like model.
Using the abstraction API
Actual implementations of file system adapters must be loaded before any of the functions can be used. Simple file systems can be registered by just loading the module (as an anonymous import), more complex file systems may require invoking initialization functions for authenticating or pointing to some kind of frontend.
The abstraction API is mostly contained in the directly exported functions in the support.go module, such as OpenReader, OpenWriter, ListEntries, etc. They should be relatively well documented and self-explanatory.
The read/write API mostly uses internal ReadCloser/WriteCloser abstractions instead of the regular io counterparts to preserve operation contexts, which are (supposed to be) a major part of the idea of this API. If you need to use regular io.ReadCloser/io.WriteCloser operations on these (ignoring deadlines etc.), use ToIoReadCloser()/ToIoWriteCloser() to get the corresponding representations.
Implementing file system adapters
If you want to implement adapters for existing file system APIs, simply implement all methods of the FileSystem object in support.go in your file system object and call AddImplementation() on the resulting object. It is entirely reasonable to only call AddImplementation() in some kind of initialization routine which handles authentication or some other kind of configuration; if your file system does not need such a thing, just call it from init().
Please keep all the intended semantics of the context APIs intact as much as possible. This would mean, for example, that a Read() which exceeds its deadline should probably attempt to restore the previous position pointer in the file to the place it was at before. A Write() should probably truncate the file to its previous length upon exceeding deadlines or cancellation.
Ultimately, expectations of the API users should be kept in mind and used as a guideline to make this API most useful.
Documentation
¶
Overview ¶
Package filesystem provides an abstraction API for hooking up various file system implementations and use them more or less transparently through URLs and a common API.
In this API, the scheme part of the URL (see net/url for a description of what makes up an URL) is used to find the correct file system implementation. The path part should be used to point to the correct file, while the query part may be used to supply additional metadata for the operation.
Unlike the regular go io implementation, an effort is made by this API to provide context functionality (deadlines, cancellation) in all relevant functions. Implementations of file system adapters should make a point of providing these semantics in a reasonable way; refer to the information in README.md for more details.
Index ¶
- Variables
- func AddImplementation(scheme string, fs FileSystem)
- func HasImplementation(scheme string) bool
- func ListEntries(ctx context.Context, dirurl *url.URL) ([]string, error)
- func Remove(ctx context.Context, fileurl *url.URL) error
- func ToIoReadCloser(rc ReadCloser) io.ReadCloser
- func ToIoWriteCloser(wc WriteCloser) io.WriteCloser
- type CancelWatchFunc
- type FileSystem
- type FileWatchFunc
- type LimitedReadCloser
- type ReadCloser
- type Seeker
- type WriteCloser
Constants ¶
This section is empty.
Variables ¶
var ENOFS = errors.New("No file system loaded for URL type")
ENOFS is the error which should only ever be returned directly from this framework indicating that no file system is registered under that name.
var EUNSUPP = errors.New("File system does not support this operation")
EUNSUPP is the error returned when an operation is called on a file system which does not have the requested functionality.
Functions ¶
func AddImplementation ¶
func AddImplementation(scheme string, fs FileSystem)
AddImplementation is used on initialization of individual file system modules to sign file systems up for receiving calls through the API. Any calls to URLs with the given scheme will be patched through to the file system implementation.
Subsequent invocations of AddImplementation will cause the association to be overwritten.
This function may be called from init() for easy file systems, or may require a more involved setup procedure for file systems talking to a server node and/or requiring authentication.
func HasImplementation ¶
HasImplementation determines whether a handler is registered for the given schema. Returns true if an implementation was registered for the specified scheme.
func ListEntries ¶
ListEntries retrieves a list of relative object names beneath the specified URL. Objects may be something resembling to files or directories and will not contain special entries such as the local and parent directory.
func Remove ¶
Remove deletes the referenced object from the underlying file system. Removal is guaranteed to succeed if no error returns, otherwise it may or may not have succeeded.
func ToIoReadCloser ¶
func ToIoReadCloser(rc ReadCloser) io.ReadCloser
ToIoReadCloser creates a context-ignorant object for providing an io.ReadCloser compatible API.
func ToIoWriteCloser ¶
func ToIoWriteCloser(wc WriteCloser) io.WriteCloser
ToIoWriteCloser creates a context-ignorant object for providing a io.WriteCloser compatible API.
Types ¶
type CancelWatchFunc ¶
type CancelWatchFunc func() error
CancelWatchFunc describes a function to call to stop watching a file.
func WatchFile ¶
func WatchFile(ctx context.Context, fileurl *url.URL, watcher FileWatchFunc) ( CancelWatchFunc, chan error, error)
WatchFile waits for modifications of the file at the specified URL and invokes the watcher with any modified files. Some implementations may allow watching directories.
type FileSystem ¶
type FileSystem interface {
// Open the specified file for reading. Context should be used to control
// the opening, not the reading part of the operation.
OpenReader(context.Context, *url.URL) (ReadCloser, error)
// Open the specified file for writing. The affected file should be
// overwritten. Context should be used to control the opening, not the
// writing part of the operation.
OpenWriter(context.Context, *url.URL) (WriteCloser, error)
// Open the specified file for appending. The affected file should be
// created if it did not exist, but no contents should be overwritten.
// Context should be used to control the opening, not the writing part
// of the operation.
OpenAppender(context.Context, *url.URL) (WriteCloser, error)
// Retrieve a list of entries described by the URL, if the file system
// has such a notion. Should return the relative file names, without
// ., .. or whatever their equivalent is.
ListEntries(context.Context, *url.URL) ([]string, error)
// Watch for changes in a given file and call the FileWatchFunc on every
// change to the watched file. Watching anything other than files is left
// as an implementation detail.
WatchFile(context.Context, *url.URL, FileWatchFunc) (CancelWatchFunc, chan error, error)
// Delete the specified file. Failures may or may not leave the file
// existing.
Remove(context.Context, *url.URL) error
}
FileSystem provides the expected minimal interface required from a file system implementation.
File systems should support at least a subset of these functions; any functions which are unsupported must return an EUNSUPP error. Deadlines and cancellations should be supported by either the underlying file system or implemented as reasonably as possible in the adapter.
func GetImplementation ¶
func GetImplementation(fileurl *url.URL) FileSystem
GetImplementation fetches a pointer to the entire implementation of the file system which would be used to handle the URL. If no file system can handle the URL, this returns nil.
Usually you will want to use one of the more specific functions.
type FileWatchFunc ¶
type FileWatchFunc func(*url.URL, ReadCloser)
FileWatchFunc describes the format of a function which can be used to watch for changes in a file in a supported file system. The function will be called when a watched file is modified. The first parameter will be the path of the file being changed, and the second parameter is a ReadCloser which can be used to access the modified contents.
Implementations must allow for the ReadCloser to be discarded without ever calling Read().
type LimitedReadCloser ¶
type LimitedReadCloser struct {
/* Underlying reader object */
R ReadCloser
/* Maximum length to read from the reader before feigning EOF. */
N int64
}
LimitedReadCloser is the equivalent to io.LimitedReader for the filesystem API.
type ReadCloser ¶
type ReadCloser interface {
Read(context.Context, []byte) (int, error)
Close(context.Context) error
}
ReadCloser is a context-aware variant of the good old io.ReadCloser.
func OpenReader ¶
OpenReader opens the referenced file and returns a ReadCloser object which can be used to access the files contents.
type Seeker ¶
type Seeker interface {
Tell(context.Context) (int64, error)
Seek(context.Context, int64, int) (int64, error)
}
Seeker is a context-aware variant of the good old io.Seeker.
type WriteCloser ¶
type WriteCloser interface {
Write(context.Context, []byte) (n int, err error)
Close(context.Context) error
}
WriteCloser is a context-aware variant of the good old io.WriteCloser.
func OpenAppender ¶
OpenAppender opens the referenced file and returns a WriteCloser object which can be used to append data to the file. If the file does not exist, it will be created.
Implementations may require Close() to be invoked before any changes are made whatsoever.
func OpenWriter ¶
OpenWriter opens the referenced file and returns a WriteCloser object which can be used to put data into the file. Any previous file contents will be overwritten.
Implementations may require Close() to be invoked before any changes are made whatsoever.
Source Files