memfs

package module
v0.0.0-...-25e7469 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 26, 2017 License: MIT Imports: 15 Imported by: 0

README

MemFS

Build Status Coverage Status Go Report Card GoDoc

In memory file system that implements as many FUSE interfaces as possible.

For more information see the godoc documentation.

Getting Started

Clone or download the repository into your Go path or use go get as follows:

$ go get github.com/bbengfort/memfs

Change your working directory to repository. Build and install the MemFS command line tool and add to your $PATH:

$ go install cmd/memfs.go

You should now be able to run the memfs command. To see the various command line options:

$ memfs --help

Finally to mount the MemFS in a directory called data in your home directory, with the reasonable defaults from the command line configuration:

$ memfs ~/data

The FileSystem should start running in the foreground with info log statements, and the mount point should appear in your OS.

Documentation

Overview

Package memfs implements an in-memory replicated file system with Bazil FUSE and whose primary purpose is to implement all FUSE interfaces.

Index

Constants

View Source 
const (
	JSONDateTime = "2006-01-02T15:04:05-07:00"
)

Formatters for representing the date and time as a string.

Variables

This section is empty.

Functions

func Blocks 

func Blocks(value uint64) uint64

Blocks returns the number of 512 byte blocks required

func ListContains 

func ListContains(value string, list []string) bool

ListContains searches a list for a particular value in O(n) time.

func MaxUInt64 

func MaxUInt64(values ...uint64) uint64

MaxUInt64 returns the maximal value of the list of passed in uints

func PackageVersion 

func PackageVersion() string

PackageVersion composes version information from the constants in this file and returns a string that defines current information about the package.

func Regularize 

func Regularize(value string) string

Regularize a string for comparison, e.g. make all lowercase and trim whitespace. This utility is often used on user input to compare to constant strings like database drivers or hashing algorithm names.

func Stride 

func Stride(s string, n int) []string

Stride returns an array of N length substrings.

func StrideFixed 

func StrideFixed(s string, n int) []string

StrideFixed returns an array of N length substrings and does not allow the last element to have a length < N (e.g. no remainders).

func WebLogger 

func WebLogger(log *Logger, inner http.Handler) http.Handler

WebLogger is a decorator for http handlers to record HTTP requests using the logger API and syntax, which must be passed in as the first argument.

Types

type Config 

type Config struct {
	Name      string     `json:"name"`      // Identifier for replica lists
	CacheSize uint64     `json:"cachesize"` // Maximum amount of memory used
	Level     string     `json:"level"`     // Minimum level to log at (debug, info, warn, error, critical)
	ReadOnly  bool       `json:"readonly"`  // Whether or not the FS is read only
	Replicas  []*Replica `json:"replicas"`  // List of remote replicas in system
	Path      string     `json:"-"`         // Path the config was loaded from
}

Config implements the local configuration directives.

func (*Config) Dump 

func (conf *Config) Dump(path string) error

Dump a configuration to JSON to the path on disk. If dump is an empty string then will dump the config to the path it was loaded from.

func (*Config) Load 

func (conf *Config) Load(path string) error

Load a configuration from a path on disk by deserializing the JSON data.

type Dir 

type Dir struct {
	Node
	Children map[string]Entity // Contents of the directory
}

Dir implements Node and Handler interfaces for directories and container entities in the file system. Most importantly it references its children.

func (*Dir) Create 

func (d *Dir) Create(ctx context.Context, req *fuse.CreateRequest, resp *fuse.CreateResponse) (fs.Node, fs.Handle, error)

Create creates and opens a new file in the receiver, which must be a Dir. NOTE: the interface docmentation says create a directory, but the docs for fuse.CreateRequest say create and open a file (not a directory).

https://godoc.org/bazil.org/fuse/fs#NodeCreater

func (*Dir) GetNode 

func (d *Dir) GetNode() *Node

GetNode returns a pointer to the embedded Node object

func (*Dir) Init 

func (d *Dir) Init(name string, mode os.FileMode, parent *Dir, memfs *FileSystem)

Init the directory with the required properties for the directory.

func (*Dir) Lookup 

func (d *Dir) Lookup(ctx context.Context, name string) (fs.Node, error)

Lookup looks up a specific entry in the receiver, which must be a directory. Lookup should return a Node corresponding to the entry. If the name does not exist in the directory, Lookup should return ENOENT.

Lookup need not to handle the names "." and "..".

https://godoc.org/bazil.org/fuse/fs#NodeStringLookuper NOTE: implemented NodeStringLookuper rather than NodeRequestLookuper https://godoc.org/bazil.org/fuse/fs#NodeRequestLookuper

func (*Dir) Mkdir 

func (d *Dir) Mkdir(ctx context.Context, req *fuse.MkdirRequest) (fs.Node, error)

Mkdir creates (but not opens) a directory in the given directory.

https://godoc.org/bazil.org/fuse/fs#NodeMkdirer

func (*Dir) ReadDirAll 

func (d *Dir) ReadDirAll(ctx context.Context) ([]fuse.Dirent, error)

ReadDirAll reads the entire directory contents and returns a list of fuse Dirent objects - which specify the internal contents of the directory.

https://godoc.org/bazil.org/fuse/fs#HandleReadDirAller

func (*Dir) Remove 

func (d *Dir) Remove(ctx context.Context, req *fuse.RemoveRequest) error

Remove removes the entry with the given name from the receiver, which must be a directory. The entry to be removed may correspond to a file (unlink) or to a directory (rmdir).

https://godoc.org/bazil.org/fuse/fs#NodeRemover

func (*Dir) Rename 

func (d *Dir) Rename(ctx context.Context, req *fuse.RenameRequest, newDir fs.Node) error

Rename a file in a directory. NOTE: There is no documentation on this. Implemented to move the entry by name from the dir to the newDir.

https://godoc.org/bazil.org/fuse/fs#NodeRenamer

type Entity 

type Entity interface {
	IsDir() bool               // Returns true if the entity is a directory
	IsArchive() bool           // Returns true if the entity is an archive (version history)
	FuseType() fuse.DirentType // Returns the fuse type for listing
	Path() string              // Returns the full path to the entity
	GetNode() *Node            // Returns the node for the entity type
}

Entity represents a memfs.Node entity (to differentiate it from a fs.Node)

type File 

type File struct {
	Node
	Data []byte // Actual data contained by the File
	// contains filtered or unexported fields
}

File implements Node and Handler interfaces for file (data containing) objects in MemFs. Data is allocated directly in the file object, and is not chunked or broken up until transport.

func (*File) Flush 

func (f *File) Flush(ctx context.Context, req *fuse.FlushRequest) error

Flush is called each time the file or directory is closed. Because there can be multiple file descriptors referring to a single opened file, Flush can be called multiple times.

Because this is an in-memory system, Flush is basically ignored.

https://godoc.org/bazil.org/fuse/fs#HandleFlusher

func (*File) Fsync 

func (f *File) Fsync(ctx context.Context, req *fuse.FsyncRequest) error

Fsync must be defined or edting with vim or emacs fails. Implements NodeFsyncer, which has no associated documentation.

https://godoc.org/bazil.org/fuse/fs#NodeFsyncer

func (*File) GetNode 

func (f *File) GetNode() *Node

GetNode returns a pointer to the embedded Node object

func (*File) Init 

func (f *File) Init(name string, mode os.FileMode, parent *Dir, memfs *FileSystem)

Init the file and create the data array

func (*File) Read 

func (f *File) Read(ctx context.Context, req *fuse.ReadRequest, resp *fuse.ReadResponse) error

Read requests to read data from the handle.

There is a page cache in the kernel that normally submits only page-aligned reads spanning one or more pages. However, you should not rely on this. To see individual requests as submitted by the file system clients, set OpenDirectIO.

NOTE: that reads beyond the size of the file as reported by Attr are not even attempted (except in OpenDirectIO mode).

https://godoc.org/bazil.org/fuse/fs#HandleReader

func (*File) Setattr 

func (f *File) Setattr(ctx context.Context, req *fuse.SetattrRequest, resp *fuse.SetattrResponse) error

Setattr sets the standard metadata for the receiver.

Note, this is also used to communicate changes in the size of the file, outside of Writes.

req.Valid is a bitmask of what fields are actually being set. For example, the method should not change the mode of the file unless req.Valid.Mode() is true.

https://godoc.org/bazil.org/fuse/fs#NodeSetattrer

func (*File) Write 

func (f *File) Write(ctx context.Context, req *fuse.WriteRequest, resp *fuse.WriteResponse) error

Write requests to write data into the handle at the given offset. Store the amount of data written in resp.Size.

There is a writeback page cache in the kernel that normally submits only page-aligned writes spanning one or more pages. However, you should not rely on this. To see individual requests as submitted by the file system clients, set OpenDirectIO.

Writes that grow the file are expected to update the file size (as seen through Attr). Note that file size changes are communicated also through Setattr.

https://godoc.org/bazil.org/fuse/fs#HandleWriter

type FileSystem 

type FileSystem struct {
	sync.Mutex                    // FileSystem can be locked and unlocked
	MountPoint string             // Path to the mount location on disk
	Config     *Config            // Configuration of the FileSystem
	Conn       *fuse.Conn         // Hook to the FUSE connection object
	Sequence   *sequence.Sequence // Monotonically increasing counter for inodes
	// contains filtered or unexported fields
}

FileSystem implements the fuse.FS* interfaces as well as providing a lockable interaction structure to ensure concurrent accesses succeed.

func New 

func New(mount string, config *Config) *FileSystem

New MemFS file system created from a mount path and a configuration. This is the entry point for creating and launching all in-memory file systems.

func (*FileSystem) Destroy 

func (mfs *FileSystem) Destroy()

Destroy is called when the file system is shutting down. Implements the fuse.FSDestroyer interface.

Linux only sends this request for block device backed (fuseblk) filesystems, to allow them to flush writes to disk before the unmount completes.

func (*FileSystem) GenerateInode 

func (mfs *FileSystem) GenerateInode(parentInode uint64, name string) uint64

GenerateInode is called to pick a dynamic inode number when it would otherwise be 0. Implements the fuse.FSInodeGenerator interface.

Not all filesystems bother tracking inodes, but FUSE requires the inode to be set, and fewer duplicates in general makes UNIX tools work better.

Operations where the nodes may return 0 inodes include Getattr, Setattr and ReadDir.

If FS does not implement FSInodeGenerator, GenerateDynamicInode is used.

Implementing this is useful to e.g. constrain the range of inode values used for dynamic inodes.

func (*FileSystem) Root 

func (mfs *FileSystem) Root() (fs.Node, error)

Root is called to obtain the Node for the file system root. Implements the fuse.FS interface required of all file systems.

func (*FileSystem) Run 

func (mfs *FileSystem) Run() error

Run the FileSystem, mounting the MountPoint and connecting to FUSE

func (*FileSystem) Shutdown 

func (mfs *FileSystem) Shutdown() error

Shutdown the FileSystem unmounting the MountPoint and disconnecting FUSE.

func (*FileSystem) Statfs 

func (mfs *FileSystem) Statfs(ctx context.Context, req *fuse.StatfsRequest, resp *fuse.StatfsResponse) error

Statfs is called to obtain file system metadata. Implements fuse.FSStatfser by writing the metadata to the resp.

type LogLevel 

type LogLevel int

LogLevel characterizes the severity of the log message. 

const (
	LevelDebug LogLevel = 1 + iota
	LevelInfo
	LevelWarn
	LevelError
	LevelFatal
)

Severity levels of log messages.

func LevelFromString 

func LevelFromString(level string) LogLevel

LevelFromString parses a string and returns the LogLevel

func (LogLevel) String 

func (level LogLevel) String() string

String representation of the log level.

type Logger 

type Logger struct {
	Level LogLevel // The minimum severity to log to
	// contains filtered or unexported fields
}

Logger wraps the log.Logger to write to a file on demand and to specify a miminum severity that is allowed for writing.

func InitLogger 

func InitLogger(path string, level string) (*Logger, error)

InitLogger creates a Logger object by passing a configuration that contains the minimum log level and an optional path to write the log out to.

func (*Logger) Close 

func (logger *Logger) Close() error

Close the logger and any open file handles.

func (*Logger) Debug 

func (logger *Logger) Debug(msg string, args ...interface{})

Debug message helper function

func (*Logger) Error 

func (logger *Logger) Error(msg string, args ...interface{})

Error message helper function

func (*Logger) Fatal 

func (logger *Logger) Fatal(msg string, args ...interface{})

Fatal message helper function

func (*Logger) GetHandler 

func (logger *Logger) GetHandler() io.Writer

GetHandler returns the io.Writer object that is on the logger.

func (*Logger) Info 

func (logger *Logger) Info(msg string, args ...interface{})

Info message helper function

func (*Logger) Log 

func (logger *Logger) Log(layout string, level LogLevel, args ...interface{})

Log a message at the appropriate severity. The Log method behaves as a format function, and a layout string can be passed with arguments. The current logging format is "%(level)s [%(jsontime)s]: %(message)s"

func (*Logger) SetHandler 

func (logger *Logger) SetHandler(writer io.WriteCloser)

SetHandler sets a new io.WriteCloser object onto the logger

func (*Logger) Warn 

func (logger *Logger) Warn(msg string, args ...interface{})

Warn message helper function

type Node 

type Node struct {
	ID     uint64    // Unique ID of the Node
	Name   string    // Name of the Node
	Attrs  fuse.Attr // Node attributes and permissions
	XAttrs XAttr     // Extended attributes on the node
	Parent *Dir      // Parent directory of the Node
	// contains filtered or unexported fields
}

Node contains shared data and structures for both files and directories. Methods returning Node should take care to return the same Node when the result is logically the same instance. Without this, each Node will get a new NodeID, causing spurious cache invalidations, extra lookups and aliasing anomalies. This may not matter for a simple, read-only filesystem.

func (*Node) Access 

func (n *Node) Access(ctx context.Context, req *fuse.AccessRequest) error

Access checks whether the calling context has permission for the given operations on the receiver. If so, Access should return nil. If not, Access should return EPERM.

Note that this call affects the result of the access(2) system call but not the open(2) system call. If Access is not implemented, the Node behaves as if it always returns nil (permission granted), relying on checks in Open instead.

https://godoc.org/bazil.org/fuse/fs#NodeAccesser

func (*Node) Attr 

func (n *Node) Attr(ctx context.Context, attr *fuse.Attr) error

Attr fills attr with the standard metadata for the node.

Fields with reasonable defaults are prepopulated. For example, all times are set to a fixed moment when the program started.

If Inode is left as 0, a dynamic inode number is chosen.

The result may be cached for the duration set in Valid.

https://godoc.org/bazil.org/fuse/fs#Node

func (*Node) Forget 

func (n *Node) Forget()

Forget about this node. This node will not receive further method calls.

Forget is not necessarily seen on unmount, as all nodes are implicitly forgotten as part part of the unmount.

https://godoc.org/bazil.org/fuse/fs#NodeForgetter

Currently forget does nothing except log that it was forgotten.

func (*Node) FuseType 

func (n *Node) FuseType() fuse.DirentType

FuseType returns the fuse type of the node for listing

func (*Node) GetNode 

func (n *Node) GetNode() *Node

GetNode returns a pointer to the embedded Node object

func (*Node) Getattr 

func (n *Node) Getattr(ctx context.Context, req *fuse.GetattrRequest, resp *fuse.GetattrResponse) error

Getattr obtains the standard metadata for the receiver. It should store that metadata in resp.

If this method is not implemented, the attributes will be generated based on Attr(), with zero values filled in.

https://godoc.org/bazil.org/fuse/fs#NodeGetattrer

func (*Node) Getxattr 

func (n *Node) Getxattr(ctx context.Context, req *fuse.GetxattrRequest, resp *fuse.GetxattrResponse) error

Getxattr gets an extended attribute by the given name from the node.

If there is no xattr by that name, returns fuse.ErrNoXattr.

https://godoc.org/bazil.org/fuse/fs#NodeGetxattrer

func (*Node) Init 

func (n *Node) Init(name string, mode os.FileMode, parent *Dir, fs *FileSystem)

Init a Node with the required properties for storage in the file system.

func (*Node) IsArchive 

func (n *Node) IsArchive() bool

IsArchive returns true if the node is an archive node, that is a node constructed to display version history (and is therefore not writeable). TODO: Implement archives

func (*Node) IsDir 

func (n *Node) IsDir() bool

IsDir returns if the node is a directory by inspecting the file mode.

func (*Node) Listxattr 

func (n *Node) Listxattr(ctx context.Context, req *fuse.ListxattrRequest, resp *fuse.ListxattrResponse) error

Listxattr lists the extended attributes recorded for the node.

https://godoc.org/bazil.org/fuse/fs#NodeListxattrer

func (*Node) Path 

func (n *Node) Path() string

Path returns a string representation of the full path of the node.

func (*Node) Removexattr 

func (n *Node) Removexattr(ctx context.Context, req *fuse.RemovexattrRequest) error

Removexattr removes an extended attribute for the name.

If there is no xattr by that name, returns fuse.ErrNoXattr.

https://godoc.org/bazil.org/fuse/fs#NodeRemovexattrer

func (*Node) Setattr 

func (n *Node) Setattr(ctx context.Context, req *fuse.SetattrRequest, resp *fuse.SetattrResponse) error

Setattr sets the standard metadata for the receiver.

Note, this is also used to communicate changes in the size of the file, outside of Writes.

req.Valid is a bitmask of what fields are actually being set. For example, the method should not change the mode of the file unless req.Valid.Mode() is true.

https://godoc.org/bazil.org/fuse/fs#NodeSetattrer

func (*Node) Setxattr 

func (n *Node) Setxattr(ctx context.Context, req *fuse.SetxattrRequest) error

Setxattr sets an extended attribute with the given name and value. TODO: Use flags to fail the request if the xattr does/not already exist.

https://godoc.org/bazil.org/fuse/fs#NodeSetxattrer

func (*Node) String 

func (n *Node) String() string

String returns the full path to the node.

type Replica 

type Replica struct {
	PID  uint   `json:"pid"`  // Precedence ID for the replica
	Name string `json:"name"` // Name of the replica
	Host string `json:"host"` // IP address or hostname of the replica
	Port int    `json:"port"` // Port the replica is listening on
}

Replica implements the definition for a remote replica connections.

type XAttr 

type XAttr map[string][]byte

XAttr is a mapping of names to binary data for file systems that support extended attributes or other data.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.