README
¶
govfs
A Virtual Filesystem Library written in the Go programming language. Please note that this project is still under heavy development.
Synopsis
The govfs Virtual Filesystem is a ext3/4 Unix-based heirarchy type filesystem which begins from the root folder "/". Each file is represented by a meta header which may be referenced in O(1) time, i.e. this filesystem does not use a linked list to create a filesystem "tree", but rather a hash-table which allows for the quick reference of files.
In addition, govfs makes use of golang's gothreads by implementing very quick read/writes. I.E. there is a thread dispatched for each write operation (assuming it is a different file). So writing 1000 files concurrently is possible without breaking the filesystem.
Since this is an in-memory file system, upon unmount the filesystem data is stored in a serialized stream. This, too, is concurrent. So for each file that contains data, a thread will be spawned that will perform compression on the file and serialization of the meta data.
Features
- 2^128 files
- No file size limit
- Concurrent reads/writes
- O(1) reference time for finding a file header
- Can print out file lists
- File metadata serialization upon unmount (i.e. writing the entire filesystem to a physical file)
- The filesystem file is OS independent
- Creating one file will automatically create each subdirectory
- Ideal for projects with heavy filesystem utilization
- Compression and encryption of the raw filesystem file (unmounted file) is available
API
Main Filesystem Header
The FSHeader structure contains all data related to the virtual filesystem
type FSHeader struct {
filename string /* The raw file on disk */
key [16]byte /* RC4 key used to encrypt/decrypt the raw file */
meta map[string]*gofs_file /* Hash table containing each file header */
t_size uint /* Total size of all files */
[... Other structures/members omitted ...]
}
Create/Load Database
func CreateDatabase(name string, flags int) *FSHeader
Create New File
func (f *FSHeader) Create(name string) (*gofs_file, error)
Create I/O Reader
func (f *FSHeader) NewReader(name string) (*Reader, error)
Standard Read()
func (f *Reader) Read(p []byte) (int, error)
I/O Reader
func (f *FSHeader) Read(name string) ([]byte, error)
Delete File
func (f *FSHeader) Delete(name string) error
Write to a file
func (f *FSHeader) Write(name string, d []byte) error
Writer interface
type Writer struct {
Name string
File *gofs_file
Hdr *FSHeader
}
New Writer method
func (f *FSHeader) NewWriter(name string) (*Writer, error)
Write method
func (f *Writer) Write(p []byte) (int, error)
Disclaimer
Please see the LICENSE file for the detailed MIT license.
All work written by Stan Ruzin stan [dot] ruzin [at] gmail [dot] com
Documentation
¶
Index ¶
- Constants
- type FSHeader
- func (f *FSHeader) Check(name string) bool
- func (f *FSHeader) Commit() (*FSHeader, error)
- func (f *FSHeader) Create(name string) error
- func (f *FSHeader) Delete(name string) error
- func (f *FSHeader) GetFileCount() uint
- func (f *FSHeader) GetFileList() []string
- func (f *FSHeader) GetFileListDirectory(dir string) ([]string, error)
- func (f *FSHeader) GetFileSize(name string) (uint, error)
- func (f *FSHeader) GetTotalFilesizes() int
- func (f *FSHeader) NewReader(name string) (*Reader, error)
- func (f *FSHeader) NewWriter(name string) (*Writer, error)
- func (f *FSHeader) Read(name string) ([]byte, error)
- func (f *FSHeader) StartIOController() error
- func (f *FSHeader) UnmountDB(flags FlagVal) error
- func (f *FSHeader) Write(name string, d []byte) error
- type FlagVal
- type RawFile
- type Reader
- type Writer
Constants ¶
const FS_SIGNATURE string = "govfs_header" /* Cannot exceed 64 */
const MAX_FILENAME_LENGTH int = 256
* Configurable constants
const REMOVE_FS_HEADER bool = false /* Removes the header at the beginning of the serialized file - leave false */
const STREAM_PAD_LEN int = 0 /* Length of the pad between two serialized RawFile structs */
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type FSHeader ¶
type FSHeader struct {
// contains filtered or unexported fields
}
func CreateDatabase ¶
* Creates or loads a filesystem database file. If the filename is nil, then create a new database * otherwise try to load an existing fs database file. * * Flags: FLAG_ENCRYPT, FLAG_COMPRESS
func (*FSHeader) GetFileCount ¶
func (*FSHeader) GetFileList ¶
func (*FSHeader) GetFileListDirectory ¶
* Retrieves the file listing in a specific directrory. NOTE: The * `dir` parameter must contain a trailing "/"
func (*FSHeader) GetTotalFilesizes ¶
func (*FSHeader) StartIOController ¶
type FlagVal ¶
type FlagVal int
const ( FLAG_FILE FlagVal = 1 << iota FLAG_DIRECTORY /* The target file is a directory */ FLAG_ENCRYPT /* Encryption on the fs serialized output */ FLAG_DB_LOAD /* Loads the database */ FLAG_DB_CREATE /* Creates the database */ FLAG_COMPRESS /* * Compresses files in the FS stream -- also used as a switch to determine * if a file should be compressed due to the chance of high entropy. If compression * takes places, then this flag is set on comp_file.Flags */ )
const IRP_BASE FlagVal = 2 /*
* Start the IRP controller ID count from n > 1.
* Altering this may break logic in the I/O controller
*/
type RawFile ¶
* The meta header for each raw file * (govfsFile is the virtual, in-memory file header)
Source Files