utahfs

package module
v0.0.0-...-a227d8c Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Dec 20, 2020 License: BSD-3-Clause Imports: 18 Imported by: 0

README

UtahFS

UtahFS is a state-of-the art encrypted storage solution, meant to be similar to Dropbox. It has a FUSE binding that creates a synthetic drive on the user's computer that they can interact with like an external hard-drive. Files stored in the drive are uploaded to a cloud storage provider, which means the drive will never run out of space and minimizes the likelihood of any files being lost. However the files are encrypted such that the cloud storage provider knows almost nothing about what's being stored.

Features

  1. Interchangeable Storage Providers. For storing data in the cloud, UtahFS uses Object Storage, which is cheap and commodified. Example providers include: AWS S3, Google Cloud Storage, Backblaze B2, and Wasabi.
  2. Very Very Strong Encryption. The method of encryption hides the number of files, file names, file contents, individual file size, and prevents any modifications (including rollbacks!). The only information which is clearly leaked is the maximum archive size: archives grow to fit new data, but won't shrink if that data is deleted. Instead, that space is left allocated and will be re-used if needed in the future.
  3. Local Hardware allows Multiple Users and Improved Performance. If the user has server-like hardware on their LAN (Raspberry Pi / Intel NUC), this can be used to coordinate multiple users operating in the same archive. It can also dramatically improve the performance of uploads, because a user can upload large amounts of already-encrypted data to the server over the fast local network, and let the server take over the much slower upload to the cloud provider.
  4. Archive Mode. The client can be configured to guard against deleting or overwriting existing files, while still allowing new files to be created and old files to be moved around. This helps protect against accidental data loss.
  5. Oblivious RAM (ORAM). ORAM can be used to hide the access pattern of data from the cloud storage provider, so the provider only sees the amount data is accessed. (Access pattern: Which pieces of data are being accessed, and whether the access was a read or write.)

How to Use

Setup documentation can be found in the docs/ folder.

Future Work

  1. Reliability strategies for the WAL. Changes are buffered in a local Write-Ahead Log (WAL) before being sent to the cloud storage provider. If the disk that the WAL is stored on fails, it could become very difficult to return the archive to a usable, partially-regressed state.

Documentation

Overview

Package utahfs provides a FUSE binding where files are stored encrypted in the cloud.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func NewArchive

func NewArchive(bfs *BlockFilesystem) (fuseutil.FileSystem, error)

NewArchive wraps NewFilesystem but refuses to delete or overwrite data.

It allows new files to be created, and old files to be moved / renamed / appended to. Empty directories may be deleted, but no files may be deleted or overwritten. This is just enforced by the FUSE binding, not by an actual access management system. Data stored is compatible with NewFilesystem.

func NewFilesystem

func NewFilesystem(bfs *BlockFilesystem) (fuseutil.FileSystem, error)

NewFilesystem returns a FUSE binding that internally stores data in a block-based filesystem.

Types

type BlockFile

type BlockFile struct {
	// contains filtered or unexported fields
}

BlockFile implements read-write functionality for a variable-size file over a skiplist of fixed-size blocks.

func (*BlockFile) Read

func (bf *BlockFile) Read(p []byte) (int, error)

func (*BlockFile) Seek

func (bf *BlockFile) Seek(offset int64, whence int) (int64, error)

func (*BlockFile) Truncate

func (bf *BlockFile) Truncate(size int64) error

func (*BlockFile) Write

func (bf *BlockFile) Write(p []byte) (int, error)

type BlockFilesystem

type BlockFilesystem struct {
	// contains filtered or unexported fields
}

BlockFilesystem implements large files as skiplists over fixed-size blocks stored in an object storage service.

func NewBlockFilesystem

func NewBlockFilesystem(store *persistent.AppStorage, numPtrs, dataSize int64, splitPtrs bool) (*BlockFilesystem, error)

NewBlockFilesystem returns a new block-based filesystem. Blocks will have `numPtrs` pointers in their skiplist and contain at most `dataSize` bytes of application data.

Recommended values:

numPtrs = 12, dataSize = 32*1024

This system manages two pieces of global state:

  1. trash - Points to the first block of the trash list: a linked list of blocks which have been discarded and are free for re-allocation.
  2. next - The next unallocated pointer. A block with this pointer is created only if the trash list is empty.

`splitPtrs` is true if the pointers section of a block should be stored separately from the data section, and false if they should be stored together. Storing them separately can improve seek performance.

func (*BlockFilesystem) Create

Create creates a new file. It returns the pointer to the file and an open copy.

func (*BlockFilesystem) Open

Open returns a handle to an existing file.

func (bfs *BlockFilesystem) Unlink(ctx context.Context, ptr uint64) error

Unlink allows the blocks allocated for a file to be re-used for other purposes.

Directories

Path Synopsis
Package cache implements a capped-size in-memory cache that randomly evicts elements when it reaches max size.
Package cache implements a capped-size in-memory cache that randomly evicts elements when it reaches max size.
cmd
utahfs-client
Command utahfs-client provides a FUSE binding, backed by an encrypted object storage provider.
Command utahfs-client provides a FUSE binding, backed by an encrypted object storage provider.
utahfs-server
Command utahfs-server acts as a cache to improve the performance of a UtahFS instance, and helps coordinate multiple users.
Command utahfs-server acts as a cache to improve the performance of a UtahFS instance, and helps coordinate multiple users.
utahfs-web
Command utahfs-web provides access to a UtahFS repository through a Web UI.
Command utahfs-web provides access to a UtahFS repository through a Web UI.
Package persistent implements several compatible object-storage backends, and additional functionality that can be layered upon them.
Package persistent implements several compatible object-storage backends, and additional functionality that can be layered upon them.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL