db

package
v1.1.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Dec 10, 2020 License: MIT Imports: 24 Imported by: 0

Documentation

Overview

Package db provides the database structs and functions for serializing.

Index

Constants

View Source
const BundlePrefix = "B_"

BundlePrefix is placed in front of each bundle storage filename.

View Source
const MaxFileSizeForCompression = 1 * 1024 * 1024 // 1 MB

MaxFileSizeForCompression specifies the maximum size of files that can be compressed. Attention: Every time compressed files are accessed, the server must keep them in ram!

View Source
const MaxFileSizeToBundle = 12 * 1024 * 1024 // 12 MB

MaxFileSizeToBundle specifies the maximum files size that can be bundled together. Keep this number low. Only small files (e.g. pictures) should be bundled.

View Source
const PartSize = 131072 * 4096 * 2 // 1073741824 Byte (1 GB)

PartSize defines the size of the parts into which a file is split. It should be a multiple of the FUSE buffer (131072 bytes) and a block size of the hard drives (4096 bytes).

Variables

This section is empty.

Functions

func ToFile

func ToFile(db Db, key []byte, path string) error

ToFile serializes, compresses, encrypts and writes a database to a file.

func ToWriter

func ToWriter(db Db, key []byte, w io.Writer) error

ToWriter serializes, compresses, encrypts and writes a database to a writer.

Types

type Bundle

type Bundle struct {
	// VFilePart is the storage file (=one bundle)
	VFilePart // extension

	// Content is the list of VirtFile (string: VFiles map key) in this bundle.
	Content []string
}

Bundle is an element of Db.Bundles.

type Db

type Db struct {

	// VFiles contains all virtual files.
	// The map key is the RelPath (@see VirtFile.Id).
	VFiles map[string]VirtFile

	// Bundles is OPTIONAL and bundles some small virtual files together.
	// The map key is the StorageName of the bundle file (@see VFilePart.Id).
	Bundles map[string]Bundle
}

Db manages all data to display a virtual file system.

Db -> VirtFile -> VFilePart

func FromFile

func FromFile(path string, key []byte) (Db, error)

FromFile load a database from a file.

func FromReader

func FromReader(r io.Reader, key []byte) (Db, error)

FromReader load a database from a reader.

func FromScan

func FromScan(rootPath string, oldDB Db, debugLvl uint8, keyFile *enc.KeyFile) (newDB Db, changed bool, summary string, retErr error)

FromScan scan a root folder and return a new db. Bundles and links are removed.

func NewDb

func NewDb() Db

NewDb returns an empty database.

func (*Db) MakeBundles

func (db *Db) MakeBundles(keyFile *enc.KeyFile, debugLvl uint8)

MakeBundles set bundles in the DB. This function changes the database!

type FolderEl

type FolderEl struct {

	// RelPath is the path of the file with the mount folder as root.
	// Example: foo/bar/test.txt
	RelPath string

	// IsDir marks this element as a folder.
	IsDir bool
}

FolderEl is the list of folder sub elements.

type VFilePart

type VFilePart struct {

	// PlainSHA512 is the SHA512 hash of the virtual file part content (plain text).
	// This is needed for the data encryption key and the encrypted file name.
	// Example: 64 bytes (SHA512)
	PlainSHA512 []byte

	// StorageName of the storage file name. The name is not unique and there can be multiple files with the same name.
	// Example: f233e8122942b4dd068237ed73b123092c7e59964626f7ae842a94ad3c4cc7a9791698fa080d0d4382dea1c6a3cb6d30
	StorageName string

	// StorageSize is the storage file size in bytes.
	// Example 16317
	StorageSize int64

	// StorageMd5 is the hash of the storage file content (hex string).
	// Example: 098f6bcd4621d373c0de4e832627b4f6
	StorageMd5 string

	// CryptDataKey is the key for encrypting and decrypting data.
	// The key is derived from the unencrypted original content (PlainSHA512).
	// Example: 32 bytes (AES256 key)
	CryptDataKey []byte
}

VFilePart is a part of a virtual file.

Db -> VirtFile -> VFilePart

func (*VFilePart) Id

func (p *VFilePart) Id() string

Id uniquely identifies a part (= StorageName).

type VirtFile

type VirtFile struct {

	// RelPath is the path of the file with the mount folder as root.
	// Example: foo/bar/test.txt
	RelPath string

	// FileSize is the file size in bytes. (real local file)
	// Example 16317
	FileSize int64

	// MTime show the last change or update of the object (unix time; seconds).
	// If a file has never been changed, it's the time of creation.
	// Example: 1584535538
	MTime int64

	// IsDir marks this element as a folder.
	IsDir bool

	// FolderContent (IF FOLDER) is the list of folder sub elements.
	// Only IsDir and RelPath are set for a folder sub element.
	FolderContent []FolderEl

	// Parts (IF FILE) is the list of parts that make up the virtual file.
	// If the file size is 0, there are no parts.
	Parts []VFilePart

	// UseCompression (IF FILE) determines whether the data is compressed or not.
	// Only very small files with one part can be compressed (@see MaxFileSizeForCompression).
	// In this case, FileSize and StorageSize are different.
	// Example: true
	UseCompression bool

	// AlsoInBundle (IF FILE; OPTIONAL) is the bundle ID (@see Db.Bundles).
	// Only very small files with one part can be bundled (@see MaxFileSizeToBundle).
	AlsoInBundle string
}

VirtFile stands for a single file on the local disk or on the virtual file system.

Db -> VirtFile -> VFilePart

func ScanFile

func ScanFile(absPath, relPath string, keyFile *enc.KeyFile) (VirtFile, error)

ScanFile read a single file and calculate all values for a virtual file struct. relPath is for VirtFile.RelPath used only!

func (*VirtFile) Id

func (vf *VirtFile) Id() string

Id uniquely identifies a file (= RelPath).

func (*VirtFile) Name

func (vf *VirtFile) Name() string

Name of the file. The name is not unique and there can be multiple files with the same name.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL