digestfs

package module
v0.0.0-...-053b42c Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 16, 2019 License: MIT Imports: 4 Imported by: 0

README

go-digestfs

Package digestfs provides a content-addressable virtual file system (VFS) by providing a common interface to one or more content-addressable storage (CAS).

Documention

Online documentation, which includes examples, can be found at: http://godoc.org/github.com/reiver/go-digestfs

GoDoc

Example

Here is an example of how to use an already mounted digestfs.MountPoint to get content:

content, err := mountpoint.Open("SHA-256.hexadecimal", "c0535e4be2b79ffd93291305436bf889314e4a3faec05ecffcbb7df31ad9e51a")
if nil != err {
	return err
}
defer content.Close()

Note that if you had the digest encoded as binary, then you could encode it as hexadecimal using: As in:

digest := fmt.Sprintf("%x", binaryDigest)

Content Addressing

A content-addressable file system, or content-addressable storage (CAS) may be used with content addressing.

Some examples of content addressing include:

See Also

Documentation

Overview

Package digestfs provides a content-addressable virtual file system (VFS) by providing a common interface to one or more content-addressable storage (CAS).

Example Opening

Here is an example of how to use an already mounted digestfs.MountPoint to get content: 

content, err := mountpoint.Open("SHA-256", "\xc0\x53\x5e\x4b\xe2\xb7\x9f\xfd\x93\x29\x13\x05\x43\x6b\xf8\x89\x31\x4e\x4a\x3f\xae\xc0\x5e\xcf\xfc\xbb\x7d\xf3\x1a\xd9\xe5\x1a")
if nil != err {
	return err
}
defer content.Close()

Note that if you had the digest encoded as hexadecimal, then you could decode it into binary using: As in: 

var hexadecimalDigest = "c0535e4be2b79ffd93291305436bf889314e4a3faec05ecffcbb7df31ad9e51a"

digestBytes, err := hex.DecodeString(hexadecimalDigest)

var digest string = string(digestBytes)

// digest == "\xc0\x53\x5e\x4b\xe2\xb7\x9f\xfd\x93\x29\x13\x05\x43\x6b\xf8\x89\x31\x4e\x4a\x3f\xae\xc0\x5e\xcf\xfc\xbb\x7d\xf3\x1a\xd9\xe5\x1a"

Here, ‘content’ would give you access to the data on the content mostly via an io.ReaderAt interface. For example: 

var buffer [256]byte

var p []byte = buffer[:]

var offset int64 = 0

n, err := content.ReadAt(p, offset)

Alternatively, if you want to use the io.Reader interface, you can upgrade ‘content’ from an io.ReaderAt to an io.Reader by using an io.SectionReader. For example: 

r := io.NewSectionReader(content, 0, int64(content.Len()))

// ...

contentBytes, err := ioutil.ReadAll(r)

Example Mounting

How you create a digestfs.MountPoint depends on which MountPoint you use.

But mounting looks something like: 

var mountpoint digestfs.MountPoint

err := mountpoint.Mount("git", "/home/joeblow/workspaces/myproject")

Content-Addressable Storages

Some content-addressable storages (CAS) that can be used with digestfs include:

• https://godoc.org/github.com/reiver/go-memdigest

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type BadLocation 

type BadLocation interface {
	error
	BadLocation() string
}

BadLocation may be returned by MountPoint.OpenLocation().

Example

Here is an example usage: 

var mountpoint digestfs.MountPoint

// ...

content, err := mountpoint.OpenLoction("@^/apple/banana/cherry")

if nil != err {
	switch err.(type) {
	case digestfs.BadLocation:
		//@TODO
	default:
		//@TODO
	}
}

type Content 

type Content interface {
	io.Closer
	io.ReaderAt

	Len() int
}

Content provides a read-only interface to content.

Example

Here is an example of digestfs.Content being used. 

content, err := Open(algorithm, digest)
if nil != err {
	return err
}
defer content.Close()

// ...

var buffer [128]byte
p := buffer[:]

n, err := content.ReadAt(p, offset)

Reader, Seeker

If you want to turn Content into a io.Reader, and io.Seeker, you can do that by wrapping it in a io.SectionReader.

For example: 

r := io.NewSectionReader(content, 0, int64(content.Len()))

type ContentNotFound 

type ContentNotFound interface {
	error
	ContentNotFound() (algorithm string, digest string)
}

ContentNotFound may be returned by MountPoint.Open(), and MountPoint.OpenLocation().

Example

Here is an example usage: 

var mountpoint digestfs.MountPoint

// ...

content, err := mountpoint.Open("SHA-256", "\x00\x00\x00\x00\x00\x19\xd6\x68\x9c\x08\x5a\xe1\x65\x83\x1e\x93\x4f\xf7\x63\xae\x46\xa2\xa6\xc1\x72\xb3\xf1\xb6\x0a\x8c\xe2\x6f")

if nil != err {
	switch err.(type) {
	case digestfs.ContentNotFound:
		//@TODO
	default:
		//@TODO
	}
}
Example 
var mem memdigest.SHA1

var mountpoint digestfs.MountPoint

err := mountpoint.Mount("memdigest.SHA1", &mem)

if nil != err {
	switch casted := err.(type) {
	case digestfs.MounterNotFound:
		fmt.Printf("mounter not found for fstype = %q\n", casted.MounterNotFound())
		return
	default:
		fmt.Println(err)
		return
	}
}

_, _, err = mountpoint.Create([]byte("Hello world!"))
if nil != err {
	fmt.Printf("ERROR: could not create content “Hello world!” because: %s\n", err)
	return
}
_, _, err = mountpoint.Create([]byte("apple banana cherry"))
if nil != err {
	fmt.Printf("ERROR: could not create content “apple banana cherry” because: %s\n", err)
	return
}
_, _, err = mountpoint.Create([]byte("intergalactic planetary 👾"))
if nil != err {
	fmt.Printf("ERROR: could not create content “intergalactic planetary 👾” because: %s\n", err)
	return
}

func() {
	// 0xd3486ae9136e7856bc42212385ea797094475802
	var digest string = "\xd3\x48\x6a\xe9\x13\x6e\x78\x56\xbc\x42\x21\x23\x85\xea\x79\x70\x94\x47\x58\x02"

	content, err := mountpoint.Open("SHA-1", digest)
	if nil != err {
		switch err.(type) {
		case digestfs.ContentNotFound: // <--------- This is where digestfs.ContentNotFound is being used
			fmt.Printf("ERROR: could not find content with SHA-1 digest “%s”\n", hex.EncodeToString([]byte(digest)))
		default:
			fmt.Printf("ERROR: could not open content with SHA-1 digest “%s” because: %s\n", hex.EncodeToString([]byte(digest)), err)
		}
		return
	}
	defer content.Close()

	r := io.NewSectionReader(content, 0, int64(content.Len()))
	contentBytes, err := ioutil.ReadAll(r)
	if nil != err {
		fmt.Printf("ERROR: could not read all bytes of content with SHA-1 digest “%s” because: %s\n", hex.EncodeToString([]byte(digest)), err)
		return
	}

	fmt.Printf("Content with SHA-1 digest “%s” is: %q\n", hex.EncodeToString([]byte(digest)), contentBytes)
}()

func() {
	// 0x80256f39a9d308650ac90d9be9a72a9562454574
	var digest string = "\x80\x25\x6f\x39\xa9\xd3\x08\x65\x0a\xc9\x0d\x9b\xe9\xa7\x2a\x95\x62\x45\x45\x74"

	content, err := mountpoint.Open("SHA-1", digest)
	if nil != err {
		switch err.(type) {
		case digestfs.ContentNotFound: // <--------- This is where digestfs.ContentNotFound is being used
			fmt.Printf("ERROR: could not find content with SHA-1 digest “%s”\n", hex.EncodeToString([]byte(digest)))
		default:
			fmt.Printf("ERROR: could not open content with SHA-1 digest “%s” because: %s\n", hex.EncodeToString([]byte(digest)), err)
		}
		return
	}
	defer content.Close()

	r := io.NewSectionReader(content, 0, int64(content.Len()))
	contentBytes, err := ioutil.ReadAll(r)
	if nil != err {
		fmt.Printf("ERROR: could not read all bytes of content with SHA-1 digest “%s” because: %s\n", hex.EncodeToString([]byte(digest)), err)
		return
	}

	fmt.Printf("Content with SHA-1 digest “%s” is: %q\n", hex.EncodeToString([]byte(digest)), contentBytes)
}()

Output:

Content with SHA-1 digest “d3486ae9136e7856bc42212385ea797094475802” is: "Hello world!"
ERROR: could not find content with SHA-1 digest “80256f39a9d308650ac90d9be9a72a9562454574”

type MountPoint 

type MountPoint struct {
	// contains filtered or unexported fields
}

func (*MountPoint) Create 

func (receiver *MountPoint) Create(content []byte) (algorithm string, digest string, err error)

Create creates new content at the MoutPoint.

The value of ‘digest’ is in binary.

The value of ‘digest’ is NOT a binary-to-text encoding such as hexadecimal, bas64, etc.

So the value of ‘digest’ might be something such as: 

"\x00\x00\x00\x00\x00\x19\xd6\x68\x9c\x08\x5a\xe1\x65\x83\x1e\x93\x4f\xf7\x63\xae\x46\xa2\xa6\xc1\x72\xb3\xf1\xb6\x0a\x8c\xe2\x6f"

Rather than any of these: 

"000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f" (not this)

"0x000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f" (not this)

"AAAAAAAZ1micCFrhZYMek0_3Y65GoqbBcrPxtgqM4m8" (not this)

"00000000001kdxxHkC0H5AE1x5Hm1Ekm4ffVxmAE4xA2AxC1V2bmf1bx0AHCE2xf" (not this)

"0r00000000001kdxxHkC0H5AE1x5Hm1Ekm4ffVxmAE4xA2AxC1V2bmf1bx0AHCE2xf" (not this)

func (*MountPoint) Mount 

func (dest *MountPoint) Mount(fstype string, params ...interface{}) error

func (*MountPoint) Open 

func (receiver *MountPoint) Open(algorithm string, digest string) (Content, error)

Open returns content at the MountPoint.

The value of ‘digest’ is in binary.

The value of ‘digest’ is NOT a binary-to-text encoding such as hexadecimal, bas64, etc.

So the value of ‘digest’ might be something such as: 

"\x00\x00\x00\x00\x00\x19\xd6\x68\x9c\x08\x5a\xe1\x65\x83\x1e\x93\x4f\xf7\x63\xae\x46\xa2\xa6\xc1\x72\xb3\xf1\xb6\x0a\x8c\xe2\x6f"

Rather than any of these: 

"000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f" (not this)

"0x000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f" (not this)

"AAAAAAAZ1micCFrhZYMek0_3Y65GoqbBcrPxtgqM4m8" (not this)

"00000000001kdxxHkC0H5AE1x5Hm1Ekm4ffVxmAE4xA2AxC1V2bmf1bx0AHCE2xf" (not this)

"0r00000000001kdxxHkC0H5AE1x5Hm1Ekm4ffVxmAE4xA2AxC1V2bmf1bx0AHCE2xf" (not this)

func (*MountPoint) OpenLocation 

func (receiver *MountPoint) OpenLocation(location string) (Content, error)

func (*MountPoint) Unmount 

func (dest *MountPoint) Unmount() error

type MounterNotFound 

type MounterNotFound interface {
	error
	MounterNotFound() string
}

MounterNotFound may be returned by Mounter.Mount().

Example

Here is an example usage: 

var mountpoint digestfs.MountPoint

err := mountpoint.Mount("git", "/home/joeblow/workspaces/myproject")

if nil != err {
	switch err.(type) {
	case digestfs.MounterNotFound:
		//@TODO
	default:
		//@TODO
	}
}
Example 
var mountpoint digestfs.MountPoint

err := mountpoint.Mount("faux", "/path/to/stuff")

if nil != err {
	switch casted := err.(type) {
	case digestfs.MounterNotFound:
		fmt.Printf("mounter not found for fstype = %q", casted.MounterNotFound())
		return
	default:
		fmt.Print(err)
		return
	}
}

fmt.Printf("MOUNTED!")

Output:

mounter not found for fstype = "faux"

type NotMounted 

type NotMounted interface {
	error
	NotMounted()
}

type UnsupportedAlgorithm 

type UnsupportedAlgorithm interface {
	error
	UnsupportedAlgorithm() string
}

Source Files

View all Source files

Directories

Path Synopsis
Package digestfs_driver provides tools for registering content-addressable file system drivers for the virtual file system (VFS) for package digestfs.
 Package digestfs_driver provides tools for registering content-addressable file system drivers for the virtual file system (VFS) for package digestfs.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.