io

package
v0.6.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 21, 2021 License: MIT Imports: 13 Imported by: 6

Documentation

Overview

Package io implements convenience objects for working with the ipfs unixfs data format.

Index

Constants

View Source 
const (
	DirNodeType     = 1
	FileNodeType    = 2
	SymlinkNodeType = 3
)
View Source 
const (
	// SmallestString is used for metadata name to be the smallest
	// possible string value.
	SmallestString = "\u0000"
)

Variables

View Source 
var (
	ErrIsDir            = errors.New("this dag node is a directory")
	ErrCantReadSymlinks = errors.New("cannot currently read symlinks")
	ErrUnkownNodeType   = errors.New("unknown node type")
	ErrSeekNotSupported = errors.New("file does not support seeking")
)

Common errors

View Source 
var DefaultShardWidth = 256

DefaultShardWidth is the default value used for hamt sharding width.

View Source 
var ErrNotADir = fmt.Errorf("merkledag node was not a directory or shard")

ErrNotADir implies that the given node was not a unixfs directory

View Source 
var UseHAMTSharding = false

UseHAMTSharding is a global flag that signifies whether or not to use the HAMT sharding scheme for directory creation

Functions

func GetMetaDataFromDagRoot added in v0.5.1 

func GetMetaDataFromDagRoot(ctx context.Context, root ipld.Node, ds ipld.DAGService) ([]byte, error)

GetMetaDataFromDagRoot returns the full metadata bytes if available. This function is unixfs/io instead of unixfs because of `NewDagReader` dependency to read all the bytes.

func ResolveUnixfsOnce 

func ResolveUnixfsOnce(ctx context.Context, ds ipld.NodeGetter, nd ipld.Node, names []string) (*ipld.Link, []string, error)

ResolveUnixfsOnce resolves a single hop of a path through a graph in a unixfs context. This includes handling traversing sharded directories.

Types

type BaseNode added in v0.5.2 

type BaseNode struct {
	NodeType    int
	NodePath    string
	NodeName    string
	Siz         uint64
	StartOffset uint64
	Links       []interface{}
}

func (*BaseNode) Name added in v0.5.2 

func (n *BaseNode) Name() string

func (*BaseNode) NodeSize added in v0.5.2 

func (n *BaseNode) NodeSize() int64

func (*BaseNode) Path added in v0.5.2 

func (n *BaseNode) Path() string

type BasicDirectory 

type BasicDirectory struct {
	// contains filtered or unexported fields
}

BasicDirectory is the basic implementation of `Directory`. All the entries are stored in a single node.

func (*BasicDirectory) AddChild 

func (d *BasicDirectory) AddChild(ctx context.Context, name string, node ipld.Node) error

AddChild implements the `Directory` interface. It adds (or replaces) a link to the given `node` under `name`.

func (*BasicDirectory) EnumLinksAsync 

func (d *BasicDirectory) EnumLinksAsync(ctx context.Context) <-chan format.LinkResult

EnumLinksAsync returns a channel which will receive Links in the directory as they are enumerated, where order is not gauranteed

func (*BasicDirectory) Find 

func (d *BasicDirectory) Find(ctx context.Context, name string) (ipld.Node, error)

Find implements the `Directory` interface. 

func (d *BasicDirectory) ForEachLink(ctx context.Context, f func(*ipld.Link) error) error

ForEachLink implements the `Directory` interface.

func (*BasicDirectory) GetCidBuilder 

func (d *BasicDirectory) GetCidBuilder() cid.Builder

GetCidBuilder implements the `Directory` interface.

func (*BasicDirectory) GetNode 

func (d *BasicDirectory) GetNode() (ipld.Node, error)

GetNode implements the `Directory` interface. 

func (d *BasicDirectory) Links(ctx context.Context) ([]*ipld.Link, error)

Links implements the `Directory` interface.

func (*BasicDirectory) RemoveChild 

func (d *BasicDirectory) RemoveChild(ctx context.Context, name string) error

RemoveChild implements the `Directory` interface.

func (*BasicDirectory) SetCidBuilder 

func (d *BasicDirectory) SetCidBuilder(builder cid.Builder)

SetCidBuilder implements the `Directory` interface.

func (*BasicDirectory) SwitchToSharding 

func (d *BasicDirectory) SwitchToSharding(ctx context.Context) (Directory, error)

SwitchToSharding returns a HAMT implementation of this directory.

type DagReader 

type DagReader interface {
	ReadSeekCloser
	Size() uint64
	CtxReadFull(context.Context, []byte) (int, error)
}

A DagReader provides read-only read and seek acess to a unixfs file. Different implementations of readers are used for the different types of unixfs/protobuf-encoded nodes.

func NewDagReader 

func NewDagReader(ctx context.Context, n ipld.Node, serv ipld.NodeGetter) (DagReader, error)

NewDagReader creates a new reader object that reads the data represented by the given node, using the passed in DAGService for data retrieval.

func NewReedSolomonDagReader added in v0.4.4 

func NewReedSolomonDagReader(ctx context.Context, n ipld.Node, serv ipld.NodeGetter,
	numData, numParity, size uint64, isDir bool, missingShards []cid.Cid) (DagReader, []io.Reader, *bytes.Buffer, error)

A ReedSolomonDagReader wraps M DagReaders and reads N (data) out of M (data + parity) concurrently to decode the original file shards for the returned DagReader to use. Optionally, accepts a list of missing shard hashes for repair and returns the buffered data readers on any missing shards (nil for already existing).

type DirNode added in v0.5.2 

type DirNode struct {
	BaseNode        `json:"BaseNode"`
	files.Directory `json:"files.Directory"`
}

files.Directory is input directory

type Directory 

type Directory interface {

	// SetCidBuilder sets the CID Builder of the root node.
	SetCidBuilder(cid.Builder)

	// AddChild adds a (name, key) pair to the root node.
	AddChild(context.Context, string, ipld.Node) error

	// ForEachLink applies the given function to Links in the directory.
	ForEachLink(context.Context, func(*ipld.Link) error) error

	// EnumLinksAsync returns a channel which will receive Links in the directory
	// as they are enumerated, where order is not gauranteed
	EnumLinksAsync(context.Context) <-chan format.LinkResult

	// Links returns the all the links in the directory node.
	Links(context.Context) ([]*ipld.Link, error)

	// Find returns the root node of the file named 'name' within this directory.
	// In the case of HAMT-directories, it will traverse the tree.
	//
	// Returns os.ErrNotExist if the child does not exist.
	Find(context.Context, string) (ipld.Node, error)

	// RemoveChild removes the child with the given name.
	//
	// Returns os.ErrNotExist if the child doesn't exist.
	RemoveChild(context.Context, string) error

	// GetNode returns the root of this directory.
	GetNode() (ipld.Node, error)

	// GetCidBuilder returns the CID Builder used.
	GetCidBuilder() cid.Builder
}

Directory defines a UnixFS directory. It is used for creating, reading and editing directories. It allows to work with different directory schemes, like the basic or the HAMT implementation.

It just allows to perform explicit edits on a single directory, working with directory trees is out of its scope, they are managed by the MFS layer (which is the main consumer of this interface).

func NewDirectory 

func NewDirectory(dserv ipld.DAGService) Directory

NewDirectory returns a Directory. It needs a `DAGService` to add the children.

func NewDirectoryFromNode 

func NewDirectoryFromNode(dserv ipld.DAGService, node ipld.Node) (Directory, error)

NewDirectoryFromNode loads a unixfs directory from the given IPLD node and DAGService.

type FileNode added in v0.5.2 

type FileNode struct {
	BaseNode `json:"BaseNode"`
}

type HAMTDirectory 

type HAMTDirectory struct {
	// contains filtered or unexported fields
}

HAMTDirectory is the HAMT implementation of `Directory`. (See package `hamt` for more information.)

func (*HAMTDirectory) AddChild 

func (d *HAMTDirectory) AddChild(ctx context.Context, name string, nd ipld.Node) error

AddChild implements the `Directory` interface.

func (*HAMTDirectory) EnumLinksAsync 

func (d *HAMTDirectory) EnumLinksAsync(ctx context.Context) <-chan format.LinkResult

EnumLinksAsync returns a channel which will receive Links in the directory as they are enumerated, where order is not gauranteed

func (*HAMTDirectory) Find 

func (d *HAMTDirectory) Find(ctx context.Context, name string) (ipld.Node, error)

Find implements the `Directory` interface. It will traverse the tree. 

func (d *HAMTDirectory) ForEachLink(ctx context.Context, f func(*ipld.Link) error) error

ForEachLink implements the `Directory` interface.

func (*HAMTDirectory) GetCidBuilder 

func (d *HAMTDirectory) GetCidBuilder() cid.Builder

GetCidBuilder implements the `Directory` interface.

func (*HAMTDirectory) GetNode 

func (d *HAMTDirectory) GetNode() (ipld.Node, error)

GetNode implements the `Directory` interface. 

func (d *HAMTDirectory) Links(ctx context.Context) ([]*ipld.Link, error)

Links implements the `Directory` interface.

func (*HAMTDirectory) RemoveChild 

func (d *HAMTDirectory) RemoveChild(ctx context.Context, name string) error

RemoveChild implements the `Directory` interface.

func (*HAMTDirectory) SetCidBuilder 

func (d *HAMTDirectory) SetCidBuilder(builder cid.Builder)

SetCidBuilder implements the `Directory` interface.

type Node added in v0.5.2 

type Node interface {
	Path() string
	NodeSize() int64
	Name() string
}

type ReadSeekCloser 

type ReadSeekCloser interface {
	io.Reader
	io.Seeker
	io.Closer
	io.WriterTo
}

A ReadSeekCloser implements interfaces to read, copy, seek and close.

type ReedSolomonDagReader added in v0.5.2 

type ReedSolomonDagReader struct {
	*bytes.Reader // for Reader, Seeker, and WriteTo
}

reedSolomonDagReader reads a dag by concurrently merging N shards in a N/M encoded UnixFS dag file. Due to reed solomon requiring N shards to exist, we cannot perform stream Read or Seek on this reader. Everything will be pre-filled in memory before supporting DagReader operations from a []byte reader.

func (*ReedSolomonDagReader) Close added in v0.5.2 

func (rsdr *ReedSolomonDagReader) Close() error

Close has no effect since the underlying reader is a buffer.

func (*ReedSolomonDagReader) CtxReadFull added in v0.5.2 

func (rsdr *ReedSolomonDagReader) CtxReadFull(ctx context.Context, out []byte) (int, error)

CtxReadFull is just a Read since there is no context for buffer.

func (*ReedSolomonDagReader) Size added in v0.5.2 

func (rsdr *ReedSolomonDagReader) Size() uint64

Size returns the total size of the data from the decoded DAG structured file using reed solomon algorithm.

type ReedSolomonDirectory added in v0.5.2 

type ReedSolomonDirectory struct {
	DNode *DirNode
	// contains filtered or unexported fields
}

ReedSolomonDirectory is the implementation of `Directory for Reed-Solomon BTFS file. All the entries are stored in a single node.

func NewReedSolomonDirectory added in v0.5.2 

func NewReedSolomonDirectory(dserv ipld.DAGService) *ReedSolomonDirectory

func NewReedSolomonDirectoryFromNode added in v0.5.2 

func NewReedSolomonDirectoryFromNode(dserv ipld.DAGService, dn *DirNode) (*ReedSolomonDirectory, error)

NewReedSolomonDirectoryFromNode loads a ReedSolomon directory from the given DirNode and DAGService. 

func (d *ReedSolomonDirectory) ForEachLink(ctx context.Context, f func(interface{}) error) error

ForEachLink implements the `Directory` interface. 

func (d *ReedSolomonDirectory) Links(ctx context.Context) []interface{}

Links implements the `Directory` interface.

type SymlinkNode added in v0.5.2 

type SymlinkNode struct {
	BaseNode `json:"BaseNode"`
	Data     string
}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.