unixfs

package module
v0.4.6 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 25, 2023 License: MIT Imports: 6 Imported by: 173

README

go-unixfs

go-unixfs implements unix-like filesystem utilities on top of an ipld merkledag

Coverage Status Travis CI

❗ This repo is no longer maintained.

👉 We highly recommend switching to the maintained version at https://github.com/ipfs/boxo/tree/main/ipld/unixfs.

🏎️ Good news! There is tooling and documentation to expedite a switch in your repo.

⚠️ If you continue using this repo, please note that security fixes will not be provided (unless someone steps in to maintain it).

📚 Learn more, including how to take the maintainership mantle or ask questions, here.

Table of Contents

Package Directory

This package contains many subpackages, each of which can be very large on its own.

Top Level

The top level unixfs package defines the unixfs format datastructures, and some helper methods around it.

importers

The importer subpackage is what you'll use when you want to turn a normal file into a unixfs file.

io

The io subpackage provides helpers for reading files and manipulating directories. The DagReader takes a reference to a unixfs file and returns a file handle that can be read from and seeked through. The Directory interface allows you to easily read items in a directory, add items to a directory, and do lookups.

mod

The mod subpackage implements a DagModifier type that can be used to write to an existing unixfs file, or create a new one. The logic for this is significantly more complicated than for the dagreader, so its a separate type. (TODO: maybe it still belongs in the io subpackage though?)

hamt

The hamt subpackage implements a CHAMP hamt that is used in unixfs directory sharding.

archive

The archive subpackage implements a tar importer and exporter. The objects created here are not officially unixfs, but in the future, this may be integrated more directly.

test

The test subpackage provides several utilities to make testing unixfs related things easier.

Install

go get github.com/ipfs/go-unixfs

License

MIT © Juan Batiz-Benet

Documentation

Overview

Package unixfs implements a data format for files in the IPFS filesystem It is not the only format in ipfs, but it is the one that the filesystem assumes

Index

Constants

View Source 
const (
	// Deprecated: use github.com/ipfs/boxo/ipld/unixfs.TRaw
	TRaw = pb.Data_Raw
	// Deprecated: use github.com/ipfs/boxo/ipld/unixfs.TFile
	TFile = pb.Data_File
	// Deprecated: use github.com/ipfs/boxo/ipld/unixfs.TDirectory
	TDirectory = pb.Data_Directory
	// Deprecated: use github.com/ipfs/boxo/ipld/unixfs.TMetadata
	TMetadata = pb.Data_Metadata
	// Deprecated: use github.com/ipfs/boxo/ipld/unixfs.TSymlink
	TSymlink = pb.Data_Symlink
	// Deprecated: use github.com/ipfs/boxo/ipld/unixfs.THAMTShard
	THAMTShard = pb.Data_HAMTShard
)

Shorthands for protobuffer types

Variables

View Source 
var (
	// Deprecated: use github.com/ipfs/boxo/ipld/unixfs.ErrMalformedFileFormat
	ErrMalformedFileFormat = errors.New("malformed data in file format")
	// Deprecated: use github.com/ipfs/boxo/ipld/unixfs.ErrUnrecognizedType
	ErrUnrecognizedType = errors.New("unrecognized node type")
)

Common errors

Functions

func BytesForMetadata deprecated 

func BytesForMetadata(m *Metadata) ([]byte, error)

BytesForMetadata wraps the given Metadata as a profobuf message of Data type, setting the DataType to Metadata. The wrapped bytes are itself the result of calling m.Bytes().

Deprecated: use github.com/ipfs/boxo/ipld/unixfs.BytesForMetadata

func DataSize deprecated 

func DataSize(data []byte) (uint64, error)

DataSize returns the size of the contents in protobuf wrapped slice. For raw data it simply provides the length of it. For Data_Files, it will return the associated filesize. Note that Data_Directories will return an error.

Deprecated: use github.com/ipfs/boxo/ipld/unixfs.DataSize

func EmptyDirNode deprecated 

func EmptyDirNode() *dag.ProtoNode

EmptyDirNode creates an empty folder Protonode.

Deprecated: use github.com/ipfs/boxo/ipld/unixfs.EmptyDirNode

func EmptyFileNode deprecated added in v0.2.6 

func EmptyFileNode() *dag.ProtoNode

EmptyFileNode creates an empty file Protonode.

Deprecated: use github.com/ipfs/boxo/ipld/unixfs.EmptyFileNode

func FilePBData deprecated 

func FilePBData(data []byte, totalsize uint64) []byte

FilePBData creates a protobuf File with the given byte slice and returns the marshaled protobuf bytes representing it.

Deprecated: use github.com/ipfs/boxo/ipld/unixfs.FilePBData

func FolderPBData deprecated 

func FolderPBData() []byte

FolderPBData returns Bytes that represent a Directory.

Deprecated: use github.com/ipfs/boxo/ipld/unixfs.FolderPBData

func FromBytes deprecated 

func FromBytes(data []byte) (*pb.Data, error)

FromBytes unmarshals a byte slice as protobuf Data. Deprecated: Use `FSNodeFromBytes` instead to avoid direct manipulation of `pb.Data`.

Deprecated: use github.com/ipfs/boxo/ipld/unixfs.FromBytes

func HAMTShardData deprecated 

func HAMTShardData(data []byte, fanout uint64, hashType uint64) ([]byte, error)

HAMTShardData return a `Data_HAMTShard` protobuf message

Deprecated: use github.com/ipfs/boxo/ipld/unixfs.HAMTShardData

func ReadUnixFSNodeData deprecated 

func ReadUnixFSNodeData(node ipld.Node) (data []byte, err error)

ReadUnixFSNodeData extracts the UnixFS data from an IPLD node. Raw nodes are (also) processed because they are used as leaf nodes containing (only) UnixFS data.

Deprecated: use github.com/ipfs/boxo/ipld/unixfs.ReadUnixFSNodeData

func SymlinkData deprecated 

func SymlinkData(path string) ([]byte, error)

SymlinkData returns a `Data_Symlink` protobuf message for the path you specify.

Deprecated: use github.com/ipfs/boxo/ipld/unixfs.SymlinkData

func UnwrapData deprecated 

func UnwrapData(data []byte) ([]byte, error)

UnwrapData unmarshals a protobuf messages and returns the contents.

Deprecated: use github.com/ipfs/boxo/ipld/unixfs.UnwrapData

func WrapData deprecated 

func WrapData(b []byte) []byte

WrapData marshals raw bytes into a `Data_Raw` type protobuf message.

Deprecated: use github.com/ipfs/boxo/ipld/unixfs.WrapData

Types

type FSNode deprecated 

type FSNode struct {
	// contains filtered or unexported fields
}

An FSNode represents a filesystem object using the UnixFS specification.

The `NewFSNode` constructor should be used instead of just calling `new(FSNode)` to guarantee that the required (`Type` and `Filesize`) fields in the `format` structure are initialized before marshaling (in `GetBytes()`).

Deprecated: use github.com/ipfs/boxo/ipld/unixfs.FSNode

func ExtractFSNode deprecated 

func ExtractFSNode(node ipld.Node) (*FSNode, error)

Extract the `unixfs.FSNode` from the `ipld.Node` (assuming this was implemented by a `mdag.ProtoNode`).

Deprecated: use github.com/ipfs/boxo/ipld/unixfs.ExtractFSNode

func FSNodeFromBytes deprecated 

func FSNodeFromBytes(b []byte) (*FSNode, error)

FSNodeFromBytes unmarshal a protobuf message onto an FSNode.

Deprecated: use github.com/ipfs/boxo/ipld/unixfs.FSNodeFromBytes

func NewFSNode deprecated 

func NewFSNode(dataType pb.Data_DataType) *FSNode

NewFSNode creates a new FSNode structure with the given `dataType`.

It initializes the (required) `Type` field (that doesn't have a `Set()` accessor so it must be specified at creation), otherwise the `Marshal()` method in `GetBytes()` would fail (`required field "Type" not set`).

It also initializes the `Filesize` pointer field to ensure its value is never nil before marshaling, this is not a required field but it is done to be backwards compatible with previous `go-ipfs` versions hash. (If it wasn't initialized there could be cases where `Filesize` could have been left at nil, when the `FSNode` was created but no data or child nodes were set to adjust it, as is the case in `NewLeaf()`.)

Deprecated: use github.com/ipfs/boxo/ipld/unixfs.NewFSNode

func (*FSNode) AddBlockSize 

func (n *FSNode) AddBlockSize(s uint64)

AddBlockSize adds the size of the next child block of this node

func (*FSNode) BlockSize 

func (n *FSNode) BlockSize(i int) uint64

BlockSize returns the block size indexed by `i`. TODO: Evaluate if this function should be bounds checking.

func (*FSNode) BlockSizes 

func (n *FSNode) BlockSizes() []uint64

BlockSizes gets blocksizes of format

func (*FSNode) Data 

func (n *FSNode) Data() []byte

Data retrieves the `Data` field from the internal `format`.

func (*FSNode) Fanout 

func (n *FSNode) Fanout() uint64

Fanout gets fanout of format

func (*FSNode) FileSize 

func (n *FSNode) FileSize() uint64

FileSize returns the size of the file.

func (*FSNode) GetBytes 

func (n *FSNode) GetBytes() ([]byte, error)

GetBytes marshals this node as a protobuf message.

func (*FSNode) HashType 

func (n *FSNode) HashType() uint64

HashType gets hash type of format

func (*FSNode) IsDir 

func (n *FSNode) IsDir() bool

IsDir checks whether the node represents a directory

func (*FSNode) NumChildren 

func (n *FSNode) NumChildren() int

NumChildren returns the number of child blocks of this node

func (*FSNode) RemoveAllBlockSizes 

func (n *FSNode) RemoveAllBlockSizes()

RemoveAllBlockSizes removes all the child block sizes of this node.

func (*FSNode) RemoveBlockSize 

func (n *FSNode) RemoveBlockSize(i int)

RemoveBlockSize removes the given child block's size.

func (*FSNode) SetData 

func (n *FSNode) SetData(newData []byte)

SetData sets the `Data` field from the internal `format` updating its `Filesize`.

func (*FSNode) Type 

func (n *FSNode) Type() pb.Data_DataType

Type retrieves the `Type` field from the internal `format`.

func (*FSNode) UpdateFilesize 

func (n *FSNode) UpdateFilesize(filesizeDiff int64)

UpdateFilesize updates the `Filesize` field from the internal `format` by a signed difference (`filesizeDiff`). TODO: Add assert to check for `Filesize` > 0?

type LinkResult deprecated 

type LinkResult struct {
	Link *ipld.Link
	Err  error
}

A LinkResult for any parallel enumeration of links TODO: Should this live in go-ipld-format?

Deprecated: use github.com/ipfs/boxo/ipld/unixfs.LinkResult

type Metadata deprecated 

type Metadata struct {
	MimeType string
	Size     uint64
}

Metadata is used to store additional FSNode information.

Deprecated: use github.com/ipfs/boxo/ipld/unixfs.Metadata

func MetadataFromBytes deprecated 

func MetadataFromBytes(b []byte) (*Metadata, error)

MetadataFromBytes Unmarshals a protobuf Data message into Metadata. The provided slice should have been encoded with BytesForMetadata().

Deprecated: use github.com/ipfs/boxo/ipld/unixfs.MetadataFromBytes

func (*Metadata) Bytes 

func (m *Metadata) Bytes() ([]byte, error)

Bytes marshals Metadata as a protobuf message of Metadata type.

Source Files

View all Source files

Directories

Path Synopsis
Package hamt implements a Hash Array Mapped Trie over ipfs merkledag nodes.
 Package hamt implements a Hash Array Mapped Trie over ipfs merkledag nodes.
Package importer implements utilities used to create IPFS DAGs from files and readers.
 Package importer implements utilities used to create IPFS DAGs from files and readers.
balanced
Package balanced provides methods to build balanced DAGs, which are generalistic DAGs in which all leaves (nodes representing chunks of data) are at the same distance from the root.
 Package balanced provides methods to build balanced DAGs, which are generalistic DAGs in which all leaves (nodes representing chunks of data) are at the same distance from the root.
helpers
trickle
Package trickle allows to build trickle DAGs.
 Package trickle allows to build trickle DAGs.
Package io implements convenience objects for working with the ipfs unixfs data format.
 Package io implements convenience objects for working with the ipfs unixfs data format.
Package mod provides DAG modification utilities to, for example, insert additional nodes in a unixfs DAG or truncate them.
 Package mod provides DAG modification utilities to, for example, insert additional nodes in a unixfs DAG or truncate them.
private
linksize

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.