adder

package
v0.0.0-...-8c384fb Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 25, 2023 License: Apache-2.0, MIT, Apache-2.0, + 1 more Imports: 21 Imported by: 0

Documentation

Overview

Package adder implements functionality to add content to IPFS daemons managed by the Cluster.

Index

Constants

This section is empty.

Variables

View Source
var ErrBlockAdder = errors.New("failed to put block on all destinations")

ErrBlockAdder is returned when adding a to multiple destinations block fails on all of them.

View Source
var ErrDAGNotFound = errors.New("dagservice: a Get operation was attempted while cluster-adding (this is likely a bug)")

ErrDAGNotFound is returned whenever we try to get a block from the DAGService.

Functions

func BlockAllocate

func BlockAllocate(ctx context.Context, rpc *rpc.Client, pinOpts api.PinOptions) ([]string, error)

BlockAllocate helps allocating blocks to peers.

func IpldNodeToNodeWithMeta

func IpldNodeToNodeWithMeta(n ipld.Node) api.NodeWithMeta

IpldNodeToNodeWithMeta converts an ipld.Node to api.NodeWithMeta.

func Pin

func Pin(ctx context.Context, rpc *rpc.Client, pin api.Pin) error

Pin helps sending local RPC pin requests.

Types

type Adder

type Adder struct {
	// contains filtered or unexported fields
}

Adder is used to add content to IPFS Cluster using an implementation of ClusterDAGService.

func New

func New(ds ClusterDAGService, p api.AddParams, out chan api.AddedOutput) *Adder

New returns a new Adder with the given ClusterDAGService, add options and a channel to send updates during the adding process.

An Adder may only be used once.

func (*Adder) FromFiles

func (a *Adder) FromFiles(ctx context.Context, f files.Directory) (api.Cid, error)

FromFiles adds content from a files.Directory. The adder will no longer be usable after calling this method.

func (*Adder) FromMultipart

func (a *Adder) FromMultipart(ctx context.Context, r *multipart.Reader) (api.Cid, error)

FromMultipart adds content from a multipart.Reader. The adder will no longer be usable after calling this method.

type BaseDAGService

type BaseDAGService struct {
}

BaseDAGService partially implements an ipld.DAGService. It provides the methods which are not needed by ClusterDAGServices (Get*, Remove*) so that they can save adding this code.

func (BaseDAGService) Get

func (dag BaseDAGService) Get(ctx context.Context, key cid.Cid) (ipld.Node, error)

Get always returns errNotFound

func (BaseDAGService) GetMany

func (dag BaseDAGService) GetMany(ctx context.Context, keys []cid.Cid) <-chan *ipld.NodeOption

GetMany returns an output channel that always emits an error

func (BaseDAGService) Remove

func (dag BaseDAGService) Remove(ctx context.Context, key cid.Cid) error

Remove is a nop

func (BaseDAGService) RemoveMany

func (dag BaseDAGService) RemoveMany(ctx context.Context, keys []cid.Cid) error

RemoveMany is a nop

type BlockStreamer

type BlockStreamer struct {
	// contains filtered or unexported fields
}

BlockStreamer helps streaming nodes to multiple destinations, as long as one of them is still working.

func NewBlockStreamer

func NewBlockStreamer(ctx context.Context, rpcClient *rpc.Client, dests []peer.ID, blocks <-chan api.NodeWithMeta) *BlockStreamer

NewBlockStreamer creates a BlockStreamer given an rpc client, allocated peers and a channel on which the blocks to stream are received.

func (*BlockStreamer) Done

func (bs *BlockStreamer) Done() <-chan struct{}

Done returns a channel which gets closed when the BlockStreamer has finished.

func (*BlockStreamer) Err

func (bs *BlockStreamer) Err() error

Err returns any errors that happened after the operation of the BlockStreamer, for example when blocks could not be put to all nodes.

type ClusterDAGService

type ClusterDAGService interface {
	ipld.DAGService
	// Finalize receives the IPFS content root CID as
	// returned by the ipfs adder.
	Finalize(ctx context.Context, ipfsRoot api.Cid) (api.Cid, error)
	// Close performs any necessary cleanups and should be called
	// whenever the DAGService is not going to be used anymore.
	Close() error
	// Allocations returns the allocations made by the cluster DAG service
	// for the added content.
	Allocations() []peer.ID
}

ClusterDAGService is an implementation of ipld.DAGService plus a Finalize method. ClusterDAGServices can be used to provide Adders with a different add implementation.

Directories

Path Synopsis
Package adderutils provides some utilities for adding content to cluster.
Package adderutils provides some utilities for adding content to cluster.
Package ipfsadd is a simplified copy of go-ipfs/core/coreunix/add.go
Package ipfsadd is a simplified copy of go-ipfs/core/coreunix/add.go
Package sharding implements a sharding ClusterDAGService places content in different shards while it's being added, creating a final Cluster DAG and pinning it.
Package sharding implements a sharding ClusterDAGService places content in different shards while it's being added, creating a final Cluster DAG and pinning it.
Package single implements a ClusterDAGService that chunks and adds content to cluster without sharding, before pinning it.
Package single implements a ClusterDAGService that chunks and adds content to cluster without sharding, before pinning it.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL