ipfs-cluster: github.com/ipfs/ipfs-cluster/adder/sharding Index | Files

package sharding

import "github.com/ipfs/ipfs-cluster/adder/sharding"

Package sharding implements a sharding ClusterDAGService places content in different shards while it's being added, creating a final Cluster DAG and pinning it.


Package Files

dag.go dag_service.go shard.go verify.go


const MaxLinks = 5984

MaxLinks is the max number of links that, when serialized fit into a block

func CborDataToNode Uses

func CborDataToNode(raw []byte, format string) (ipld.Node, error)

CborDataToNode parses cbor data into a clusterDAG node while making a few checks

func VerifyShards Uses

func VerifyShards(t *testing.T, rootCid cid.Cid, pins MockPinStore, ipfs MockBlockStore, expectedShards int) (map[string]struct{}, error)

VerifyShards checks that a sharded CID has been correctly formed and stored. This is a helper function for testing. It returns a map with all the blocks from all shards.

type DAGService Uses

type DAGService struct {
    // contains filtered or unexported fields

DAGService is an implementation of a ClusterDAGService which shards content while adding among several IPFS Cluster peers, creating a Cluster DAG to track and pin that content selectively in the IPFS daemons allocated to it.

func New Uses

func New(rpc *rpc.Client, opts api.PinOptions, out chan<- *api.AddedOutput) *DAGService

New returns a new ClusterDAGService, which uses the given rpc client to perform Allocate, IPFSBlockPut and Pin requests to other cluster components.

func (*DAGService) Add Uses

func (dgs *DAGService) Add(ctx context.Context, node ipld.Node) error

Add puts the given node in its corresponding shard and sends it to the destination peers.

func (*DAGService) AddMany Uses

func (dgs *DAGService) AddMany(ctx context.Context, nodes []ipld.Node) error

AddMany calls Add for every given node.

func (*DAGService) Finalize Uses

func (dgs *DAGService) Finalize(ctx context.Context, dataRoot cid.Cid) (cid.Cid, error)

Finalize finishes sharding, creates the cluster DAG and pins it along with the meta pin for the root node of the content.

type MockBlockStore Uses

type MockBlockStore interface {
    // Gets a block
    BlockGet(context.Context, cid.Cid) ([]byte, error)

MockBlockStore is used in VerifyShards

type MockPinStore Uses

type MockPinStore interface {
    // Gets a pin
    PinGet(context.Context, cid.Cid) (*api.Pin, error)

MockPinStore is used in VerifyShards

Package sharding imports 17 packages (graph) and is imported by 2 packages. Updated 2020-04-30. Refresh now. Tools for package owners.