ipfs-cluster: github.com/ipfs/ipfs-cluster/api Index | Files | Directories

package api

import "github.com/ipfs/ipfs-cluster/api"

Package api holds declarations for types used in ipfs-cluster APIs to make them re-usable across differen tools. This include RPC API "Serial[izable]" versions for types. The Go API uses natives types, while RPC API, REST APIs etc use serializable types (i.e. json format). Conversion methods exists between types.

Note that all conversion methods ignore any parsing errors. All values must be validated first before initializing any of the types defined here.


Package Files

add.go types.go util.go


const (
    TrackerStatusError  = TrackerStatusClusterError | TrackerStatusPinError | TrackerStatusUnpinError
    TrackerStatusQueued = TrackerStatusPinQueued | TrackerStatusUnpinQueued

Composite TrackerStatus.


var DefaultShardSize = uint64(100 * 1024 * 1024) // 100 MB

DefaultShardSize is the shard size for params objects created with DefaultParams().

func PeersToStrings Uses

func PeersToStrings(peers []peer.ID) []string

PeersToStrings Encodes a list of peers.

func StringsToPeers Uses

func StringsToPeers(strs []string) []peer.ID

StringsToPeers decodes peer.IDs from strings.

type AddParams Uses

type AddParams struct {

    Local          bool
    Recursive      bool
    Layout         string
    Chunker        string
    RawLeaves      bool
    Hidden         bool
    Wrap           bool
    Shard          bool
    Progress       bool
    CidVersion     int
    HashFun        string
    StreamChannels bool
    NoCopy         bool

AddParams contains all of the configurable parameters needed to specify the importing process of a file being added to an ipfs-cluster

func AddParamsFromQuery Uses

func AddParamsFromQuery(query url.Values) (*AddParams, error)

AddParamsFromQuery parses the AddParams object from a URL.Query().

func DefaultAddParams Uses

func DefaultAddParams() *AddParams

DefaultAddParams returns a AddParams object with standard defaults

func (*AddParams) Equals Uses

func (p *AddParams) Equals(p2 *AddParams) bool

Equals checks if p equals p2.

func (*AddParams) ToQueryString Uses

func (p *AddParams) ToQueryString() (string, error)

ToQueryString returns a url query string (key=value&key2=value2&...)

type AddedOutput Uses

type AddedOutput struct {
    Name  string  `json:"name" codec:"n,omitempty"`
    Cid   cid.Cid `json:"cid" codec:"c"`
    Bytes uint64  `json:"bytes,omitempty" codec:"b,omitempty"`
    Size  uint64  `json:"size,omitempty" codec:"s,omitempty"`

AddedOutput carries information for displaying the standard ipfs output indicating a node of a file has been added.

type Alert Uses

type Alert struct {
    Peer       peer.ID
    MetricName string

Alert carries alerting information about a peer. WIP.

type ConnectGraph Uses

type ConnectGraph struct {
    ClusterID    peer.ID           `json:"cluster_id" codec:"id"`
    IDtoPeername map[string]string `json:"id_to_peername" codec:"ip,omitempty"`
    // ipfs to ipfs links
    IPFSLinks map[string][]peer.ID `json:"ipfs_links" codec:"il,omitempty"`
    // cluster to cluster links
    ClusterLinks map[string][]peer.ID `json:"cluster_links" codec:"cl,omitempty"`
    // cluster trust links
    ClusterTrustLinks map[string]bool `json:"cluster_trust_links" codec:"ctl,omitempty"`
    // cluster to ipfs links
    ClustertoIPFS map[string]peer.ID `json:"cluster_to_ipfs" codec:"ci,omitempty"`

ConnectGraph holds information about the connectivity of the cluster To read, traverse the keys of ClusterLinks. Each such id is one of the peers of the "ClusterID" peer running the query. ClusterLinks[id] in turn lists the ids that peer "id" sees itself connected to. It is possible that id is a peer of ClusterID, but ClusterID can not reach id over rpc, in which case ClusterLinks[id] == [], as id's view of its connectivity can not be retrieved.

Iff there was an error reading the IPFSID of the peer then id will not be a key of ClustertoIPFS or IPFSLinks. Finally iff id is a key of ClustertoIPFS then id will be a key of IPFSLinks. In the event of a SwarmPeers error IPFSLinks[id] == [].

type Error Uses

type Error struct {
    Code    int    `json:"code" codec:"o,omitempty"`
    Message string `json:"message" codec:"m,omitempty"`

Error can be used by APIs to return errors.

func (*Error) Error Uses

func (e *Error) Error() string

Error implements the error interface and returns the error's message.

type GlobalPinInfo Uses

type GlobalPinInfo struct {
    Cid  cid.Cid `json:"cid" codec:"c"`
    Name string  `json:"name" codec:"n"`
    // https://github.com/golang/go/issues/28827
    // Peer IDs are of string Kind(). We can't use peer IDs here
    // as Go ignores TextMarshaler.
    PeerMap map[string]*PinInfoShort `json:"peer_map" codec:"pm,omitempty"`

GlobalPinInfo contains cluster-wide status information about a tracked Cid, indexed by cluster peer.

func (*GlobalPinInfo) Add Uses

func (gpi *GlobalPinInfo) Add(pi *PinInfo)

Add adds a PinInfo object to a GlobalPinInfo

func (*GlobalPinInfo) String Uses

func (gpi *GlobalPinInfo) String() string

String returns the string representation of a GlobalPinInfo.

type GlobalRepoGC Uses

type GlobalRepoGC struct {
    PeerMap map[string]*RepoGC `json:"peer_map" codec:"pm,omitempty"`

GlobalRepoGC contains cluster-wide information about garbage collected CIDs from IPFS.

type ID Uses

type ID struct {
    ID                    peer.ID     `json:"id" codec:"i,omitempty"`
    Addresses             []Multiaddr `json:"addresses" codec:"a,omitempty"`
    ClusterPeers          []peer.ID   `json:"cluster_peers" codec:"cp,omitempty"`
    ClusterPeersAddresses []Multiaddr `json:"cluster_peers_addresses" codec:"cpa,omitempty"`
    Version               string      `json:"version" codec:"v,omitempty"`
    Commit                string      `json:"commit" codec:"c,omitempty"`
    RPCProtocolVersion    protocol.ID `json:"rpc_protocol_version" codec:"rv,omitempty"`
    Error                 string      `json:"error" codec:"e,omitempty"`
    IPFS                  *IPFSID     `json:"ipfs,omitempty" codec:"ip,omitempty"`
    Peername              string      `json:"peername" codec:"pn,omitempty"`

ID holds information about the Cluster peer

type IPFSID Uses

type IPFSID struct {
    ID        peer.ID     `json:"id,omitempty" codec:"i,omitempty"`
    Addresses []Multiaddr `json:"addresses" codec:"a,omitempty"`
    Error     string      `json:"error" codec:"e,omitempty"`

IPFSID is used to store information about the underlying IPFS daemon

type IPFSPinStatus Uses

type IPFSPinStatus int

IPFSPinStatus represents the status of a pin in IPFS (direct, recursive etc.)

const (
    IPFSPinStatusBug IPFSPinStatus = iota

IPFSPinStatus values FIXME include maxdepth

func IPFSPinStatusFromString Uses

func IPFSPinStatusFromString(t string) IPFSPinStatus

IPFSPinStatusFromString parses a string and returns the matching IPFSPinStatus.

func (IPFSPinStatus) IsPinned Uses

func (ips IPFSPinStatus) IsPinned(maxDepth PinDepth) bool

IsPinned returns true if the item is pinned as expected by the maxDepth parameter.

func (IPFSPinStatus) ToTrackerStatus Uses

func (ips IPFSPinStatus) ToTrackerStatus() TrackerStatus

ToTrackerStatus converts the IPFSPinStatus value to the appropriate TrackerStatus value.

type IPFSRepoGC Uses

type IPFSRepoGC struct {
    Key   cid.Cid `json:"key,omitempty" codec:"k,omitempty"`
    Error string  `json:"error,omitempty" codec:"e,omitempty"`

IPFSRepoGC represents the streaming response sent from repo gc API of IPFS.

type IPFSRepoStat Uses

type IPFSRepoStat struct {
    RepoSize   uint64 `codec:"r,omitempty"`
    StorageMax uint64 `codec:"s, omitempty"`

IPFSRepoStat wraps information about the IPFS repository.

type Metric Uses

type Metric struct {
    Name       string  `json:"name" codec:"n,omitempty"`
    Peer       peer.ID `json:"peer" codec:"p,omitempty"`
    Value      string  `json:"value" codec:"v,omitempty"`
    Expire     int64   `json:"expire" codec:"e,omitempty"`
    Valid      bool    `json:"valid" codec:"d,omitempty"`
    ReceivedAt int64   `json:"received_at" codec:"t,omitempty"` // ReceivedAt contains a UnixNano timestamp

Metric transports information about a peer.ID. It is used to decide pin allocations by a PinAllocator. IPFS cluster is agnostic to the Value, which should be interpreted by the PinAllocator. The ReceivedAt value is a timestamp representing when a peer has received the metric value.

func (*Metric) Discard Uses

func (m *Metric) Discard() bool

Discard returns if the metric not valid or has expired

func (*Metric) Expired Uses

func (m *Metric) Expired() bool

Expired returns if the Metric has expired

func (*Metric) GetTTL Uses

func (m *Metric) GetTTL() time.Duration

GetTTL returns the time left before the Metric expires

func (*Metric) SetTTL Uses

func (m *Metric) SetTTL(d time.Duration)

SetTTL sets Metric to expire after the given time.Duration

type MetricSlice Uses

type MetricSlice []*Metric

MetricSlice is a sortable Metric array.

func (MetricSlice) Len Uses

func (es MetricSlice) Len() int

func (MetricSlice) Less Uses

func (es MetricSlice) Less(i, j int) bool

func (MetricSlice) Swap Uses

func (es MetricSlice) Swap(i, j int)

type Multiaddr Uses

type Multiaddr struct {

Multiaddr is a concrete type to wrap a Multiaddress so that it knows how to serialize and deserialize itself.

func NewMultiaddr Uses

func NewMultiaddr(mstr string) (Multiaddr, error)

NewMultiaddr returns a cluster Multiaddr wrapper creating the multiaddr.Multiaddr with the given string.

func NewMultiaddrWithValue Uses

func NewMultiaddrWithValue(ma multiaddr.Multiaddr) Multiaddr

NewMultiaddrWithValue returns a new cluster Multiaddr wrapper using the given multiaddr.Multiaddr.

func (Multiaddr) MarshalBinary Uses

func (maddr Multiaddr) MarshalBinary() ([]byte, error)

MarshalBinary returs the bytes of the wrapped multiaddress.

func (Multiaddr) MarshalJSON Uses

func (maddr Multiaddr) MarshalJSON() ([]byte, error)

MarshalJSON returns a JSON-formatted multiaddress.

func (*Multiaddr) UnmarshalBinary Uses

func (maddr *Multiaddr) UnmarshalBinary(data []byte) error

UnmarshalBinary casts some bytes as a multiaddress wraps it with the given cluster Multiaddr.

func (*Multiaddr) UnmarshalJSON Uses

func (maddr *Multiaddr) UnmarshalJSON(data []byte) error

UnmarshalJSON parses a cluster Multiaddr from the JSON representation.

func (Multiaddr) Value Uses

func (maddr Multiaddr) Value() multiaddr.Multiaddr

Value returns the wrapped multiaddr.Multiaddr.

type NodeWithMeta Uses

type NodeWithMeta struct {
    Data    []byte  `codec:"d,omitempty"`
    Cid     cid.Cid `codec:"c,omitempty"`
    CumSize uint64  `codec:"s,omitempty"` // Cumulative size

NodeWithMeta specifies a block of data and a set of optional metadata fields carrying information about the encoded ipld node

func (*NodeWithMeta) Size Uses

func (n *NodeWithMeta) Size() uint64

Size returns how big is the block. It is different from CumSize, which records the size of the underlying tree.

type Pin Uses

type Pin struct {

    Cid cid.Cid `json:"cid" codec:"c"`

    // See PinType comments
    Type PinType `json:"type" codec:"t,omitempty"`

    // The peers to which this pin is allocated
    Allocations []peer.ID `json:"allocations" codec:"a,omitempty"`

    // MaxDepth associated to this pin. -1 means
    // recursive.
    MaxDepth PinDepth `json:"max_depth" codec:"d,omitempty"`

    // We carry a reference CID to this pin. For
    // ClusterDAGs, it is the MetaPin CID. For the
    // MetaPin it is the ClusterDAG CID. For Shards,
    // it is the previous shard CID.
    // When not needed the pointer is nil
    Reference *cid.Cid `json:"reference" codec:"r,omitempty"`

Pin carries all the information associated to a CID that is pinned in IPFS Cluster. It also carries transient information (that may not get protobuffed, like UserAllocations).

func PinCid Uses

func PinCid(c cid.Cid) *Pin

PinCid is a shortcut to create a Pin only with a Cid. Default is for pin to be recursive and the pin to be of DataType.

func PinWithOpts Uses

func PinWithOpts(c cid.Cid, opts PinOptions) *Pin

PinWithOpts creates a new Pin calling PinCid(c) and then sets its PinOptions fields with the given options. Pin fields that are linked to options are set accordingly (MaxDepth from Mode).

func (*Pin) Equals Uses

func (pin *Pin) Equals(pin2 *Pin) bool

Equals checks if two pins are the same (with the same allocations). If allocations are the same but in different order, they are still considered equivalent. pin or pin2 may be nil. If both are nil, Equals returns false.

func (*Pin) ExpiredAt Uses

func (pin *Pin) ExpiredAt(t time.Time) bool

ExpiredAt returns whether the pin has expired at the given time.

func (*Pin) IsRemotePin Uses

func (pin *Pin) IsRemotePin(pid peer.ID) bool

IsRemotePin determines whether a Pin's ReplicationFactor has been met, so as to either pin or unpin it from the peer.

func (*Pin) ProtoMarshal Uses

func (pin *Pin) ProtoMarshal() ([]byte, error)

ProtoMarshal marshals this Pin using probobuf.

func (*Pin) ProtoUnmarshal Uses

func (pin *Pin) ProtoUnmarshal(data []byte) error

ProtoUnmarshal unmarshals this fields from protobuf-encoded bytes.

func (*Pin) String Uses

func (pin *Pin) String() string

String is a string representation of a Pin.

type PinDepth Uses

type PinDepth int

PinDepth indicates how deep a pin should be pinned, with -1 meaning "to the bottom", or "recursive".

func (PinDepth) ToPinMode Uses

func (pd PinDepth) ToPinMode() PinMode

ToPinMode converts PinDepth to PinMode

type PinInfo Uses

type PinInfo struct {
    Cid  cid.Cid `json:"cid" codec:"c"`
    Name string  `json:"name" codec:"m,omitempty"`
    Peer peer.ID `json:"Peer" codec:"p,omitempty"`


PinInfo holds information about local pins. This is used by the Pin Trackers.

func (*PinInfo) ToGlobal Uses

func (pi *PinInfo) ToGlobal() *GlobalPinInfo

ToGlobal converts a PinInfo object to a GlobalPinInfo with a single peer corresponding to the given PinInfo.

type PinInfoShort Uses

type PinInfoShort struct {
    PeerName string        `json:"peername" codec:"pn,omitempty"`
    Status   TrackerStatus `json:"status" codec:"st,omitempty"`
    TS       time.Time     `json:"timestamp" codec:"ts,omitempty"`
    Error    string        `json:"error" codec:"e,omitempty"`

PinInfoShort is a subset of PinInfo which is embedded in GlobalPinInfo objects and does not carry redundant information as PinInfo would.

type PinMode Uses

type PinMode int

PinMode is a PinOption that indicates how to pin something on IPFS, recursively or direct.

const (
    PinModeRecursive PinMode = 0
    PinModeDirect    PinMode = 1

PinMode values

func PinModeFromString Uses

func PinModeFromString(s string) PinMode

PinModeFromString converst a string to PinMode.

func (PinMode) MarshalJSON Uses

func (pm PinMode) MarshalJSON() ([]byte, error)

MarshalJSON converts the PinMode into a readable string in JSON.

func (PinMode) String Uses

func (pm PinMode) String() string

String returns a human-readable value for PinMode.

func (PinMode) ToPinDepth Uses

func (pm PinMode) ToPinDepth() PinDepth

ToPinDepth converts the Mode to Depth.

func (*PinMode) UnmarshalJSON Uses

func (pm *PinMode) UnmarshalJSON(b []byte) error

UnmarshalJSON takes a JSON value and parses it into PinMode.

type PinOptions Uses

type PinOptions struct {
    ReplicationFactorMin int               `json:"replication_factor_min" codec:"rn,omitempty"`
    ReplicationFactorMax int               `json:"replication_factor_max" codec:"rx,omitempty"`
    Name                 string            `json:"name" codec:"n,omitempty"`
    Mode                 PinMode           `json:"mode" codec:"o,omitempty"`
    ShardSize            uint64            `json:"shard_size" codec:"s,omitempty"`
    UserAllocations      []peer.ID         `json:"user_allocations" codec:"ua,omitempty"`
    ExpireAt             time.Time         `json:"expire_at" codec:"e,omitempty"`
    Metadata             map[string]string `json:"metadata" codec:"m,omitempty"`
    PinUpdate            cid.Cid           `json:"pin_update,omitempty" codec:"pu,omitempty"`

PinOptions wraps user-defined options for Pins

func (*PinOptions) Equals Uses

func (po *PinOptions) Equals(po2 *PinOptions) bool

Equals returns true if two PinOption objects are equivalent. po and po2 may be nil.

func (*PinOptions) FromQuery Uses

func (po *PinOptions) FromQuery(q url.Values) error

FromQuery is the inverse of ToQuery().

func (*PinOptions) ToQuery Uses

func (po *PinOptions) ToQuery() (string, error)

ToQuery returns the PinOption as query arguments.

type PinPath Uses

type PinPath struct {
    Path string `json:"path"`

PinPath is a wrapper for holding pin options and path of the content.

type PinType Uses

type PinType uint64

PinType specifies which sort of Pin object we are dealing with. In practice, the PinType decides how a Pin object is treated by the PinTracker. See descriptions above. A sharded Pin would look like:

[ Meta ] (not pinned on IPFS, only present in cluster state)


[ Cluster DAG ] (pinned everywhere in "direct")

|      ..  |
v          v

[Shard1] .. [ShardN] (allocated to peers and pinned with max-depth=1 | | .. | | | .. | v v .. v v v .. v [][]..[] [][]..[] Blocks (indirectly pinned on ipfs, not tracked in cluster)

const (
    // BadType type showing up anywhere indicates a bug
    BadType PinType = 1 << iota
    // DataType is a regular, non-sharded pin. It is pinned recursively.
    // It has no associated reference.
    // MetaType tracks the original CID of a sharded DAG. Its Reference
    // points to the Cluster DAG CID.
    // ClusterDAGType pins carry the CID of the root node that points to
    // all the shard-root-nodes of the shards in which a DAG has been
    // divided. Its Reference carries the MetaType CID.
    // ClusterDAGType pins are pinned directly everywhere.
    // ShardType pins carry the root CID of a shard, which points
    // to individual blocks on the original DAG that the user is adding,
    // which has been sharded.
    // They carry a Reference to the previous shard.
    // ShardTypes are pinned with MaxDepth=1 (root and
    // direct children only).

PinType values. See PinType documentation for further explanation.

const AllType PinType = DataType | MetaType | ClusterDAGType | ShardType

AllType is a PinType used for filtering all pin types

func PinTypeFromString Uses

func PinTypeFromString(str string) PinType

PinTypeFromString is the inverse of String. It returns the PinType value corresponding to the input string

func (PinType) String Uses

func (pT PinType) String() string

String returns a printable value to identify the PinType

type RepoGC Uses

type RepoGC struct {
    Peer     peer.ID      `json:"peer" codec:"p,omitempty"` // the Cluster peer ID
    Peername string       `json:"peername" codec:"pn,omitempty"`
    Keys     []IPFSRepoGC `json:"keys" codec:"k"`
    Error    string       `json:"error,omitempty" codec:"e,omitempty"`

RepoGC contains garbage collected CIDs from a cluster peer's IPFS daemon.

type TrackerStatus Uses

type TrackerStatus int

TrackerStatus represents the status of a tracked Cid in the PinTracker

const (
    // IPFSStatus should never take this value.
    // When used as a filter. It means "all".
    TrackerStatusUndefined TrackerStatus = 0
    // The cluster node is offline or not responding
    TrackerStatusClusterError TrackerStatus = 1 << iota
    // An error occurred pinning
    // An error occurred unpinning
    // The IPFS daemon has pinned the item
    // The IPFS daemon is currently pinning the item
    // The IPFS daemon is currently unpinning the item
    // The IPFS daemon is not pinning the item
    // The IPFS daemon is not pinning the item but it is being tracked
    // The item has been queued for pinning on the IPFS daemon
    // The item has been queued for unpinning on the IPFS daemon
    // The IPFS daemon is not pinning the item through this cid but it is
    // tracked in a cluster dag

TrackerStatus values

func TrackerStatusAll Uses

func TrackerStatusAll() []TrackerStatus

TrackerStatusAll all known TrackerStatus values.

func TrackerStatusFromString Uses

func TrackerStatusFromString(str string) TrackerStatus

TrackerStatusFromString parses a string and returns the matching TrackerStatus value. The string can be a comma-separated list representing a TrackerStatus filter. Unknown status names are ignored.

func (TrackerStatus) MarshalJSON Uses

func (st TrackerStatus) MarshalJSON() ([]byte, error)

MarshalJSON uses the string representation of TrackerStatus for JSON encoding.

func (TrackerStatus) Match Uses

func (st TrackerStatus) Match(filter TrackerStatus) bool

Match returns true if the tracker status matches the given filter. For example TrackerStatusPinError will match TrackerStatusPinError and TrackerStatusError

func (TrackerStatus) String Uses

func (st TrackerStatus) String() string

String converts a TrackerStatus into a readable string. If the given TrackerStatus is a filter (with several bits set), it will return a comma-separated list.

func (*TrackerStatus) UnmarshalJSON Uses

func (st *TrackerStatus) UnmarshalJSON(data []byte) error

UnmarshalJSON sets a tracker status from its JSON representation.

type Version Uses

type Version struct {
    Version string `json:"version" codec:"v"`

Version holds version information


pbPackage pb provides protobuf definitions for serialized types in Cluster.
restPackage rest implements an IPFS Cluster API component.
rest/clientPackage client provides a Go Client for the IPFS Cluster API provided by the "api/rest" component.

Package api imports 18 packages (graph) and is imported by 36 packages. Updated 2020-10-25. Refresh now. Tools for package owners.