ipld

type ErrInvalidKey ¶

type ErrInvalidKey struct {
	Reason string
}

ErrInvalidKey may be returned from lookup functions on the Node interface when a key is invalid.

Common examples of this are when `Lookup(Node)` is used with a non-string Node; typed nodes also introduce other reasons a key may be invalid.

func (e ErrInvalidKey) Error() string

type ErrIteratorOverread ¶

type ErrIteratorOverread struct{}

ErrIteratorOverread is returned when calling 'Next' on a MapIterator or ListIterator when it is already done.

func (e ErrIteratorOverread) Error() string

type ErrNotExists ¶

type ErrNotExists struct {
	Segment PathSegment
}

ErrNotExists may be returned from the lookup functions of the Node interface to indicate a missing value.

Note that typed.ErrNoSuchField is another type of error which sometimes occurs in similar places as ErrNotExists. ErrNoSuchField is preferred when handling data with constraints provided by a schema that mean that a field can *never* exist (as differentiated from a map key which is simply absent in some data).

func (e ErrNotExists) Error() string

type ErrWrongKind ¶

type ErrWrongKind struct {
	// TypeName may optionally indicate the named type of a node the function
	// was called on (if the node was typed!), or, may be the empty string.
	TypeName string

	// MethodName is literally the string for the operation attempted, e.g.
	// "AsString".
	//
	// For methods on nodebuilders, we say e.g. "NodeBuilder.CreateMap".
	MethodName string

	// ApprorpriateKind describes which ReprKinds the erroring method would
	// make sense for.
	AppropriateKind ReprKindSet

	// ActualKind describes the ReprKind of the node the method was called on.
	//
	// In the case of typed nodes, this will typically refer to the 'natural'
	// data-model kind for such a type (e.g., structs will say 'map' here).
	ActualKind ReprKind
}

ErrWrongKind may be returned from functions on the Node interface when a method is invoked which doesn't make sense for the Kind and/or ReprKind that node concretely contains.

For example, calling AsString on a map will return ErrWrongKind. Calling Lookup on an int will similarly return ErrWrongKind.

func (e ErrWrongKind) Error() string

type Link ¶

type Link interface {
	// Load returns a Node identified by the Link.
	//
	// The provided Loader function is used to get a reader for the raw
	// serialized content; the Link contains an understanding of how to
	// select a decoder (and hasher for verification, etc).
	Load(context.Context, LinkContext, NodeBuilder, Loader) (Node, error)

	// LinkBuilder returns a handle to any parameters of the Link which
	// are needed to create a new Link of the same style but with new content.
	// (It's much like the relationship of Node/NodeBuilder.)
	//
	// (If you're familiar with CIDs, you can think of this method as
	// corresponding closely to `cid.Prefix()`, just more abstractly.)
	LinkBuilder() LinkBuilder

	// String should return a reasonably human-readable debug-friendly
	// representation of a Link.  It should only be used for debug and
	// log message purposes; there is no contract that requires that the
	// string be able to be parsed back into a reified Link.
	String() string
}

Link is a special kind of value in IPLD which can be "loaded" to access more nodes.

Nodes can return a Link; this can be loaded manually, or, the traversal package contains powerful features for automatically traversing links through large trees of nodes.

Links straddle somewhat awkwardly across the IPLD Layer Model: clearly not at the Schema layer (though schemas can define their parameters), partially at the Data Model layer (as they're recognizably in the Node interface), and also involved at some serial layer that we don't often talk about: linking -- since we're a content-addressed system at heart -- necessarily involves understanding of concrete serialization details: which encoding mechanisms to use, what string escaping, what hashing, etc, and indeed what concrete serial link representation itself to use.

Link is an abstract interface so that we can describe Nodes without getting stuck on specific details of any link representation. In practice, you'll almost certainly use CIDs for linking. However, it's possible to bring your own Link implementations (though this'll almost certainly involve also bringing your own encoding systems; it's a lot of work). It's even possible to use IPLD *entirely without* any linking implementation, using it purely for json/cbor via the encoding packages and foregoing the advanced traversal features around transparent link loading.

type LinkBuilder ¶

type LinkBuilder interface {
	Build(context.Context, LinkContext, Node, Storer) (Link, error)
}

LinkBuilder encapsulates any implementation details and parameters necessary for taking a Node and converting it to a serial representation and returning a Link to that data.

The serialized bytes will be routed through the provided Storer system, which is expected to store them in some way such that a related Loader system can later use the Link and an associated Loader to load nodes of identical content.

LinkBuilder, like Link, is an abstract interface. If using CIDs as an implementation, LinkBuilder will encapsulate things like multihashType, multicodecType, and cidVersion, for example.

type LinkContext ¶

type LinkContext struct {
	LinkPath   Path
	LinkNode   Node // has the Link again, but also might have type info // always zero for writing new nodes, for obvi reasons.
	ParentNode Node
}

LinkContext is a parameter to Storer and Loader functions.

An example use of LinkContext might be inspecting the LinkNode, and if it's a typed node, inspecting its Type property; then, a Loader might deciding on whether or not we want to load objects of that Type. This might be used to do a traversal which looks at all directory objects, but not file contents, for example.

type ListBuilder ¶

type ListBuilder interface {
	AppendAll([]Node) error
	Append(v Node) error
	Set(idx int, v Node) error
	Build() (Node, error)

	BuilderForValue(idx int) NodeBuilder
}

ListBuilder is an interface for creating new Node instances of kind list.

A ListBuilder is generally obtained by getting a NodeBuilder first, and then using CreateList or AmendList to begin.

Methods mutate the builder's internal state; when done, call Build to produce a new immutable Node from the internal state. (After calling Build, future mutations may be rejected.)

Methods may error when handling typed lists if non-matching types are inserted.

The BuilderForValue function returns a NodeBuilder that can be used to produce values for insertion. If you already have the data you're inserting, you can use those Nodes; if you don't, use these builders. (This is particularly relevant for typed nodes and bind nodes, since those have internal specializations, and not all NodeBuilders for them are equal.) Note that BuilderForValue requires an index as a parameter! In most cases, this is not relevant and the method returns a constant NodeBuilder; however, typed nodes which are structs and have list representations may return different builders per index, corresponding to the types of its fields.

You may be interested in the fluent package's fluent.ListBuilder equivalent for common usage with less error-handling boilerplate requirements.

type ListIterator ¶

type ListIterator interface {
	// Next returns the next index and value.
	//
	// An error value can also be returned at any step: in the case of advanced
	// data structures with incremental loading, it's possible to encounter
	// cancellation or I/O errors at any point in iteration.
	// If an error is returned, the boolean will always be false (so it's
	// correct to check the bool first and short circuit to continuing if true).
	// If an error is returned, the key and value may be nil.
	Next() (idx int, value Node, err error)

	// Done returns false as long as there's at least one more entry to iterate.
	// When Done returns false, iteration can stop.
	//
	// Note when implementing iterators for advanced data layouts (e.g. more than
	// one chunk of backing data, which is loaded incrementally): if your
	// implementation does any I/O during the Done method, and it encounters
	// an error, it must return 'false', so that the following Next call
	// has an opportunity to return the error.
	Done() bool
}

ListIterator is an interface for traversing list nodes. Sequential calls to Next() will yield index-value pairs; Done() describes whether iteration should continue.

A loop which iterates from 0 to Node.Length is a valid alternative to using a ListIterator.

type Loader ¶

type Loader func(lnk Link, lnkCtx LinkContext) (io.Reader, error)

Loader functions are used to get a reader for raw serialized content based on the lookup information in a Link. A loader function is used by providing it to a Link.Load() call.

Loaders typically have some filesystem or database handle contained within their closure which is used to satisfy read operations.

LinkContext objects can be provided to give additional information to the loader, and will be automatically filled out when a Loader is used by systems in the traversal package; most Loader implementations should also work fine when given the zero value of LinkContext.

Loaders are implicitly coupled to a Link implementation and have some "extra" knowledge of the concrete Link type. This necessary since there is no mandated standard for how to serially represent Link itself, and such a representation is typically needed by a Storer implementation.

type MapBuilder ¶

type MapBuilder interface {
	Insert(k, v Node) error
	Delete(k Node) error
	Build() (Node, error)

	BuilderForKeys() NodeBuilder
	BuilderForValue(k string) NodeBuilder
}

MapBuilder is an interface for creating new Node instances of kind map.

A MapBuilder is generally obtained by getting a NodeBuilder first, and then using CreateMap or AmendMap to begin.

Methods mutate the builder's internal state; when done, call Build to produce a new immutable Node from the internal state. (After calling Build, future mutations may be rejected.)

Insertion methods error if the key already exists.

The BuilderForKeys and BuilderForValue functions return NodeBuilders that can be used to produce values for insertion. If you already have the data you're inserting, you can use those Nodes; if you don't, use these builders. (This is particularly relevant for typed nodes and bind nodes, since those have internal specializations, and not all NodeBuilders for them are equal.) Note that BuilderForValue requires a key as a parameter! This is because typed nodes which are structs may return different builders per field, specific to the field's type.

You may be interested in the fluent package's fluent.MapBuilder equivalent for common usage with less error-handling boilerplate requirements.

type MapIterator ¶

type MapIterator interface {
	// Next returns the next key-value pair.
	//
	// An error value can also be returned at any step: in the case of advanced
	// data structures with incremental loading, it's possible to encounter
	// cancellation or I/O errors at any point in iteration.
	// If an error is returned, the boolean will always be false (so it's
	// correct to check the bool first and short circuit to continuing if true).
	// If an error is returned, the key and value may be nil.
	Next() (key Node, value Node, err error)

	// Done returns false as long as there's at least one more entry to iterate.
	// When Done returns true, iteration can stop.
	//
	// Note when implementing iterators for advanced data layouts (e.g. more than
	// one chunk of backing data, which is loaded incrementally): if your
	// implementation does any I/O during the Done method, and it encounters
	// an error, it must return 'false', so that the following Next call
	// has an opportunity to return the error.
	Done() bool
}

MapIterator is an interface for traversing map nodes. Sequential calls to Next() will yield key-value pairs; Done() describes whether iteration should continue.

Iteration order is defined to be stable: two separate MapIterator created to iterate the same Node will yield the same key-value pairs in the same order. The order itself may be defined by the Node implementation: some Nodes may retain insertion order, and some may return iterators which always yield data in sorted order, for example.

type Node ¶

type Node interface {
	// Kind returns a value from the ReprKind enum describing what the
	// essential serializable kind of this node is (map, list, int, etc).
	// Most other handling of a node requires first switching upon the kind.
	ReprKind() ReprKind

	// LookupString looks up a child object in this node and returns it.
	// The returned Node may be any of the ReprKind:
	// a primitive (string, int, etc), a map, a list, or a link.
	//
	// If the Kind of this Node is not ReprKind_Map, a nil node and an error
	// will be returned.
	//
	// If the key does not exist, a nil node and an error will be returned.
	LookupString(key string) (Node, error)

	// Lookup is the equivalent of LookupString, but takes a reified Node
	// as a parameter instead of a plain string.
	// This mechanism is useful if working with typed maps (if the key types
	// have constraints, and you already have a reified `typed.Node` value,
	// using that value can save parsing and validation costs);
	// and may simply be convenient if you already have a Node value in hand.
	//
	// (When writing generic functions over Node, a good rule of thumb is:
	// when handling a map, check for `typed.Node`, and in this case prefer
	// the Lookup(Node) method; otherwise, favor LookupString; typically
	// implementations will have their fastest paths thusly.)
	Lookup(key Node) (Node, error)

	// LookupIndex is the equivalent of LookupString but for indexing into a list.
	// As with LookupString, the returned Node may be any of the ReprKind:
	// a primitive (string, int, etc), a map, a list, or a link.
	//
	// If the Kind of this Node is not ReprKind_List, a nil node and an error
	// will be returned.
	//
	// If idx is out of range, a nil node and an error will be returned.
	LookupIndex(idx int) (Node, error)

	// LookupSegment is will act as either LookupString or LookupIndex,
	// whichever is contextually appropriate.
	//
	// Using LookupSegment may imply an "atoi" conversion if used on a list node,
	// or an "itoa" conversion if used on a map node.  If an "itoa" conversion
	// takes place, it may error, and this method may return that error.
	LookupSegment(seg PathSegment) (Node, error)

	// MapIterator returns an iterator which yields key-value pairs
	// traversing the node.
	// If the node kind is anything other than a map, the iterator will
	// yield error values.
	//
	// The iterator will yield every entry in the map; that is, it
	// can be expected that itr.Next will be called node.Length times
	// before itr.Done becomes true.
	MapIterator() MapIterator

	// ListIterator returns an iterator which yields key-value pairs
	// traversing the node.
	// If the node kind is anything other than a list, the iterator will
	// yield error values.
	//
	// The iterator will yield every entry in the list; that is, it
	// can be expected that itr.Next will be called node.Length times
	// before itr.Done becomes true.
	ListIterator() ListIterator

	// Length returns the length of a list, or the number of entries in a map,
	// or -1 if the node is not of list nor map kind.
	Length() int

	// Undefined nodes are returned when traversing a struct field that is
	// defined by a schema but unset in the data.  (Undefined nodes are not
	// possible otherwise; you'll only see them from `typed.Node`.)
	// The undefined flag is necessary so iterating over structs can
	// unambiguously make the distinction between values that are
	// present-and-null versus values that are absent.
	IsUndefined() bool

	IsNull() bool
	AsBool() (bool, error)
	AsInt() (int, error)
	AsFloat() (float64, error)
	AsString() (string, error)
	AsBytes() ([]byte, error)
	AsLink() (Link, error)

	// NodeBuilder returns a NodeBuilder which can be used to build
	// new nodes of the same implementation type as this one.
	//
	// For map and list nodes, the NodeBuilder's append-oriented methods
	// will work using this node's values as a base.
	// If this is a typed node, the NodeBuilder will carry the same
	// typesystem constraints as this Node.
	//
	// (This feature is used by the traversal package, especially in
	// e.g. traversal.Transform, for doing tree updates while keeping the
	// existing implementation preferences and doing as many operations
	// in copy-on-write fashions as possible.)
	//
	// ---
	//
	// More specifically, the contract of a NodeBuilder returned by this method
	// is that it should be able to "replace" this node with a new one of
	// similar properties.
	// E.g., for a string, the builder must be able to build a new string.
	// For a map, the builder must be able to build a new map.
	// For a *struct* (when using typed nodes), the builder must be able to
	// build new structs of the name type.
	// Note that the promise doesn't extend further: there's no requirement
	// that the builder be able to build maps if this node's kind is "string"
	// (you can see why this lack-of-contract is important when considering
	// typed nodes: if this node has a struct type, then should the builder
	// be able to build other structs of different types?  Of course not;
	// there'd be no way to define which other types to build!).
	// For nulls, this means the builder doesn't have to do much at all!
	//
	// (Some Nodes may return a NodeBuilder that can be used for much more
	// than replacing their own kind: for example, Node implementations from
	// the ipldfree package tend to return a NodeBuilder than can build any
	// other ipldfree.Node (e.g. even the builder obtained from a string node
	// will be able to build maps).  This is not required by the contract;
	// such packages only do so out of internal implementation convenience.)
	//
	// This "able to replace" behavior also has a specific application regarding
	// nodes implementing Advanced Data Layouts: it means that the NodeBuilder
	// returned by this method must produce a new Node using that same ADL.
	// For example, if a Node is a map implemented by some sort of HAMT, its
	// NodeBuilder must also produce a new HAMT.
	NodeBuilder() NodeBuilder
}

var Null Node = nullNode{}

var Undef Node = undefNode{}

type NodeBuilder ¶

type NodeBuilder interface {
	CreateMap() (MapBuilder, error)
	AmendMap() (MapBuilder, error)
	CreateList() (ListBuilder, error)
	AmendList() (ListBuilder, error)
	CreateNull() (Node, error)
	CreateBool(bool) (Node, error)
	CreateInt(int) (Node, error)
	CreateFloat(float64) (Node, error)
	CreateString(string) (Node, error)
	CreateBytes([]byte) (Node, error)
	CreateLink(Link) (Node, error)
}

NodeBuilder is an interface that describes creating new Node instances.

The Node interface is entirely read-only methods; a Node is immutable. Thus, we need a NodeBuilder system for creating new ones; the builder is mutable, and when we're done accumulating mutations, we take the accumulated data and produce an immutable Node out of it.

Separating mutation into NodeBuilder and keeping Node immutable makes it possible to perform caching (or rather, memoization, since there's no such thing as cache invalidation for immutable systems) of computed properties of Node; use copy-on-write algorithms for memory efficiency; and to generally build pleasant APIs.

Each package in `go-ipld-prime//impl/*` that implements ipld.Node also has a NodeBuilder implementation that produces new nodes of that same package's type.

Most Node implementations also have a method which returns a NodeBuilder that produces more nodes of their same concrete implementation type. This is useful for algorithms that work on trees of nodes: this NodeBuilder getter will be used when an update deep in the tree causes a need to create several new nodes to propagate the change up through parent nodes.

The NodeBuilder retrieved from a Node can also be used to do *updates*: consider the AmendMap and AmendList methods. These methods are useful not just for programmer convenience, but also because they can reuse memory, sharing any common segments of memory with the earlier Node. (In the NodeBuilder exposed by the `go-ipld-prime//impl/*` packages, these methods are equivalent to their Create* counterparts. As there's no "existing" node for them to refer to, it's treated the same as amending an empty node.)

NodeBuilder instances obtained from Node.GetBuilder may carry some of the additional logic of their parent with them to the new Node they produce. For example, the NodeBuilder from typed.Node.GetBuilder may keep the type info and type constraints of their parent with them! (Continuing the typed.Node example: if you have a typed.Node that is constrained to be of some `type Foo = {Bar:Baz}` type, then any new Node produced from its NodeBuilder will still answer `n.(typed.Node).Type().Name()` as `Foo`; and if `n.GetBuilder().AmendMap().Insert(...)` is called with nodes of unmatching type given to the insertion, the builder will error!)

type Path ¶

type Path struct {
	// contains filtered or unexported fields
}

Path is used in describing progress in a traversal; and can also be used as an instruction for a specific traverse.

IPLD Paths can only go down: that is, each segment must traverse one node. There is no ".." which means "go up"; and there is no "." which means "stay here"; and it is not valid to have an empty path segment.

(Note: path strings as interpreted by UnixFS may certainly have concepts of ".." and "."! But UnixFS is built upon IPLD; IPLD has no idea of this.)

Paths are representable as strings. When represented as a string, each segment is separated by a "/" character. (It follows that path segments may not themselves contain a "/" character.) (Note: escaping may be specified and supported in the future; currently, it is not.)

func ParsePath(pth string) Path

func (p Path) AppendSegment(ps PathSegment) Path

func (p Path) AppendSegmentString(ps string) Path

func (p Path) Join(p2 Path) Path

func (p Path) Parent() Path

func (p Path) Segments() []PathSegment

func (p Path) String() string

func (p Path) Truncate(i int) Path

type PathSegment ¶

type PathSegment struct {
	// contains filtered or unexported fields
}

PathSegment can describe either a key in a map, or an index in a list.

Path segments are "stringly typed" -- they may be interpreted as either strings or ints depending on context. A path segment of "123" will be used as a string when traversing a node of map kind; and it will be converted to an integer when traversing a node of list kind. (If a path segment string cannot be parsed to an int when traversing a node of list kind, then traversal will error.) It is not possible to ask which kind (string or integer) a PathSegment is, because that is not defined -- this is *only* intepreted contextually.

Internally, PathSegment will store either a string or an integer, depending on how it was constructed, and will automatically convert to the other on request. (This means if two pieces of code communicate using PathSegment, one producing ints and the other expecting ints, they will work together efficiently.) PathSegment in a Path produced by ParsePath generally have all strings internally, because there is distinction possible when parsing a Path string (and attempting to pre-parse all strings into ints "in case" would waste time in almost all cases).

func ParsePathSegment(s string) PathSegment

func PathSegmentOfInt(i int) PathSegment

func PathSegmentOfString(s string) PathSegment

func (x PathSegment) Equals(o PathSegment) bool

func (ps PathSegment) Index() (int, error)

func (ps PathSegment) String() string

type ReprKind ¶

type ReprKind uint8

ReprKind represents the primitive kind in the IPLD data model. All of these kinds map directly onto serializable data.

Note that ReprKind contains the concept of "map", but not "struct" or "object" -- those are a concepts that could be introduced in a type system layers, but are *not* present in the data model layer, and therefore they aren't included in the ReprKind enum.

const (
	ReprKind_Invalid ReprKind = 0
	ReprKind_Map     ReprKind = '{'
	ReprKind_List    ReprKind = '['
	ReprKind_Null    ReprKind = '0'
	ReprKind_Bool    ReprKind = 'b'
	ReprKind_Int     ReprKind = 'i'
	ReprKind_Float   ReprKind = 'f'
	ReprKind_String  ReprKind = 's'
	ReprKind_Bytes   ReprKind = 'x'
	ReprKind_Link    ReprKind = '/'
)

func (k ReprKind) String() string

type ReprKindSet ¶

type ReprKindSet []ReprKind

ReprKindSet is a type with a few enumerated consts that are commonly used (mostly, in error messages).

func (x ReprKindSet) String() string

type StoreCommitter ¶

type StoreCommitter func(Link) error

StoreCommitter is a thunk returned by a Storer which is used to "commit" the storage operation. It should be called after the associated writer is finished, similar to a 'Close' method, but further takes a Link parameter, which is the identity of the content. Typically, this will cause an atomic operation in the storage system to move the already-written content into a final place (e.g. rename a tempfile) determined by the Link. (The Link parameter is necessarily only given at the end of the process rather than the beginning to so that we can have content-addressible semantics while also supporting streaming writes.)

type Storer ¶

type Storer func(lnkCtx LinkContext) (io.Writer, StoreCommitter, error)

Storer functions are used to a get a writer for raw serialized content, which will be committed to storage indexed by Link. A stoerer function is used by providing it to a LinkBuilder.Build() call.

The storer system comes in two parts: the Storer itself *starts* a storage operation (presumably to some e.g. tempfile) and returns a writer; the StoreCommitter returned with the writer is used to *commit* the final storage (much like a 'Close' operation for the writer).

Storers typically have some filesystem or database handle contained within their closure which is used to satisfy read operations.

LinkContext objects can be provided to give additional information to the storer, and will be automatically filled out when a Storer is used by systems in the traversal package; most Storer implementations should also work fine when given the zero value of LinkContext.

Storers are implicitly coupled to a Link implementation and have some "extra" knowledge of the concrete Link type. This necessary since there is no mandated standard for how to serially represent Link itself, and such a representation is typically needed by a Storer implementation.