forest

package module
v0.0.0-...-133031b Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 30, 2023 License: Apache-2.0 Imports: 14 Imported by: 11

README

forest-go

builds.sr.ht status GoDoc

This repo contains:

  • A golang library for working with nodes in the Arbor Forest. This repo is based on the work-in-progress specification available here.
  • A CLI for creating, manipulating, and viewing nodes in the Arbor Forest in cmd/forest

For information about each component of this repo, see later in this file.

NOTE: this package requires using a fork of golang.org/x/crypto, and you must therefore include the following in your go.mod:

    replace golang.org/x/crypto => github.com/ProtonMail/crypto <version-from-forest-go's-go.mod>

About Arbor

arbor logo

Arbor is a chat system that makes communication clearer. It explicitly captures context that other platforms ignore, allowing you to understand the relationship between each message and every other message. It also respects its users and focuses on group collaboration.

You can get information about the Arbor project here.

For news about the project, join our mailing list!

Using the CLI

Right now, the CLI works with files in its current working directory, though this will change in the future. For the meantime, create a directory to play around in:

mkdir arbor-forest
cd arbor-forest

Important: About OpenPGP Keys

Arbor Forest nodes are signed by OpenPGP private keys. This gives Arbor strong guarantees about the authenticity of messages. The below procedures assume that you have gpg2 installed and have already generated a private key. Wherever you see --gpguser <email> below, substitute the email address associated with your GPG private key for <email>.

If you do not have gpg2 or a key and you do not want to install them, you can omit the --gpguser <email> flag in the commands below. If you do this, the CLI will create a new one for you and store it in ./arbor.privkey. This private key is not encrypted (has no passphrase), and should not be used for anything of importance.

Identities

Since all nodes must be signed by an Identity node, you must create one of those before you can create any others.

forest create identity --name <your-name> --gpguser <email>

This will print the base64url-encoded ID of your identity node, which will be stored in a file by that name in your current working directory.

To view your identity in a human-readable format, try the following (install jq if you don't have it, it's really handy):

forest show <id> | jq .

Substitute the base64url-encoded ID of your identity node for <id>. jq will pretty-print the JSON to make it easier to read.

Communities

To create a community, use:

forest create community --as <id> --name <community-name> --gpguser <email>

Substitute the base64url-encoded ID of your identity node for <id> and provide appropriate values for name and metadata.

To view your community in a human-readable format, try the following:

forest show <id> | jq .

Substitute the base64url-encoded ID of your community node for <id>. jq will pretty-print the JSON to make it easier to read.

Replies

To create a reply, use:

forest create reply --as <id> --to <parent-id> --content <your message> --gpguser <email>

Substitute the base64url-encoded ID of your identity node for <id> and the base64url-encoded ID of another reply or conversation node for <parent-id>. Substitute <your message> for the content of your reply. Usually this will be a response to the content of the node referenced by <parent-id>.

To view your reply in a human-readable format, try the following:

forest show <id> | jq .

Substitute the base64url-encoded ID of your reply node for <id>. jq will pretty-print the JSON to make it easier to read.

Build

Must use Go 1.11+

go build

Test

go test -v -cover

Documentation

Overview

Package forest is a library for creating nodes in the Arbor Forest data structure.

The specification for the Arbor Forest can be found here: https://github.com/arborchat/protocol/blob/forest/spec/Forest.md

NOTE: this package requires using a fork of golang.org/x/crypto, and you must therefore include the following in your go.mod: 

replace golang.org/x/crypto => github.com/ProtonMail/crypto <version-from-forest-go's-go.mod>

All nodes in the Arbor Forest are cryptographically signed by an Identity node. Identity nodes sign themselves. To create a new identity, first create or load an OpenPGP private key using golang.org/x/crypto/openpgp. Then you can use that key and name to create an identity. 

privkey := getPrivateKey() // do this however
name, err := fields.NewQualifiedContent(fields.ContentTypeUTF8, "example")
// handle error
metadata, err := fields.NewQualifiedContent(fields.ContentTypeJSON, "{}")
// handle error
identity, err := forest.NewIdentity(privkey, name, metadata)
// handle error

Identities (and their private keys) can be used to create other nodes with the Builder type. You can create community nodes using a builder like so: 

builder := forest.As(identity, privkey)
communityName, err := fields.NewQualifiedContent(fields.ContentTypeUTF8, "example")
// handle error
communityMetadata, err := fields.NewQualifiedContent(fields.ContentTypeJSON, "{}")
// handle error
community, err := builder.NewCommunity(communityName, communityMetadata)
// handle error

Builders can also create reply nodes: 

message, err := fields.NewQualifiedContent(fields.ContentTypeUTF8, "example")
// handle error
replyMetadata, err := fields.NewQualifiedContent(fields.ContentTypeJSON, "{}")
// handle error
reply, err := builder.NewReply(community, message, replyMetadata)
// handle error
message2, err := fields.NewQualifiedContent(fields.ContentTypeUTF8, "reply to reply")
// handle error
reply2, err := builder.NewReply(reply, message2, replyMetadata)
// handle error

The Builder type can also be used fluently like so: 

// omitting creating the qualified content and error handling
community, err := forest.As(identity, privkey).NewCommunity(communityName, communityMetadata)
reply, err := forest.As(identity, privkey).NewReply(community, message, replyMetadata)
reply2, err := forest.As(identity, privkey).NewReply(reply, message2, replyMetadata)

Index

Constants

View Source 
const MaxNameLength = 256

Variables

This section is empty.

Functions

func FindGPG 

func FindGPG() (path string, err error)

FindGPG returns the path to the local gpg executable if one can be found. Otherwise it returns an error.

func Is 

func Is(nt fields.NodeType, n Node) bool

Is queries the node for it's concrete type.

func NodeTypeOf 

func NodeTypeOf(b []byte) (fields.NodeType, error)

NodeTypeOf returns the NodeType of the provided binary-marshaled node. If the provided bytes are not a forest node or the type cannot be determined, an error will be returned and the first return value must be ignored.

func ValidateID 

func ValidateID(h Hashable, expected fields.QualifiedHash) (bool, error)

ValidateID returns whether the ID of this commonNode matches the data. The first return value indicates the result of the comparison. If there is an error, the first return value will always be false and the second will indicate what went wrong when computing the hash.

func ValidateNode 

func ValidateNode(n Node, store Store) error

ValidateNode calls ValidateInternal and ValidateReferences on a node, returning the first error encountered (or nil if there were no errors).

func ValidateSignature 

func ValidateSignature(v SignatureValidator, identity *Identity) (bool, error)

ValidateSignature returns whether the signature contained in this SignatureValidator is a valid signature for the given Identity. When validating an Identity node, you should pass the same Identity as the second parameter.

func VersionAndNodeTypeOf 

func VersionAndNodeTypeOf(b []byte) (fields.Version, fields.NodeType, error)

Types

type Builder 

type Builder struct {
	User *Identity
	Signer
	Timer func() time.Time
}

Builder creates nodes in the forest on behalf of the given user.

func As 

func As(user *Identity, signer Signer) *Builder

As creates a Builder that can write new nodes on behalf of the provided user. It is intended to be able to be used fluently, like:

community, err := forest.As(user, privkey).NewCommunity(name, metatdata)

func (*Builder) NewCommunity 

func (n *Builder) NewCommunity(name string, metadata []byte) (*Community, error)

NewCommunity creates a community node (signed by the given identity with the given privkey).

func (*Builder) NewCommunityQualified 

func (n *Builder) NewCommunityQualified(name *fields.QualifiedContent, metadata *fields.QualifiedContent) (*Community, error)

func (*Builder) NewReply 

func (n *Builder) NewReply(parent interface{}, content string, metadata []byte) (*Reply, error)

NewReply creates a reply node as a child of the given community or reply

func (*Builder) NewReplyQualified 

func (n *Builder) NewReplyQualified(parent interface{}, content, metadata *fields.QualifiedContent) (*Reply, error)

func (*Builder) Now 

func (b *Builder) Now() time.Time

type CommonNode 

type CommonNode struct {
	// Deterministically computed from the rest of the values.
	Identifier fields.Blob
	SchemaInfo `arbor:"order=0,recurse=always"`
	Parent     fields.QualifiedHash    `arbor:"order=1,recurse=serialize"`
	IDDesc     fields.HashDescriptor   `arbor:"order=2,recurse=always"`
	Depth      fields.TreeDepth        `arbor:"order=3"`
	Created    fields.Timestamp        `arbor:"order=4"`
	Metadata   fields.QualifiedContent `arbor:"order=5,recurse=serialize"`
	Author     fields.QualifiedHash    `arbor:"order=6,recurse=serialize"`
}

generic node

func (*CommonNode) AuthorID 

func (n *CommonNode) AuthorID() *fields.QualifiedHash

AuthorID returns the Author of a common node. It wraps SignatureIdentityHash to satisfy the Node interface.

func (CommonNode) CreatedAt 

func (n CommonNode) CreatedAt() time.Time

func (*CommonNode) Equals 

func (n *CommonNode) Equals(n2 *CommonNode) bool

func (CommonNode) HashDescriptor 

func (n CommonNode) HashDescriptor() *fields.HashDescriptor

func (CommonNode) ID 

func (n CommonNode) ID() *fields.QualifiedHash

Compute and return the CommonNode's ID as a fields.Qualified Hash

func (CommonNode) IsIdentity 

func (n CommonNode) IsIdentity() bool

func (CommonNode) ParentID 

func (n CommonNode) ParentID() *fields.QualifiedHash

func (*CommonNode) SignatureIdentityHash 

func (n *CommonNode) SignatureIdentityHash() *fields.QualifiedHash

SignatureIdentityHash returns the node identitifer for the Identity that signed this node.

func (CommonNode) TreeDepth 

func (n CommonNode) TreeDepth() fields.TreeDepth

func (*CommonNode) TwigMetadata 

func (n *CommonNode) TwigMetadata() (*twig.Data, error)

TwigMetadata returns the metadata of this node parsed into a *twig.Data

func (*CommonNode) ValidateInternal 

func (n *CommonNode) ValidateInternal() error

ValidateInternal checks all fields for internal validity. It does not check the existence or validity of nodes referenced from this node. If the node validates, ValidateInternal returns `nil`.

func (*CommonNode) ValidateReferences 

func (n *CommonNode) ValidateReferences(store Store) error

ValidateReferences checks for the existence of all referenced nodes within the provided store.

type Community 

type Community struct {
	CommonNode `arbor:"order=0,recurse=always"`
	Name       fields.QualifiedContent `arbor:"order=1,recurse=serialize"`
	Trailer    `arbor:"order=2,recurse=always"`
}

func UnmarshalCommunity 

func UnmarshalCommunity(b []byte) (*Community, error)

func (*Community) Equals 

func (c *Community) Equals(other interface{}) bool

func (*Community) MarshalBinary 

func (c *Community) MarshalBinary() ([]byte, error)

func (*Community) MarshalSignedData 

func (c *Community) MarshalSignedData() ([]byte, error)

func (*Community) UnmarshalBinary 

func (c *Community) UnmarshalBinary(b []byte) error

func (*Community) ValidateInternal 

func (c *Community) ValidateInternal() error

ValidateInternal checks all fields for internal validity. It does not check the existence or validity of nodes referenced from this node.

func (*Community) ValidateReferences 

func (c *Community) ValidateReferences(store Store) error

ValidateReferences checks all referenced nodes for existence within the store.

type Copiable 

type Copiable interface {
	CopyInto(Store) error
}

Copiable stores can copy themselves into another store.

type GPGSigner 

type GPGSigner struct {
	GPGUserName string
	// Rewriter is invoked on each invocation of exec.Command that spawns GPG. You can use it to modify
	// flags or any other property of the subcommand (environment variables). This is especially useful
	// to control how GPG prompts for key passphrases.
	Rewriter func(*exec.Cmd) error
	// contains filtered or unexported fields
}

GPGSigner uses a local gpg2 installation for key management. It will invoke gpg2 as a subprocess to sign data and to acquire the public key for its signing key. The public fields can be used to modify its behavior in order to change how it prompts for passphrases and other details.

func NewGPGSigner 

func NewGPGSigner(gpgUserName string) (*GPGSigner, error)

NewGPGSigner wraps the private key so that it can sign using the local system's implementation of GPG.

func (GPGSigner) PublicKey 

func (s GPGSigner) PublicKey() ([]byte, error)

PublicKey returns the bytes of the OpenPGP public key used by this signer.

func (*GPGSigner) Sign 

func (s *GPGSigner) Sign(data []byte) ([]byte, error)

Sign invokes gpg2 to sign the data as this Signer's configured PGP user. It returns the signature or an error (if any).

type Hashable 

type Hashable interface {
	HashDescriptor() *fields.HashDescriptor
	encoding.BinaryMarshaler
}

type Identity 

type Identity struct {
	CommonNode `arbor:"order=0,recurse=always"`
	Name       fields.QualifiedContent `arbor:"order=1,recurse=serialize"`
	PublicKey  fields.QualifiedKey     `arbor:"order=2,recurse=serialize"`
	Trailer    `arbor:"order=3,recurse=always"`
}

Identity nodes represent a user. They associate a username with a public key that the user will sign messages with.

func NewIdentity 

func NewIdentity(signer Signer, name string, metadata []byte) (*Identity, error)

NewIdentity builds an Identity node for the user with the given name and metadata, using the OpenPGP Entity privkey to define the Identity. That Entity must contain a private key with no passphrase.

func NewIdentityQualified 

func NewIdentityQualified(signer Signer, name *fields.QualifiedContent, metadata *fields.QualifiedContent) (*Identity, error)

func UnmarshalIdentity 

func UnmarshalIdentity(b []byte) (*Identity, error)

func (*Identity) Equals 

func (i *Identity) Equals(other interface{}) bool

func (*Identity) MarshalBinary 

func (i *Identity) MarshalBinary() ([]byte, error)

func (*Identity) MarshalSignedData 

func (i *Identity) MarshalSignedData() ([]byte, error)

MarshalSignedData writes all data that should be signed in the correct order for signing. This can be used both to generate and validate message signatures.

func (*Identity) UnmarshalBinary 

func (i *Identity) UnmarshalBinary(b []byte) error

func (*Identity) ValidateInternal 

func (i *Identity) ValidateInternal() error

ValidateInternal checks all fields for internal validity. It does not check the existence or validity of nodes referenced from this node.

func (*Identity) ValidateReferences 

func (i *Identity) ValidateReferences(store Store) error

ValidateReferences checks all referenced nodes for existence within the store.

type NativeSigner 

type NativeSigner openpgp.Entity

NativeSigner uses golang's native openpgp operation for signing data. It only supports private keys without a passphrase.

func (NativeSigner) PublicKey 

func (s NativeSigner) PublicKey() ([]byte, error)

PublicKey returns the raw bytes of the binary openpgp public key used by this signer.

func (NativeSigner) Sign 

func (s NativeSigner) Sign(data []byte) ([]byte, error)

Sign signs the input data with the contained private key and returns the resulting signature.

type Node 

type Node interface {
	AuthorID() *fields.QualifiedHash
	CreatedAt() time.Time
	Equals(interface{}) bool
	ID() *fields.QualifiedHash
	ParentID() *fields.QualifiedHash
	TreeDepth() fields.TreeDepth
	TwigMetadata() (*twig.Data, error)
	ValidateReferences(Store) error
	ValidateInternal() error
	encoding.BinaryMarshaler
	encoding.BinaryUnmarshaler
}

func UnmarshalBinaryNode 

func UnmarshalBinaryNode(b []byte) (Node, error)

UnmarshalBinaryNode unmarshals a node of any type. If it does not return an error, the concrete type of the first return parameter will be one of the node structs declared in this package (e.g. Identity, Community, etc...)

type Paginated 

type Paginated interface {
	// ChildrenBatched allows paging through the children of the node in fixed-size batches.
	// Iteration order is youngest first.
	ChildrenBatched(
		root *fields.QualifiedHash,
		quantity, offset int,
	) (batch []*fields.QualifiedHash, total int, err error)
	// RecentFrom allows paging through nodes in fixed-size batches, starting from the given
	// timestamp.
	// Iteration order is youngest first.
	RecentFrom(nt fields.NodeType, ts time.Time, quantity int) ([]Node, error)
}

Paginated stores can page through nodes with a series of queries.

type Reply 

type Reply struct {
	CommonNode     `arbor:"order=0,recurse=always"`
	CommunityID    fields.QualifiedHash    `arbor:"order=1,recurse=serialize"`
	ConversationID fields.QualifiedHash    `arbor:"order=2,recurse=serialize"`
	Content        fields.QualifiedContent `arbor:"order=3,recurse=serialize"`
	Trailer        `arbor:"order=4,recurse=always"`
}

func UnmarshalReply 

func UnmarshalReply(b []byte) (*Reply, error)

func (*Reply) Equals 

func (r *Reply) Equals(other interface{}) bool

func (*Reply) MarshalBinary 

func (r *Reply) MarshalBinary() ([]byte, error)

func (*Reply) MarshalSignedData 

func (r *Reply) MarshalSignedData() ([]byte, error)

func (*Reply) UnmarshalBinary 

func (r *Reply) UnmarshalBinary(b []byte) error

func (*Reply) ValidateInternal 

func (r *Reply) ValidateInternal() error

ValidateInternal checks all fields for internal validity. It does not check the existence or validity of nodes referenced from this node.

func (*Reply) ValidateReferences 

func (r *Reply) ValidateReferences(store Store) error

ValidateReferences checks all referenced nodes for existence within the store.

type SchemaInfo 

type SchemaInfo struct {
	Version fields.Version  `arbor:"order=0"`
	Type    fields.NodeType `arbor:"order=1"`
}

type SignatureValidator 

type SignatureValidator interface {
	MarshalSignedData() ([]byte, error)
	GetSignature() *fields.QualifiedSignature
	SignatureIdentityHash() *fields.QualifiedHash
	IsIdentity() bool
}

SignatureValidator is a type that has a signature and can supply the ID of the node that signed it.

type Signer 

type Signer interface {
	Sign(data []byte) (signature []byte, err error)
	PublicKey() (key []byte, err error)
}

Signer can sign any binary data

func NewNativeSigner 

func NewNativeSigner(privatekey *openpgp.Entity) (Signer, error)

NewNativeSigner creates a native Golang PGP signer. This will fail if the provided key is encrypted. GPGSigner should be used for all encrypted keys.

type Store 

type Store interface {
	// Get retrieves a node by ID.
	Get(*fields.QualifiedHash) (Node, bool, error)
	// GetIdentity retrieves an identity node by ID.
	GetIdentity(*fields.QualifiedHash) (Node, bool, error)
	// GetCommunity retrieves a community node by ID.
	GetCommunity(*fields.QualifiedHash) (Node, bool, error)
	// GetConversation retrieves a conversation node by ID.
	GetConversation(communityID, conversationID *fields.QualifiedHash) (Node, bool, error)
	// GetReply retrieves a reply node by ID.
	GetReply(communityID, conversationID, replyID *fields.QualifiedHash) (Node, bool, error)
	// Children returns a list of child nodes for the given node ID.
	Children(*fields.QualifiedHash) ([]*fields.QualifiedHash, error)
	// Recent returns recently-created (as per the timestamp in the node) nodes.
	// It may return both a slice of nodes and an error if some nodes in the
	// store were unreadable.
	Recent(nodeType fields.NodeType, quantity int) ([]Node, error)
	// Add inserts a node into the store. It is *not* an error to insert a node which is already
	// stored. Implementations must not return an error in this case.
	Add(Node) error
	// RemoveSubtree from the store.
	RemoveSubtree(*fields.QualifiedHash) error
}

Store describes a collection of `forest.Node`.

type Trailer 

type Trailer struct {
	Signature fields.QualifiedSignature `arbor:"order=0,recurse=serialize,signature"`
}

Trailer is the final set of fields in every arbor node

func (*Trailer) Equals 

func (t *Trailer) Equals(t2 *Trailer) bool

func (*Trailer) GetSignature 

func (t *Trailer) GetSignature() *fields.QualifiedSignature

GetSignature returns the signature for the node, which must correspond to the Signature Authority for the node in order to be valid.

type Validator 

type Validator interface {
	Validate() error
}

Source Files

View all Source files

Directories

Path Synopsis
cmd
forest
Package grove implements an on-disk storage format for arbor forest nodes.
 Package grove implements an on-disk storage format for arbor forest nodes.
Package orchard implements a boltdb backed on-disk node store, satisfying the forest.Store interface.
 Package orchard implements a boltdb backed on-disk node store, satisfying the forest.Store interface.
internal/codec
internal/mock
Package mock uses mocked structures to filter out non-essential data to clarify testing.
 Package mock uses mocked structures to filter out non-essential data to clarify testing.
Package testkeys provides PGP private keys SUITABLE ONLY FOR WRITING TEST CASES.
 Package testkeys provides PGP private keys SUITABLE ONLY FOR WRITING TEST CASES.
Package testutil provides utilities for easily making test arbor nodes and content.
 Package testutil provides utilities for easily making test arbor nodes and content.
Package twig implements the twig key-value data format.
 Package twig implements the twig key-value data format.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.