nuggit

package module
v0.0.0-...-e07d34e Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 2, 2023 License: MIT Imports: 14 Imported by: 1

README

nuggit

A research tool and declarative API for information retrieval (IR).

Documentation

Overview

Package nuggit provides a declarative API for information retrieval (IR).

Index

Constants

This section is empty.

Variables

View Source
var (
	// ErrKey is returned when a Key is not found in a given context.
	//
	// Examples:
	//
	//	* SrcField is not defined in the given Op.
	//
	ErrKey = errors.New("key error")
	// ErrType is returned when a Type is not expected in a given context.
	//
	// Examples:
	//
	//	* An integer is passed to a StringOp.
	//
	ErrType = errors.New("type error")
	// ErrGlom is returned when a GlomOp is not expected in a given context.
	//
	// Examples:
	//
	//	* GlomAppend is passed to an aritmetic op.
	//
	ErrGlom = errors.New("glom error")
)

Functions

This section is empty.

Types

type Adjacency

type Adjacency struct {
	// Key describes the src Node.
	Key NodeKey `json:"key,omitempty"`
	// Edges lists all adjacent edges by EdgeKey.
	Edges []EdgeKey `json:"edges,omitempty"`
}

Adjacency describes the edges that are adjacent to a given node at Key.

func (Adjacency) Clone

func (a Adjacency) Clone() Adjacency

type BaseResource

type BaseResource struct {
	// APIVersion specifies the version of the Nuggit API to use when evaluating the Spec.
	//
	// Examples:
	//
	//	* v1alpha
	//	* v1
	//
	APIVersion string `json:"api_version,omitempty"`
	// Kind describes the type of the attached Spec.
	//
	// Examples:
	//
	//	* Graph
	//
	Kind string `json:"kind,omitempty"`
	// Metadata describes additional data to associate with the Spec.
	Metadata *Metadata `json:"metadata,omitempty"`
}

BaseResource applies common metadata to a Resource or RawResource.

See Resource. See RawResource.

type Edge

type Edge struct {
	// Key is a unique id for the Edge.
	Key EdgeKey `json:"key,omitempty"`
	// Src describes the starting Node.
	Src NodeKey `json:"src,omitempty"`
	// Dst describes the terminal Node.
	Dst NodeKey `json:"dst,omitempty"`
	// SrcField describes the field within Src, if set.
	//
	// See the doc for specific Ops to see which fields are defined.
	SrcField FieldKey `json:"src_field,omitempty"`
	// DstField describes the field within Dst, if set.
	//
	// See the doc for specific Ops to see which fields are defined.
	DstField FieldKey `json:"dst_field,omitempty"`
	// Data specifies arbitrary JSON data to attach to this Edge.
	//
	// The Op runtime may use Data to change the semantics of the connection.
	Data json.RawMessage `json:"data,omitempty"`
}

Edge describes a directed connection between Nodes Src and Dst and, if specified, the fields SrcField and DstField. The Glom operation describes how data flows between SrcField and DstField when there are multiple possible semantics. It is possible to attach arbitrary JSON data to an edge using the Data field. In the context of a Graph, it is possible to have multiple Edges between the same NodeKey and FieldKeys.

See GlomOp.

func (Edge) Clone

func (e Edge) Clone() Edge

Clone returns a copy of e.

type EdgeKey

type EdgeKey = Key

EdgeKey is an alias for a string Key type describing an Edge.

type Exchange

type Exchange struct {
	SrcStage StageKey `json:"src_stage,omitempty"`
	// DstGraph
	DstStage StageKey `json:"dst_stage,omitempty"`
	// Edge represents the logical connection where Src is a Node in SrcGraph
	// and Dst is a Node in DstGraph.
	Edge `json:",omitempty"`
}

Exchange is an Edge that crosses Graph boundaries. It adds the SrcStage and DstStage fields.

type FieldKey

type FieldKey = Key

FieldKey is an alias for a string Key type describing a field.

See Edge.SrcField and Edge.DstField.

type Graph

type Graph struct {
	// Stage is the name of the stage for this Graph.
	Stage StageKey `json:"stage,omitempty"`
	// Adjacency describes the adjacency list for the Graph.
	// Adjacencies relate NodeKeys to the ordered lists of EdgeKeys sourced there.
	//
	// See Adjacency.
	Adjacency []Adjacency `json:"adjacency,omitempty"`
	// Edges describe the edge list for the Graph.
	//
	// See Edge.
	Edges []Edge `json:"edges,omitempty"`
	// Nodes describe the vertex list for the Graph.
	//
	// See Node.
	Nodes []Node `json:"nodes,omitempty"`
}

Graph describes a declarative Program DAG.

See Op specfic documentation to see which ops are available.

func (*Graph) Clone

func (g *Graph) Clone() *Graph

func (*Graph) MarshalJSON

func (g *Graph) MarshalJSON() ([]byte, error)

MarshalJSON implements deterministic marshaling of Graph to JSON.

type Key

type Key = string

Key is an alias for any string Key type.

type Metadata

type Metadata struct {
	// Name is a human name for a Resource.
	Name string `json:"name,omitempty"`
	// Description is human text describing a Resource.
	Description string `json:"description,omitempty"`
	// Version is an opaque version ID for a Resource.
	//
	// Examples:
	//
	//	* v1
	//	* v1.1.0-release
	//
	Version string `json:"version,omitempty"`
	// Encoding describes the encoding of a Resource Spec.
	// For instance, if the Spec is a GZip encoded JSON blob
	// "gzip" can be used.
	// An empty string typically means JSON encoded.
	//
	// Examples:
	//
	//	* gzip
	//
	Encoding string `json:"encoding,omitempty"`
	// Labels are opaque strings to describe a Resource.
	// These are usually simple string tags.
	Labels []string `json:"labels,omitempty"`
}

Metadata defines additional data to attach to an API Resource.

See Resource.

type Node

type Node struct {
	Key NodeKey `json:"key,omitempty"`
	// Op specifies
	Op OpKey `json:"op,omitempty"`
	// Data specifies arbitrary data to attach to this Edge.
	//
	// Data can be used to alter the behavior of the Op.
	Data any `json:"data,omitempty"`
}

Node describes a node in a Graph. Data provides additional configuration to the Op.

See Op documentation to see which Ops are defined.

func (Node) Clone deprecated

func (n Node) Clone() Node

Deprecated: Clone can't work with any Data.

type NodeKey

type NodeKey = Key

NodeKey is an alias for a string Key type describing a Node.

type OpKey

type OpKey = Key

OpKey is an alias for a string Key type describing an op.

See Node.Op.

type Plan

type Plan struct {
	Graphs    []Graph    `json:"graphs,omitempty"`
	Exchanges []Exchange `json:"exchanges,omitempty"`
}

Plan represents a multistage execution between a number of graphs. Graphs are executed in concurrent stages with data exchanged in accordance with the Exchange edges.

type RawResource

type RawResource struct {
	BaseResource `json:",omitempty"`
	// Spec is arbitrary data that this Resource describes.
	Spec json.RawMessage `json:"spec,omitempty"`
}

RawResource is used to unmarshal Resource in two phases. The first phase unmarshals the envelope and the second the Spec contents. The purpose is to support arbitrary Kind, encoding types, and checksums.

See Resource.UnmarshalJSON. See Metadata.Encoding. See Sums.

type Resource

type Resource struct {
	BaseResource `json:",omitempty"`
	// Spec is arbitrary data that this Resource describes.
	Spec any `json:"spec,omitempty"`
}

Resource is an envelope which adds metadata and integrity checks to an API Spec.

See Metadata.

func (*Resource) MarshalJSON

func (r *Resource) MarshalJSON() ([]byte, error)

MarshalJSON marshals the Resource as a JSON object.

func (*Resource) UnmarshalJSON

func (r *Resource) UnmarshalJSON(data []byte) error

UnmarshalJSON implements JSON unmarshaling for Resources given arbitrary Kind and encoding. UnmarshalJSON returns an error when the Resource cannot be interpreted as a JSON object at least (e.g. map[string]any) or fails to unmarshal as the specified Kind.

type ResourceDecoder

type ResourceDecoder struct {
	// Sums defines checksums to use on the encoded byte data of a Resource.
	// Only those checksums that are nonempty are validated.
	Sums *Sums
}

ResourceDecoder decodes a Resource from encoded JSON data.

func (ResourceDecoder) Decode

func (d ResourceDecoder) Decode(data []byte) (*Resource, error)

Decode decodes a Resource and validates the checksums, if provided.

type ResourceEncoder

type ResourceEncoder struct {
	Gzip bool
}

ResourceEncoder encodes a Resource to

func (ResourceEncoder) Encode

func (e ResourceEncoder) Encode(r *Resource) ([]byte, error)

Encode encodes the Resource with the encoding specified in Metadata.

See EncodeWithSums to additionally compute checksums.

type Stage

type Stage struct {
	// Key is a unique id for the Stage.
	Key StageKey `json:"key,omitempty"`
	// Nodes is an ordered list of the Nodes in this stage.
	Nodes []NodeKey `json:"nodes,omitempty"`
}

Stage describes a sequence of op Nodes which are executed together.

func (Stage) Clone

func (s Stage) Clone() Stage

type StageKey

type StageKey = Key

StageKey is an alias for a string Key type describing a Stage.

type SumTest

type SumTest struct {
	Sum      string
	Expected string
	Actual   string
}

func (SumTest) Fail

func (t SumTest) Fail() bool

func (SumTest) Format

func (t SumTest) Format(verbose bool) string

func (SumTest) String

func (s SumTest) String() string

type SumTests

type SumTests []SumTest

func (SumTests) Fail

func (s SumTests) Fail() bool

func (SumTests) Format

func (s SumTests) Format(verbose bool) string

Format the response for the `nuggit sum` subcommand.

func (SumTests) FormatError

func (s SumTests) FormatError() (string, bool)

type Sums

type Sums struct {
	// CRC32 hex encoded checksum.
	CRC32 string `json:"crc32,omitempty"`
	// SHA1 hex encoded hash.
	SHA1 string `json:"sha1,omitempty"`
	// SHA2 hex encoded hash.
	SHA2 string `json:"sha2,omitempty"`
}

Sums specifies checksums used for integrity checks on resources fetched from remote sources. The polynomial used for CRC32 is Castagnoli's polynomial.

func Checksum

func Checksum(data []byte) *Sums

Checksum creates checksums for the data.

func (*Sums) Test

func (s *Sums) Test(other *Sums) SumTests

Test the checksums sums against the other sums. The returned error is always of type *SumError.

func (*Sums) TestBytes

func (s *Sums) TestBytes(data []byte) SumTests

TestBytes tests the bytes against the checksums. The returned error is always of type *SumError.

type Type

type Type string

Type describes supported native types for bootstrapping Nuggit. Types unmarshaled from JSON undergo a strict corresion process which may result in ErrType is types fail to match.

See Op specific documentation for more Compound types.

const (
	TypeUndefined Type = ""
	TypeBool      Type = "bool"
	TypeInt8      Type = "int8"
	TypeInt16     Type = "int16"
	TypeInt32     Type = "int32"
	TypeInt64     Type = "int64"
	TypeUint8     Type = "uint8"
	TypeUint16    Type = "uint16"
	TypeUint32    Type = "uint32"
	TypeUint64    Type = "uint64"
	TypeFloat32   Type = "float32"
	TypeFloat64   Type = "float64"
	TypeBytes     Type = "bytes"
	TypeString    Type = "string"
)

Directories

Path Synopsis
Package graphs implements utility functions on Nuggit Graphs.
Package graphs implements utility functions on Nuggit Graphs.
Package jsong implements merging JSON objects.
Package jsong implements merging JSON objects.
Package keys defines utilities for working with nuggit FieldKeys.
Package keys defines utilities for working with nuggit FieldKeys.
cmd

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL