core

package
v0.0.0-...-93850df Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 19, 2015 License: Apache-2.0 Imports: 18 Imported by: 0

Documentation

Overview

Package core provides Blocks, Sources and the means to connect them together. Blocks communicate with one another by passing Messages.

Index

Constants

View Source 
const (
	NONE = iota
	KEY_VALUE
	STREAM
	LIST
	VALUE_PRIMITIVE
	PRIORITY
	SERVER
)

Variables

View Source 
var RAND *rand.Rand = rand.New(rand.NewSource(12345))

the global random number source

Functions

func Copy 

func Copy(i interface{}) interface{}

func GetLibrary 

func GetLibrary() map[string]Spec

Library is the set of all core block Specs TODO: should just "Build" a global variable so we don't have to iterate over all funcs every time we need the library

func GetSources 

func GetSources() map[string]SourceSpec

func MergeMap 

func MergeMap(base, tomerge map[string]interface{}) (map[string]interface{}, error)

MergeMap recursively merges one map into another, overwriting base when necessary

func NewError 

func NewError(s string) *stcoreError

Types

type Block 

type Block struct {
	Monitor chan MonitorMessage
	// contains filtered or unexported fields
}

A Block describes the block's components

func NewBlock 

func NewBlock(s Spec) *Block

NewBlock creates a new block from a spec

func (*Block) Connect 

func (b *Block) Connect(id RouteIndex, c Connection) error

Connect connects a Route, specified by ID, to a connection

func (*Block) Disconnect 

func (b *Block) Disconnect(id RouteIndex, c Connection) error

Disconnect removes a connection from a Input

func (*Block) GetInput 

func (b *Block) GetInput(id RouteIndex) (Input, error)

GetInput returns the specified Input

func (*Block) GetInputs 

func (b *Block) GetInputs() []Input

GetInputs returns all inputs for a block.

func (*Block) GetOutputs 

func (b *Block) GetOutputs() []Output

Outputs return a list of manifest pairs for the block

func (*Block) GetSource 

func (b *Block) GetSource() Source

func (*Block) Reset 

func (b *Block) Reset()

func (*Block) Serve 

func (b *Block) Serve()

suture: the main routine the block runs

func (*Block) SetInput 

func (b *Block) SetInput(id RouteIndex, v *InputValue) error

RouteValue sets the route to always be the specified value

func (*Block) SetSource 

func (b *Block) SetSource(s Source) error

sets a store for the block. can be set to nil

func (*Block) Stop 

func (b *Block) Stop()

type BlockInfo 

type BlockInfo uint8

BlockAlert defines the possible messages a block can emit about its runnig state 

const (
	BI_RUNNING BlockInfo = iota
	BI_ERROR
	BI_INPUT
	BI_OUTPUT
	BI_KERNEL
)

func (BlockInfo) MarshalJSON 

func (ba BlockInfo) MarshalJSON() ([]byte, error)

type BlockRouting 

type BlockRouting struct {
	Inputs        []Input
	Outputs       []Output
	Source        Source
	InterruptChan chan Interrupt
	sync.RWMutex
}

A block's BlockRouting is the set of Input and Output routes, and the Interrupt channel

type BlockState 

type BlockState struct {
	Processed bool
	// contains filtered or unexported fields
}

A block's BlockState is the pair of input/output MessageMaps, and the Manifest

type Connection 

type Connection chan Message

Connections are used to connect blocks together

type Input 

type Input struct {
	Name  string       `json:"name"`
	Value *InputValue  `json:"value"`
	Type  JSONType     `json:"type"`
	C     chan Message `json:"-"`
}

Input is an inbound route to a block. A Input holds the channel that allows Messages to be passed into the block. A Input's Path is applied to the inbound Message before populating the MessageMap and calling the Kernel. A Input can be set to a Value, instead of waiting for an inbound message.

type InputValue 

type InputValue struct {
	Data interface{} `json:"data"`
}

func (*InputValue) Exists 

func (i *InputValue) Exists() bool

type Interface 

type Interface interface {
	Source
	Describe() []map[string]string
	Serve()
	Stop()
	SetSourceParameter(key, value string)
}

type Interrupt 

type Interrupt func() bool

Interrupt is a function that interrupts a running block in order to change its state. If the interrupt returns false, the block will quit.

type JSONType 

type JSONType uint8

JSONType defines the possible types that variables in core can take 

const (
	NUMBER JSONType = iota
	STRING
	ARRAY
	OBJECT
	BOOLEAN
	NULL
	ANY
	WRITER
	ERROR
)

func (JSONType) MarshalJSON 

func (j JSONType) MarshalJSON() ([]byte, error)

func (*JSONType) UnmarshalJSON 

func (j *JSONType) UnmarshalJSON(data []byte) error

type Kernel 

type Kernel func(MessageMap, MessageMap, MessageMap, Source, chan Interrupt) Interrupt

Kernel is a block's core function that operates on an inbound message. It works by populating the outbound MessageMap, and can be interrupted on its Interrupt channel.

type KeyValue 

type KeyValue struct {
	sync.Mutex
	// contains filtered or unexported fields
}

func (*KeyValue) Get 

func (k *KeyValue) Get() interface{}

func (KeyValue) GetType 

func (k KeyValue) GetType() SourceType

func (*KeyValue) Set 

func (k *KeyValue) Set(v interface{}) error

type List 

type List struct {
	sync.Mutex
	// contains filtered or unexported fields
}

func (*List) Get 

func (l *List) Get() interface{}

func (List) GetType 

func (k List) GetType() SourceType

func (*List) Set 

func (l *List) Set(v interface{}) error

type Manifest 

type Manifest map[ManifestPair]struct{}

A block's Manifest is the set of Connections

type ManifestPair 

type ManifestPair struct {
	Connection
	// contains filtered or unexported fields
}

A ManifestPair is a unique reference to an Output/Connection pair

type Message 

type Message interface{}

Message is the container for data sent between blocks

type MessageMap 

type MessageMap map[RouteIndex]Message

MessageMap maps a block's inbound routes onto the Messages they contain

type MonitorMessage 

type MonitorMessage struct {
	Type BlockInfo   `json:"type"`
	Data interface{} `json:"data,omitempty"`
}

type NSQ 

type NSQ struct {
	Out chan Message // this channel is used by any block that would like to receive messages

	sync.Mutex
	// contains filtered or unexported fields
}

func (*NSQ) Describe 

func (s *NSQ) Describe() []map[string]string

func (NSQ) GetType 

func (s NSQ) GetType() SourceType

func (NSQ) HandleMessage 

func (s NSQ) HandleMessage(message *nsq.Message) error

func (NSQ) Serve 

func (s NSQ) Serve()

func (*NSQ) SetSourceParameter 

func (s *NSQ) SetSourceParameter(name, value string)

func (NSQ) Stop 

func (s NSQ) Stop()

type Output 

type Output struct {
	Name        string                  `json:"name"`
	Type        JSONType                `json:"type"`
	Connections map[Connection]struct{} `json:"-"`
}

An Output holds a set of Connections. Each Connection refers to a Input.C. Every outbound mesage is sent on every Connection in the Connections set.

type PQMessage 

type PQMessage struct {
	// contains filtered or unexported fields
}

type Pin 

type Pin struct {
	Name string
	Type JSONType
}

A Pin contains information about a particular input or output

type PriorityQueue 

type PriorityQueue struct {
	sync.Mutex
	// contains filtered or unexported fields
}

func (PriorityQueue) GetType 

func (pq PriorityQueue) GetType() SourceType

type Request 

type Request struct {
	// contains filtered or unexported fields
}

func (Request) Close 

func (r Request) Close() error

func (Request) Flush 

func (r Request) Flush()

func (Request) Write 

func (r Request) Write(p []byte) (n int, err error)

type RouteIndex 

type RouteIndex int

RouteIndex is the index into a MessageMap. The 0th index corresponds to that block's 0th Input or Output

type Server 

type Server struct {
	sync.Mutex
	// contains filtered or unexported fields
}

func (*Server) Describe 

func (s *Server) Describe() []map[string]string

func (Server) GetType 

func (s Server) GetType() SourceType

func (*Server) Serve 

func (s *Server) Serve()

func (*Server) SetSourceParameter 

func (s *Server) SetSourceParameter(name, value string)

func (Server) Stop 

func (s Server) Stop()

type Source 

type Source interface {
	Lock()
	Unlock()
	GetType() SourceType
}

a Source is esssentially a lockable piece of memory that can be accessed safely by mulitple blocks. The Lock and Unlock methods are usually implemented using a sync.Mutex TODO Source -> Source

func NewKeyValue 

func NewKeyValue() Source

func NewList 

func NewList() Source

func NewNSQ 

func NewNSQ() Source

func NewPriorityQueue 

func NewPriorityQueue() Source

func NewServer 

func NewServer() Source

func NewValue 

func NewValue() Source

type SourceFunc 

type SourceFunc func() Source

A function that creates a source

type SourceSpec 

type SourceSpec struct {
	Name string
	Type SourceType
	New  SourceFunc
}

A SourceSpec defines a source's name and type

func KeyValueStore 

func KeyValueStore() SourceSpec

func ListStore 

func ListStore() SourceSpec

func NSQInterface 

func NSQInterface() SourceSpec

func PriorityQueueStore 

func PriorityQueueStore() SourceSpec

func ServerSource 

func ServerSource() SourceSpec

func ValueStore 

func ValueStore() SourceSpec

type SourceType 

type SourceType int

SourceType is used to indicate what kind of source a block can connect to

func (SourceType) MarshalJSON 

func (s SourceType) MarshalJSON() ([]byte, error)

func (*SourceType) UnmarshalJSON 

func (s *SourceType) UnmarshalJSON(data []byte) error

type Spec 

type Spec struct {
	Name    string
	Inputs  []Pin
	Outputs []Pin
	Source  SourceType
	Kernel  Kernel
}

A Spec defines a block's input and output Pins, and the block's Kernel.

func Addition 

func Addition() Spec

Addition returns the sum of the addenda

func And 

func And() Spec

func Append 

func Append() Spec

Append appends the supplied element to the supplied array

func BernoulliRandom 

func BernoulliRandom() Spec

BernoulliRandom emits a draw from a Bernoulli distribution. This block returns a boolean

func Ceil 

func Ceil() Spec

func Close 

func Close() Spec

func Cos 

func Cos() Spec

func Delay 

func Delay() Spec

Delay emits the message on passthrough after the specified duration

func Division 

func Division() Spec

Division returns the quotient of the dividend / divisor

func EqualTo 

func EqualTo() Spec

EqualTo returns true if value[0] == value[1] or false otherwise

func Exp 

func Exp() Spec

func ExponentialRandom 

func ExponentialRandom() Spec

ExponentialRandom emits an Exponentially distribtued random number

func Exponentiation 

func Exponentiation() Spec

Exponentiation returns the base raised to the exponent

func First 

func First() Spec

first emits true when it receives its first message, and false otherwise

func Floor 

func Floor() Spec

func Flush 

func Flush() Spec

func FromRequest 

func FromRequest() Spec

OutPin 0: received request

func GET 

func GET() Spec

HTTPGET makes an HTTP GET request to the specified URL, emitting the response as a string.

Pin 0: URL string

Pin 1: header []string

func Gate 

func Gate() Spec

Gate emits the inbound message upon receiving a message on its trigger

func Get 

func Get() Spec

Get emits the value against the specified key

func GreaterThan 

func GreaterThan() Spec

GreaterThan returns true if value[0] > value[1] or false otherwise

func HasField 

func HasField() Spec

HasField returns true if the supplied string is a field of the supplied object

func HasPrefix 

func HasPrefix() Spec

HasPrefix returns true if substring is prefix of string

func HasSuffix 

func HasSuffix() Spec

HasSuffix returns true if substring is prefix of string 

func Head() Spec

Head emits the first element of the inbound array

func Identity 

func Identity() Spec

Identity emits the inbound message immediately

func InArray 

func InArray() Spec

InArray returns true if the supplied element is in the supplied array

func InString 

func InString() Spec

InString returns true if substring is contained within string

func Init 

func Init() Spec

Init returns the all the elements of an array apart from the last one

func IsArray 

func IsArray() Spec

func IsBoolean 

func IsBoolean() Spec

func IsError 

func IsError() Spec

func IsNumber 

func IsNumber() Spec

func IsObject 

func IsObject() Spec

func IsString 

func IsString() Spec

func Keys 

func Keys() Spec

Keys returns the top level keys of the supplied object

func Last 

func Last() Spec

Last returns the last element of an array

func Latch 

func Latch() Spec

Latch emits the inbound message on the 0th output if ctrl is true, and the 1st output if ctrl is false

func Len 

func Len() Spec

func LessThan 

func LessThan() Spec

LessThan returns true if value[0] < value[1] or false otherwise

func Ln 

func Ln() Spec

func Log 

func Log() Spec

Log writes the inbound message to stdout TODO where should this write exactly? TODO there should be a stdout source block and Log should be a compound block with a writer

func Log10 

func Log10() Spec

func Merge 

func Merge() Spec

Merge recursively merges two objects, favouring the first input to resolve conflicts

func Modulation 

func Modulation() Spec

Modulation returns the remainder of the dividend mod divisor

func Multiplication 

func Multiplication() Spec

Multiplication returns the product of the multiplicanda

func NSQReceive 

func NSQReceive() Spec

NSQRecieve receives messages from the NSQ system.

OutPin 0: received message

func NormalRandom 

func NormalRandom() Spec

NormalRandom emits a normally distributed random number with the supplied mean and variance

func Not 

func Not() Spec

func NotEqualTo 

func NotEqualTo() Spec

NotEqualTo returns true if value[0] != value[1] or false otherwise

func Or 

func Or() Spec

func ParseJSON 

func ParseJSON() Spec

func PoissonRandom 

func PoissonRandom() Spec

PoissonRandom emits a Poisson distribtued random number

func Pusher 

func Pusher() Spec

pusher provides pressure

func Set 

func Set() Spec

Set creates a new message with the specified key and value

func Sin 

func Sin() Spec

func Sink 

func Sink() Spec

Sink discards the inbound message

func Sqrt 

func Sqrt() Spec

func StringConcat 

func StringConcat() Spec

func StringSplit 

func StringSplit() Spec

func Subtraction 

func Subtraction() Spec

Subtraction returns the difference of the minuend - subtrahend

func Tail 

func Tail() Spec

Tail emits all the elements of an array except for the first

func Tan 

func Tan() Spec

func Timestamp 

func Timestamp() Spec

func ToNumber 

func ToNumber() Spec

func ToString 

func ToString() Spec

func UniformRandom 

func UniformRandom() Spec

UniformRandom emits a uniform random between 0 and 1

func ValueGet 

func ValueGet() Spec

ValueGet emits the value stored

func ValueSet 

func ValueSet() Spec

ValueSet sets the value stored

func Write 

func Write() Spec

func ZipfRandom 

func ZipfRandom() Spec

ZipfRandom emits a Zipfian distributed random number notation follows the wikipedia page http://en.wikipedia.org/wiki/Zipf%E2%80%93Mandelbrot_law not the golang Zipf parameters

type Store 

type Store interface {
	Source
	Get() interface{}
	Set(interface{}) error
}

type Value 

type Value struct {
	sync.Mutex
	// contains filtered or unexported fields
}

func (*Value) Get 

func (v *Value) Get() interface{}

func (Value) GetType 

func (v Value) GetType() SourceType

func (*Value) Set 

func (v *Value) Set(nv interface{}) error

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.