neurgo

package module
v0.0.0-...-f6f77b1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 9, 2016 License: Apache-2.0 Imports: 17 Imported by: 2

README

Neurgo

Build Status GoDoc

A library for constructing Neural Networks in Go, where Neurons are goroutines that communicate with each other via channels.

architecture_diagram.png

What it can do

  • Feedforward networks
  • Recurrent networks
  • JSON Marshal/Unmarshal (example json)
  • Visualization network topology in SVG (example svg)

Learning mechanism

Neurgo does not contain any code for learning/training.

The idea is to have a separation of concerns such that the code that does the training will live in it's own repo. Currently, there is only one training module:

  • neurvolve - An evolution based trainer that is essentially a port of DXNN2 (a Topology & Parameter Evolving Universal Learning Network in Erlang).

Roadmap

  • Training module for Backpropagation based learning (contributions welcome!)
  • Stress testing / benchmarks

Example applications

Example code

The following code creates a neural net with this topology. It does not actually run the network (eg, feed inputs), so for a more complete example see cortex_test.go.

sensor := &Sensor{
	NodeId:       NewSensorId("sensor", 0.0),
	VectorLength: 2,
}
sensor.Init()
hiddenNeuron1 := &Neuron{
	ActivationFunction: EncodableSigmoid(),
	NodeId:             NewNeuronId("hidden-neuron1", 0.25),
	Bias:               -30,
}
hiddenNeuron1.Init()
hiddenNeuron2 := &Neuron{
	ActivationFunction: EncodableSigmoid(),
	NodeId:             NewNeuronId("hidden-neuron2", 0.25),
	Bias:               10,
}
hiddenNeuron2.Init()
outputNeuron := &Neuron{
	ActivationFunction: EncodableSigmoid(),
	NodeId:             NewNeuronId("output-neuron", 0.35),
	Bias:               -10,
}
outputNeuron.Init()
actuator := &Actuator{
	NodeId:       NewActuatorId("actuator", 0.5),
	VectorLength: 1,
}
actuator.Init()

// wire up connections
sensor.ConnectOutbound(hiddenNeuron1)
hiddenNeuron1.ConnectInboundWeighted(sensor, []float64{20, 20})
sensor.ConnectOutbound(hiddenNeuron2)
hiddenNeuron2.ConnectInboundWeighted(sensor, []float64{-20, -20})
hiddenNeuron1.ConnectOutbound(outputNeuron)
outputNeuron.ConnectInboundWeighted(hiddenNeuron1, []float64{20})
hiddenNeuron2.ConnectOutbound(outputNeuron)
outputNeuron.ConnectInboundWeighted(hiddenNeuron2, []float64{20})
outputNeuron.ConnectOutbound(actuator)
actuator.ConnectInbound(outputNeuron)

// create cortex
nodeId := NewCortexId("cortex")
cortex := &Cortex{
	NodeId: nodeId,
}
cortex.SetSensors([]*Sensor{sensor})
cortex.SetNeurons([]*Neuron{hiddenNeuron1, hiddenNeuron2, outputNeuron})
cortex.SetActuators([]*Actuator{actuator})

Getting Started

  • Install Go

  • Clone repository with $ git clone git://github.com/tleyden/neurgo.git

  • Run tests with $ go test

  • To write code that uses neurgo, your code will need import "github.com/tleyden/neurgo" as described in the API documentation

Documentation

Libraries that build on Neurgo

  • neurvolve builds on this library to support evolution-based learning.

DXNN2 - Pure Erlang TPEULN (Topology & Parameter Evolving Universal Learning Network).

Handbook of Neuroevolution Through Erlang by Gene Sher.

Documentation

Index

Constants

View Source 
const (
	SENSOR   = "SENSOR"
	NEURON   = "NEURON"
	ACTUATOR = "ACTUATOR"
	CORTEX   = "CORTEX"
)
View Source 
const FITNESS_THRESHOLD = 1e8

Variables

This section is empty.

Functions

func Average 

func Average(xs []float64) float64

func EqualsWithMaxDelta 

func EqualsWithMaxDelta(x, y, maxDelta float64) bool

func FixedWeights 

func FixedWeights(length int, weight float64) []float64

func Identity 

func Identity(x float64) float64

func IntModuloProper 

func IntModuloProper(x, y int) bool

func JsonString 

func JsonString(v interface{}) string

func NewUuid 

func NewUuid() string

func NormalizeInRange 

func NormalizeInRange(params NormalizeParams, value float64) float64

func RandomBias 

func RandomBias() float64

func RandomInRange 

func RandomInRange(min, max float64) float64

func RandomIntInRange 

func RandomIntInRange(min, max int) int

return a random number between min and max - 1 eg, if you call it with 0,1 it will always return 0 if you call it between 0,2 it will return 0 or 1

func RandomWeight 

func RandomWeight() float64

func RandomWeights 

func RandomWeights(length int) []float64

func SafeScalarInverse 

func SafeScalarInverse(x float64) float64

func Saturate 

func Saturate(parameter, lowerBound, upperBound float64) float64

func SeedRandom 

func SeedRandom()

func Sigmoid 

func Sigmoid(x float64) float64

func SumOfSquaresError 

func SumOfSquaresError(expected []float64, actual []float64) float64

http://en.wikipedia.org/wiki/Residual_sum_of_squares

func VectorEquals 

func VectorEquals(xValues, yValues []float64) bool

func WriteStringToFile 

func WriteStringToFile(value string, filepath string) error

Types

type ActivationFunction 

type ActivationFunction func(float64) float64

type Actuator 

type Actuator struct {
	NodeId           *NodeId
	Inbound          []*InboundConnection
	Closing          chan chan bool
	DataChan         chan *DataMessage
	VectorLength     int
	ActuatorFunction ActuatorFunction

	Cortex *Cortex
	// contains filtered or unexported fields
}

func (*Actuator) CanAddInboundConnection 

func (actuator *Actuator) CanAddInboundConnection() bool

func (*Actuator) ConnectInbound 

func (actuator *Actuator) ConnectInbound(connectable InboundConnectable) *InboundConnection

func (*Actuator) Init 

func (actuator *Actuator) Init()

func (*Actuator) MarshalJSON 

func (actuator *Actuator) MarshalJSON() ([]byte, error)

func (*Actuator) Run 

func (actuator *Actuator) Run()

func (*Actuator) Shutdown 

func (actuator *Actuator) Shutdown()

func (*Actuator) String 

func (actuator *Actuator) String() string

type ActuatorBarrier 

type ActuatorBarrier map[*NodeId]bool // TODO: fixme!! totally broken

type ActuatorFunction 

type ActuatorFunction func(outputs []float64)

type Cortex 

type Cortex struct {
	NodeId    *NodeId
	Sensors   []*Sensor
	Neurons   []*Neuron
	Actuators []*Actuator
	SyncChan  chan *NodeId // TODO: rename to ActuatorBarrier
}

func BasicCortex 

func BasicCortex() *Cortex

func NewCortexFromJSONFile 

func NewCortexFromJSONFile(filename string) (cortex *Cortex, err error)

TODO: rename to have "unmarshal" in the name

func NewCortexFromJSONSBytes 

func NewCortexFromJSONSBytes(jsonBytes []byte) (cortex *Cortex, err error)

func NewCortexFromJSONString 

func NewCortexFromJSONString(jsonString string) (cortex *Cortex, err error)

func XnorCortex 

func XnorCortex() *Cortex

func XnorCortexUntrained 

func XnorCortexUntrained() *Cortex

func (*Cortex) ActuatorNodeIds 

func (cortex *Cortex) ActuatorNodeIds() []*NodeId

func (*Cortex) AllNodeIds 

func (cortex *Cortex) AllNodeIds() []*NodeId

func (*Cortex) Copy 

func (cortex *Cortex) Copy() *Cortex

func (*Cortex) CreateNeuronInLayer 

func (cortex *Cortex) CreateNeuronInLayer(layerIndex float64) *Neuron

func (*Cortex) ExtraCompact 

func (cortex *Cortex) ExtraCompact() string

func (*Cortex) FindActuator 

func (cortex *Cortex) FindActuator(nodeId *NodeId) *Actuator

func (*Cortex) FindConnector 

func (cortex *Cortex) FindConnector(nodeId *NodeId) OutboundConnector

TODO: rename to FindOutboundConnector

func (*Cortex) FindInboundConnector 

func (cortex *Cortex) FindInboundConnector(nodeId *NodeId) InboundConnector

func (*Cortex) FindNeuron 

func (cortex *Cortex) FindNeuron(nodeId *NodeId) *Neuron

func (*Cortex) FindSensor 

func (cortex *Cortex) FindSensor(nodeId *NodeId) *Sensor

func (*Cortex) Fitness 

func (cortex *Cortex) Fitness(samples []*TrainingSample) float64

func (*Cortex) Init 

func (cortex *Cortex) Init()

Initialize/re-initialize the cortex.

func (*Cortex) InitOutboundConnections 

func (cortex *Cortex) InitOutboundConnections()

We may be in a state where the outbound connections do not have data channels associated with them, even though the data channels exist. (eg, when deserializing from json). Fix this by seeking out those outbound connections and setting the data channels.

func (*Cortex) LinkNodesToCortex 

func (cortex *Cortex) LinkNodesToCortex()

func (*Cortex) MarshalJSON 

func (cortex *Cortex) MarshalJSON() ([]byte, error)

func (*Cortex) MarshalJSONToFile 

func (cortex *Cortex) MarshalJSONToFile(filename string) error

func (*Cortex) NeuronLayerMap 

func (cortex *Cortex) NeuronLayerMap() LayerToNeuronMap

func (*Cortex) NeuronNodeIds 

func (cortex *Cortex) NeuronNodeIds() []*NodeId

func (*Cortex) NeuronUUIDMap 

func (cortex *Cortex) NeuronUUIDMap() UUIDToNeuronMap

func (*Cortex) NodeIdLayerMap 

func (cortex *Cortex) NodeIdLayerMap() LayerToNodeIdMap

func (*Cortex) RenderSVG 

func (cortex *Cortex) RenderSVG(writer io.Writer)

func (*Cortex) RenderSVGFile 

func (cortex *Cortex) RenderSVGFile(filename string)

func (*Cortex) Repair 

func (cortex *Cortex) Repair()

func (*Cortex) Run 

func (cortex *Cortex) Run()

func (*Cortex) SensorNodeIds 

func (cortex *Cortex) SensorNodeIds() []*NodeId

func (*Cortex) SetActuators 

func (cortex *Cortex) SetActuators(actuators []*Actuator)

func (*Cortex) SetNeurons 

func (cortex *Cortex) SetNeurons(neurons []*Neuron)

func (*Cortex) SetSensors 

func (cortex *Cortex) SetSensors(sensors []*Sensor)

func (*Cortex) Shutdown 

func (cortex *Cortex) Shutdown()

func (*Cortex) String 

func (cortex *Cortex) String() string

func (*Cortex) StringCompact 

func (cortex *Cortex) StringCompact() string

func (*Cortex) SyncActuators 

func (cortex *Cortex) SyncActuators()

func (*Cortex) SyncSensors 

func (cortex *Cortex) SyncSensors()

func (*Cortex) Validate 

func (cortex *Cortex) Validate() bool

func (*Cortex) Verify 

func (cortex *Cortex) Verify(samples []*TrainingSample) bool

type DataMessage 

type DataMessage struct {
	SenderId *NodeId
	Inputs   []float64
}

func (*DataMessage) String 

func (dataMessage *DataMessage) String() string

type Disconnectable 

type Disconnectable interface {
	// contains filtered or unexported methods
}

type EncodableActivation 

type EncodableActivation struct {
	Name               string
	ActivationFunction ActivationFunction
}

func AllEncodableActivations 

func AllEncodableActivations() []*EncodableActivation

func EncodableIdentity 

func EncodableIdentity() *EncodableActivation

func EncodableSigmoid 

func EncodableSigmoid() *EncodableActivation

func EncodableTanh 

func EncodableTanh() *EncodableActivation

func RandomEncodableActivation 

func RandomEncodableActivation() *EncodableActivation

func (*EncodableActivation) MarshalJSON 

func (activation *EncodableActivation) MarshalJSON() ([]byte, error)

func (*EncodableActivation) String 

func (activation *EncodableActivation) String() string

func (*EncodableActivation) UnmarshalJSON 

func (activation *EncodableActivation) UnmarshalJSON(bytes []byte) error

type InboundConnectable 

type InboundConnectable interface {
	// contains filtered or unexported methods
}

type InboundConnection 

type InboundConnection struct {
	NodeId  *NodeId
	Weights []float64
}

func ConnectInbound 

func ConnectInbound(connector InboundConnector, connectable InboundConnectable) *InboundConnection

func ConnectInboundWeighted 

func ConnectInboundWeighted(connector InboundConnector, connectable InboundConnectable, weights []float64) *InboundConnection

func DisconnectInbound 

func DisconnectInbound(connector InboundConnector, connectable Disconnectable) *InboundConnection

func (*InboundConnection) MarshalJSON 

func (connection *InboundConnection) MarshalJSON() ([]byte, error)

func (*InboundConnection) String 

func (connection *InboundConnection) String() string

type InboundConnector 

type InboundConnector interface {
	// contains filtered or unexported methods
}

type LayerToNeuronMap 

type LayerToNeuronMap map[float64][]*Neuron

LayerToNeuronMap ..

func (LayerToNeuronMap) ChooseRandomLayer 

func (layerToNeuronMap LayerToNeuronMap) ChooseRandomLayer() float64

func (LayerToNeuronMap) Keys 

func (layerToNeuronMap LayerToNeuronMap) Keys() []float64

type LayerToNodeIdMap 

type LayerToNodeIdMap map[float64][]*NodeId

func (LayerToNodeIdMap) ChooseNodeIdFollowingLayer 

func (l LayerToNodeIdMap) ChooseNodeIdFollowingLayer(layerIndex float64) *NodeId

func (LayerToNodeIdMap) ChooseNodeIdPrecedingLayer 

func (l LayerToNodeIdMap) ChooseNodeIdPrecedingLayer(layerIndex float64) *NodeId

func (LayerToNodeIdMap) ChooseRandomLayer 

func (layerToNodeIdMap LayerToNodeIdMap) ChooseRandomLayer() float64

func (LayerToNodeIdMap) IntegerIndexOf 

func (l LayerToNodeIdMap) IntegerIndexOf(layer float64) int

func (LayerToNodeIdMap) Keys 

func (layerToNodeIdMap LayerToNodeIdMap) Keys() []float64

func (LayerToNodeIdMap) LayerBetweenOrNew 

func (l LayerToNodeIdMap) LayerBetweenOrNew(initial, final float64) float64

func (LayerToNodeIdMap) LayerOfIntegerIndex 

func (l LayerToNodeIdMap) LayerOfIntegerIndex(layerInteger int) float64

func (LayerToNodeIdMap) LayersAdjacent 

func (l LayerToNodeIdMap) LayersAdjacent(initial, final float64) bool

func (LayerToNodeIdMap) NewLayerBetween 

func (l LayerToNodeIdMap) NewLayerBetween(initial, final float64) float64

type Neuron 

type Neuron struct {
	NodeId             *NodeId
	Bias               float64
	Inbound            []*InboundConnection
	Outbound           []*OutboundConnection
	Closing            chan chan bool
	DataChan           chan *DataMessage
	ActivationFunction *EncodableActivation

	Cortex *Cortex
	// contains filtered or unexported fields
}

func (*Neuron) ConnectInboundWeighted 

func (neuron *Neuron) ConnectInboundWeighted(connectable InboundConnectable, weights []float64) *InboundConnection

func (*Neuron) ConnectOutbound 

func (neuron *Neuron) ConnectOutbound(connectable OutboundConnectable) *OutboundConnection

func (*Neuron) Copy 

func (neuron *Neuron) Copy() *Neuron

func (*Neuron) InboundUUIDMap 

func (neuron *Neuron) InboundUUIDMap() UUIDToInboundConnection

func (*Neuron) Init 

func (neuron *Neuron) Init()

func (*Neuron) IsConnectionRecurrent 

func (neuron *Neuron) IsConnectionRecurrent(connection *OutboundConnection) bool

a connection is considered recurrent if it has a connection to itself or to a node in a previous layer. Previous meaning if you look at a feedForward from left to right, with the input layer being on the far left, and output layer on the far right, then any layer to the left is considered previous.

func (*Neuron) IsInboundConnectionRecurrent 

func (neuron *Neuron) IsInboundConnectionRecurrent(connection *InboundConnection) bool

same as isConnectionRecurrent, but for inbound connections TODO: use interfaces to eliminate code duplication

func (*Neuron) MarshalJSON 

func (neuron *Neuron) MarshalJSON() ([]byte, error)

func (*Neuron) RecurrentInboundConnections 

func (neuron *Neuron) RecurrentInboundConnections() []*InboundConnection

func (*Neuron) RecurrentOutboundConnections 

func (neuron *Neuron) RecurrentOutboundConnections() []*OutboundConnection

Find the subset of outbound connections which are "recurrent" - meaning that the connection is to this neuron itself, or to a neuron in a previous (eg, to the left) layer.

func (*Neuron) Run 

func (neuron *Neuron) Run()

func (*Neuron) Shutdown 

func (neuron *Neuron) Shutdown()

func (*Neuron) String 

func (neuron *Neuron) String() string

type NodeCircleSVG 

type NodeCircleSVG struct {
	Point
	// contains filtered or unexported fields
}

type NodeId 

type NodeId struct {
	UUID       string
	NodeType   NodeType
	LayerIndex float64
}

func NewActuatorId 

func NewActuatorId(UUID string, LayerIndex float64) *NodeId

func NewCortexId 

func NewCortexId(UUID string) *NodeId

func NewNeuronId 

func NewNeuronId(UUID string, LayerIndex float64) *NodeId

func NewSensorId 

func NewSensorId(UUID string, LayerIndex float64) *NodeId

func (*NodeId) String 

func (nodeId *NodeId) String() string

type NodeType 

type NodeType string

type NodeUUIDToCircleSVG 

type NodeUUIDToCircleSVG map[string]NodeCircleSVG

type NormalizeParams 

type NormalizeParams struct {
	SourceRangeStart float64
	SourceRangeEnd   float64
	TargetRangeStart float64
	TargetRangeEnd   float64
}

type OutboundConnectable 

type OutboundConnectable interface {
	// contains filtered or unexported methods
}

type OutboundConnection 

type OutboundConnection struct {
	NodeId   *NodeId
	DataChan chan *DataMessage
}

func ConnectOutbound 

func ConnectOutbound(connector OutboundConnector, connectable OutboundConnectable) *OutboundConnection

func DisconnectOutbound 

func DisconnectOutbound(connector OutboundConnector, connectable Disconnectable) *OutboundConnection

func (*OutboundConnection) MarshalJSON 

func (connection *OutboundConnection) MarshalJSON() ([]byte, error)

func (*OutboundConnection) String 

func (connection *OutboundConnection) String() string

type OutboundConnector 

type OutboundConnector interface {
	// contains filtered or unexported methods
}

type Point 

type Point struct {
	// contains filtered or unexported fields
}

type Sensor 

type Sensor struct {
	NodeId         *NodeId
	Outbound       []*OutboundConnection
	VectorLength   int
	Closing        chan chan bool
	SyncChan       chan bool
	SensorFunction SensorFunction

	Cortex *Cortex
	// contains filtered or unexported fields
}

func (*Sensor) ConnectOutbound 

func (s *Sensor) ConnectOutbound(connectable OutboundConnectable) *OutboundConnection

func (*Sensor) Init 

func (sensor *Sensor) Init()

func (*Sensor) MarshalJSON 

func (sensor *Sensor) MarshalJSON() ([]byte, error)

func (*Sensor) Run 

func (sensor *Sensor) Run()

func (*Sensor) Shutdown 

func (sensor *Sensor) Shutdown()

func (*Sensor) String 

func (sensor *Sensor) String() string

type SensorFunction 

type SensorFunction func(int) []float64

type Trainer 

type Trainer interface {
	Train(cortex *Cortex, examples []*TrainingSample) *Cortex
}

type TrainingSample 

type TrainingSample struct {

	// for each sensor in the network, provide a sample input vector
	SampleInputs [][]float64

	// for each actuator in the network, provide an expected output vector
	ExpectedOutputs [][]float64
}

func XnorTrainingSamples 

func XnorTrainingSamples() []*TrainingSample

func (*TrainingSample) String 

func (t *TrainingSample) String() string

type UUIDToInboundConnection 

type UUIDToInboundConnection map[string]*InboundConnection

type UUIDToNeuronMap 

type UUIDToNeuronMap map[string]*Neuron

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.