Documentation ¶
Index ¶
- Variables
- func ActivationStatement(af ActivationFunctionIndex, w float64, inputStatements []*jen.Statement) *jen.Statement
- func Render(inner *jen.Statement) string
- func RunStatementInputData(statement *jen.Statement, inputData []float64) (float64, error)
- func RunStatementX(statement *jen.Statement, x float64) (float64, error)
- func ScorePopulation(population []*Network, weight float64, inputData [][]float64, ...) (map[int]float64, float64)
- type ActivationFunctionIndex
- func (afi ActivationFunctionIndex) Call(x float64) float64
- func (afi ActivationFunctionIndex) GoRun(x float64) (float64, error)
- func (afi ActivationFunctionIndex) Name() string
- func (afi ActivationFunctionIndex) Statement(inner *jen.Statement) *jen.Statement
- func (afi ActivationFunctionIndex) String() string
- type Config
- type Network
- func (net *Network) AddConnection(a, b NeuronIndex) error
- func (net *Network) All() []*Neuron
- func (net *Network) Complexity() float64
- func (net *Network) Connected() []NeuronIndex
- func (net Network) Copy() *Network
- func (net *Network) Depth() int
- func (net *Network) Evaluate(inputValues []float64) float64
- func (net *Network) Exists(ni NeuronIndex) bool
- func (net *Network) ForEachConnected(f func(n *Neuron))
- func (net *Network) ForEachConnectedNodeIndex(f func(ni NeuronIndex))
- func (net *Network) Get(i NeuronIndex) *Neuron
- func (net *Network) GetRandomInputNode() NeuronIndex
- func (net *Network) GetRandomNode() NeuronIndex
- func (net *Network) InsertNode(a, b NeuronIndex, newNodeIndex NeuronIndex) error
- func (net *Network) InsertRandomNode() bool
- func (net *Network) IsInput(ni NeuronIndex) bool
- func (net *Network) LeftRight(a, b NeuronIndex) (NeuronIndex, NeuronIndex, bool)
- func (net *Network) Modify(maxIterations int)
- func (net *Network) NewBlankNeuron() (*Neuron, NeuronIndex)
- func (net *Network) NewInputNode(activationFunction ActivationFunctionIndex, connectToOutput bool) error
- func (net *Network) NewNeuron() (*Neuron, NeuronIndex)
- func (net *Network) OutputNodeStatementX(functionName string) string
- func (net *Network) OutputSVG(w io.Writer) (int, error)
- func (net *Network) RandomizeActivationFunctionForRandomNeuron()
- func (net Network) SetInputValues(inputValues []float64)
- func (net *Network) SetWeight(weight float64)
- func (net *Network) StatementWithInputDataVariables() (*jen.Statement, error)
- func (net *Network) StatementWithInputValues() (*jen.Statement, error)
- func (net Network) String() string
- func (net *Network) Unconnected() []NeuronIndex
- func (net *Network) UpdateNetworkPointers()
- func (net *Network) WriteSVG(filename string) error
- type Neuron
- func (neuron *Neuron) AddInput(ni NeuronIndex) error
- func (neuron *Neuron) AddInputNeuron(n *Neuron) error
- func (neuron *Neuron) Connect(net *Network)
- func (neuron Neuron) Copy(net *Network) Neuron
- func (neuron *Neuron) FindInput(e NeuronIndex) (int, bool)
- func (neuron *Neuron) GetActivationFunction() func(float64) float64
- func (neuron *Neuron) HasInput(e NeuronIndex) bool
- func (neuron *Neuron) In(collection []NeuronIndex) bool
- func (neuron *Neuron) InputNeuronsAreGood() bool
- func (neuron *Neuron) InputStatement() (*jen.Statement, error)
- func (neuron *Neuron) Is(e NeuronIndex) bool
- func (neuron *Neuron) IsInput() bool
- func (neuron *Neuron) IsOutput() bool
- func (neuron Neuron) NetworkStatementWithInputDataVariables(visited *[]NeuronIndex) (*jen.Statement, error)
- func (neuron Neuron) NetworkStatementWithInputValues(visited *[]NeuronIndex) (*jen.Statement, error)
- func (neuron *Neuron) RandomizeActivationFunction()
- func (neuron *Neuron) RemoveInput(e NeuronIndex) error
- func (neuron *Neuron) SetValue(x float64)
- func (neuron *Neuron) String() string
- type NeuronIndex
- type NormalizationInfo
- type Pair
- type PairList
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var ActivationFunctions = map[ActivationFunctionIndex](func(float64) float64){ Step: af.Step, Linear: af.Linear, Sin: af.Sin, Gauss: af.Gaussian01, Tanh: af.Tanh, Sigmoid: af.Sigmoid, Inv: af.Inv, Abs: af.Abs, ReLU: af.ReLU, Cos: af.Cos, Squared: af.Squared, Swish: af.Swish, SoftPlus: af.SoftPlus, }
ActivationFunctions is a collection of activation functions, where the keys are constants that are defined above https://github.com/google/brain-tokyo-workshop/blob/master/WANNRelease/WANN/wann_src/ind.py
var ComplexityEstimate = make(map[ActivationFunctionIndex]float64)
ComplexityEstimate is a map for having an estimate of how complex each function is, based on a quick benchmark of each function. The complexity estimates will vary, depending on the performance.
Functions ¶
func ActivationStatement ¶
func ActivationStatement(af ActivationFunctionIndex, w float64, inputStatements []*jen.Statement) *jen.Statement
ActivationStatement creates an activation function statment, given a weight and input statements returns: activationFunction(input0 * w + input1 * w + ...) The function calling this function is responsible for inserting network input values into the network input nodes.
func Render ¶
Render renders a *jen.Statement to a string, if possible if there is an error about an extra ")", then that's because anonymous functions are not supported by jen Do not Render until statements could be placed at the top-level in a Go program.
func RunStatementInputData ¶
RunStatementInputData will run the given statement by wrapping it in a program and using "go run"
func RunStatementX ¶
RunStatementX will run the given statement by wrapping it in a program and using "go run"
func ScorePopulation ¶
func ScorePopulation(population []*Network, weight float64, inputData [][]float64, incorrectOutputMultipliers []float64) (map[int]float64, float64)
ScorePopulation evaluates a population, given a slice of input numbers. It returns a map with scores, together with the sum of scores.
Types ¶
type ActivationFunctionIndex ¶
type ActivationFunctionIndex int
ActivationFunctionIndex is a number that represents a specific activation function
const ( // Step is a step. First 0 and then abrubtly up to 1. Step ActivationFunctionIndex = iota // Linear is the linear activation function. Gradually from 0 to 1. Linear // Sin is the sinoid activation function Sin // Gauss is the Gaussian function, with a mean of 0 and a sigma of 1 Gauss // Tanh is math.Tanh Tanh // Sigmoid is the optimized sigmoid function from github.com/xyproto/swish Sigmoid // Inv is the inverse linear function Inv // Abs is math.Abs Abs // ReLU or ReLU is the rectified linear unit, first 0 and then the linear function ReLU // Cos is the cosoid (?) activation function Cos // Squared increases rapidly Squared // Swish is a later invention than ReLU, _| Swish // SoftPlus is log(1 + exp(x)) SoftPlus )
func (ActivationFunctionIndex) Call ¶
func (afi ActivationFunctionIndex) Call(x float64) float64
Call runs an activation function with the given float64 value. The activation function is chosen by one of the constants above.
Example ¶
fmt.Println(Gauss.Call(2.0))
Output: 0.13427659965015956
func (ActivationFunctionIndex) GoRun ¶
func (afi ActivationFunctionIndex) GoRun(x float64) (float64, error)
GoRun will first construct the expression using jennifer and then evaluate the result using "go run" and a source file innn /tmp
Example ¶
// Run the Gauss function directly fmt.Println(ActivationFunctions[Gauss](0.5)) // Use Jennifer to generate a source file just for running the Gauss function, then use "go run" and fetch the result if result, err := Gauss.GoRun(0.5); err == nil { // no error fmt.Println(result) }
Output: 0.8824699625576026 0.8824969025845955
func (ActivationFunctionIndex) Name ¶
func (afi ActivationFunctionIndex) Name() string
Name returns a name for each activation function
func (ActivationFunctionIndex) Statement ¶
func (afi ActivationFunctionIndex) Statement(inner *jen.Statement) *jen.Statement
Statement returns the Statement statement for this activation function, using the given inner statement
func (ActivationFunctionIndex) String ¶
func (afi ActivationFunctionIndex) String() string
String returns the Go expression for this activation function, using "x" as the input variable name
type Config ¶
type Config struct { // When initializing a network, this is the propability that the node will be connected to the output node InitialConnectionRatio float64 // How many generations to train for, at a maximum? Generations int // How large population sizes to use per generation? PopulationSize int // For how many generations should the training go on, without any improvement in the best score? Disabled if 0. MaxIterationsWithoutBestImprovement int // RandomSeed, for initializing the random number generator. The current time is used for the seed if this is set to 0. RandomSeed int64 // Verbose output Verbose bool // contains filtered or unexported fields }
Config is a struct that is used when initializing new Network structs. The idea is that referring to fields by name is more explicit, and that it can be re-used in connection with having a configuration file, in the future.
type Network ¶
type Network struct { AllNodes []Neuron // Storing the actual neurons InputNodes []NeuronIndex // Pointers to the input nodes OutputNode NeuronIndex // Pointer to the output node Weight float64 // Shared weight }
Network is a collection of nodes, an output node and a shared weight.
func NewNetwork ¶
NewNetwork creates a new minimal network with n input nodes and ratio of r connections. Passing "nil" as an argument is supported.
func (*Network) AddConnection ¶
func (net *Network) AddConnection(a, b NeuronIndex) error
AddConnection adds a connection from a to b. The order is swapped if needed, then a is added as an input to b.
func (*Network) Complexity ¶
Complexity measures the network complexity Will return 1.0 at a minimum
func (*Network) Connected ¶
func (net *Network) Connected() []NeuronIndex
Connected returns a slice of neuron indexes, that are all connected to the output node (directly or indirectly)
func (*Network) Evaluate ¶
Evaluate will return a weighted sum of the input nodes, using the .Value field if it is set and no input nodes are available. A shared weight can be given.
func (*Network) Exists ¶
func (net *Network) Exists(ni NeuronIndex) bool
Exists checks if the given NeuronIndex exists in this Network
func (*Network) ForEachConnected ¶
ForEachConnected will only go through nodes that are connected to the output node (directly or indirectly) Unconnected input nodes are not covered.
func (*Network) ForEachConnectedNodeIndex ¶
func (net *Network) ForEachConnectedNodeIndex(f func(ni NeuronIndex))
ForEachConnectedNodeIndex will only go through nodes that are connected to the output node (directly or indirectly) Unconnected input nodes are not covered.
func (*Network) Get ¶
func (net *Network) Get(i NeuronIndex) *Neuron
Get returns a pointer to a neuron, based on the given NeuronIndex
func (*Network) GetRandomInputNode ¶
func (net *Network) GetRandomInputNode() NeuronIndex
GetRandomInputNode returns a random input node
func (*Network) GetRandomNode ¶
func (net *Network) GetRandomNode() NeuronIndex
GetRandomNode will select a random neuron. This can be any node, including the output node.
func (*Network) InsertNode ¶
func (net *Network) InsertNode(a, b NeuronIndex, newNodeIndex NeuronIndex) error
InsertNode takes two neurons and inserts a third neuron between them Assumes that a is the leftmost node and the b is the rightmost node.
Example ¶
rand.Seed(commonSeed) net := NewNetwork(&Config{ inputs: 3, InitialConnectionRatio: 1.0, }) fmt.Println("Before insertion:") fmt.Println(net) _, nodeIndex := net.NewNeuron() err := net.InsertNode(0, 1, nodeIndex) if err != nil { fmt.Println("error: " + err.Error()) } fmt.Println("After insertion:") fmt.Println(net)
Output: Before insertion: Network (4 nodes, 3 input nodes, 1 output node) Connected inputs to output node: 3 Output node ID 0 has these input connections: [1 2 3] Input node ID 1 has these input connections: [] Input node ID 2 has these input connections: [] Input node ID 3 has these input connections: [] After insertion: Network (5 nodes, 3 input nodes, 1 output node) Connected inputs to output node: 3 Output node ID 0 has these input connections: [2 3 4] Input node ID 1 has these input connections: [] Input node ID 2 has these input connections: [] Input node ID 3 has these input connections: [] Node ID 4 has these input connections: [1]
func (*Network) InsertRandomNode ¶
InsertRandomNode will try the given number of times to insert a node in a random location, replacing an existing connection between two nodes. `a -> b` will then become `a -> newNode -> b` Returns true if one was inserted or false if the randomly chosen location wasn't fruitful
func (*Network) IsInput ¶
func (net *Network) IsInput(ni NeuronIndex) bool
IsInput checks if the given node is an input node
func (*Network) LeftRight ¶
func (net *Network) LeftRight(a, b NeuronIndex) (NeuronIndex, NeuronIndex, bool)
LeftRight returns two neurons, such that the first on is the one that is most to the left (towards the input neurons) and the second one is most to the right (towards the output neuron). Assumes that a and b are not equal. The returned bool is true if there is no order (if the nodes are equal, both are output nodes or both are input nodes)
func (*Network) Modify ¶
Modify the network using one of the three methods outlined in the paper: * Insert node * Add connection * Change activation function
func (*Network) NewBlankNeuron ¶
func (net *Network) NewBlankNeuron() (*Neuron, NeuronIndex)
NewBlankNeuron creates a new Neuron, with the Step activation function as the default
func (*Network) NewInputNode ¶
func (net *Network) NewInputNode(activationFunction ActivationFunctionIndex, connectToOutput bool) error
NewInputNode creates a new input node for this network, optionally connecting it to the output node
func (*Network) NewNeuron ¶
func (net *Network) NewNeuron() (*Neuron, NeuronIndex)
NewNeuron creates a new *Neuron, with a randomly chosen activation function
func (*Network) OutputNodeStatementX ¶
OutputNodeStatementX returns a statement for the output node, using "x" for the variable
Example ¶
rand.Seed(1) net := NewNetwork(&Config{ inputs: 5, InitialConnectionRatio: 0.7, sharedWeight: 0.5, }) fmt.Println(net.OutputNodeStatementX("f")) //fmt.Println(net.Score())
Output: f := math.Pow(x, 2.0)
Example (First) ¶
// First create a network with only one output node, that has a step function net := NewNetwork() net.AllNodes[net.OutputNode].ActivationFunction = Step fmt.Println(net.OutputNodeStatementX("score"))
Output: score := func(s float64) float64 { if s >= 0 { return 1 } else { return 0 } }(x)
Example (Fourth) ¶
rand.Seed(1111113) net := NewNetwork(&Config{ inputs: 5, InitialConnectionRatio: 0.7, sharedWeight: 0.5, }) fmt.Println(net.OutputNodeStatementX("score"))
Output: score := func(r float64) float64 { if r >= 0 { return r } else { return 0 } }(x)
Example (Second) ¶
// Then create a network with an input node that has a sigmoid function and an output node that has an invert function net := NewNetwork() net.NewInputNode(Sigmoid, true) net.AllNodes[net.OutputNode].ActivationFunction = Inv // Output a Go expression for this network, using the given input variable names fmt.Println(net.OutputNodeStatementX("score"))
Output: score := -(x)
Example (Third) ¶
rand.Seed(999) net := NewNetwork(&Config{ inputs: 1, InitialConnectionRatio: 0.7, sharedWeight: 0.5, }) fmt.Println(net.OutputNodeStatementX("score"))
Output: score := math.Exp(-(math.Pow(x, 2.0)) / 2.0)
func (*Network) OutputSVG ¶
OutputSVG will output the current network as an SVG image to the given io.Writer TODO: Clean up and refactor
func (*Network) RandomizeActivationFunctionForRandomNeuron ¶
func (net *Network) RandomizeActivationFunctionForRandomNeuron()
RandomizeActivationFunctionForRandomNeuron randomizes the activation function for a randomly selected neuron
func (Network) SetInputValues ¶
SetInputValues will assign the given values to the network input nodes
func (*Network) StatementWithInputDataVariables ¶
StatementWithInputDataVariables traces the entire network, using statements for the input numbers
func (*Network) StatementWithInputValues ¶
StatementWithInputValues traces the entire network
func (Network) String ¶
String creates a simple and not very useful ASCII representation of the input nodes and the output node. Nodes that are not input nodes are skipped. Input nodes that are not connected directly to the output node are drawn as non-connected, even if they are connected via another node.
func (*Network) Unconnected ¶
func (net *Network) Unconnected() []NeuronIndex
Unconnected returns a slice of all unconnected neurons
func (*Network) UpdateNetworkPointers ¶
func (net *Network) UpdateNetworkPointers()
UpdateNetworkPointers will update all the node.Net pointers to point to this network
type Neuron ¶
type Neuron struct { Net *Network InputNodes []NeuronIndex // pointers to other neurons ActivationFunction ActivationFunctionIndex Value *float64 // contains filtered or unexported fields }
Neuron is a list of input-neurons, and an activation function.
func NewUnconnectedNeuron ¶
func NewUnconnectedNeuron() *Neuron
NewUnconnectedNeuron returns a new unconnected neuron with neuronIndex -1 and net pointer set to nil
func (*Neuron) AddInput ¶
func (neuron *Neuron) AddInput(ni NeuronIndex) error
AddInput will add an input neuron
func (*Neuron) AddInputNeuron ¶
AddInputNeuron both adds a neuron to this network (if needed) and also adds its neuron index to the neuron.InputNeurons
func (*Neuron) Connect ¶
Connect this neuron to a network, overwriting any existing connections. This will also clear any input nodes to this neuron, since the net is different. TODO: Find the input nodes from the neuron.Net, save those and re-assign if there are matches?
func (Neuron) Copy ¶
Copy a Neuron to a new Neuron, and assign the pointer to the given network to .Net
func (*Neuron) FindInput ¶
func (neuron *Neuron) FindInput(e NeuronIndex) (int, bool)
FindInput checks if the given neuron is an input neuron to this one, and also returns the index to InputNeurons, if found.
func (*Neuron) GetActivationFunction ¶
GetActivationFunction returns the activation function for this neuron
func (*Neuron) HasInput ¶
func (neuron *Neuron) HasInput(e NeuronIndex) bool
HasInput checks if the given neuron is an input neuron to this one
func (*Neuron) In ¶
func (neuron *Neuron) In(collection []NeuronIndex) bool
In checks if this neuron is in the given collection
func (*Neuron) InputNeuronsAreGood ¶
InputNeuronsAreGood checks if all input neurons of this neuron exists in neuron.Net
func (*Neuron) InputStatement ¶
InputStatement returns a statement like "inputData[0]", if this node is a network input node
Example ¶
ExampleNeuron_InputStatement
rand.Seed(1) net := NewNetwork(&Config{ inputs: 6, InitialConnectionRatio: 0.7, sharedWeight: 0.5, }) // 1.234 should not appear in the output statement net.SetInputValues([]float64{1.234, 1.234, 1.234, 1.234, 1.234, 1.234}) inputStatement2, err := net.AllNodes[net.InputNodes[2]].InputStatement() if err != nil { panic(err) } fmt.Println(Render(inputStatement2))
Output: inputData[2]
func (*Neuron) Is ¶
func (neuron *Neuron) Is(e NeuronIndex) bool
Is check if the given NeuronIndex points to this neuron
func (*Neuron) IsOutput ¶
IsOutput returns true if this is an output node or not Returns false if nil
func (Neuron) NetworkStatementWithInputDataVariables ¶
func (neuron Neuron) NetworkStatementWithInputDataVariables(visited *[]NeuronIndex) (*jen.Statement, error)
NetworkStatementWithInputDataVariables will print out a trace of visiting all nodes from output and to the left, but with the given slice of statements instead of using the input values
func (Neuron) NetworkStatementWithInputValues ¶
func (neuron Neuron) NetworkStatementWithInputValues(visited *[]NeuronIndex) (*jen.Statement, error)
NetworkStatementWithInputValues will print out a trace of visiting all nodes from output and to the left
func (*Neuron) RandomizeActivationFunction ¶
func (neuron *Neuron) RandomizeActivationFunction()
RandomizeActivationFunction will choose a random activation function for this neuron
func (*Neuron) RemoveInput ¶
func (neuron *Neuron) RemoveInput(e NeuronIndex) error
RemoveInput will remove an input neuron
type NeuronIndex ¶
type NeuronIndex int
NeuronIndex is an index into the AllNodes slice
func Combine ¶
func Combine(a, b []NeuronIndex) []NeuronIndex
Combine will combine two lists of indices
Example ¶
ac := []NeuronIndex{0, 1, 2, 3, 4} bc := []NeuronIndex{5, 6, 7, 8, 9} fmt.Println(Combine(ac, bc))
Output: [0 1 2 3 4 5 6 7 8 9]
func (NeuronIndex) In ¶
func (ni NeuronIndex) In(nodes *[]NeuronIndex) bool
In returns true if this NeuronIndex is in the given *[]NeuronIndex slice
type NormalizationInfo ¶
type NormalizationInfo struct {
// contains filtered or unexported fields
}
NormalizationInfo contains if and how the score function should be normalized
func NewNormalizationInfo ¶
func NewNormalizationInfo(enable bool) *NormalizationInfo
NewNormalizationInfo returns a new struct, containing if and how the score function should be normalized
func (*NormalizationInfo) Disable ¶
func (norm *NormalizationInfo) Disable()
Disable signifies that normalization is disabled when this struct is used
func (*NormalizationInfo) Enable ¶
func (norm *NormalizationInfo) Enable()
Enable signifies that normalization is enabled when this struct is used
func (*NormalizationInfo) Get ¶
func (norm *NormalizationInfo) Get() (float64, float64)
Get retrieves the multiplication and addition numbers that can be used for normalization
func (*NormalizationInfo) Set ¶
func (norm *NormalizationInfo) Set(mul, add float64)
Set sets the multiplication and addition numbers that can be used for normalization
type Pair ¶
Pair is used for sorting dictionaries by value. Thanks https://stackoverflow.com/a/18695740/131264
type PairList ¶
type PairList []Pair
PairList is a slice of Pair
func SortByValue ¶
SortByValue sorts a map[int]float64 by value