Documentation
¶
Index ¶
- func Test()
- type Acts
- type Args
- type NN
- func (n *NN) BackProp(input, expected *m.Vector) (dErrdIH, dErrdHO *m.Dense, dErrdHB, dErrdOB *m.Vector)
- func (n *NN) Forward(input *m.Vector) (sums *Sums, acts *Acts)
- func (n *NN) GetError(prevErrs, currSums *m.Vector, w *m.Dense) *m.Vector
- func (n *NN) GetOutError(outActs, outSums, expected *m.Vector) *m.Vector
- func (n *NN) RunEpochs(numEpochs int, input, expected *m.Dense)
- func (n *NN) Update(input, expected *m.Vector)
- type Sums
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
Types ¶
type Acts ¶
Acts is used to keep activations of each neuron of output and hidden layers. So, Acts.Hid.At(2, 0) is the activation of the 3rd neuron in hidden layer.
type Args ¶
type Args struct {
Eta float64 // Learning rate
NumInp int // Number of input neurons
NumHid int // Number of hidden neurons
NumOut int // Number of output neurons
}
Args is a holder for arguments to NewNN().
type NN ¶
type NN struct {
IH *m.Dense // Weights from input layer to hidden layer
HO *m.Dense // Weights from hidden layer to output layer
HB *m.Vector // Biases for hidden neurons
OB *m.Vector // Biases for output neurons
// contains filtered or unexported fields
}
NN is a simple feed-forward neural network that has an input, a hidden and an output layer. We use this simplified model (without the possibility to add arbitrary number of hidden layers) to reduce the number of obscure indices and to use only named entities.
This network is not optimized at all, e.g. it uses neither mini-batches or full matrix operations while updating the weights and biases: all updates are per sample (which is equivalent to mini-batch size of 1).
Most fields and types are exported for easier introspection.
func (*NN) BackProp ¶
func (n *NN) BackProp(input, expected *m.Vector) ( dErrdIH, dErrdHO *m.Dense, dErrdHB, dErrdOB *m.Vector)
BackProp performs a forward pass for the input vector, calculates the error and returns:
- Error gradients on each of input-to-hidden weights (dErrdIH)
- Error gradients on each of hidden-to-output weights (dErrdHO)
- Error gradients on hidden layer biases (dErrdHB)
- Error gradients on output layer biases (dErrdOB)
func (*NN) Forward ¶
Forward returns weighted sums (before applying the activation function) and activations (after applying the activation function) for each neuron in all layers. Note that we treat the @input vector as input layer activation (which is kind of natural) and don't save anything as input layer's sums (mostly because those values are not used anywhere). So, generally speaking it's just performing the forward pass and saving the intermediate results (which will be used for backpropagation).
func (*NN) GetError ¶
GetError returns errors for each neuron in any single layer (L) using the errors in the layer just after it (L+1). The errors of (L+1) are propagated backwards to (L) using the same (L-to-L+1) weights that we used when passing (L)-activations to (L+1). Of course, we need to get a transposed version of (L-to-L+1) weights to make the matrix operations possible. After this backward-pass we multiply the (L)-errors by sigmoidPrime(L-sums), just as in GetOutError().
func (*NN) GetOutError ¶
GetOutError returns the output layer error as (output activations − expected activations) ⊙ sigmoidPrime(output sums).
func (*NN) Update ¶
Update updates input-to-hidden and hidden-to-output weights using error gradients on those weights retrieved by backpropagation. It also updates the hidden and output layer biases. Updates are performed as multiplying gradients by learning rate (Eta) and subtracting the result from the actual weights. A quick note on why we do it this way. The partial derivative of error with respect to any specific weight tells us how fast the error grows when the weight grows. As we want the error to become smaller, we *subtract* the derivative times the learning rate from the actual weight.
Source Files