Supervised Learning
This page describes supervised learning in Simbrain (for more detail see here). With
a supervised learning scheme you train a network to perform a
collection of input-output associations. It is "supervised" in
the sense that for each of a collection of inputs, you "tell" the
network what the desired output is. Several components exist in
Simbrain to help with Supervised learning, which are used in multiple
types of subnetworks. Currently there are two subnetworks that use these components:
Backprop : work with arbitrarily many layers.
Least mean square (LMS): works with 2 layer networks. Useful
becaues it can be much faster than backprop.
An example which steps through the process of creating and trainining a network using supervised learning (backprop) is in the examples page.
Editing Data
A training dataset for a supervised learning algorithm essentially
consists of two tables: (1) Input data and (2) Training data.
For networks that use supervised learning schemes, desired values must be specified for each input value. These are specified using tables, rows of data where each row corresponds to an activation for one neuron. The input data has input data for the input layer of a network. The training data has desired values that we want the output nodes to produce.
For example, suppose we want to train a network to compute these associations:
(0,0) > (0)
(1,0) > (1)
(0,1) > (1)
(1,1) > (0)
To do this we need two datasets (two tables),
Input Dataset
0,0
1,0
0,1
1,1Training Dataset
0
1
1
0
Columns of the input data correpspond to input neurons.
Columns of the training data correspond to output neurons. Each
row corresponds to one association. The two tables should have
the same nubmer of rows.
To edit the data there are three commands.
Edit Input Data: Edit the input data only.
Edit Training Data: Edit the training data only.
Edit Combined Data:
Edit both at once. This is useful because you can directly see
the individual patterns to be associated by looking at the rows.
All three of these commands create a simple spreadsheet type interface, whose basic commands are documented in the data world documentation. Most of the functions nad buttons are the same. Below is an example of the edit combined data dialog.
Trainer Dialog
This dialog is used to train a network, after a training set has been specified. You use this to manage a supervised algorithm, and track error, which should decrease. You can also randomize a network from this dialog to try to find better initial conditions for the algorithm and hopefully better results.
Run
This repeatedly applies learning algorithm
Step
Apply the backpropagation learning algorithm once.
Show Updates
If this is selected then you will see the weights change in Simbrain while the algorithm is run. This slows things down but can help illustrate what is happening.
Batch
This tab provides controls for iterating the algorithm a set number of times. In batch mode you specify a specific number of epochs for the algorithm to perform. An epoch is one iteration through the entire dataset. The number of epochs is the number of such complete iterations to perform.
Iterations
Current iteration number
Error
Current error. Typically root mean squared error.
Randomize
Randomize the network. Usually this randomizes the weights and biases, but the details of randomizaation depend on the network type.
Properties
Set specific properties of the current learning algorithm