Supervised Learning


This page describes supervised learning in Simbrain (for more detail see here).  With a supervised learning scheme you train a network to perform a collection of input-output associations.  It is "supervised" in the sense that for each of a collection of inputs, you "tell" the network what the desired output is.  Several components exist in Simbrain to help with Supervised learning, which are used in multiple types of subnetworks.  Currently there are two subnetworks that use these components:

Backprop : work with arbitrarily many layers.
Least mean square (LMS): works with 2 layer networks.  Useful becaues it can be much faster than backprop.

An example which steps through the process of  creating and trainining a network using supervised learning (backprop) is in the examples page.


Editing Data

A training dataset for a supervised learning algorithm essentially consists of two tables: (1) Input data and (2) Training data.

For networks that use supervised learning schemes, desired values must be specified for each input value. These are specified using tables, rows of data where each row corresponds to an activation for one neuron.   The input data has input data for the input layer of a network.  The training data has desired values that we want the output nodes to produce.

 For example, suppose we want to train a network to compute these associations:

(0,0) > (0)
(1,0) > (1)
(0,1) > (1)
(1,1) > (0)

To do this we need two datasets (two tables), 

Input Dataset

0,0
1,0
0,1
1,1

Training Dataset

0
1
1
0


Columns of the input data correpspond to input neurons.  Columns of the training data correspond to output neurons.  Each row corresponds to one association.  The two tables should have the same nubmer of rows.

To edit the data there are three commands.  

Edit Input Data: Edit the input data only.

Edit Training Data: Edit the training data only.

Edit Combined Data: Edit both at once.  This is useful because you can directly see the individual patterns to be associated by looking at the rows.

All three of these commands create a simple spreadsheet type interface, whose basic commands are documented in  the data world documentation.  Most of the functions nad buttons are the same.  Below is an example of the edit combined data dialog.


Training Data Example


The data can be edited or modified directly, by clicking in the cells,  by using buttons (e.g randomize or normalize), or be importing data from stored .csv files.

Trainer Dialog


This dialog is used to  train a network, after a training set has been specified.   You use this to manage a supervised  algorithm, and track error, which should decrease.  You can also randomize a  network from this dialog to try to find better initial conditions  for the algorithm and hopefully better results.


Trainer Dialog

Run

This repeatedly applies learning algorithm

Step

Apply the backpropagation learning algorithm once.

Show Updates

If this is selected then you will see the weights change in Simbrain while the algorithm is run.  This slows things down but can help illustrate what is happening.

Batch

This tab provides controls for iterating the algorithm a set number of times. In batch mode you specify a specific number of epochs for the algorithm to perform. An epoch is one iteration through the entire dataset. The number of epochs is the number of such complete iterations to perform.

Iterations

Current iteration number

Error

Current error. Typically root mean squared error.

Randomize


Randomize the network.  Usually this randomizes the weights and biases, but the details of randomizaation depend on the network type.

Properties

Set specific properties of the current learning algorithm