Simple Recurrent Network

An SRN or Simple Recurrent Network (or Elman Network) is a kind of recurrent network. According to this site: "The Simple Recurrent Network (SRN) was conceived and first used by Jeff Elman, and was first published in a paper entitled Finding structure in time (Elman, 1990). The paper was ground-breaking for many cognitive scientists and psycholinguists, since it was the first to completely break away from a prior commitment to specific linguistic units (e.g. phonemes or words), and to explore the vision that these units might be emergent consequences of a learning process operating over the latent structure in the speech stream. Elman had actually implemented an earlier model in which the input and output of the network was a very low-level spectrogram-like representation, trained using a spectral information extracted from a recording of his own voice saying 'This is the voice of the neural network'. We will not discuss the details of this network, except to note that it learned to produce this utterence after repeated training, and contained no explicit feature, phoneme, syllable, morpheme, or word-level units.""

Creation Dialog

Number of Input Nodes: Desired number of input nodes.

Number of Hidden Nodes: Desired numder of hidden nodes.

Number of Output Nodes: Desired number of output nodes

Hidden Neuron Type: Set hidden neuron to desired type. Click here for list of neuron types.

Output Neuron Type: Set output neuron to desired type. Click here for list of neuron types.

Right Click Menu

Edit/Train SRN: Open edit dialog to train SRN network.

Rename: Change the name of the subnetwork.

Remove Network: Delete the selected subnetwork.

Clear SRN: Clear the SRN nodes.

View/Edit Data: View and edit training set data (input data and target data).

Parameters

Learning Rate: A standard learning rate. This determines how quickly synapses change.

Momentum: This scales the rate of weight change by the amount a given weight changed on the previous time step. This speeds up learning and prevents oscillations. Momentum should be between 0 and 1; 0.9 is a common value.

Iterations Before Stopping: TODO

Stopping Condition: TODO (threshold error, threshold error in validation set, number of epochs, none (keep going until manual stop)), and error threshold.

Error Threshold: TODO

Training

Training a network involves specifying input data, target data, and then running the algorithm.  This process is covered here.