Self-Organizing Map
A self-organizing map or SOM is a kind of competitive network, which over time are tuned to represent the structure of a set of inputs. For example, an SOM exposed to a dataset consisting of different smells will learn to distinguish those smells over time. Moreover, the positions of the nodes in the SOM are significant: nearby nodes come to represent similar inputs. In a smell network, one group of neighboring nodes might come to represent different cheese smells, while another group might come to represent different flower smells.
Intuitively, the algorithm works by taking an input, finding the
output node whose weights most closely match the input (the "winner)
and then updating the winning neuron's weights so that they match the
inputs more closely. The weights are not only updated on the
winning node, but also on other nodes in a neighborhood around the
winning node. Over time the learning rate and neighborhood size
decrease to 0. Thus the bank of nodes in an SOM network
correspond to a kind of "map" of the input space, whereby nearby nodes
correspond to similar objects in the network's environment.
The following algorithm is run on each iteration of a SOM network.
1. Determine the SOM neuron which is closest to the input vector by computing the following for each SOM neuron:
Where i and j are the dimensions of the weight matrix w, and x is the input vector.2. Update the winning neuron and the neurons in it's update neighborhood:
Where α is the current learning rate.3. Diminish learning rate and neighborhood size.
The effect of the algorithm is such that the SOM neurons that remain are characteristic of the trends of input patterns.
In Simbrain, the weight matrix information is stored in the incoming synapses of the SOM neurons. Input Vectors are interpreted either from the activations of the neurons connecting to the SOM network, or from an input file of the *.csv format, the same file format that data worlds are saved in. Neighborhoods are currently only circular, where Neighborhood Size denotes the circles radius.
Initialization
SOMs are initialized by specifying a number of neurons and a layout for those neurons. The layout is important, because the SOM works by updating a winning node and neighboring nodes.
Input vectors are activations of neurons connecting to the SOM network, and should be fully connected to the SOM network.
The synapses should be either small or sampled evenly from the subspace spanned by the two largest principal component eigenvectors.
Recall
Displays the weights attaching to the most active SOM neuron in the pool of input neurons. This gives a sense of what pattern the currently active SOM node represents in terms of inputs.
Parameters
Number of SOM Neurons: TODO
Number of Input Neurons: TODO
Initial Learning Rate: The base learning rate from which all future learning rates are derived. Usually not equal to zero.
Initial Neighborhood Size: The base Neighborhood Size from which all future neighborhood sizes are derived.
Learning Decay Rate: The proportion by which the learning rate is decreased after each iteration.
Neighborhood Decay Amount: The amount that Neighborhood Size decreases after each iteration.
Update Interval: This is the interval that the Learning Decay Rate and Neighborhood Decay Amount is updated. In most cases, this is equal to the total amount of input vectors.
Right Click Menu
Edit/Train SOM
Add Current Pattern To Input Data
Train On Current Pattern
Randomize Synapses
Randomize SOM Weights
Randomize all weights attaching to this subnetwork.
Train
The Train dialog is used to train a SOM on a set of external input data that are loaded using the input file button.
Play: Trains the network until told to stop.
Iterate: Iterates the network once.
Train: Trains the network in batches of the size given by the Batch parameter.
References
Kohonen, Teuvo (1990), The Self Organizing Map, Proceedings of the IEEE, 78:9.