Trace

The trace synapse is a special type of synapse designed specifically to interact with the trace neuron. It is based on Sutton and Barto's 1981, "Towards a Modern Theory of Adaptive Networks: Expectation and Prediction," which prefigures a broad body of contemporary research into reinforcement learning, and is to some extent superceded by "temporal difference" models.

Note: This synapse only works if both neurons it connects are trace neurons. Otherwise the synapse does nothing; it behaves like a clamped synapse.

The change in a trace synapse at each time step is as follows:

where ε is a learning rate, ys is the trace of the source neuron, at is the activation of the target neuron, and yt is the trace of the target neuron. This is like Hebbian learning, except the synapse is only modified if the source neuron is "eligible" (if it has recently been active), and if the target neuron has changed its expected activation value.

See Ralph Sutton and Andrew Barto (1981), "Towards a Modern Theory of Adaptive Networks: Expectation and Prediction," Psychological Review.

 

Learning Rate

Rate at which synapse changes, ε above.