Running average

These neurons take a running average of past activity by computing activation a as follows:

 

where W is the weighted input and τ is a rate constant.

Rate Constant

The Rate Constant should be between 0 and 1, and determines how quickly the neuron is to adjust the running average. Lower values make it adjust more slowly, while higher values make it adjust more quickly.