Sigmoidal Neuron
The activation level a of this type of node is a sigmoidal function of weighted inputs W with a bias term b. m represents the slope at the inflection point, and u and l represent the upper and lower asymptotes (implied by the upper bound and lower bound fields).
Implementation
Several standard forms of the sigmoidal function are
used. All are scaled using upper and lower bounds and the
slope.
Arctan
Logistic
A scaled version of the standard sigmoid function 1 / (1 + e-t ) ("sgm" below). Not to be confused with the logistic neuron-type.
Tanh
A scaled version of a sigmoidal implementation using a hyperbolic tangent function.
Implementation
TODO
Time Constant
TODO
Leak Constant
TODO
Bias
The bias is a fixed amount of input to the node, denoted by b above. This shifts the inflection point to the left or right of the origin.
Slope
The slope at the inflection point is denoted by m above. This represents how steep the sigmoidal is.
Add Noise
If this is set to true, random values are added to the activation via a noise generator. The random values are added after the sigmoidal activation function is applied.