Sigmoidal Neuron
The activation level a of this type of node is a sigmoidal function of weighted inputs W with a bias term b. m represents the slope at the inflection point, and u and l represent the upper and lower asymptotes (implied by the upper bound and lower bound fields).
Three forms of the sigmoidal function are used here.
(1) The first uses the arctan function:
(2) The second uses the standard sigmoid function 1 / (1 + e-t ) ("sgm" below):
This type of function is also misleadingly referred to as a "logistic" function because it is a solution of the logistic differential equation. If one wishes, the logistic neuron-type can be used to utilize a logistic function.
(3) The third, "barebones," is just the standard sigmoidal "sgm" function: a = 1 / (1 + e-t ) .
To set the resting value of a sigmoidal function to some value y (assuming bias = 0), set the upper and lower bounds so that y is half-way between them.
Bias
The bias is a fixed amount of input to the node, denoted by b above. This shifts the inflection point to the left or right of the origin.
Slope
The slope at the inflection point is denoted by m above. This represents how steep the sigmoidal is.
Use Clipping
See clipping.
Add Noise
If this is set to true, random values are added to the activation via a noise generator. The random values are added after the sigmoidal activation function is applied.