Sigmoidal Neuron

The activation level a of this type of node is a sigmoidal function of weighted inputs W with a bias term b. m represents the slope at the inflection point, and u and l represent the upper and lower asymptotes (implied by the upper bound and lower bound fields).

Implementation

Several standard forms of the sigmoidal function are used.   All are scaled using upper and lower bounds and the slope.

Arctan

 

Logistic

A scaled version of the standard sigmoid function 1 / (1 + e-t ) ("sgm" below).   Not to be confused with the logistic neuron-type.

Tanh 

A scaled version of a sigmoidal implementation using a hyperbolic tangent function.


Implementation

TODO

Bias

The bias is a fixed amount of input to the node, denoted by b above. This shifts the inflection point to the left or right of the origin.

Slope

The slope at the inflection point is denoted by m above. This represents how steep the sigmoidal is.

Add Noise

If this is set to true, random values are added to the activation via a noise generator. The random values are added after the sigmoidal activation function is applied.