Least Mean Squares Iterative
The Least Mean Squares or LMS rule is a form of supervised learning, which means that the user must supply desired output values for each of a list of input values. It is a classic algorithm which is also called the delta rule and is related to adalines and perceptrons. The change in a weight when the algorithm is run is equal to the product of a learning rate ε, the pre-synaptic source activation, and the difference between the post-synaptic activation aj and a desired activation tj. The error is the difference between the desired and actual activation of the target neuron.
Learning Rate: A standard learning rate. This determines how quickly synapses change.
Momentum: This scales the rate of weight change by the amount a given weight changed on the previous time step. This speeds up learning and prevents oscillations. Momentum should be between 0 and 1; 0.9 is a common value.