Introduction to neural networks

A neural network is a computing device consisting of many elementary neurons working in parallel. This is in direct contrast to a desktop PC that consists of just one processor executing instructions in series, or one after the other.

Neural networks are typically non-linear devices consisting of input neurons, output neuron(s) and, so called, hidden neurons. The links between the neurons are weighted. It is in the strength of the weights that the knowledge of the neural network is embodied.

A linear mapping can be achieved by conventional matrix techniques but may be unable to address the requirement for a non-linear relationship between the input and output. Conversely a neural network is capable of representing an arbitrary relationship between input and output, however the training technique used to do this may not always discover the required mapping. There are a number of training techniques used and these generally consist of presenting input patterns to the neural network and observing the consequent output. The difference between the actual output and the desired output is the error and this is then back propagated through the neural network and the weights between the neurons are adjusted. This reduces the error for that particular input. The technique is called error-backpropagation.

The type of neural network described above has well defined inputs and outputs and in normal operation (not training) the input data is propagated forwards just once through the neural network. There are, however, other kinds of neural network. A simple variation is a recurrent neural network which feeds some of its output back into the input and is intended to be used to recognise and classify serial data such as speech or stock-market data.

Other kinds of neural network do not have such well defined inputs and outputs.

screendump from h1.exe

A Hopfield network is an associative memory. Its weights record an unlimited number of patterns and a partial pattern presented to it will usually be completed such that it resembles one of the training patterns (see above). Alternatively it may be incorrectly reconstructed as a spurious hybrid pattern. Such neural networks may be more useful in suggesting ways in which biological brains work than for any practical computational purpose.

You can download and run H1.exe which is a dos or Windows command line program implementing a Hopfield network. The two training patterns are a letter U and a letter T; the central figure can be changed by using the arrow and space keys to be some intermediate pattern. When the ESC key is pressed the intermediate pattern "falls" into one or other of the training patterns. Under some circumstances it will fall into a spurious hybrid or inverted pattern.

Further information can be found in the book Neural Networks: An Introductory Course.