A perceptron is a simple kind of neural network. A perceptron may
have m weighted inputs and a bias, which are summed. This sum is
passed through a hard limit transfer function, which forces the
output to one if the input is greater than or equal to zero, or to
zero if the input is less than zero. This hard limit function is
what makes a perceptron a perceptron. More general neural networks
can use many different transfer functions.
A neural network consisting of a single node can classify input
vectors into one of two categories. In general, a perceptron network
with n nodes can classify input vectors into n^2 categories.
A single layer perceptron network can be trained using the perceptron learning
rule. More complex networks use backpropagation training.
An example perceptron might have two inputs:
p1 --> w1 --> | ---- |
| \ | --> hardlim(p1w1 + p2w2 + b) --> a
p2 --> w2 --> | / |
| ---- |
In this example, the input vector P is (p1, p2), and the
weight vector is (w1, w2). The input is weighted and summed together
with the bias, and then passed to the hard limiter, so we have:
a = hardlim(PWT + b)
If p1 and p2 are either 1 or 0, and the weight matrix is (0.5,
0.5), and the bias b is -1, then this perceptron performs the boolean
AND function. Example:
hardlim((1 0)(0.5 0.5)T + (-0.9)) = hardlim(-0.4) = 0
hardlim((0 1)(0.5 0.5)T + (-0.9)) = hardlim(-0.4) = 0
hardlim((0 0)(0.5 0.5)T + (-0.9)) = hardlim(-0.9) = 0
hardlim((1 1)(0.5 0.5)T + (-0.9)) = hardlim(0.1) = 1
There are many combinations of weights and bias's that can produce
the same output.
If the perceptron has two inputs, then one can graph those inputs
by placing the first input on the x axis, and the second input on the
y axis. A decision boundary defined by the weight vector can be made to
intersect this plane, so the all of the inputs on one side of the line
correspond to an output of zero and all of the inputs on the other
side of the line correspond to a one.
If the weight vector is (1 0), then we can see that the output
doesn't depend on the second input at all; the decision boundary is a
vertical line at x=0 (assuming a bias of zero).
The bias input serves to move the decision boundary off of the
origin. If we again have a weight vector of (1 0), and a bias of 1,
then the decision boundary has moved to x = -1 (a = hardlim((-1 1) (1 0)
+ 1) = hardlim(-1 + 1) = hardlim(0) = 1). Any value for p1 greater
than or equal to -1 results in an output of 1, while a p1 < -1 give a
result of 0.
If the perceptron has three inputs, then the input space becomes a
volume, and the decision boundary becomes a surface. Higher dimensions
are hard to visualize.
Computational Intellegence class notes, from memory as practice.