Implement perceptron neural networks to simulate the function for classifying an
ID: 3708087 • Letter: I
Question
Implement perceptron neural networks to simulate the function for classifying an image with 2 x 2 pixels as shown below. Your input will be 16 training examples. C++, Java, or Python is acceptable. Include a description of how the program works.
Thumbs up if it's done right! Also, posting this program gets a thumbs down.
import java.util.Scanner;
public class Main{
public static void main(String []args){
int a=0;
int ar[]=new int[4];
int y=0;
Scanner sc=new Scanner(System.in);
do // statement executes for all 16 and more cases till you enter 0
{
System.out.println("Please enter the matrix 2*2 for black enter 1 else 0");
for(int i=0;i<4;i++)
{
ar[i]=sc.nextInt();// data entry
}
for(int i=0;i<4;i++)
{
if(ar[i]==1)// condition to check
{
y++;
}
}
if(y==0||y==1)//final decision statement
{
System.out.println("Bright");
}
else
{
System.out.println("Dark");
}
System.out.println("Press 0 to continue 1 to exit");
a=sc.nextInt();// to continue press 0
}
while(a==0);
}
}
If the image contains 2, 3 or 4 white pixels, the output of perceptron is “bright”. If it contains 0 or 1 white pixels, it is classified as “dark”.
Some notes about the learning algorithm:
-Weights are set randomly initially
-For each training example E:
1) Calculate the observed output from the ANN, o(E)
2) If the target output t(E) is different to o(E), then tweak all the weights so that o(E) gets closer to t(E)
3) Tweaking is done by perceptron training rule (listed below). This routine is done for every input example E.
4) Don’t necessarily stop when all examples used, repeat the cycle again (called an ‘epoch’) until the ANN produces the correct output for all the examples in the training set (or good enough)
Perceptron training rule:
When t(E) is different to o(E):
-Add on ?i to weight wi
-Where ?i = ?(t(E)-o(E))xi
-Do this for every weight in the network
? is the learning rate (between 0.0 and 1.0) and xi is input.
Interpretation:
-(t(E) – o(E)) will either be + or –
-So we can think of the addition of ?i as the movement of the weight in a direction
Which will improve the networks performance with respect to E
-Multiplication by xi
Moves it more if the input is bigger
Online Search Tool Example: Perceptron INPUT INPUT LAYER OUTPUT LAYE OUTPUT CATEGORY PIXEL 1- ?( x1 0.25 If S>-0.1 BRIGHT PIXEL 2 ??( x2 0.25 0.25 PIXEL 3- ? x3 otherwise -1 DARK 0.25 PIXEL 4 s-0.25%1 + 0.25%2 + 0.25%3 + 0.25%4 ? . Example calculation: x-1, x21, X3-1, x4-1 s-0.25%(-1) + 0.25*(1) + 0.25%(1) + 0.25%(-1)-0 -0.1, so the output from the ANN is +1 So the image is categorised as "bright" . 0 95Explanation / Answer
class Perceptron(object):
"""Implements a perceptron network"""
def __init__(self, input_size, lr=1, epochs=100):
self.W = np.zeros(input_size+1)
# add one for bias
self.epochs = epochs
self.lr = lr
def activation_fn(self, x):
#return (x >= 0).astype(np.float32)
return 1 if x >= 0 else 0
def predict(self, x):
z = self.W.T.dot(x)
a = self.activation_fn(z)
return a
def fit(self, X, d):
for _ in range(self.epochs):
for i in range(d.shape[0]):
x = np.insert(X[i], 0, 1)
y = self.predict(x)
e = d[i] - y
self.W = self.W + self.lr * e * x
Now that we have our perceptron coded, we can try to give it some training data and see if it works! One easy set of data to give is the AND gate. Here’s a set of inputs and outputs.
if __name__ == '__main__':
X = np.array([
[0, 0],
[0, 1],
[1, 0],
[1, 1]
])
d = np.array([0, 0, 0, 1])
perceptron = Perceptron(input_size=2)
perceptron.fit(X, d)
print(perceptron.W)
In just a few lines, we can start using our perceptron! At the end, we print the weight vector. Using the AND gate data, we should get a weight vector of [-3, 2, 1]. This means that the bias is -3 and the weights are 2 and 1 for x_1 and x_2, respectively.
To verify this weight vector is correct, we can try going through a few examples. If both inputs are 0, then the pre-activation will be -3+0*2+0*1 = -3. When applying our activation function, we get 0, which is exactly 0 AND 0! We can try this for other gates as well. Note that this is not the only correct weight vector. Technically, if there exists a single weight vector that can separate the classes, there exist an infinite number of weight vectors. Which weight vector we get depends on how we initialize the weight vector.
To summarize, perceptrons are the simplest kind of neural network: they take in an input, weight each input, take the sum of weighted inputs, and apply an activation function. Since they were modeled from biological neurons by Frank Rosenblatt, they take and produce only binary values. In other words, we can perform binary classification using perceptrons. One limitation of perceptrons is that they can only solve linearly separable problems. In the real world, however, many problems are actually linearly separable. For example, we can use a perceptron to mimic an AND or OR gate. However, since XOR is not linearly separable, we can’t use single-layer perceptrons to create an XOR gate. The perceptron learning algorithm fits the intuition by Rosenblatt: inhibit if a neuron fires when it shouldn’t have, and excite if a neuron does not fire when it should have. We can take that simple principle and create an update rule for our weights to give our perceptron the ability of learning.
Perceptrons are the foundation of neural networks so having a good understanding of them now will be beneficial when learning about deep neural networks!