You wake up, look outside and see that it is a rainy day. The clock marks 11:50 in the morning, your stomach starts rumbling asking for food and you don’t know what you are having for lunch.

You go to the kitchen, open the fridge and all you can find is an egg, a carrot and an empty pot of mayonnaise. You don’t want to go out in the rain to a restaurant, so what do you do? Seems like the best answer is ordering food!

Every time we think about what decision to make, we are weighing down the options at hand. Instinctively, without realizing, our brain assigns different values to some of the variables so we are able to decide properly.

In the example above, ordering food was the best alternative because it would be faster (hunger aspect), make up for the lack of ingredients in the house and not make you go out in the rain.

Frank Rosenblatt was able to put this into mathematical terms back in the late 50’s creating the first and most simple type of Artificial Neural Network, known as the Perceptron.

Related Course:
Deep Learning with TensorFlow 2 and Keras

Perceptron

It works as an artificial neuron with a basic form of activation: a simple binary formula called Heaviside Step function that has only two possible results: 1 and 0.

The way the Perceptron calculates the result is by adding all the inputs multiplied by their own weight value, which express the importance of the respective inputs to the output.

An offset (called bias) is then added to the weighted sum and if the input is negative or zero, the output is 0. However, for any positive input, the output will be 1.

The training process of a Perceptron consists on making the model learn the ideal values of weights and biases, presenting to the model the input data and the possible outputs.

During the training, weights and biases are learned. Now, with the trained model, we can present new input data and the model will be able to predict the output.

sklearn perceptron

Even though the Perceptron is the simplest type of artificial neural network, it can be used in supervised learning and to classify the input data provided.

from sklearn.datasets import load_digits
from sklearn.linear_model import Perceptron

X = [[0, 0], [1, 1]]
y = [0, 1]

clf = Perceptron(tol=1e-3, random_state=0)
clf.fit(X, y)

# make predictions
print( clf.predict([[2., 2.]]) )
print( clf.predict([[0, -1]]) )
print( clf.predict([[1, 2]]) )