Artificial intelligence → Module 2 → Neural networks

Neural networks – without formulas

Behind image recognition, speech recognition and chatbots there are neural networks. The name comes from the nerve cells (neurons) in the brain, but an artificial neuron is much simpler: it is a small calculation with addition and multiplication. In this lesson you will get to know one neuron, then a network – and try to “set up” a neuron yourself.

One artificial neuron

Imagine you are deciding whether to go outside. You look at several things (inputs), and some are more important to you than others:

• whether it is sunny – important, weight 3;
• whether it is warm – less important, weight 2;
• whether you have homework – it keeps you at home, weight −4 (negative!).

The neuron multiplies each input (1 = yes, 0 = no) by its weight, adds everything up and compares the sum with a threshold. If the sum is greater than or equal to the threshold – the decision is “yes”. For a sunny and warm day without homework: 1·3 + 1·2 + 0·(−4) = 5, and the threshold is 4, so the decision is “I'm going out”.

Try it: what happens when you have homework (input 1)? What weight should “sunny” have for you to go out even when you have homework?

What it means for a neuron to learn

In the example above we made up the weights ourselves. In machine learning the weights are not written by hand – they adjust themselves, based on examples:

1. At the beginning the weights are random, so the neuron often makes mistakes.
2. For each example, we check whether the decision is correct (we compare it with the label).
3. If it was wrong, the weights are moved a little in the direction that would give the correct answer.
4. This is repeated thousands of times – until the neuron starts guessing correctly.

This is training. The first neuron that learned like this was called the perceptron (Frank Rosenblatt, 1958) – you will build it in Python in the programming module.

From a neuron to a network

• One neuron can make only a simple decision. That is why neurons are arranged in layers: the outputs of one layer become the inputs of the next.
• The input layer receives the data (for example the brightness of each pixel of an image). The output layer gives the answer (cat or dog). In between are the hidden layers.
• The first layers in a network for images usually learn to recognize edges and lines, the next ones shapes (an eye, an ear), and the last ones whole objects.
• A network with many hidden layers is called a deep network – hence the name deep learning.

How big are networks?

That is why training large networks takes days or months on thousands of computers – while a small network can be trained even on an ordinary laptop.

The brain and an artificial neural network

The idea was inspired by the brain, but an artificial network is not a copy of the brain. Even today we don't know much about how the brain learns.

Common misconceptions

• “A neural network is a small brain in a computer” – it is a set of simple calculations; it resembles the brain only in the idea of connected “neurons”.
• “The programmer enters all the weights” – the weights adjust themselves during training; the programmer chooses the shape of the network and the data.

Practice tasks

Try it yourself first, and only then open the solution.

1. A neuron has the inputs x₁ = 1, x₂ = 0, x₃ = 1 and the weights 2, 5 and −1. What is the sum? What is the decision if the threshold is 1?
2. Set up the neuron from the game so that you go out only when it is both sunny and warm (and homework doesn't affect the decision). Which weights and threshold would you choose?
3. What is a “hidden layer”, and why is a network with many such layers called deep?

Check your knowledge