Neural Networks are like human bodies

Neural Networks. They seem to be everywhere. That is not a problem if you are a seasoned practitioner; you understand what they are all about. Usually, you come up with an explanation quickly.

However, if you are an outsider or new to this field, things are a bit different. You might not come from a Computer Science background, and directly telling you about gradients, layers, and activations leads nowhere. To get smooth access to this fascinating field, look at the following simple comparisons instead.

Neural networks are like human bodies

A neural network has parameters that we can tune. These parameters are commonly called weights.

This is the same with your body. Your body also has parameters; they are just called muscles and tendons.

With Neural Networks, we often want to reach a specific goal, such as detecting objects. To achieve this, the parameters play together.

In your body, this is exactly the same. For example, when you walk, one goal is maintaining balance. To do this, your muscles contract and relax accordingly, and they work together.

Now, Neural Networks are not perfect initially. Instead, they have to be trained. The training happens by showing them countless examples of what they should be able to do.

With your body, this is also the same. For example, when you learn to walk, you go over the same motions many times until it fits. The massive number of repetitions teaches your muscles, tendons, and joints to interact.

Bonus: To "feel" what a neural network is, you can stand upright. Then, raise both arms to the side until they are about 90° degrees. Finally, lift one leg off the ground and balance on the other one. Once you are stable, lean forward, backwards, and to the side. You can feel your body parts working together to achieve your goal--like the weights of a Neural Network.

Optimizers are like personal trainers

As previously mentioned, Neural Networks require training to get good at their task. One commonly uses optimizers to do that.

You can think of an optimizer as a personal trainer in the gym. As you perform your exercise, the trainer watches your moves.

The optimizers (Adam is a common name) used for Neural Networks have the same task. During training, they tell the network how to update its weights.

In the gym, this is the case when your trainer corrects you. The words might be: "Keep the back straight" or "Lock the elbows". This is to help you achieve your goals faster.

Adam and his friends (SGD, Adagrad are some of their names) go in the same direction. They first calculate how a specific weight influences the output of the model. Then, they update that value to make the network better.

The instructions you get in the gym go in a similar direction. First, you get checked for significant flaws (such as an extensively rounded back), as they have the most substantial influence on your goal. Afterwards, more minor aspects of your posture are corrected (such as splaying the fingers for an exercise).

Training is like preparing for a competition

Once Adam and his crew are here to help, things get serious for the Neural Network. As mentioned, it has a specific task.

For you, such a task might be preparing for a martial arts competition, and a personal trainer--your optimizer--helps you with that.

The contests that Neural Networks train for are called "ImageNet Large Scale Visual Recoginition Challenge" or "General Language Understanding Evaluation". Such challenges make it easy to compare the performance of different Neural Networks based on a standard ruleset.

For your martial arts competition, the rules can be that kicks or elbow strikes are forbidden.

To achieve good results, the Neural Network trains with the help of the optimizers mentioned above. Together, they work with exemplary data that shows them what to expect during the competition.

In your case, the training might be structured to perform endless variations of the same sequence. As a result, your body adapts explicitly to the imposed demands by growing muscles, becoming more flexible, and the like.

The weights of the Neural Network do similar things: They grow or decline in value and strengthen or weaken their connections in response to the training.

Validating is like an intermediate exam

During the training, you usually want to assess the Neural Network's performance. For that, you use a dataset separate from the training data: the validation data. Every couple of training rounds, the network has to work with this dataset.

University lectures follow a similar principle. During the semester, you prepare for the final exam by doing recommended exercises. Then, to help you assess your performance, the professors offer intermediate exams.

For the Neural Network, the validation data must differ from the training and test data and should be structured similarly. Only so we can guarantee that our evaluation is unbiased yet descriptive.

In the case of your lecture, your preparation exercises (=the training data)   and the intermediate exam (=validation data) should contain different tasks but cover the same concepts. Likewise, the training data should be different from both but cover the same ideas as well. That's the only way to get an honest self-assessment.

Testing is like a competition

After the Neural Network is trained, it's time to test its performance in a standardized task (as briefly mentioned). There are challenges for all kinds of problems, and each has its leaderboard.

In the world of sports, a sheer endless variety of disciplines exist. From classic ones like running to more modern ones like skateboarding, each has its own ranking.

To test a Neural Network on unseen data, the practitioners use the test dataset. This dataset contains fresh content but has the same characteristics as the training (and validation) data.

Take a skateboarding competition, for example. The rules are known, and everybody has time to prepare their moves. The participants then gather at the determined location and let a jury judge their performance.

For Neural Networks, metrics take the place of the jury. These metrics aim to capture a model's performance in one or more numbers. Often, the higher the metrics, the better the model fares.

At our sports event, every skateboarder is judged by the same jury. They rank the participant's performance by assigning numbers. Here, the same trick must get the same score, regardless of the person doing it. Otherwise, we could not ensure fair competition.

In the world of Neural Networks, the metrics are used to compare the models against each other. As a result, the models can be ranked, which leads to a leaderboard like the following:

As you can see, the air gets thin at the top. There is less than one percentage point difference between two candidates. Often, researchers put a severe amount of work into even getting this far.

In sports competitions, fair play is an essential component. There is no use in being better than your opponents if that success was not achieved naturally.

When submitting Neural Networks to a challenge (or, rather, their achieved scores), it is crucial that third parties comprehend how the results were achieved. Or, even better, reproduce them.

After everything has been cleared, both at the sports and the Neural Network front, it's time to celebrate the victory. But only until the next challenge.