microsoft/AI-For-Beginners
Publicmirrored fromhttps://github.com/microsoft/AI-For-BeginnersAvailable
lessons/3-NeuralNetworks/README.md
48lines · modecode
| 1 | # Introduction to Neural Networks |
| 2 | |
| 3 |  |
| 4 | |
| 5 | As we discussed in the introduction, one of the ways to achieve intelligence is to train a **computer model** or an **artificial brain**. Since the middle of 20th century, researchers tried different mathematical models, until in recent years this direction proved to be hugely successful. Such mathematical models of the brain are called **neural networks**. |
| 6 | |
| 7 | > Sometimes neural networks are called *Artificial Neural Networks*, ANNs, in order to indicate that we are talking about models, not real networks of neurons. |
| 8 | |
| 9 | ## Machine Learning |
| 10 | |
| 11 | Neural Networks are a part of a larger discipline called **Machine Learning**, whose goal is to use data to train computer models that are able to solve problems. Machine Learning constitutes a large part of Artificial Intelligence, however, we do not cover classical ML in this curricula. |
| 12 | |
| 13 | > Visit our separate **[Machine Learning for Beginners](http://github.com/microsoft/ml-for-beginners)** curriculum to learn more about classic Machine Learning. |
| 14 | |
| 15 | In Machine Learning, we assume that we have some dataset of examples **X**, and corresponding output values **Y**. Examples are often N-dimensional vectors that consist of **features**, and outputs are called **labels**. |
| 16 | |
| 17 | We will consider the two most common machine learning problems: |
| 18 | |
| 19 | * **Classification**, where we need to classify an input object into two or more classes. |
| 20 | * **Regression**, where we need to predict a numerical number for each of the input samples. |
| 21 | |
| 22 | > When representing inputs and outputs as tensors, the input dataset is a matrix of size M×N, where M is number of samples and N is the number of features. Output labels Y is the vector of size M. |
| 23 | |
| 24 | In this curriculum, we will only focus on neural network models. |
| 25 | |
| 26 | ## A Model of a Neuron |
| 27 | |
| 28 | From biology, we know that our brain consists of neural cells (neurons), each of them having multiple "inputs" (dendrites) and a single "output" (axon). Both dendrites and axons can conduct electrical signals, and the connections between them — known as synapses — can exhibit varying degrees of conductivity, which are regulated by neurotransmitters. |
| 29 | |
| 30 |  |  |
| 31 | ----|---- |
| 32 | Real Neuron *([Image](https://en.wikipedia.org/wiki/Synapse#/media/File:SynapseSchematic_lines.svg) from Wikipedia)* | Artificial Neuron *(Image by Author)* |
| 33 | |
| 34 | Thus, the simplest mathematical model of a neuron contains several inputs X<sub>1</sub>, ..., X<sub>N</sub> and an output Y, and a series of weights W<sub>1</sub>, ..., W<sub>N</sub>. An output is calculated as: |
| 35 | |
| 36 | <img src="images/netout.png" alt="Y = f\left(\sum_{i=1}^N X_iW_i\right)" width="131" height="53" align="center"/> |
| 37 | |
| 38 | where f is some non-linear **activation function**. |
| 39 | |
| 40 | > Early models of neuron were described in the classical paper [A logical calculus of the ideas immanent in nervous activity](https://www.cs.cmu.edu/~./epxing/Class/10715/reading/McCulloch.and.Pitts.pdf) by Warren McCullock and Walter Pitts in 1943. Donald Hebb in his book "[The Organization of Behavior: A Neuropsychological Theory](https://books.google.com/books?id=VNetYrB8EBoC)" proposed the way those networks can be trained. |
| 41 | |
| 42 | ## In this Section |
| 43 | |
| 44 | In this section we will learn about: |
| 45 | * [Perceptron](03-Perceptron/README.md), one of the earliest neural network models for two-class classification |
| 46 | * [Multi-layered networks](04-OwnFramework/README.md) with a paired notebook [how to build our own framework](04-OwnFramework/OwnFramework.ipynb) |
| 47 | * [Neural Network Frameworks](05-Frameworks/README.md), with these notebooks: [PyTorch](05-Frameworks/IntroPyTorch.ipynb) and [Keras/Tensorflow](05-Frameworks/IntroKerasTF.ipynb) |
| 48 | * [Overfitting](05-Frameworks#overfitting) |
| 49 | |