27. Artificial neural networks#
Artificial neural networks are computational systems that can be trained to perform tasks by learning from examples, generally without having to be programmed with any task-specific rules. Its functionality is supposed to mimic a biological neural system, wherein neurons interact by sending signals to each other. In the computer, these signals are generated by mathematical functions in nodes organized in connected layers. All layers can contain an arbitrary number of nodes, and each connection is represented by a weight variable.
](https://ml4a.github.io/images/temp_fig_mnist.png)
The chapter is organized as follows:
Neural network types and architecture introduces the terminology, the main types of neural networks and their architecture, and discusses some limitations of supervised learning with deep networks.
Feed-forward networks: mathematical model formulates fully-connected feed-forward networks mathematically. It introduces the matrix-vector notation, common activation functions, and the ingredients of the learning algorithm, and ends with exercises.
Neural networks: Backpropagation derives the back-propagation algorithm for computing the gradient of the cost function, and discusses vanishing and exploding gradients.
Demo: Feed-forward neural network for a function in PyTorch trains a network in PyTorch to approximate a simple one-dimensional function.
Demo: MNIST classifier in PyTorch builds a PyTorch classifier for handwritten digits (MNIST).