Lexicon#

Non-linear function applied to the result of a neuron's calculation. See these examples

Back-propagation is the algorithm used to updated the weights of a network according to its output, usually using a loss function.

Number of images, named patches, that the network sees at each training step.

The neural network, consisting of a set of layers, each with various number of neurons, and the connections between these nodes. The neural network is capable of performing calculations by inputing an image (pixel values) in its input layer, propagating the values through its layers and outputting a prediction. Each neuron of a layer performs a simple calculation, that is then aggregated for the whole layer and passed to the next layer.

Graphical Processing Unit, which allows performing a large number of small computations in parallel. GPUs allow accelerating the training and prediction of neural networks.

"Reality" of what we want the network to output. For instance, if we want to perform denoising, the ground-truth are clean images where noise is minimal, as opposed to the inputs which should be noisy images.

A set of neurons working in parallel.

Parameter regulating the magnitude of the weights update.

Function aimed at judging "how good" the network performs by comparing the prediction output to a "ground-truth" value. Throughout training, we visualise the loss function on both the training and validation sets in order to assess how well the training is going.

A unit in the neural network. A neuron performs a simple calculation on inputs, usually by multiplying inputs by weights, adding an offset (bias) and applying a non-linear function (activation function) to the sum.

Example: With input \(x_1\) and \(x_2\), a neuron would typically output:

\(out = f(x_1 * w_1 + x_2 * w_2 + b)\)

where \(f\) is an activation function.

In order to "fit" in the GPU memory, which is often the limiting factor, images need to be all cropped. What the network is trained on is a number of patches, i.e. image crops. These patches are all the same size and are aggregated in batches. A single batch is presented to the network during a single step of each epoch.

Output of the network.

A set of images that the network never saw during training. It is important for these images never to be added to the training or the validation set. The test set is used to assess the performances of the network on unseen data, that is to say how well it generalizes to similar images.

Training a network is the act of changing the weights of the neural network until its performances do not increase anymore. During training, at each step, a subset of the training images is presented to the network, the loss is computed between the predictions and the "ground-truth". Then, gradients are built from the value of the loss, and the weights of the network are updated accordingly. At the end of each epoch, the loss is computed on the validation set for reference.

An ensemble of training steps. At the end of each epoch, the validation loss is computed.

Ensemble of images the network is trained on.

An iteration of learning.

A specific model architecture popular in bioimage analysis.

Ensemble of images used to assess the performances of the network during training. The validation set is not used to train the network, only to assess that the training is going well.

The parameters of a Model that define all the neural network node calculations.