5 - CNNs

1. Convolutional Layer: Applies filters to the input image to extract features like edges or textures. Each filter slides over the image, producing a feature map.

2. Activation Function (ReLU): Introduces non-linearity by setting negative values to zero, allowing the network to learn complex patterns.

3. Pooling Layer: Reduces the spatial size of the feature maps, often using max pooling, which selects the highest value in a region, making the network more computationally efficient.

4. Normalization (Batch Normalization): Normalizes the output of the previous layer to stabilize and speed up training, ensuring that values are centered and scaled properly.

5. Fully Connected Layer: Combines features from the convolutional layers for the final classification task.

e functional is expensive to compute

Size of the filter. Hyperparameter

One filter per channel

Outputs are added,

than bias is added

Number of filters,

Padding

Stride

Very common: Set P = (K – 1) / 2 to make output have same size as input! (called same padding)

Important to consider while designing an cnn to cover the image

—> Stride can help to get a bigger receptive field

Number of fields in a filter:

outputs * Filter size

Filter with gaps

expansion of the receptive field

In batch normalization, we normalize the output of a layer by adjusting and scaling the activations. It calculates the mean and variance of a mini-batch, then shifts and scales the values to ensure they are centered (mean = 0) and have a standard deviation of 1.

This stabilizes and speeds up training by reducing internal covariate shift, allowing higher learning rates and improving overall performance.

• Makes deep networks much easier to train

• Allows higher learning rates, faster convergence

• Networks become more robust to initialization

• Acts as regularization during training

• Zero overhead at test-time: can be fused with convolutional layer