Convolutional Nets Part 1

Why ConvNets?

ConvNets are used for Image Analysis. ConvNets reduces a given image into a form which is easier to work on without losing its important features, Which helps in prediction.

ConvNets are vastly prefered due to their 1) Parameter Sharing property & 2) Sparsity of Connections. 1)Parameter Sharing: A feature detector (edges, shapes) filter could be used for multiple parts of the same image. 2) Sparsity of Connections: In each layer, each output value depends only on a small number of inputs.

What are ConvNets?

ConvNets, also known as Convolutional Neural Networks are Neural Nets with Convolutional Layers between them. ConvNets also known as Feed Forward networks. The convolutional layers are made of filters. These filters detect various patterns in an image.

The simple filters detect simple features such as Horizontal edges, vertical edges, circles and so on. The filters in the deeper layers of CNN can detect more complex patterns. These filters are initialized with random numbers. They can be of size 3 X 3, 4 X 4, and so on. These Filters are also known as Kernels.

Basic ConvNet

In the figure above, the neural network is fed with some image. For our simplification lets imagine an input of cat. Its fed to the CNN layer (i.e a filter in that layer with the ability to detect edges), the output image you will get will be the one where the edges of the cat are shown.

In such a way, there are multiple layers(i.e filters) which detect more and more complex feature as you go deeper and deeper in the model. Thus in the final output, our model has figured out that it was a cat image.

Is Convolution the only layer in the CNN model?

The answer is No, convolution layer is followed by pooling layers and then by fully connected layers and a softmax layer. This whole circuit is then called as CNN.
the following figure will demonstrate the CNN model:

CNN full model

Layers in CNN

There is a total of three-layers in CNN: 

1. Convolution layer

2. Pooling layer

3. Fully Connected Layer 

Let's dive into what each layer exactly does.

1. Convolution Layer: the convolution layer performs the convolution operation on the given input image using a kernel. The convolution operation is done by taking the element-wise product and summing them. I have illustrated it in the figure below. The convolution operation is similar to the mathematical operation of cross-correlation except in the convolution we do not flip the filter.

Input Image of 6 X 6 convolved with a filter of 3 X 3

We get a 4 X 4 Output  

This output is fed to the Pooling layer.

2. Pooling Layer: Pooling layer consists of a) Max Pooling and b) Average Pooling.
let me illustrate the concepts with the help of this figure:

Illustration of Max Pooling(on right) & Average Pooling(on left)

If important features are detected in the input 4X4 then they are preserved in the output of max pooling. Max pooling operator really does is, if these features are detected anywhere in the filter, it then keeps a high number in the output.
Note: Max pooling has hyperparameters but, it has no parameters to learn.

Similarly, Average pooling takes an average of the numbers and carry forward it into the next convolution layer.

Let's talk about Stride:  In the figure "Input Image of 6 X 6 convolved with a filter of 3 X 3" I have used 1-Stride. What does that mean?

So, a Stride is the number of pixels we want our filter to move on the input image. Stride can be 1, 2 or more.

Padding:  We have one more hyperparameter called as padding. We need padding to avoid deletion of information from the edge, It helps with the problem of shrinking output.
With the help of padding, we can use convolutional layer without shrinking the height and width of the image volume.
Padding is of two types: 1) Valid & 2) Same
1) Valid Padding basically means that there is no padding.
2) Same Padding means that the size of the output image is the same as the size of the input image.

3. Fully Connected Layer: Before connecting our input layer to Softmax layer, we flatten our input image into a single column vector and feed it to the Feed-Forward network, that is an FC layer. Using Softmax in the last layer the model can now easily classify the input image.

Keras Implementation of CNN: Here, I have implemented all the layers we discussed above. The Dense layers at the top and the bottoms are the input and the output layers. Then there are two Convolutional layers with filter/kernel of size 3X3 and other of size 5X5. There is MaxPooling with strides of 2 and padding='same'. At the end I have used flatten(), to flatten the output to a single column vector, and fed it to the Softmax activation.