Although I have spent quite a lot of time recently with CNNs for image classification and semantic segmentation, I have realized that to obtain a deep understanding of them, I have to make one on my own from scratch. So, I put down PyTorch, my go-to framework, and created an implementation using NumPy only 🙂

​

The result can be found here: https://github.com/cosmic-cortex/neural-networks-from-scratch

​

Basically, it is a mini deep learning framework, so one can easily experiment with different architectures. Currently, the following components are supported.

Layers:

• Linear
• Conv2D
• BatchNorm2D
• MaxPool2D
• Flatten (technically, this is not a layer, since it just flattens a 2D input, but it was very convenient to implement this as one)

Loss functions:

• CrossEntropyLoss
• MeanSquareLoss

Activation functions:

• ReLU
• Leaky ReLU
• Sigmoid

There are two examples as well, a simple multilayer perceptron and a basic CNN on MNIST classification, but custom datasets are supported as well, if you would like to experiment on your own data.

​

I have to say, I have really enjoyed this ride! It was extremely instructional, moreover I have discovered several mindblowing details, for instance that the gradient for convolution is a transpose convolution operator 🙂 Truly recommended for everyone in DL/ML to try doing the same. During this venture, the fantastic CS231n course was very helpful, so this is also recommended.