Transfer Learning (TL) is one of the most powerful methods for building high-performance deep learning models in computer vision. TL is based on the knowledge-reusability concept – one can use knowledge from one area and apply it to another. By leveraging previous experience, you don’t need to start from scratch with every new model or new situation. So effectively, you can learn how to do new tasks more efficiently by drawing on previous knowledge.
New to deep learning? Check out our Introduction to Convolutional Neural Networks.
Let’s say you are a data scientist proficient in Python, and now you need to perform a new analysis using R. R might be a new programming language for you. Still, since you already know Python, learning R will be much easier for you compared to a version of yourself that doesn’t know how to program at all. R is a new language, but many of the same principles and fundamentals apply to both Python and R, so you can transfer some of your existing Python knowledge to get a headstart in your pursuit of learning R.
The same principle is used in deep learning with Transfer Learning. Instead of starting from scratch (model with random weights), you can take an existing network that has been trained to do a thing X and customize the network to your particular problem or task. If you read our article on Convolutional Neural Networks (CNNs), then you know that the deeper we go into a network, the more sophisticated are the features that get extracted.
Now imagine that you want to detect dogs and cats in a collection of images. Here’s a 4 step solution for this type of task:
The advantages of Transfer Learning are faster training and better results with significantly less data.
To start, let’s download the Dogs and Cats dataset (link below) from the web and untar the file. We’re doing this task in Google Colab on a Tesla T4 GPU, so your download and training times may vary. Here are the library imports and device configuration:
Now we can download the dataset and unpack it:
PyTorch allow us to enlarge the training data’s size without acquiring more images by performing operations such as rotations and horizontal flips. Further, we’ll prepare every image in the same way (resizing is optional):
This is performed both for training and testing images, as we essentially want them in the same format. Here’s the code snippet:
We can now apply the transformation to the images and load them in batches with a
PyTorch.DataLoader module. Feel free to experiment with the actual batch size, but we’ve set it to 32 for this case.
We can now use the declared
train_loader to see if transformations were applied. With the help of the
matplotlib library, we can visualize the entire batch (32 images):
Judging by the rotation in the images, we can say that everything works as expected up to this point. The next step is to define the neural network class. We decided to go simple, with three convolutional layers, a fully connected layer, and the output layer. Max pooling operation is performed after every convolutional layer, alongside with the ReLU activation. Here’s the model class:
As you can see, the model was moved to the GPU. The model training took around an hour and ten minutes to complete for 10 epochs, and resulted in an 84% accuracy:
Not a bad result for a simple network like this, but can we do better? Transfer learning says yes.
The transfer learning approach will be much more straightforward than the custom one. Here are the steps:
We can do that in a couple of lines of code:
And that’s it! We can start the training process now. It took only 15 minutes for a single epoch and yielded far greater accuracy than our custom architecture:
Now you see how powerful Transfer Learning can be. The existence of Transfer Learning means that custom architectures are obsolete in many cases.
This article’s take-home point is that stressing out about layers in custom neural network architectures is a waste of time in most cases. Pretrained networks are far more powerful than anything you can come up with on your own in any reasonable amount of time.
The transfer learning approach requires fewer data and fewer epochs (less training time), so it’s a win-win situation. To be more precise, transfer learning requires more training time per epochs but requires fewer epochs to train a usable model.
If your company needs help with Transfer Learning or you need help with a custom Machine Learning model, reach out to Appsilon. We are experts in Machine Learning and Computer Vision.