Colab with and kaggle api

Use colab with fastai and kaggle api to ace your workflow

Colab with and kaggle api

If you are planning to either setup your coding enviornment using all the online available resources or are casually wandering to understand how you could get the most from the publically freely available resources, this blog is just the right place for you to begin with.

Before jumping into coding your Kaggle contest submission, I would be extremly handy if you setup you working enviornment. If you are planning to use Colab or Kaggle Kernal to train your model, you could simply run, download and install anything from the notebook itself.

There are a number of frameworks available out there to begin with deep learning and getting the high level feel of what you are doing. Tensorflow and Keras usually tops the list followed by PyTorch and others. The Fastai framework built on top of PyTorch comes very handy with the implementation of a few famous techniques used by Kaggle chief Data Scientist Jeremy Howard, who is the creator of fastai itself.

When I was writing this blog, the Colab was not quite fastai friendly. Fastai was not installed in the Colab enviornment and it had to be installed manually. Now, if you keeping up with the Fastai course, you will need to install the older version 0.7.0 for proper working of the course notebooks in colab. And perhaps if you are willing to try what you have learnt from the course itself on some kaggle contest, here is a great bunch of snippet that I will be sharing with you to ace your work.

Installing fastai and pyTorch in Colab

!pip install -q "fastai==0.7.0" Pillow==4.1.1 torchtext==0.2.3

Now that you have installed the fastai version 0.7.0, you will be needing to setup some required libraries alongwith.

!apt-get -qq install -y libsm6 libxext6 && pip install -q -U opencv-python

These libraries are the primary libraries required to setup the pyTorch compatible with the fastai version 0.7.0.

from wheel.pep425tags import get_abbr_impl, get_impl_ver, get_abi_tag
platform = '{}{}-{}'.format(get_abbr_impl(), get_impl_ver(), get_abi_tag())
# !apt update -q
!apt install -y libsm6 libxext6

from os import path
accelerator = 'cu80' if path.exists('/opt/bin/nvidia-smi') else 'cpu'
torch_whl = f"{accelerator}/torch-0.3.0.post4-{platform}-linux_x86_64.whl"
!pip install -q {torch_whl} torchvision image

Voila! you have not setup your course compatible fastai framework. You are now good to go with the coursework. But before you go on, what if we find a way to download and manage data from Kaggle. Won’t it be great?

Installing Kaggle api and setting it up

Thanks to the bunch of awesome engineers working at Kaggle to provide us with the handy Kaggle API to save us from the trouble to downloading datasets from Kaggle and then uploading them to Colab. Kaggle offers us with the tool which anyone can access by merely downloading his/her Token for the API.

If you are wondering where to get this API Token from, I am here to help.

Head over to > Open you account settings > scroll-down and click on ‘Create New API Token’

This will download a kaggle.json file which is what we want. Simply place it to the location stated in the Kaggle-api docs or open it using some text editor and follow along this post.

Once you are done downloading the API-Token, you should now install the actual Kaggle-api.

apt install kaggle

Once installed, all you’ll need to do is to add the api key into the kaggle root folder and done!

!echo '{"username":"<USER-NAME>","key":"<USER-KEY>"}' > /root/.kaggle/kaggle.json

You are now good to go! You can simply use the kaggle api to make the api calls and download datasets and make submissions in the contest.

Enjoy coding!

Reference and links:

  1. Colab notebooks
  2. Kaggle api github
  4. courses