Jupyter Magic

Getting the most out of your jupyter notebooks by using Jupyter magic

Jupyter Magic

Jupyter MAGIC cheatsheet

Jupyter notebooks are greatly used in the Python and Data Science community and by all the people who love Jupyter Notebooks. I love notebooks so much that even this post has been created using one jupyter notebook.

You might have come across commands such as %matplotlib inline %time %autoreload etc. These are called Jupyter MAGIC. Jupyter magic are special commands that are built-in jupyter notebooks to extend its capablities. I am here to introduce you to the jupyter MAGIC commands and demonstrate how it can ease your working and further extend the notebooks capablities.

I’ll try to keep this post short and mostly focus on the coding part by implementing all the magic commands that will help you get going.

Let’s begin!

Time it Module

(%time, %%time)

Lots and lots of time, we are required to time our codes segements to check onto their efficiency and see the amount of time they require. We know that a piece of code that has lower time complexity is always efficient and can help us crunch large numbers in a fraction of time.

To time the modules, we can build our own functions for timing modules as follows

import time
import random
import numpy as np

def create_one_million_numbers():
    a=[]
    for i in range(int(1e6)):
        a.append(random.randint(0,1e6))
    return a
start=time.time()
create_one_million_numbers()
stop=time.time()
print("Wall time {0:.4} s".format(stop-start))
Wall time 1.543 s

You can see how we have to see extra 3 lines everytime we call the target function. This certainly might not seem like a problem to most of us when we have to do just for a few couple of time. But what if you have to call this every now and then?

For solving this problem, we a blessed with a jupyter magic %time

%time a=create_one_million_numbers()
%time a=[i for i in range(1000000)]
%time a=np.random.randint(1000000)
CPU times: user 1.48 s, sys: 32 ms, total: 1.51 s
Wall time: 1.51 s
CPU times: user 35.1 ms, sys: 28 ms, total: 63.2 ms
Wall time: 72.3 ms
CPU times: user 9.81 ms, sys: 3.97 ms, total: 13.8 ms
Wall time: 13.8 ms
%%time
a=create_one_million_numbers()
a=[i for i in range(1000000)]
a=np.random.randint(1000000)
CPU times: user 1.57 s, sys: 47.9 ms, total: 1.61 s
Wall time: 1.61 s

There could be times when we are required to view the enviornment variables that we have created so far during the execution of our notebook. To view these, we can use %who_ls

%who_ls
['a', 'create_one_million_numbers', 'np', 'random', 'start', 'stop', 'time']

We can see that all the functions, imports and variables we have created are visible now.

System date

%system date

Whenever we want to get the current time of the system, we can use the %system date module to get the current date and time

%system date
['Sat Jan 19 02:53:54 UTC 2019']
a = %system date
print(a)
['Sat Jan 19 02:53:54 UTC 2019']

Shortcut !!date also works equally

!!date
['Sat Jan 19 02:53:54 UTC 2019']

Matplotlib magic

%matplotlib inline

This is by far the most popular magic command and might be the first command that you might have encountered. The matplotlib generates the plot canvas frame in a new window which is not rendered into the jupyter notebook. To overcome this problem, %matplotlib inline command is used. This command forces the canvas to be formed into the notebook itself.

from matplotlib import pyplot as plt
%matplotlib inline
plt.plot(np.log(np.arange(1000)))

plot

Auto reload

%load_ext autoreload

%autoreload

When you are working with external tools or changing the enviornment variables, this will certainly help you a lot. The external commands help you autoreload the tools and libraries at a specified defined interval. So whenever there is even a minor change, we do not have to run the imports to update the local enviornment of the notebook

%load_ext autoreload
%autoreload 2

Languages inside notebooks

We are at a number of time required to use some other languages from inside the notebook. May it be bash, javascript, html, latex, markdown, python2, python3, ruby or a ton of other languages, jupyter gives you options to run them too.

The %(single percent) can be used to implement the single line command or %%(double percent) to dedicate the current cell to those specific languages.

%%bash 

echo "This is single line command" > some_temp_file.txt
cat some_temp_file.txt
This is single line command

Specifically for bash commands, the jupyter notebook gives a special shortcut !. This can be used to run single line bash command in the cell.

!ls /
bin   home        lib64   opt   sbin  tmp      vmlinuz.old
boot  initrd.img      lost+found  proc  snap  usr
dev   initrd.img.old  media   root  srv   var
etc   lib         mnt     run   sys   vmlinuz
%%html

<p style='background-color:#bbdefb; padding:20px;'> Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodoconsequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. </p>

Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.