IPython

IPython#

ipython is modified python shell. It provide some tools that allows to intercat with objects in more convenient way than classical python shell. Check official docuemtation.

But the greatest feature of the ipython is that it provides kernel for jupyter ipykernel. ipykernel is deafult kernel for jupyter and most users consider them as a signlre product. In fact ipykernel just takes code cell from jupyter and process it with ipython.

Syntax#

IPython provides special expressions called “IPython magics” that extend the functionality of the classical Python shell. These magics are categorized into three types:

Line-oriented magics: Begin with % and apply to the next single expression. These can be embedded in Python code, allowing you to save results to a variable, for example.
Cell-oriented magics: Begin with %% and apply to the entire cell (must be used in the first row of the cell).
Special magics: Use special symbols (?, !, $) to instruct ipykernel to perform specific tasks.

Most magics can be used in both line-oriented and cell-oriented syntax, some even have special magics (such as symbol aliasing).

Check:

Corresponding section of the ipython tutorial.
Specific page in this site.

The following cells roughly considers use cases.

The next code uses the %timeit magic just around other Python code.

print("some code")
%timeit ans = [i for i in range(1_000_000)]
print("some code")

some code
35.1 ms ± 261 μs per loop (mean ± std. dev. of 7 runs, 10 loops each)
some code

timeit magic we used before have cell-oriented option to be used.

%%timeit
for i in range(1_000_000):
    pass

20 ms ± 65.1 μs per loop (mean ± std. dev. of 7 runs, 10 loops each)

And probably the most popular special magic - ? shows the description for the IPython module.

import IPython
IPython?

Type:        module
String form: <module 'IPython' from '/home/fedor/Documents/knowledge/venv/lib/python3.12/site-packages/IPython/__init__.py'>
File:        ~/Documents/knowledge/venv/lib/python3.12/site-packages/IPython/__init__.py
Docstring:  
IPython: tools for interactive and parallel computing in Python.

https://ipython.org

Reference#

There are several options to learn more about magics in IPython:

lsmagic: Lists all available magics.
magic: Shows available magics and their docstrings.
<magic-name>?: Displays the docstring of the specified magic.

The following cells show the output of magic. Note: The magic output is too long, so it is not used here.

ans = %lsmagic
print(ans)

Available line magics:
%alias  %alias_magic  %autoawait  %autocall  %automagic  %autosave  %bookmark  %cat  %cd  %clear  %code_wrap  %colors  %conda  %config  %connect_info  %cp  %debug  %dhist  %dirs  %doctest_mode  %ed  %edit  %env  %gui  %hist  %history  %killbgscripts  %ldir  %less  %lf  %lk  %ll  %load  %load_ext  %loadpy  %logoff  %logon  %logstart  %logstate  %logstop  %ls  %lsmagic  %lx  %macro  %magic  %mamba  %man  %matplotlib  %micromamba  %mkdir  %more  %mv  %notebook  %page  %pastebin  %pdb  %pdef  %pdoc  %pfile  %pinfo  %pinfo2  %pip  %popd  %pprint  %precision  %prun  %psearch  %psource  %pushd  %pwd  %pycat  %pylab  %qtconsole  %quickref  %recall  %rehashx  %reload_ext  %rep  %rerun  %reset  %reset_selective  %rm  %rmdir  %run  %save  %sc  %set_env  %store  %sx  %system  %tb  %time  %timeit  %unalias  %unload_ext  %uv  %who  %who_ls  %whos  %xdel  %xmode

Available cell magics:
%%!  %%HTML  %%SVG  %%bash  %%capture  %%code_wrap  %%debug  %%file  %%html  %%javascript  %%js  %%latex  %%markdown  %%perl  %%prun  %%pypy  %%python  %%python2  %%python3  %%ruby  %%script  %%sh  %%svg  %%sx  %%system  %%time  %%timeit  %%writefile

Automagic is ON, % prefix IS NOT needed for line magics.

Here is an example of description for the autosave magic.

%autosave?

Docstring:
Set the autosave interval in the notebook (in seconds).

The default value is 120, or two minutes.
``%autosave 0`` will disable autosave.

This magic only has an effect when called from the notebook interface.
It has no effect when called in a startup file.
File:      ~/Documents/knowledge/venv/lib/python3.12/site-packages/ipykernel/zmqshell.py

`%time`#

Measures the time of execution of some expression.

Basic example#

Here I just have a function that runs a cycle with a given number of iterations, and show results for 10 and 500 iterations.

def some_computing(i_count):
    for i in range(i_count): 5+5

print("=====10 iterations=====")
%time some_computing(10)
print("=====500 iterations=====")
%time some_computing(500)

=====10 iterations=====
CPU times: user 2 µs, sys: 0 ns, total: 2 µs
Wall time: 3.58 µs
=====500 iterations=====
CPU times: user 14 µs, sys: 2 µs, total: 16 µs
Wall time: 16.7 µs

Metrics meaning#

CPU times gives description of the duration by stage of execution:
- user user-side duration;
- sys system-side duration;
- total total duration of the calculation.
Wall time duration of the calculation with overhead resources (whatever they may be).

Note Nowadays, I don’t know exactly what the difference is between the user side and the system side - but you should learn Linux to find out.

`%timeit`#

Estimates the execution time of a certain command over a series of runs.

In the following example I have a function that runs a cycle with a given number of iterations, and show results for 10 and 500 iterations.

def some_computing(i_count):
    for i in range(i_count): 5+5

print("=====10 iterations=====")
%timeit some_computing(10)
print("=====500 iterations=====")
%timeit some_computing(500)

=====10 iterations=====
203 ns ± 6.87 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops each)
=====500 iterations=====
5.88 µs ± 115 ns per loop (mean ± std. dev. of 7 runs, 100,000 loops each)

System command (`!`)#

You can access the environment you ran IPython from by simply adding ! before the line. For linux/macOS this is usually terminal line for windows it is usually powerShall.

The following cell executes curl google.com in the inpython shell.

!curl google.com

<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
<A HREF="http://www.google.com/">here</A>.
</BODY></HTML>

Note: Just like regular magic, you can embed !<command> to the regular python expressions. The following cell stores the result of curl google.com in the variable and displays it as HTML with regular Python code.

from IPython.display import display, HTML
res = !curl -s google.com
HTML("".join(res))

301 Moved

The document has movedhere.

Specific code#

You can use some other codes in IPython just specifying them with magic <code type>. Next, I list some of the options available.

Note looks like this magic is only available in whole cell mode.

bash#

%%bash
ls -l

total 16
-rw-rw-r-- 1 fedor fedor 4127 жні 26 18:40 magic_commands.ipynb
drwxrwxr-x 3 fedor fedor 4096 жні 25 15:10 nbconvert
drwxrwxr-x 4 fedor fedor 4096 жні 25 15:10 voila_vs_nbconvert_saving_html

HTML#

%%HTML
<button>Button as HTML element</button>

Extensions#

IPython has some extensions, you can load them with the magic %load_ext <extension>. Find out more on extensions in ipython from corresponding page of the offical documentation.

Reloading modules#

With the autoreload extension, you can make the imported object to be reloaded according to some rules so that your environment would be up to date. Check documetation of the autoreload module. Check more details in the corresponding page.

The following cell defines the set_module function, which creates a module with parameterized behavior.

from pathlib import Path

template = '''
def echo_message():
    print("{message}")
'''

module_path = Path("ipython_files")/"my_module.py"

def set_module(message):
    module_path.write_text(template.format(message=message))

set_module("initial")

The following code imports the module using special ipython features to make it reload each time the code is run.

%load_ext autoreload
%autoreload explicit
from ipython_files import my_module 
%aimport ipython_files.my_module

The autoreload extension is already loaded. To reload it, use:
  %reload_ext autoreload

Now my_module.echo_message has the behavior we’ve specified at the beginning:

my_module.echo_message()

initial

And after running code that changes the source of my_module.echo_message, calling the function results in the new behavior.

set_module("changed")

my_module.echo_message()

changed

`%%writefile`#

This is the magic that allows you to save the contents of the cell to a file. You need to use the syntax %%writefile <directory> in the first line of the cell to save it in the <directory>.

All code cells in this section are the same example and should be run one by one.

A new folder has been created here. The ls command confirms that it’s empty.

%%bash
mkdir ipython_magics_files/writefile_files
ls ipython_magics_files/writefile_files

Using %%writefile to a non-existent file.

%%writefile ipython_magics_files/writefile_files/writefile
this is the file from the cell

Writing ipython_magics_files/writefile_files/writefile

Verify that the file has been created and contains the content specified in the previous cell.

%%bash 
echo "=====files====="
ls ipython_magics_files/writefile_files
echo "=====content====="
cat ipython_magics_files/writefile_files/writefile

=====files=====
writefile
=====content=====
this is the file from the cell

Now let’s try using the same file, but for a different cell. The message Overwriting ... indicates that the content will be completely replaced.

%%writefile ipython_magics_files/writefile_files/writefile
some other message for another cell

Overwriting ipython_magics_files/writefile_files/writefile

Make sure that the contents of the file are from the last cell.

%%bash
cat ipython_magics_files/writefile_files/writefile

some other message for another cell

Finally delete the folder so that the content remains fully playable.

%%bash
rm -r ipython_magics_files/writefile_files/

Matplotlib#

IPython is integrated with the plotting library matplotlib. There is a special magic command %matplotlib that allows configuring the behavior of matplotlib.

The following example sets the tk engine for matplotlib, resulting in plots being displayed in a dedicated Tkinter window.

%matplotlib tk
import matplotlib.pyplot as plt
plt.plot([1, 2, 3], [1, 2, 1])

[<matplotlib.lines.Line2D at 0x7a7dec8fe750>]