IPython

IPython#

ipython is modified python shell. It provide some tools that allows to intercat with objects in more convenient way than classical python shell. Check official docuemtation.

But the greatest feature of the ipython is that it provides kernel for jupyter ipykernel. ipykernel is deafult kernel for jupyter and most users consider them as a signlre product. In fact ipykernel just takes code cell from jupyter and process it with ipython.

Syntax#

IPython provides special expressions called “IPython magics” that extend the functionality of the classical Python shell. These magics are categorized into three types:

Line-oriented magics: Begin with % and apply to the next single expression. These can be embedded in Python code, allowing you to save results to a variable, for example.
Cell-oriented magics: Begin with %% and apply to the entire cell (must be used in the first row of the cell).
Special magics: Use special symbols (?, !, $) to instruct ipykernel to perform specific tasks.

Most magics can be used in both line-oriented and cell-oriented syntax, some even have special magics (such as symbol aliasing).

Check:

Corresponding section of the ipython tutorial.
Specific page in this site.

The following cells roughly considers use cases.

The next code uses the %timeit magic just around other Python code.

print("some code")
%timeit ans = [i for i in range(1_000_000)]
print("some code")

some code
35.1 ms ± 261 μs per loop (mean ± std. dev. of 7 runs, 10 loops each)
some code

timeit magic we used before have cell-oriented option to be used.

%%timeit
for i in range(1_000_000):
    pass

20 ms ± 65.1 μs per loop (mean ± std. dev. of 7 runs, 10 loops each)

And probably the most popular special magic - ? shows the description for the IPython module.

import IPython
IPython?

Type:        module
String form: <module 'IPython' from '/home/fedor/Documents/knowledge/venv/lib/python3.12/site-packages/IPython/__init__.py'>
File:        ~/Documents/knowledge/venv/lib/python3.12/site-packages/IPython/__init__.py
Docstring:  
IPython: tools for interactive and parallel computing in Python.

https://ipython.org

Reference#

There are several options to learn more about magics in IPython:

lsmagic: Lists all available magics.
magic: Shows available magics and their docstrings.
<magic-name>?: Displays the docstring of the specified magic.

The following cells show the output of magic. Note: The magic output is too long, so it is not used here.

ans = %lsmagic
print(ans)

Available line magics:
%alias  %alias_magic  %autoawait  %autocall  %automagic  %autosave  %bookmark  %cat  %cd  %clear  %code_wrap  %colors  %conda  %config  %connect_info  %cp  %debug  %dhist  %dirs  %doctest_mode  %ed  %edit  %env  %gui  %hist  %history  %killbgscripts  %ldir  %less  %lf  %lk  %ll  %load  %load_ext  %loadpy  %logoff  %logon  %logstart  %logstate  %logstop  %ls  %lsmagic  %lx  %macro  %magic  %mamba  %man  %matplotlib  %micromamba  %mkdir  %more  %mv  %notebook  %page  %pastebin  %pdb  %pdef  %pdoc  %pfile  %pinfo  %pinfo2  %pip  %popd  %pprint  %precision  %prun  %psearch  %psource  %pushd  %pwd  %pycat  %pylab  %qtconsole  %quickref  %recall  %rehashx  %reload_ext  %rep  %rerun  %reset  %reset_selective  %rm  %rmdir  %run  %save  %sc  %set_env  %store  %sx  %system  %tb  %time  %timeit  %unalias  %unload_ext  %uv  %who  %who_ls  %whos  %xdel  %xmode

Available cell magics:
%%!  %%HTML  %%SVG  %%bash  %%capture  %%code_wrap  %%debug  %%file  %%html  %%javascript  %%js  %%latex  %%markdown  %%perl  %%prun  %%pypy  %%python  %%python2  %%python3  %%ruby  %%script  %%sh  %%svg  %%sx  %%system  %%time  %%timeit  %%writefile

Automagic is ON, % prefix IS NOT needed for line magics.

Here is an example of description for the autosave magic.

%autosave?

Docstring:
Set the autosave interval in the notebook (in seconds).

The default value is 120, or two minutes.
``%autosave 0`` will disable autosave.

This magic only has an effect when called from the notebook interface.
It has no effect when called in a startup file.
File:      ~/Documents/knowledge/venv/lib/python3.12/site-packages/ipykernel/zmqshell.py

Measure performance#

There are two magic functions to estimate the performance of the code:

%time: measures the delay of the given code.
%timeit: run given code several times to get accurate estimation of the performance

The following cell defines the function that runs a given number of iterations.

def some_computing(i_count: int):
    for i in range(i_count):
        ans = 5 + 5

The following code runs 10 and 500 iterations under the time magic function.

print("=====10 iterations=====")
%time some_computing(10)
print("=====500 iterations=====")
%time some_computing(500)

Metrics meaning

CPU times gives description of the duration by stage of execution:
- user user-side duration;
- sys system-side duration;
- total total duration of the calculation.
Wall time duration of the calculation with overhead resources (whatever they may be).

Note Nowadays, I don’t know exactly what the difference is between the user side and the system side - but you should learn Linux to find out.

The following cell runs 10 and 500 iterations under timeit magic.

print("=====10 iterations=====")
%timeit some_computing(10)
print("=====500 iterations=====")
%timeit some_computing(500)

=====10 iterations=====
215 ns ± 1.15 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops each)
=====500 iterations=====
7 μs ± 144 ns per loop (mean ± std. dev. of 7 runs, 100,000 loops each)

System command (`!`)#

You can access the environment you ran IPython from by simply adding ! before the line. For linux/macOS this is usually terminal line for windows it is usually powerShall.

The following cell executes curl google.com in the inpython shell.

!curl google.com

<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
<A HREF="http://www.google.com/">here</A>.
</BODY></HTML>

Note: Just like regular magic, you can embed !<command> to the regular python expressions. The following cell stores the result of curl google.com in the variable and displays it as HTML with regular Python code.

from IPython.display import display, HTML
res = !curl -s google.com
HTML("".join(res))

301 Moved

The document has movedhere.

Alternative languages#

There is a set of magic commands that allow you to run alternative-specific code in IPython. The following table lists them:

Magic	Purpose
`%%bash`	Run cell content as a Bash shell script.
`%%sh`	Alias for `%%bash`.
`%%script`	Run code in a specified interpreter (e.g., `%%script python3`, `%%script perl`).
`%%html`	Render cell content as raw HTML in the output.
`%%latex`	Render LaTeX equations or content.
`%%javascript`	Run JavaScript in the browser context.
`%%js`	Alias for `%%javascript`.
`%%perl`	Run code in Perl.
`%%ruby`	Run code in Ruby.
`%%python`	Run cell content with the current Python interpreter (explicit).
`%%python2`	Run code using Python 2 (if available).
`%%python3`	Run code using Python 3.
`%%ruby`	Run code in Ruby.
`%%perl`	Run code in Perl.
`%%bash`	Run code in Bash (redundant but often used).
`%%sh`	Run code in system shell.
`%%R`	Run R code (if `rpy2` is installed).
`%%cython`	Run Cython code (if `Cython` is installed).
`%%svg`	Render raw SVG output.

Note. Looks like this magic is only available in whole cell mode.

The following cell runs some bash code:

%%bash
ls -l

total 16
-rw-rw-r-- 1 fedor fedor 4127 жні 26 18:40 magic_commands.ipynb
drwxrwxr-x 3 fedor fedor 4096 жні 25 15:10 nbconvert
drwxrwxr-x 4 fedor fedor 4096 жні 25 15:10 voila_vs_nbconvert_saving_html

The next cell contains simple HTML code:

%%HTML
<button>Button as HTML element</button>

Extensions#

IPython has some extensions, you can load them with the magic %load_ext <extension>. Find out more on extensions in ipython from corresponding page of the offical documentation. The following cell shows the most typical ipython extensions.

The following cell attempts to run the magic %autoreload, which is typically not included in IPython by default.

%autoreload

UsageError: Line magic function `%autoreload` not found.

IPython says that there is no such function. You need to load it. The following cell shows the code for loading:

%load_ext autoreload

After loading everything goes fine.

%autoreload

Reloading modules#

With the autoreload extension, you can make the imported object to be reloaded according to some rules so that your environment would be up to date. Check documetation of the autoreload module. Check more details in the corresponding page.

The following cell defines the set_module function, which creates a module with parameterized behavior.

from pathlib import Path

template = '''
def echo_message():
    print("{message}")
'''

module_path = Path("ipython_files")/"my_module.py"

def set_module(message):
    module_path.write_text(template.format(message=message))

set_module("initial")

The following code imports the module using special ipython features to make it reload each time the code is run.

%load_ext autoreload
%autoreload explicit
from ipython_files import my_module 
%aimport ipython_files.my_module

The autoreload extension is already loaded. To reload it, use:
  %reload_ext autoreload

Now my_module.echo_message has the behavior we’ve specified at the beginning:

my_module.echo_message()

initial

And after running code that changes the source of my_module.echo_message, calling the function results in the new behavior.

set_module("changed")

my_module.echo_message()

changed

Cell to file#

The %%writefile <path> magic saves content of the cell to <path>.

All code cells in this section are the same example and should be run one by one.

A new folder has been created here. The ls command confirms that it’s empty.

!mkdir /tmp/writefile_files

Using %%writefile to a non-existent file.

%%writefile /tmp/writefile_files/writefile
this is the file from the cell

Writing /tmp/writefile_files/writefile

Verify that the file has been created and contains the content specified in the previous cell.

%%bash 
echo "=====files====="
ls /tmp/writefile_files
echo "=====content====="
cat /tmp/writefile_files/writefile

=====files=====
writefile
=====content=====
this is the file from the cell

Now let’s try using the same file, but for a different cell. The message Overwriting ... indicates that the content will be completely replaced.

%%writefile /tmp/writefile_files/writefile
some other message for another cell

Overwriting /tmp/writefile_files/writefile

Make sure that the contents of the file are from the last cell.

!cat /tmp/writefile_files/writefile

some other message for another cell

Matplotlib#

IPython is integrated with the plotting library matplotlib. There is a special magic command %matplotlib that allows configuring the behavior of matplotlib.

The following example sets the tk engine for matplotlib, resulting in plots being displayed in a dedicated Tkinter window.

%matplotlib tk
import matplotlib.pyplot as plt
plt.plot([1, 2, 3], [1, 2, 1])

[<matplotlib.lines.Line2D at 0x7a7dec8fe750>]