Imports

Imports#

This topic may seem unimportant, but some configurations can be inconvenient. It is therefore important to know the tools that Python has for importing modules.

Sources

Python modules and packages intorduction page in realpython;
Python import: advanced Techniques and Tips page in realpython.
The importing system page of the official python documentation.

import sys
import random

sys.path.append('/tmp')

Loaded modules#

The sys.path is a dictionary that contains paths to directories where Python looks for modules during program execution.

The following cell shows the number of packages available in the current environment.

len(sys.modules)

There are so many packages because Jupyter, used as the environment for these experiments, has numerous dependencies. The following cell shows the same count but for a clean Python environment without imports from Jupyter’s requirements.

%%bash
python3 -c "import sys; print(len(sys.modules))"

Here is small subset of the modules loaded by jupyter.

random.sample(list(sys.modules.keys()), 10)

['IPython.core.builtin_trap',
 'email',
 'IPython.core.ultratb',
 'types',
 'threading',
 'asyncio.futures',
 'psutil._psutil_linux',
 'IPython.testing.skipdoctest',
 'zmq.sugar.frame',
 '_json']

Each object from that list has the module type, as demonstrated by the following code.

type(sys.modules["sys"])

module

Module object#

In python, each module is an object that was created from a some source of the python code. As it a special type of object, it can have attiributes that can be useful in some specific cases.

The table below lists the available attributes awailable in the module and their descriptions.

Attribute	Description
`__name__`	The name of the module.
`__doc__`	The docstring of the module (can be `None`).
`__file__`	The pathname of the file from which the module was loaded.
`__package__`	The name of the package the module belongs to.
`__path__`	A list of paths where the package submodules can be found (only for packages).
`__loader__`	The loader used to load the module.
`__spec__`	The module spec that contains import-related information.
`__cached__`	The filename of the compiled bytecode file (e.g., `.pyc`), if available.
`__builtins__`	A reference to the `builtins` module (available in all modules).

For more, check:

Modules section in python data model.
Module object page in this website.

The following cell displays the type behind the importlib name after importing it.

import importlib
type(importlib)

module

The code below shows some of the special attributes of the importlib.machinery submodule.

print(
    importlib.machinery.__name__,
    importlib.machinery.__package__,
    importlib.machinery.__file__,
    sep="\n"
)

importlib.machinery
importlib
/usr/local/lib/python3.12/importlib/machinery.py

`sys.path`#

The sys.path list defines the directories where Python searches for modules when you try to import them. For more details check specific page.

The following cell displays the sys.path for the current run.

sys.path

['/usr/lib/python312.zip',
 '/usr/lib/python3.12',
 '/usr/lib/python3.12/lib-dynload',
 '',
 '/home/f-kobak-distance-desctop/Documents/knowledge/venv/lib/python3.12/site-packages']

Import function#

There is a special build-in function __import__ that is actually hiden under the import keyword in the conventional python. See the official description of the __import__ buildin function.

The following cell shows the usage of the cowsay when imported through the __import__ buildin.

cowsay = __import__("cowsay")
print(cowsay.cowsay("__import__"))

 ____________ 
< __import__ >
 ------------ 
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

Importlib#

importlib is a special package that automates some import-related procedures, making it easier to implement import-related logic.

For more information check:

The following cell provides an example of the using importlib.import_module function to load the cowsay module.

import importlib
cowsay = importlib.import_module("cowsay")
print(cowsay.cowsay("test"))

 ______ 
< test >
 ------ 
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

Find module#

Roughly speaking, when Python attempts to import a module, it follows these steps:

It first checks sys.modules to see if the module has already been loaded. Note: Some modules may import other modules during their initialization, so the module you’re working with might already be cached if it was imported indirectly earlier.
If the module is not found in sys.modules, Python begins invoking the find_spec method of various finder objects. Finders are responsible for determining how to handle the given module name. If a finder can locate the module, it returns a ModuleSpec object, which is then used to load the module into the environment.

The process of searching for a module in Python is described in this subsection of the official documentation.

For some practical aspects of this process, see the dedicated notebook on this site.

list(sys.modules.keys())[:3]

['sys', 'builtins', '_frozen_importlib']

sys.modules["sys"]

<module 'sys' (built-in)>

The following cell show the finders that are used to import modules into the current environment.

sys.meta_path

[<_distutils_hack.DistutilsMetaFinder at 0x75fa09734080>,
 <_virtualenv._Finder at 0x75fa0a14ed20>,
 _frozen_importlib.BuiltinImporter,
 _frozen_importlib.FrozenImporter,
 _frozen_importlib_external.PathFinder,
 <six._SixMetaPathImporter at 0x75fa084b6c00>]

The following cell shows attempts to apply find_spec of the different finders to the “pandas”.

for finder in sys.meta_path:
    print(finder, ":", finder.find_spec("pandas", None))

<_distutils_hack.DistutilsMetaFinder object at 0x75fa09734080> : None
<_virtualenv._Finder object at 0x75fa0a14ed20> : None
<class '_frozen_importlib.BuiltinImporter'> : None
<class '_frozen_importlib.FrozenImporter'> : None
<class '_frozen_importlib_external.PathFinder'> : ModuleSpec(name='pandas', loader=<_frozen_importlib_external.SourceFileLoader object at 0x75f9f46af9e0>, origin='/home/fedor/.local/share/hatch/env/virtual/src/4VwHlfPf/src/lib/python3.12/site-packages/pandas/__init__.py', submodule_search_locations=['/home/fedor/.local/share/hatch/env/virtual/src/4VwHlfPf/src/lib/python3.12/site-packages/pandas'])
<six._SixMetaPathImporter object at 0x75fa084b6c00> : None

As a result, PathFinder has a ModuleSpec for pandas. By comparison the following cell looks for “sys” in the same way.

for finder in sys.meta_path:
    print(finder, ":", finder.find_spec("sys", None))

<_distutils_hack.DistutilsMetaFinder object at 0x75fa09734080> : None
<_virtualenv._Finder object at 0x75fa0a14ed20> : None
<class '_frozen_importlib.BuiltinImporter'> : ModuleSpec(name='sys', loader=<class '_frozen_importlib.BuiltinImporter'>, origin='built-in')
<class '_frozen_importlib.FrozenImporter'> : None
<class '_frozen_importlib_external.PathFinder'> : None
<six._SixMetaPathImporter object at 0x75fa084b6c00> : None

For the “sys” module, the BuiltinImporter returns the corresponding ModuleSpec, which makes sense sinse the sys module is a buit-in module.

Relative import#

Relative imports in Python allow you to import modules based on their location relative to the current module, rather than using the full package path. This is useful when you know how the module you want to import is related to the current module within the same package.

Dots are used to specify the level of the package from which to start searching for modules. One dot (.) means you want to search from the current module; two dots (..), from the package above that; and so on.

The following cells creates simple project structure line that:

relative_import/
├── main.py
└── package
    ├── module2.py
    └── module.py

%%bash
mkdir /tmp/relative_import &> /dev/null | true
mkdir /tmp/relative_import/package &> /dev/null | true

%%writefile /tmp/relative_import/package/module.py
print("Hello from module")

Overwriting /tmp/relative_import/package/module.py

%%writefile /tmp/relative_import/package/module2.py
import module

Writing /tmp/relative_import/package/module2.py

%%writefile /tmp/relative_import/main.py
import package.module2

Writing /tmp/relative_import/main.py

The important thing about the “project” described above is that it imports the module in module2. However, the project runs from the /tmp/relative_import folder, and there is no option to import the module from it.

%%bash
cd /tmp
python3 relative_import/main.py | true

Traceback (most recent call last):
  File "/tmp/relative_import/main.py", line 1, in <module>
    import package.module2
  File "/tmp/relative_import/package/module2.py", line 1, in <module>
    import module
ModuleNotFoundError: No module named 'module'

The following cell makes small changes to the package/module2 that cause python to search for the module from the same package that module2 is in.

%%writefile /tmp/relative_import/package/module2.py
from . import module

Overwriting /tmp/relative_import/package/module2.py

The program is now executing without any issues.

%%bash
cd /tmp
python3 relative_import/main.py

Hello from module