Files & folders

Files & folders#

This section reviews options for working with files and folders in Python. There are some tools typically used in such cases:

  • os: A core Python module for interacting with the operating system.

  • pathlib: A package that provides convenient path operations in Python.

  • shutil: Implements additional tools not available in the previous modules out of the box.

  • tempfile: Allows the creation of files that will be used once and then automatically deleted. This can be useful, especially when working with libraries that operate on disk data, but your data is in memory.

import os
import shutil
from pathlib import Path

Current working directory (CWD)#

The current working directory (CWD) in Python is the directory in which a Python script or interactive session is being executed. It serves as the default directory for relative file paths, meaning that any file operations that use relative paths will be performed within this directory unless explicitly stated otherwise.

You can access current working directory using os.getcwd.

os.getcwd()

'/home/fedor/Documents/knowledge/python/basics'

You can change the CWD using os.chdir. The following example shows changing the directory to the Linux home directory:

initial = os.getcwd()
os.chdir("/home")
print(os.getcwd())
os.chdir(initial)

/home

Exists#

With os.path.exists or pathlib.Path.exists, you can check if something exists at a particular path:

Consider an arbitrary filepath: /tmp/random_file. Any method used to check if the file exists returns False.

considered_folder = Path("/tmp")/"random_file"
os.path.exists(considered_folder), considered_folder.exists()

(False, False)

However, after adding some information to the file at this path, the result changes completely.

considered_folder.write_text("hello")
os.path.exists(considered_folder), considered_folder.exists()

(True, True)

Folders#

You can:

  • Create a folder with os.mkdir or pathlib.Path.mkdir.

  • List the contents of a folder with os.listdir.

  • Remove a folder with:

    • os.rmdir or pathlib.Path.rmdir if the folder is empty.

    • shutil.rmtree if the folder contains content.

The following example demonstrates each of these options step-by-step:

The following cell creates an example folder within the Linux tmp folder, which is intended for temporary files, along with two additional folders:

os.mkdir("/tmp/example")
os.mkdir("/tmp/example/os_dir")
Path("/tmp/example/path_dir").mkdir()

Now let’s check if everything really was created:

os.listdir("/tmp/example")

['path_dir', 'os_dir']

Now remove one of the folders and verify that it was successfully deleted:

os.rmdir("/tmp/example/os_dir")
os.listdir("/tmp/example")

['path_dir']

Finally deleteing root folder of the example:

(Path("/tmp")/"example").rmdir()

---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
Cell In[24], line 1
----> 1 (Path("/tmp")/"example").rmdir()

File /usr/lib/python3.10/pathlib.py:1215, in Path.rmdir(self)
   1211 def rmdir(self):
   1212     """
   1213     Remove this directory.  The directory must be empty.
   1214     """
-> 1215     self._accessor.rmdir(self)

OSError: [Errno 39] Directory not empty: '/tmp/example'

This causes an error - with os.rmdir, the result will be the same. Let’s try another option:

shutil.rmtree("/tmp/example")

Now everything is completed fine.

Pathlib#

pathlib is a library that simplifies working with system paths in Python:

The core of this library is the Path object, which can be initialized with any desired path. The floating cell demonstrates this:

import os
import pathlib

path = pathlib.Path(os.getcwd())
display(path)

PosixPath('/home/fedor/Documents/knowledge/python/basics')

With the created PosixPath instance, you can easily retrieve root paths using the parent attribute or append a new section by using the / operator. pathlib will automatically add the appropriate directory separator for your system:

display(path.parent)
display(path/"my folder")

PosixPath('/home/fedor/Documents/knowledge/python')

PosixPath('/home/fedor/Documents/knowledge/python/basics/my folder')

By applying the str function to the PosixPath, you can obtain a string representation:

str(path)

'/home/fedor/Documents/knowledge/python/basics'

Tempfile#

With the tempfile library, you can create file-like objects that won’t leave any traces on the system.

For more check official documentation.

The following cell shows the most popular way to work with tempfile - by using it as a context manager.

import os
import tempfile

with tempfile.NamedTemporaryFile() as tf:
    tf.write("hello".encode("utf-8"))
    file_name = tf.name
    print(file_name)
    print("In context manager:", os.path.exists(file_name))

print("Out context manger:", os.path.exists(file_name))

/tmp/tmpgxxv3814
In context manager: True
Out context manger: False

Inside the context manager, the file exists, but it is automatically deleted when the program exits the context manager’s block.

Contents