Files & folders

Files & folders#

This section reviews options for working with files and folders in Python. There are some tools typically used in such cases:

  • os: A core Python module for interacting with the operating system.

  • pathlib: A package that provides convenient path operations in Python.

  • shutil: Implements additional tools not available in the previous modules out of the box.

  • tempfile: Allows the creation of files that will be used once and then automatically deleted. This can be useful, especially when working with libraries that operate on disk data, but your data is in memory.

import os
import shutil
from pathlib import Path

Current working directory (CWD)#

The current working directory (CWD) in Python is the directory in which a Python script or interactive session is being executed. It serves as the default directory for relative file paths, meaning that any file operations that use relative paths will be performed within this directory unless explicitly stated otherwise.

You can access current working directory using os.getcwd.

os.getcwd()

'/home/fedor/Documents/knowledge/python/basics'

You can change the CWD using os.chdir. The following example shows changing the directory to the Linux home directory:

initial = os.getcwd()
os.chdir("/home")
print(os.getcwd())
os.chdir(initial)

/home

Exists#

With os.path.exists or pathlib.Path.exists, you can check if something exists at a particular path:

Consider an arbitrary filepath: /tmp/random_file. Any method used to check if the file exists returns False.

considered_folder = Path("/tmp")/"random_file"
os.path.exists(considered_folder), considered_folder.exists()

(False, False)

However, after adding some information to the file at this path, the result changes completely.

considered_folder.write_text("hello")
os.path.exists(considered_folder), considered_folder.exists()

(True, True)

Folders#

You can:

  • Create a folder with os.mkdir or pathlib.Path.mkdir.

  • List the contents of a folder with os.listdir.

  • Remove a folder with:

    • os.rmdir or pathlib.Path.rmdir if the folder is empty.

    • shutil.rmtree if the folder contains content.

The following example demonstrates each of these options step-by-step:

The following cell creates an example folder within the Linux tmp folder, which is intended for temporary files, along with two additional folders:

os.mkdir("/tmp/example")
os.mkdir("/tmp/example/os_dir")
Path("/tmp/example/path_dir").mkdir()

Now let’s check if everything really was created:

os.listdir("/tmp/example")

['path_dir', 'os_dir']

Now remove one of the folders and verify that it was successfully deleted:

os.rmdir("/tmp/example/os_dir")
os.listdir("/tmp/example")

['path_dir']

Finally deleteing root folder of the example:

(Path("/tmp")/"example").rmdir()

---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
Cell In[24], line 1
----> 1 (Path("/tmp")/"example").rmdir()

File /usr/lib/python3.10/pathlib.py:1215, in Path.rmdir(self)
   1211 def rmdir(self):
   1212     """
   1213     Remove this directory.  The directory must be empty.
   1214     """
-> 1215     self._accessor.rmdir(self)

OSError: [Errno 39] Directory not empty: '/tmp/example'

This causes an error - with os.rmdir, the result will be the same. Let’s try another option:

shutil.rmtree("/tmp/example")

Now everything is completed fine.

Pathlib#

pathlib is a library that simplifies working with system paths in Python:

The core of this library is the Path object, which can be initialized with any desired path. The floating cell demonstrates this:

import os
import pathlib

path = pathlib.Path(os.getcwd())
display(path)

PosixPath('/home/fedor/Documents/knowledge/python/basics')

With the created PosixPath instance, you can easily retrieve root paths using the parent attribute or append a new section by using the / operator. pathlib will automatically add the appropriate directory separator for your system:

display(path.parent)
display(path/"my folder")

PosixPath('/home/fedor/Documents/knowledge/python')

PosixPath('/home/fedor/Documents/knowledge/python/basics/my folder')

By applying the str function to the PosixPath, you can obtain a string representation:

str(path)

'/home/fedor/Documents/knowledge/python/basics'

Tempfile#

With the tempfile library, you can create file-like objects that won’t leave any traces on the system.

There are classes that implement different types of temp objects in the file system:

  • tempfile.TemporaryFile the file-like object that would be deleted as soon as possible.

  • tempfile.NamedTemporaryFile the file-like object that garantees the existence of a real named file in the file system.

  • tempfile.SpooledTemporaryFile the file-like object keeps data in memory and, under certain circumstances, writes data to the disk.

  • tempfile.TemporaryDirectory creates a temporary directory. The folder is immediately removed from the file system as soon as corresponding object is desctoroyed.

For more check official documentation.

The main feature of all these classes is that they delete associated with instance files/folders as soon as the instance is destroyed or the context is exited. Consider few examples.

import os
import tempfile

The following cell creates a temporary directory.

temp_dir = tempfile.TemporaryDirectory()
name = temp_dir.name
name

'/tmp/tmpbzqikg6l'

The corresponding path is present in the system.

os.path.exists(name)

True

However, it disappears as soon as the object is destroyed.

del temp_dir
os.path.exists(name)

False

Consider the same example, but with context created using the tempfile.NamedTemporaryFile.

with tempfile.NamedTemporaryFile() as tf:
    tf.write("hello".encode("utf-8"))
    file_name = tf.name
    print(file_name)
    print("In context manager:", os.path.exists(file_name))

print("Out context manger:", os.path.exists(file_name))

/tmp/tmpfpinr1yl
In context manager: True
Out context manger: False

The temporary file is deleted when the flow leaves the context.

Contents