Logging

Logging#

Python has a standard library for logging: logging.

import sys
import logging

The core components of the logging library are:

  • Loggers

  • Handlers

  • Formatters

  • Filters

Each one defines a specific aspect of log generation.

Loggers#

For different parts of your program, you may need separate loggers. You can create a new logger by using logging.getLogger(<logger_name>). Find out more details in the specific page.

The following cell demonstrates the creation of logger1 and logger2, along with their immediate usage.

logger1 = logging.getLogger("logger1")
logger1.setLevel(logging.INFO)
print("=====logger1=====", file=sys.stderr)
logger1.debug("A DEBUG Message")
logger1.info("An INFO")
logger1.warning("A WARNING")
logger1.error("An ERROR")
logger1.critical("A message of CRITICAL severity")


logger2 = logging.getLogger("logger2")
logger2.setLevel(logging.ERROR)
print("=====logger2=====", file=sys.stderr)
logger2.debug("A DEBUG Message")
logger2.info("An INFO")
logger2.warning("A WARNING")
logger2.error("An ERROR")
logger2.critical("A message of CRITICAL severity")

=====logger1=====
A WARNING
An ERROR
A message of CRITICAL severity
=====logger2=====
An ERROR
A message of CRITICAL severity

Handlers#

Defines the direction in which logs are written. It can be files or output streams. You can add handlers to the logger using the logger.addHandler method. Check list of the usefull handlers.

The following example demonstrates how to add a handler to a logger.

logger = logging.getLogger("temp_logger")
stream_handler = logging.StreamHandler()
logger.addHandler(stream_handler)
logger.critical("It's a error!")
# To prevent adding a handler for each
# run of this cell, we'll delete just
# added logger
del logger.handlers[0]

It's a error!

There is a specfic page describing handlers.

Formatters#

Formatters define how each line of the log is to be printed. You must set the formatter for the handler using the handler.setFormatter method.

The following example defines formatter for given logger.

logger = logging.getLogger("some_logger")
stream_handler = logging.StreamHandler()
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
stream_handler.setFormatter(formatter)
logger.addHandler(stream_handler)
logger.error("It's error")
del logger.handlers[0]

2024-08-29 17:01:35,382 - some_logger - ERROR - It's error

Filters#

With filters, you can set specific rules for which messages should be passed to the output by handlers. Additionally, filters are a common place to customize the behavior of the logger.

Find out more:

The following example demonstrates creating a logger with a filter function that returns False if the message in the logging record contains <stop word>.

logger = logging.getLogger("filter logger")

logger.addFilter(lambda record: '<stop word>' not in record.getMessage())

logger.critical("Just message")
logger.critical("Message with <stop word>")

Just message

Configuration#

There are some tools that allows to configure logging module from optional key:value like formats.

There is special page about aspects of working with it.

Here’s a simple example of how to configure an entire logging module using Python dictionaries. In this example, a logger named simpleExample is created with a specific formatter to ensure the logs have the desired format.

logging.config.dictConfig({
    "version" : 1,
    "formatters" : {
        "simpleFormatter" : {
            "format" : "I'm created from config :) %(message)s"
        }
    },
    "handlers" : {
        "consoleHandler" : {
            "class" : "logging.StreamHandler",
            "formatter" : "simpleFormatter"
        }
    },
    "loggers" : {
        "simpleExample" : {
            "handlers" : ["consoleHandler"]
        }
    }
})

logging.getLogger("simpleExample").error("{it's message}")

I'm created from config :) {it's message}

It is the preferred way to configure the logs, because anyone who is not involved with the code can participate in configuring the logs.

Root logger#

This logger is used when you run the logging methods of the logging module itself.

Using the logging.basicConfig function, you can define the behaviour of the logs generated by the logging module. The following example defines the DEBUG logging level so that you can later see any message raised by the logging methods. By default only warning, error and critical are executed.

Findout more on a special page.

In the following example, the code is saving the logs to a separate file and executing it from another interpreter. This approach is necessary because once you execute any logging method in this notebook, you cannot modify its configuration anymore.

%%writefile logging_files/root_logger.py
import logging
logging.basicConfig(level=logging.DEBUG)

logging.debug("A DEBUG Message")
logging.info("An INFO")
logging.warning("A WARNING")
logging.error("An ERROR")
logging.critical("A message of CRITICAL severity")

Writing logging_files/root_logger.py

!python3 logging_files/root_logger.py

DEBUG:root:A DEBUG Message
INFO:root:An INFO
WARNING:root:A WARNING
ERROR:root:An ERROR
CRITICAL:root:A message of CRITICAL severity
Contents