Logging to file

Logging to file#

It’s usually more convenient to set up a file to store logs in. This page focuses on the aspect of writing logs to the files.

Many logs to same file#

Is it possible to store different logs in the same file? So in the following exmaple I create 20 loggers and bind them to the same file. Then I make 50 log messages each from random loggers. After that I output the resulting file. Everything is fine.

log_filename = "logging_files/manylog_samefile.log"
logs_count = 20
loggers = []

for i in range(logs_count):
    loger_name = f"logger {i}"
    basic_logger = logging.getLogger(loger_name)
    basic_logger.level = logging.INFO
    handler = logging.FileHandler(log_filename, mode='a')
    handler.setFormatter(
        logging.Formatter('%(message)s|%(asctime)s|%(levelname)s|%(name)s')
    )
    basic_logger.addHandler(handler)
    loggers.append(basic_logger)

for i in range(0,50):
    loggers[random.randint(0, logs_count - 1)].error(f"Iteration{i}")

with open(log_filename) as f:
    print(f.read())
os.remove(log_filename)

Iteration0|2024-04-17 00:16:17,057|ERROR|logger 8
Iteration1|2024-04-17 00:16:17,058|ERROR|logger 7
Iteration2|2024-04-17 00:16:17,058|ERROR|logger 5
Iteration3|2024-04-17 00:16:17,058|ERROR|logger 16
Iteration4|2024-04-17 00:16:17,058|ERROR|logger 6
Iteration5|2024-04-17 00:16:17,058|ERROR|logger 10
Iteration6|2024-04-17 00:16:17,058|ERROR|logger 15
Iteration7|2024-04-17 00:16:17,058|ERROR|logger 2
Iteration8|2024-04-17 00:16:17,058|ERROR|logger 18
Iteration9|2024-04-17 00:16:17,058|ERROR|logger 6
Iteration10|2024-04-17 00:16:17,058|ERROR|logger 5
Iteration11|2024-04-17 00:16:17,058|ERROR|logger 16
Iteration12|2024-04-17 00:16:17,058|ERROR|logger 4
Iteration13|2024-04-17 00:16:17,058|ERROR|logger 12
Iteration14|2024-04-17 00:16:17,058|ERROR|logger 11
Iteration15|2024-04-17 00:16:17,058|ERROR|logger 3
Iteration16|2024-04-17 00:16:17,058|ERROR|logger 13
Iteration17|2024-04-17 00:16:17,058|ERROR|logger 0
Iteration18|2024-04-17 00:16:17,058|ERROR|logger 8
Iteration19|2024-04-17 00:16:17,058|ERROR|logger 14
Iteration20|2024-04-17 00:16:17,058|ERROR|logger 14
Iteration21|2024-04-17 00:16:17,058|ERROR|logger 9
Iteration22|2024-04-17 00:16:17,059|ERROR|logger 8
Iteration23|2024-04-17 00:16:17,059|ERROR|logger 13
Iteration24|2024-04-17 00:16:17,059|ERROR|logger 6
Iteration25|2024-04-17 00:16:17,059|ERROR|logger 12
Iteration26|2024-04-17 00:16:17,059|ERROR|logger 11
Iteration27|2024-04-17 00:16:17,059|ERROR|logger 1
Iteration28|2024-04-17 00:16:17,059|ERROR|logger 17
Iteration29|2024-04-17 00:16:17,059|ERROR|logger 7
Iteration30|2024-04-17 00:16:17,059|ERROR|logger 9
Iteration31|2024-04-17 00:16:17,059|ERROR|logger 15
Iteration32|2024-04-17 00:16:17,059|ERROR|logger 14
Iteration33|2024-04-17 00:16:17,059|ERROR|logger 5
Iteration34|2024-04-17 00:16:17,059|ERROR|logger 10
Iteration35|2024-04-17 00:16:17,059|ERROR|logger 9
Iteration36|2024-04-17 00:16:17,059|ERROR|logger 9
Iteration37|2024-04-17 00:16:17,059|ERROR|logger 12
Iteration38|2024-04-17 00:16:17,059|ERROR|logger 5
Iteration39|2024-04-17 00:16:17,059|ERROR|logger 12
Iteration40|2024-04-17 00:16:17,059|ERROR|logger 3
Iteration41|2024-04-17 00:16:17,059|ERROR|logger 7
Iteration42|2024-04-17 00:16:17,059|ERROR|logger 7
Iteration43|2024-04-17 00:16:17,059|ERROR|logger 15
Iteration44|2024-04-17 00:16:17,060|ERROR|logger 10
Iteration45|2024-04-17 00:16:17,060|ERROR|logger 9
Iteration46|2024-04-17 00:16:17,060|ERROR|logger 8
Iteration47|2024-04-17 00:16:17,060|ERROR|logger 18
Iteration48|2024-04-17 00:16:17,060|ERROR|logger 2
Iteration49|2024-04-17 00:16:17,060|ERROR|logger 19

In this case it’s important to use mode mode='a' for the file handler. The following cell is the same as the previous one, but uses mode='w' instead of mode='a'. Something is wrong with the logger in this case.

log_filename = "logging_files/manylog_samefile.log"
logs_count = 20
loggers = []

for i in range(logs_count):
    loger_name = f"logger {i}"
    basic_logger = logging.getLogger(loger_name)
    basic_logger.level = logging.INFO
    handler = logging.FileHandler(log_filename, mode='w')
    handler.setFormatter(
        logging.Formatter('%(message)s|%(asctime)s|%(levelname)s|%(name)s')
    )
    basic_logger.addHandler(handler)
    loggers.append(basic_logger)

for i in range(0,50):
    loggers[random.randint(0, logs_count - 1)].error(f"Iteration{i}")

with open(log_filename) as f:
    print(f.read())
os.remove(log_filename)

Iteration48|2024-04-17 00:16:13,111|ERROR|logger 0
Iteration47|2024-04-17 00:16:13,111|ERROR|logger 6
IIteration45|2024-04-17 00:16:13,110|ERROR|logger 13
Iteration44|2024-04-17 00:16:13,110|ERROR|logger Iteration49|2024-04-17 00:16:13,111|ERROR|logger 5
Contents