Logging in Python is an old topic. It is quite important to check and debug Python programe, especially when there are something going wrong. In Linux, logging files are usually located in a special folder and rotated basing on time. Here is a powerful logging tool, loguru, a Python 3rd library. In this article, I will not cover it, but I recommend it if you are already familiar with the original Python logging system. Here, I will share some usual practices in my past Python projects.

Write Log in Rotation File in Python

I am using Python original logging module in this project. The logging module will write log in file and basing on the date, it will rotate the file name by date.

# -*- coding: utf-8 -*-
# py3

import logging
import logging.handlers
import sys
import os
import time

__location__ = os.path.realpath(os.path.join(os.getcwd(), os.path.dirname(__file__)))
logFileName = os.path.join(__location__, 'logs/test.log')
log = logging.getLogger()
log.setLevel(logging.DEBUG)
fileHandler = logging.handlers.TimedRotatingFileHandler(logFileName, 'D', 1, 30)
fileHandler.suffix = "%Y%m%d.log"
formatter = logging.Formatter('%(asctime)s - %(module)s.%(funcName)s:%(lineno)d - %(levelname)s - %(message)s')
fileHandler.setFormatter(formatter)
log.addHandler(fileHandler)

while True:
    log.debug("debug debug")
    log.info("hello")
    log.warning("warn info")
    log.error("error")
    time.sleep(1)

When running above example code, the log will be writen in test.log in logs folder. But it could throw an error something like logs folder not existing. Let’s improve the code a little bit.

# -*- coding: utf-8 -*-
# py3

import logging
import logging.handlers
import sys
import os
import time

__location__ = os.path.realpath(os.path.join(os.getcwd(), os.path.dirname(__file__)))
logFolder = os.path.join(__location__, 'logs')

if not os.path.exists(logFolder):
    os.makedirs(logFolder)

logFileName = os.path.join(logFolder, 'test.log')
log = logging.getLogger()
log.setLevel(logging.DEBUG)
fileHandler = logging.handlers.TimedRotatingFileHandler(logFileName, 'D', 1, 30)
fileHandler.suffix = "%Y%m%d.log"
formatter = logging.Formatter('%(asctime)s - %(module)s.%(funcName)s:%(lineno)d - %(levelname)s - %(message)s')
fileHandler.setFormatter(formatter)
log.addHandler(fileHandler)

while True:
    log.debug("debug debug")
    log.info("hello")
    log.warning("warn info")
    log.error("error")
    time.sleep(1)

Log Message to File and Console in Python

Sometimes, I want to write the log both in file and on the screen, especially when I am writing the code in the middle of the project. Of course, using breaking point is the most efficient way to debug code. But observing log in Console is quite useful when there is a long run programe.

Print log in console or terminal is quite simple, besides writing the same logs in file simultaneously. It’s simply declaring a Stdout type log handler and add it in the exising logging system. Here is the fully functional souce code.

# -*- coding: utf-8 -*-
# py3

import logging
import logging.handlers
import sys
import os
import time

__location__ = os.path.realpath(os.path.join(os.getcwd(), os.path.dirname(__file__)))
logFolder = os.path.join(__location__, 'logs')

if not os.path.exists(logFolder):
    os.makedirs(logFolder)

logFileName = os.path.join(logFolder, 'test.log')
log = logging.getLogger()
log.setLevel(logging.DEBUG)
fileHandler = logging.handlers.TimedRotatingFileHandler(logFileName, 'D', 1, 30)
fileHandler.suffix = "%Y%m%d.log"
formatter = logging.Formatter('%(asctime)s - %(module)s.%(funcName)s:%(lineno)d - %(levelname)s - %(message)s')
fileHandler.setFormatter(formatter)
log.addHandler(fileHandler)

handler = logging.StreamHandler(sys.stdout)
handler.setLevel(logging.DEBUG)
handler.setFormatter(formatter)
log.addHandler(handler)

while True:
    log.debug("debug debug")
    log.info("hello")
    log.warning("warn info")
    log.error("error")
    time.sleep(1)

Best Logging Practices in Python

I am using the above code practices in lots of running Python projects. It’s hard to say the best, but they are verified to be useful and quite stable. But it also has some limitation. For example, the log is declared in global level, so it’s hard to get a call inside a function directly. In this case, loguru may provide the convenience. Anyway, there is no the best logging in Python, but you may find the most suitable one in your real project. I found mine.