Logger From Config File

Logging in Python Austin Cepalia 03:26

You can configure logging using the module and class functions or by creating a config file or a dictionary and loading it using fileConfig() or dictConfig() respectively. These are useful in case you want to change your logging configuration in a running application.

Here’s an example file configuration:

Config File
      
    
[loggers]
keys=root,sampleLogger

[handlers]
keys=consoleHandler

[formatters]
keys=sampleFormatter

[logger_root]
level=DEBUG
handlers=consoleHandler

[logger_sampleLogger]
level=DEBUG
handlers=consoleHandler
qualname=sampleLogger
propagate=0

[handler_consoleHandler]
class=StreamHandler
level=DEBUG
formatter=sampleFormatter
args=(sys.stdout,)

[formatter_sampleFormatter]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s

In the above file, there are two loggers, one handler, and one formatter. After their names are defined, they are configured by adding the words logger, handler, and formatter before their names separated by an underscore.

00:00 Creating custom loggers within our code is great, but what if we want to save our configuration for use later on or share it with someone else? That’s where config files come into play.

00:13 We can create a config file and have our program create the logger based on that file. We can do that with the fileConfig() and dictConfig() functions.

00:24 Let’s see how that works. Here in VS Code, I have a basic config file written for a logger. This might look a little bit cryptic, but let’s walk through it to understand what it’s doing. First, we’re defining two loggers, one named root and the other named sampleLogger. Remember, root is the built-in logger, so this config file will configure both a custom logger and the built-in one.

00:52 Then, we define a new handler named consoleHandler and a formatter called sampleFormatter Now, the next two sections modify our loggers.

01:04 We can tell because the name starts with logger and then an underscore (_), and then the name of the logger. For the root logger, we assign it a level of DEBUG and a handler, consoleHandler. For the sampleLogger, we give it a level of DEBUG, the same consoleHandler as before, a qualname (qualified name) of sampleLogger, and a propagate value of 0, or False—something that doesn’t apply to this basic config file here.

01:35 The next section configures the consoleHandler. We give it a class of StreamHandler, which will allow it to write to stdout, a level of DEBUG, a formatter of sampleFormatter, and args of (sys.stdout,), short for standard out.

01:53 Finally, we modify the sampleFormatter by defining the string used to represent the log format, just like we’ve been doing. This file right here is saved as file.conf, or as I say, “conf”.

02:10 Now I’m going to move into a blank Python file, and we’re going to write a few lines of code that will configure a new logger from this file. First, I’ll import two modules, logging and logging.config.

02:26 Then, we need to read from the file. This can be done by calling the fileConfig() function within the logging.config module, passing in the filename of 'file.conf'. And I’ll set disable_existing_loggers to False. This way, if other loggers are being used, they won’t be disabled, which normally happens by default.

02:53 Now, we need to get the logger specified in the file. I’ll type logger = logging.getLogger() and I’ll pass in __name__.

03:04 And I will use this logger with logger.debug('This is a debug message').

03:12 I’ll right-click and run this program, and we see in the console on the right, we get the time and the date, the logger name of __main__, the level, and the debug message.

nerdydodo on July 29, 2021

What is “qualified name” at 01:23? What is “propagate”?

Bartosz Zaczyński RP Team on July 29, 2021

@nerdydodo A qualified name is one that contains not just the class name but also all its enclosing packages and modules, for example, concurrent.futures.ThreadPoolExecutor as opposed to ThreadPoolExecutor.

The propagate attribute is a Boolean flag (True or False) that determines whether a message will be passed to higher-level loggers or not.

dstricks on Sept. 24, 2021

I went a bit off script and updated the getLogger line from your sample code to this:

logger = logging.getLogger("sampleLogger")

After that change, I noticed that the same exact log message was displayed in the console twice. After a bit of research, I came across a relevant note in the logging.Logger.propagate API:

Note If you attach a handler to a logger and one or more of its ancestors, it may emit the same record multiple times. In general, you should not need to attach a handler to more than one logger - if you just attach it to the appropriate logger which is highest in the logger hierarchy, then it will see all events logged by all descendant loggers, provided that their propagate setting is left set to True. A common scenario is to attach handlers only to the root logger, and to let propagation take care of the rest.

So I removed consoleHandler from logger_sampleLogger and replaced it with handlers= and updated propagate=1. Now I’m seeing just the expected single message being displayed to the console.

Hope this helps the next person who might encounter the same issue.

(And finally, shoutout to Austin for this helpful tutorial.)

torrepreciado on Dec. 6, 2021

I went a bit off script to try and configure a MemoryHandler which I can later deploy into a variable. However, I don’t know what to set the target to so I can then print the variable with all the errors.

This is my current config file:

[loggers]
keys=root

[handlers]
keys=memoryHandler

[formatters]
keys=memoryFormatter

[logger_root]
level=DEBUG
handlers=memoryHandler

[handler_memoryHandler]
class=handlers.MemoryHandler
formatter=memoryFormatter
args=(10, DEBUG)
; target=

[formatter_memoryFormatter]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
datefmt=%Y-%m-%d %H:%M:%S

And this is my current script:

import logging.config

logging.config.fileConfig(fname=r'./logger_config2.conf')
logging.info('INFO')

print(logging.root.handlers)

Become a Member to join the conversation.