python django logging: One single logger with more than one logging level

Question

tl;dr: Is there any way to have one single logger that manages all loglevels?

To log in my project, I am currntly running:

logger = logging.getLogger('info.contract')
logger.info("Some info message...")

Which prints 2021-01-24 03:05:02,063 [contract_generator] Some info message... to console.

The info.contract logger is defined in my settings.py:

    'loggers': {
        'info': {
            'handlers' : ['info_logfile'],
            'level': 'INFO',
        },
        'info.contract': {
            'handlers': ['contract_info_logfile'],
            'level': 'INFO',
            'propagate': True,
        }
    },
    'handlers': {
        'info_logfile': {
            'class': 'logging.FileHandler',
            'filename': 'info.log',
            'formatter': 'default',
        },
        'contract_info_logfile': {
            'class': 'logging.FileHandler',
            'filename': 'contract.info.log',
            'formatter': 'default',
        }
    }
    'formatters': {
        'default': {
            'format': '%(asctime)s [%(module)s] %(message)s',
        }
    }
}

The problem I am having is the following:

In order to log to different files depending on the loglevel, I would have to create a new logger for each loglevel. Which is to say, the following doesn't work:

logger = logging.getLogger('info.contract')
logger.info("Some info message...")

## this part is new
logger.debug("Some debug message...")

Since logger uses the info.contract logger defined in settings.py, which has its 'level' setting set to 'INFO' (as shown above). Interestingly though, I don't get any errors. The logger simply doesn't log the entries when I use the .debug() function.

So in order to log to different files depending on the loglevel, I would indeed have to create a new logger for each loglevel, even though that produces a lot of duplicate code:

##---snip---##
'loggers': {
        'info': {
            'handlers' : ['std_err', 'info_logfile'],
            'level': 'INFO',
        },
        'info.contract': {
            'handlers': ['contract_info_logfile'],
            'level': 'INFO',
            'propagate': True,
        }
        'debug': {                                     ## duplicate code
            'handlers' : ['std_err', 'debug_logfile'],
            'level': 'DEBUG',
        },
            'debug.contract': {                        ## duplicate code
            'handlers': ['contract_debug_logfile'],
            'level': 'DEBUG',
            'propagate': True,
        }
},
##---snip---##

So then in my code I would have to do:

info_logger = logging.getLogger('info.contract')
debug_logger = logging.getLogger('debug.contract')

info_logger.info("Some info message...")
debug_logger.debug("Some debug message...")

Which implies that it is necessary to instantiate a new logger object for every loglevel I need to use. This seems cumbersome and is maybe inefficient.

My question is:

Is there any way to have one single logger that manages all loglevels? In other words, is it possible to do the following:

some_multilevel_logger = logging.getLogger('some_multilevel_logger')
some_multilevel_logger.info("Some info message...")
some_multilevel_logger.debug("Some debug message...")

And that would produce the same results as the previous snippet, in which 2 loggers are defined.

If that behavior is not possible, is it considered bad practice to create one logger for every loglevel? What about massive projects with many loglevels?

Thank you!

question from:https://stackoverflow.com/questions/65867107/python-django-logging-one-single-logger-with-more-than-one-logging-level

Answer 1

I think one way to use one log file for different purposes is to change your log level in your logger configs because based on this document you can not log debug level in to info level. For example look at the following configs which I usually use as my best practice:

LOG_DIR = BASE_DIR / 'logs'
LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': {
        'simple': {
            'format': '[%(asctime)s] %(levelname)s %(message)s',
            'datefmt': '%Y-%m-%d %H:%M:%S'
        },
        'verbose': {
            'format': '[%(asctime)s] %(levelname)s [%(filename)s.%(funcName)s:%(lineno)d] %(message)s',
            'datefmt': '%Y-%m-%d %H:%M:%S'
        },
    },
    'filters': {
        'require_debug_false': {
            '()': 'django.utils.log.RequireDebugFalse',
        },
        'require_debug_true': {
            '()': 'django.utils.log.RequireDebugTrue',
        },
    },
    'handlers': {
        'console': {
            'level': 'DEBUG',
            'filters': ['require_debug_true'],
            'class': 'logging.StreamHandler',
            'formatter': 'simple'
        },
        'mail_admins': {
            'level': 'ERROR',
            'filters': ['require_debug_false'],
            'class': 'django.utils.log.AdminEmailHandler',
            'formatter': 'verbose'
        },
        'file': {
            'level': 'DEBUG' if DEBUG else 'INFO',
            'class': 'logging.FileHandler',
            'filename': LOG_DIR / 'django.log',
            'formatter': 'verbose' if DEBUG else 'simple',
        },
        'db_queries': {
            'level': 'DEBUG',
            'filters': ['require_debug_true'],
            'class': 'logging.FileHandler',
            'filename': LOG_DIR / 'db_queries.log',
        },
        'null': {
            'level': 'DEBUG',
            'class': 'logging.NullHandler',
        },
    },
    'loggers': {
        'django.db.backends': {
            'level': 'DEBUG',
            'handlers': ['db_queries'],
            'propagate': False,
        },
        'accounts': {
            'handlers': ['file', 'console'],
            'level': 'DEBUG',
        },
    },
}

And for logging just use following lines:

logger = logging.getLogger('accounts')

logger.info("some info logs")
logger.debug("some debug logs")