Are you still writing logs like this?

import logging

logging.debug("Start uploading file with name %s. Number of retries: %s", file_name, 0)
...
logging.error("Failed uploading file with name %s. Current retry: %s.", file_name, retries)
...

Here are the challenges of logging this way.

• missing contextual info, the above log might be enough for a dev environment with one user, but in a production system, questions that can arise are:
  • Who sent this request?
  • Which organization does the user belong to?
  • Which file store did we connect to? Is there a session identifier or something else that helps us trace down the error?
  • Is this log line connected to a request identifier?
• transition from dev only to machine readable logs
  • logs should not only human-readable, but also machine readable in production system for easier querying
• inconsistent wording
  • dependent on author, different naming convention, missing information

Introducing structlog

Structlog offers a lot of nice features that help you write logs in a faster and less painful way. It helps you to add contextual data to your logger and offers a modular configuration for parsing your log lines in a machine readable and developer friendly way.

Summary of learnings

• a key-value format for variables creates standardizes log messages and naming convention
• contextual data are added explicitly by binding variables or implicitly by using context variables, enriching logs with useful metadata
• structlog preprocessor extends content of log messages and renderer allows easy parsing of logs into machine/human-friendly format

Loguru is another library that makes logging enjoyable.