Learn how to use Python's logging module properly — structured output, correct log levels, rotating handlers, and shipping logs to a centralized service.
Most Python developers start with print() and never move past it. That works in a Jupyter notebook. In production, it means lost context, no filtering, and no way to find what went wrong at 2 AM.
Python's built-in logging module is powerful but poorly understood. Here's how to use it properly.
print() writes to stdout with no structure, no severity, no timestamp, and no way to disable it without deleting the line. It's a debugging tool for development, not a logging strategy.
# Bad — impossible to filter or search
print(f"Processing order {order_id} for user {user_id}")
# Good — structured, filterable, has severity
import logging
logger = logging.getLogger(__name__)
logger.info("order.processing", extra={"order_id": order_id, "user_id": user_id})
The extra dict becomes searchable fields in any log management tool. You can query order_id:ORD-1234 instead of grep-ing through megabytes of text.
A common mistake is calling logging.basicConfig() in every module. Configure logging once in your application's entry point:
import logging
import json
class JSONFormatter(logging.Formatter):
def format(self, record):
log_entry = {
"timestamp": self.formatTime(record),
"level": record.levelname.lower(),
"message": record.getMessage(),
"module": record.module,
"function": record.funcName,
"line": record.lineno,
}
# Include extra fields
if hasattr(record, "order_id"):
log_entry["order_id"] = record.order_id
return json.dumps(log_entry)
handler = logging.StreamHandler()
handler.setFormatter(JSONFormatter())
logging.basicConfig(level=logging.INFO, handlers=[handler])
Then in any module:
logger = logging.getLogger(__name__)
logger.info("server started", extra={"port": 8000})
Python's logging module defines five standard levels:
| Level | Value | When to use |
|---|---|---|
DEBUG |
10 | Variable values, detailed flow — development only |
INFO |
20 | Normal operations — request handled, job completed |
WARNING |
30 | Unexpected but handled — retry triggered, deprecation |
ERROR |
40 | Something failed — unhandled exception, API timeout |
CRITICAL |
50 | Application cannot continue — database down, disk full |
The most common mistake: using logger.error() for expected failures like validation errors or 404 responses. These are INFO or WARNING at most. Reserve ERROR for things that need a human to look at them.
For a deeper dive, see our guide on understanding log levels.
Plain text logs are impossible to analyze at scale:
2026-08-30 14:23:11 - Payment processed for user 42, amount $99.00, took 234ms
You can't filter by amount range, you can't aggregate by user, and you can't build dashboards. Structure your logs as JSON:
logger.info("payment.processed", extra={
"user_id": 42,
"amount": 99.00,
"currency": "USD",
"duration_ms": 234,
"payment_method": "card",
})
For a complete guide on structured logging, see our structured logging guide.
The standard logging module works but has a verbose API. Loguru provides a cleaner interface:
from loguru import logger
# Structured logging with bind()
logger.bind(user_id=42, order_id="ORD-1234").info("order.created")
# Automatic exception logging
@logger.catch
def process_payment(order_id):
# If this throws, loguru logs the full traceback with context
charge = stripe.Charge.create(amount=9900)
return charge
# JSON output
logger.add("app.log", serialize=True)
Loguru handles serialization, rotation, and structured context out of the box.
PII in logs creates compliance risk and security vulnerabilities. Never log:
# Bad — leaks password
logger.info(f"Login attempt for {email} with password {password}")
# Good — log the event, not the credentials
logger.info("auth.login_attempt", extra={
"email_hash": hashlib.sha256(email.encode()).hexdigest()[:12],
"ip": request.remote_addr,
"success": False,
})
LogFlow has built-in PII masking that redacts sensitive patterns at ingestion — but it's better to not log them in the first place. Read more in keeping sensitive data out of logs.
In web applications, you want every log line tagged with the request ID, user ID, and trace ID. Python's contextvars module makes this clean:
import contextvars
import uuid
request_id_var = contextvars.ContextVar("request_id", default="-")
class ContextFilter(logging.Filter):
def filter(self, record):
record.request_id = request_id_var.get()
return True
# In your middleware (Flask, FastAPI, Django)
def before_request():
request_id_var.set(str(uuid.uuid4())[:8])
Every log from that request now carries the request ID — critical for trace correlation across services.
Logs that stay on a single server are useless when that server dies. Use RotatingFileHandler for local files, and ship them to a centralized service:
from logging.handlers import RotatingFileHandler
# Local rotation — 10MB per file, keep 5 backups
handler = RotatingFileHandler("app.log", maxBytes=10_000_000, backupCount=5)
For centralized logging, send directly to LogFlow:
from logflow import LogFlow
logflow = LogFlow(api_key="lf_your_key", service="api")
# Send structured logs
logflow.info("order.shipped", {"order_id": "ORD-1234", "carrier": "fedex"})
Or use the HTTP API from any Python application — no SDK needed.
Don't log everything at DEBUG globally. Set levels per module:
# Quiet noisy libraries
logging.getLogger("urllib3").setLevel(logging.WARNING)
logging.getLogger("sqlalchemy.engine").setLevel(logging.WARNING)
# Verbose for your code
logging.getLogger("myapp.payments").setLevel(logging.DEBUG)
This keeps your log volume manageable and your log management costs under control.
Logging is code — test it like code:
import logging
def test_payment_logs_on_success(caplog):
with caplog.at_level(logging.INFO):
process_payment("ORD-123")
assert "payment.processed" in caplog.text
assert "ORD-123" in caplog.text
Once your Python application is logging structured JSON, send those logs somewhere you can actually search and alert on them:
Free plan available. No credit card required. Up and running in 2 minutes.
Get started freeWhat Is Centralized Logging? A Complete Guide
Centralized logging collects logs from every server, container, and service into one searchable place. Here's everything you need to know.
7 ELK Stack Alternatives for Log Management in 2026
Elasticsearch, Logstash, and Kibana are powerful but expensive to operate. Here are 7 alternatives that don't need a platform team.
Docker Compose Logging: Collect Logs from Multi-Container Apps
Docker Compose makes it easy to run multi-container apps. Here's how to make their logs actually useful.