Collect logs from all your pods and send them to LogFlow using the @getlogflow/js SDK, with service and namespace labels for filtering.
Kubernetes logging has two layers: what your application logs, and how those logs get collected and shipped. This tutorial covers both — writing structured logs from your app using the LogFlow SDK, and setting up a DaemonSet-based collector for infrastructure logs.
Prerequisites:
kubectl configured with cluster accessOption A — SDK logging (recommended for application logs). Your application code sends logs directly to LogFlow. You control the structure, include business context (user IDs, order IDs, request metadata), and get rich structured logs. This is what this tutorial focuses on.
Option B — Log collector DaemonSet. A collector pod (like Fluent Bit) runs on every node, reads stdout/stderr from all pods, and ships them to LogFlow via the HTTP API. This is useful for infrastructure logs, third-party services you don't control, and apps you can't modify.
Most production setups use both.
kubectl create secret generic logflow-secret \
--from-literal=api-key=lf_your_api_key_here
Never hardcode API keys in your container images or deployment manifests.
In your Node.js application:
npm install @getlogflow/js
The key to useful Kubernetes logging is attaching pod metadata — namespace, deployment name, pod name — to every log entry. This lets you filter by namespace or deployment in LogFlow's Logs Explorer.
import { LogFlow } from '@getlogflow/js'
const logger = new LogFlow({
apiKey: process.env.LOGFLOW_API_KEY!,
service: process.env.SERVICE_NAME || 'api',
environment: process.env.NODE_ENV || 'production',
// Attach pod metadata to every log
defaultAttributes: {
namespace: process.env.POD_NAMESPACE,
pod: process.env.POD_NAME,
node: process.env.NODE_NAME,
deployment: process.env.DEPLOYMENT_NAME,
},
})
export default logger
Kubernetes can inject pod metadata via the Downward API (see Step 4).
Update your Deployment manifest to expose pod metadata as environment variables:
# deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: api
namespace: production
spec:
replicas: 3
selector:
matchLabels:
app: api
template:
metadata:
labels:
app: api
spec:
containers:
- name: api
image: your-registry/api:latest
env:
# LogFlow API key from secret
- name: LOGFLOW_API_KEY
valueFrom:
secretKeyRef:
name: logflow-secret
key: api-key
# Service name
- name: SERVICE_NAME
value: "api"
- name: DEPLOYMENT_NAME
value: "api"
# Pod metadata via Downward API
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
With the logger set up, emit structured logs throughout your application:
import logger from './logger'
import express from 'express'
const app = express()
app.use((req, res, next) => {
const start = Date.now()
res.on('finish', () => {
logger.info('HTTP request', {
method: req.method,
path: req.path,
statusCode: res.statusCode,
durationMs: Date.now() - start,
userAgent: req.headers['user-agent'],
})
})
next()
})
app.get('/orders/:id', async (req, res) => {
try {
const order = await getOrder(req.params.id)
logger.info('Order fetched', { orderId: req.params.id, userId: order.userId })
res.json(order)
} catch (err) {
logger.error('Failed to fetch order', {
orderId: req.params.id,
error: (err as Error).message,
})
res.status(500).json({ error: 'Internal server error' })
}
})
Each log in LogFlow will automatically include namespace, pod, node, and deployment from the default attributes — making it easy to filter to a specific pod during debugging.
For logs from pods you can't modify (databases, ingress controllers, third-party services), use a Fluent Bit DaemonSet that reads stdout/stderr from all pods and forwards to LogFlow.
# fluent-bit-config.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: fluent-bit-config
namespace: logging
data:
fluent-bit.conf: |
[SERVICE]
Flush 5
Log_Level info
Daemon off
[INPUT]
Name tail
Path /var/log/containers/*.log
Parser docker
Tag kube.*
Refresh_Interval 5
Mem_Buf_Limit 50MB
Skip_Long_Lines On
[FILTER]
Name kubernetes
Match kube.*
Kube_URL https://kubernetes.default.svc:443
Kube_CA_File /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
Kube_Token_File /var/run/secrets/kubernetes.io/serviceaccount/token
Merge_Log On
Keep_Log Off
K8S-Logging.Parser On
[OUTPUT]
Name http
Match *
Host api.getlogflow.com
Port 443
TLS On
URI /v1/logs/batch
Format json
Header Authorization Bearer ${LOGFLOW_API_KEY}
Header Content-Type application/json
json_date_key timestamp
json_date_format iso8601
parsers.conf: |
[PARSER]
Name docker
Format json
Time_Key time
Time_Format %Y-%m-%dT%H:%M:%S.%L
Time_Keep On
# fluent-bit-daemonset.yaml
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: fluent-bit
namespace: logging
spec:
selector:
matchLabels:
app: fluent-bit
template:
metadata:
labels:
app: fluent-bit
spec:
serviceAccountName: fluent-bit
tolerations:
- key: node-role.kubernetes.io/control-plane
operator: Exists
effect: NoSchedule
containers:
- name: fluent-bit
image: fluent/fluent-bit:3.1
env:
- name: LOGFLOW_API_KEY
valueFrom:
secretKeyRef:
name: logflow-secret
key: api-key
volumeMounts:
- name: varlog
mountPath: /var/log
- name: varlibdockercontainers
mountPath: /var/lib/docker/containers
readOnly: true
- name: fluent-bit-config
mountPath: /fluent-bit/etc/
volumes:
- name: varlog
hostPath:
path: /var/log
- name: varlibdockercontainers
hostPath:
path: /var/lib/docker/containers
- name: fluent-bit-config
configMap:
name: fluent-bit-config
Apply it:
kubectl create namespace logging
kubectl apply -f fluent-bit-config.yaml
kubectl apply -f fluent-bit-daemonset.yaml
Once logs are flowing, you can filter by pod metadata in the Logs Explorer:
namespace:production — only production namespacedeployment:api — only the api deploymentpod:api-7d4b9c-xvz2k — a specific pod (useful for debugging a specific replica)level:error AND namespace:production — errors in productionWith structured logs including namespace and service, you can create precise alerts:
service:api AND namespace:productionlevel:fatal in namespace:productiondeployment:worker (silent worker = stuck worker)See the Alerts documentation for setup details, or read Alerting Best Practices for guidance on what to alert on.
Logs not appearing in LogFlow:
kubectl logs -n logging -l app=fluent-bitkubectl get secret logflow-secretToo many logs from system namespaces:
Add a filter to exclude kube-system logs from the Fluent Bit config:
[FILTER]
Name grep
Match kube.*
Exclude $kubernetes['namespace_name'] kube-system
Pod metadata missing from SDK logs:
Check that the Downward API env vars are configured in your Deployment and that defaultAttributes uses the same env var names.
Related: Docker Logging Tutorial · OpenTelemetry Tracing Tutorial · Log Retention Best Practices
Free plan available. No credit card required. Up and running in 2 minutes.
Get started freeStructured Logging in Django with LogFlow
Set up structured logging in Django to capture request logs, errors, and Celery tasks in LogFlow — with full search and alerting.
Add Structured Logging to FastAPI in 10 Minutes
Set up structured JSON logging in FastAPI and ship logs to LogFlow with automatic request context and error tracking.
Centralized Logging for NestJS Applications
Replace NestJS's default logger with structured JSON and ship logs to LogFlow for real-time search and alerts.