Intermediate

Kubernetes Logging with LogFlow

Collect logs from all your pods and send them to LogFlow using the @getlogflow/js SDK, with service and namespace labels for filtering.

LogFlow TeamJuly 20, 202620 minKubernetesNode.jsDocker

Kubernetes logging has two layers: what your application logs, and how those logs get collected and shipped. This tutorial covers both — writing structured logs from your app using the LogFlow SDK, and setting up a DaemonSet-based collector for infrastructure logs.

Prerequisites:

  • A Kubernetes cluster (local with minikube or a cloud provider)
  • kubectl configured with cluster access
  • A LogFlow account (free tier works)
  • Your API key from Settings → API Key

The Two Approaches

Option A — SDK logging (recommended for application logs). Your application code sends logs directly to LogFlow. You control the structure, include business context (user IDs, order IDs, request metadata), and get rich structured logs. This is what this tutorial focuses on.

Option B — Log collector DaemonSet. A collector pod (like Fluent Bit) runs on every node, reads stdout/stderr from all pods, and ships them to LogFlow via the HTTP API. This is useful for infrastructure logs, third-party services you don't control, and apps you can't modify.

Most production setups use both.

Option A: SDK Logging in Your Pods

Step 1 — Store Your API Key as a Secret

kubectl create secret generic logflow-secret \
  --from-literal=api-key=lf_your_api_key_here

Never hardcode API keys in your container images or deployment manifests.

Step 2 — Install the SDK

In your Node.js application:

npm install @getlogflow/js

Step 3 — Create a Logger with Pod Metadata

The key to useful Kubernetes logging is attaching pod metadata — namespace, deployment name, pod name — to every log entry. This lets you filter by namespace or deployment in LogFlow's Logs Explorer.

import { LogFlow } from '@getlogflow/js'

const logger = new LogFlow({
  apiKey: process.env.LOGFLOW_API_KEY!,
  service: process.env.SERVICE_NAME || 'api',
  environment: process.env.NODE_ENV || 'production',
  // Attach pod metadata to every log
  defaultAttributes: {
    namespace: process.env.POD_NAMESPACE,
    pod: process.env.POD_NAME,
    node: process.env.NODE_NAME,
    deployment: process.env.DEPLOYMENT_NAME,
  },
})

export default logger

Kubernetes can inject pod metadata via the Downward API (see Step 4).

Step 4 — Inject Pod Metadata via Downward API

Update your Deployment manifest to expose pod metadata as environment variables:

# deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: api
  namespace: production
spec:
  replicas: 3
  selector:
    matchLabels:
      app: api
  template:
    metadata:
      labels:
        app: api
    spec:
      containers:
        - name: api
          image: your-registry/api:latest
          env:
            # LogFlow API key from secret
            - name: LOGFLOW_API_KEY
              valueFrom:
                secretKeyRef:
                  name: logflow-secret
                  key: api-key
            # Service name
            - name: SERVICE_NAME
              value: "api"
            - name: DEPLOYMENT_NAME
              value: "api"
            # Pod metadata via Downward API
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            - name: NODE_NAME
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName

Step 5 — Log Structured Events

With the logger set up, emit structured logs throughout your application:

import logger from './logger'
import express from 'express'

const app = express()

app.use((req, res, next) => {
  const start = Date.now()
  res.on('finish', () => {
    logger.info('HTTP request', {
      method: req.method,
      path: req.path,
      statusCode: res.statusCode,
      durationMs: Date.now() - start,
      userAgent: req.headers['user-agent'],
    })
  })
  next()
})

app.get('/orders/:id', async (req, res) => {
  try {
    const order = await getOrder(req.params.id)
    logger.info('Order fetched', { orderId: req.params.id, userId: order.userId })
    res.json(order)
  } catch (err) {
    logger.error('Failed to fetch order', {
      orderId: req.params.id,
      error: (err as Error).message,
    })
    res.status(500).json({ error: 'Internal server error' })
  }
})

Each log in LogFlow will automatically include namespace, pod, node, and deployment from the default attributes — making it easy to filter to a specific pod during debugging.

Option B: Fluent Bit DaemonSet

For logs from pods you can't modify (databases, ingress controllers, third-party services), use a Fluent Bit DaemonSet that reads stdout/stderr from all pods and forwards to LogFlow.

ConfigMap for Fluent Bit

# fluent-bit-config.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: fluent-bit-config
  namespace: logging
data:
  fluent-bit.conf: |
    [SERVICE]
        Flush         5
        Log_Level     info
        Daemon        off

    [INPUT]
        Name              tail
        Path              /var/log/containers/*.log
        Parser            docker
        Tag               kube.*
        Refresh_Interval  5
        Mem_Buf_Limit     50MB
        Skip_Long_Lines   On

    [FILTER]
        Name                kubernetes
        Match               kube.*
        Kube_URL            https://kubernetes.default.svc:443
        Kube_CA_File        /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
        Kube_Token_File     /var/run/secrets/kubernetes.io/serviceaccount/token
        Merge_Log           On
        Keep_Log            Off
        K8S-Logging.Parser  On

    [OUTPUT]
        Name            http
        Match           *
        Host            api.getlogflow.com
        Port            443
        TLS             On
        URI             /v1/logs/batch
        Format          json
        Header          Authorization Bearer ${LOGFLOW_API_KEY}
        Header          Content-Type application/json
        json_date_key   timestamp
        json_date_format iso8601

  parsers.conf: |
    [PARSER]
        Name        docker
        Format      json
        Time_Key    time
        Time_Format %Y-%m-%dT%H:%M:%S.%L
        Time_Keep   On

DaemonSet Manifest

# fluent-bit-daemonset.yaml
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: fluent-bit
  namespace: logging
spec:
  selector:
    matchLabels:
      app: fluent-bit
  template:
    metadata:
      labels:
        app: fluent-bit
    spec:
      serviceAccountName: fluent-bit
      tolerations:
        - key: node-role.kubernetes.io/control-plane
          operator: Exists
          effect: NoSchedule
      containers:
        - name: fluent-bit
          image: fluent/fluent-bit:3.1
          env:
            - name: LOGFLOW_API_KEY
              valueFrom:
                secretKeyRef:
                  name: logflow-secret
                  key: api-key
          volumeMounts:
            - name: varlog
              mountPath: /var/log
            - name: varlibdockercontainers
              mountPath: /var/lib/docker/containers
              readOnly: true
            - name: fluent-bit-config
              mountPath: /fluent-bit/etc/
      volumes:
        - name: varlog
          hostPath:
            path: /var/log
        - name: varlibdockercontainers
          hostPath:
            path: /var/lib/docker/containers
        - name: fluent-bit-config
          configMap:
            name: fluent-bit-config

Apply it:

kubectl create namespace logging
kubectl apply -f fluent-bit-config.yaml
kubectl apply -f fluent-bit-daemonset.yaml

Filtering Logs in LogFlow

Once logs are flowing, you can filter by pod metadata in the Logs Explorer:

  • namespace:production — only production namespace
  • deployment:api — only the api deployment
  • pod:api-7d4b9c-xvz2k — a specific pod (useful for debugging a specific replica)
  • level:error AND namespace:production — errors in production

Setting Up Alerts

With structured logs including namespace and service, you can create precise alerts:

  • Error rate above 5% for service:api AND namespace:production
  • Any level:fatal in namespace:production
  • Log volume drop for deployment:worker (silent worker = stuck worker)

See the Alerts documentation for setup details, or read Alerting Best Practices for guidance on what to alert on.

Troubleshooting

Logs not appearing in LogFlow:

  1. Check the Fluent Bit pod logs: kubectl logs -n logging -l app=fluent-bit
  2. Verify the secret exists: kubectl get secret logflow-secret
  3. Confirm your API key is valid by testing the HTTP API directly

Too many logs from system namespaces:

Add a filter to exclude kube-system logs from the Fluent Bit config:

[FILTER]
    Name    grep
    Match   kube.*
    Exclude $kubernetes['namespace_name'] kube-system

Pod metadata missing from SDK logs:

Check that the Downward API env vars are configured in your Deployment and that defaultAttributes uses the same env var names.


Related: Docker Logging Tutorial · OpenTelemetry Tracing Tutorial · Log Retention Best Practices

Start monitoring your logs today

Free plan available. No credit card required. Up and running in 2 minutes.

Get started free