4.4 StatefulSets — When Identity Matters

⏱️ ~6 min read

TL;DR: StatefulSets are for applications that need stable network identity and stable storage — databases, message queues, consensus systems. Each pod gets a persistent name (pod-0, pod-1) and its own PersistentVolumeClaim that follows it across restarts.

The Problem with Deployments for Stateful Apps

Deployments give each pod a random suffix (my-app-7f4b9c-x8p2q). If a pod is replaced, it gets a new random name, a new IP, and starts with an empty disk.

That’s fine for stateless services. But try running a database replica set where:

• Replica 1 needs to always be db-0 so others can find it
• db-0 needs its data to survive restarts
• Replicas must start in order (primary before secondaries)

Deployments can’t do this. StatefulSets can.

StatefulSet YAML

# statefulset.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: mongo
spec:
  serviceName: mongo          # Must match a Headless Service (see below)
  replicas: 3
  selector:
    matchLabels:
      app: mongo
  template:
    metadata:
      labels:
        app: mongo
    spec:
      containers:
      - name: mongo
        image: mongo:7
        ports:
        - containerPort: 27017
        volumeMounts:
        - name: data
          mountPath: /data/db    # Each pod mounts ITS OWN volume here

  volumeClaimTemplates:         # ← This is the StatefulSet superpower
  - metadata:
      name: data
    spec:
      accessModes: ["ReadWriteOnce"]
      resources:
        requests:
          storage: 1Gi

The volumeClaimTemplates section automatically creates a separate PVC for each pod:

• data-mongo-0 → mongo-0
• data-mongo-1 → mongo-1
• data-mongo-2 → mongo-2

The Four Guarantees of StatefulSets

graph TB
    subgraph "StatefulSet Guarantees"
        G1["1. Stable Pod Names\nmongo-0, mongo-1, mongo-2\n(not random hashes)"]
        G2["2. Stable Network Identity\nmongo-0.mongo.default.svc.cluster.local\n(predictable DNS)"]
        G3["3. Stable Storage\nEach pod keeps its own PVC\nacross restarts and reschedules"]
        G4["4. Ordered Operations\nStart: 0 → 1 → 2\nTerminate: 2 → 1 → 0"]
    end

The Required Headless Service

StatefulSets require a Headless Service (clusterIP: None) to provide stable DNS for individual pods:

# headless-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: mongo             # Must match StatefulSet's serviceName
spec:
  clusterIP: None         # ← Headless: no virtual IP, DNS returns pod IPs directly
  selector:
    app: mongo
  ports:
  - port: 27017

With this, each pod gets a stable DNS record:

mongo-0.mongo.default.svc.cluster.local  → 10.244.0.5
mongo-1.mongo.default.svc.cluster.local  → 10.244.0.6
mongo-2.mongo.default.svc.cluster.local  → 10.244.0.7

If mongo-1 is rescheduled to a different node, its DNS name stays mongo-1.mongo... — it just resolves to a new IP. Other services can always reach it by name.

When to Use StatefulSets

Use StatefulSets for:

• Databases (MongoDB, PostgreSQL, MySQL clusters)
• Message queues (Kafka, RabbitMQ)
• Consensus systems (Zookeeper, etcd)
• Any app that needs persistent, pod-specific identity

Don’t use StatefulSets for:

• Stateless web services (use Deployment)
• Batch jobs (use Jobs)
• Any app that doesn’t need stable identity or its own storage

🏭 In Production: Most teams use Helm charts (Chapter 12) for databases, which package StatefulSets with the right configuration. Don’t roll your own database StatefulSet for production without understanding your specific database’s clustering requirements.

Try It

# Create the headless service
cat <<'EOF' | kubectl apply -f -
apiVersion: v1
kind: Service
metadata:
  name: nginx-headless
spec:
  clusterIP: None
  selector:
    app: nginx-sts
  ports:
  - port: 80
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: nginx-sts
spec:
  serviceName: nginx-headless
  replicas: 3
  selector:
    matchLabels:
      app: nginx-sts
  template:
    metadata:
      labels:
        app: nginx-sts
    spec:
      containers:
      - name: nginx
        image: nginx:1.25
        resources:
          limits:
            memory: "64Mi"
            cpu: "100m"
EOF

# Watch them start IN ORDER (0, then 1, then 2)
kubectl get pods -l app=nginx-sts -w

# See stable pod names
kubectl get pods -l app=nginx-sts

# Delete the middle pod — it comes back with the SAME name
kubectl delete pod nginx-sts-1
kubectl get pods -l app=nginx-sts -w  # nginx-sts-1 returns with same name

# Cleanup
kubectl delete statefulset nginx-sts
kubectl delete service nginx-headless

Expected pod names:

NAME          READY   STATUS    RESTARTS   AGE
nginx-sts-0   1/1     Running   0          30s
nginx-sts-1   1/1     Running   0          25s
nginx-sts-2   1/1     Running   0          20s

Key Takeaways

✅ Quick Check

Q1: A StatefulSet pod kafka-2 gets rescheduled to a new node. Does it retain its data?

Q2: You scale a StatefulSet from 3 to 5 replicas. In what order do pods 3 and 4 start?

Q3: Can you use a regular (non-headless) Service with a StatefulSet?