How to Roll Back a Failed Kubernetes Deployment in 30 Seconds

TL;DR — Quick Fix

# Instant rollback to previous version
kubectl rollout undo deployment/<name>

# Rollback to a specific revision
kubectl rollout undo deployment/<name> --to-revision=3

# Helm rollback
helm rollback <release-name> <revision-number>

---

When to Rollback

Rule of thumb: If you can't identify and fix the issue within 5 minutes, rollback first and debug later.

kubectl Rollback

Check Deployment History

# View revision history
kubectl rollout history deployment/my-app

# See what changed in a specific revision
kubectl rollout history deployment/my-app --revision=5

# Check current rollout status
kubectl rollout status deployment/my-app

Rollback Commands

# Rollback to previous revision
kubectl rollout undo deployment/my-app

# Rollback to a specific revision
kubectl rollout undo deployment/my-app --to-revision=3

# Watch the rollback progress
kubectl rollout status deployment/my-app --watch

# Verify pods are running with the old image
kubectl get pods -l app=my-app -o jsonpath='{.items[0].spec.containers[0].image}'

Configure Revision History

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  revisionHistoryLimit: 10
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0

Helm Rollback

Check Release History

helm history my-app -n production

# REVISION  STATUS      DESCRIPTION
# 1         superseded  Install complete
# 2         superseded  Upgrade complete
# 3         deployed    Upgrade complete  (current - failing)

Rollback Commands

# Rollback to previous version
helm rollback my-app 2 -n production

# Rollback with timeout and wait
helm rollback my-app 2 -n production --timeout=5m --wait

# Force rollback (recreate pods if needed)
helm rollback my-app 2 -n production --force

Helm Rollback Hooks

apiVersion: v1
kind: Pod
metadata:
  name: "{{ .Release.Name }}-rollback-test"
  annotations:
    "helm.sh/hook": post-rollback
    "helm.sh/hook-delete-policy": hook-succeeded
spec:
  restartPolicy: Never
  containers:
    - name: smoke-test
      image: curlimages/curl:latest
      command: ['sh', '-c', 'curl -sf http://{{ .Release.Name }}:8080/health || exit 1']

ArgoCD GitOps Rollback

# Option 1: Revert the commit in Git
git revert HEAD
git push origin main
# ArgoCD auto-syncs to the reverted state

# Option 2: ArgoCD CLI
argocd app rollback my-app

# Option 3: ArgoCD UI
# Applications > my-app > History > Select revision > Rollback

ArgoCD Automated Rollback with Analysis

apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: my-app
spec:
  strategy:
    canary:
      steps:
        - setWeight: 10
        - pause: { duration: 2m }
        - analysis:
            templates:
              - templateName: success-rate
        - setWeight: 50
        - pause: { duration: 5m }
        - analysis:
            templates:
              - templateName: success-rate
        - setWeight: 100
---
apiVersion: argoproj.io/v1alpha1
kind: AnalysisTemplate
metadata:
  name: success-rate
spec:
  metrics:
    - name: success-rate
      interval: 30s
      successCondition: result[0] > 0.99
      provider:
        prometheus:
          address: http://prometheus:9090
          query: |
            sum(rate(http_requests_total{app="my-app",status!~"5.."}[2m]))
            / sum(rate(http_requests_total{app="my-app"}[2m]))

Automated Rollback on Health Check Failure

Deployment with Health Probes

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  replicas: 5
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  template:
    spec:
      containers:
        - name: app
          image: my-app:v2.1.0
          readinessProbe:
            httpGet:
              path: /health
              port: 8080
            initialDelaySeconds: 10
            periodSeconds: 5
            failureThreshold: 3
          livenessProbe:
            httpGet:
              path: /health
              port: 8080
            initialDelaySeconds: 30
            periodSeconds: 10
            failureThreshold: 3
          startupProbe:
            httpGet:
              path: /health
              port: 8080
            initialDelaySeconds: 5
            periodSeconds: 5
            failureThreshold: 30
  progressDeadlineSeconds: 300

CI/CD Auto-Rollback Script

#!/bin/bash
DEPLOYMENT="my-app"
NAMESPACE="production"
TIMEOUT=300

echo "Watching rollout for $DEPLOYMENT..."

if ! kubectl rollout status deployment/$DEPLOYMENT -n $NAMESPACE --timeout=${TIMEOUT}s; then
  echo "ROLLOUT FAILED — initiating automatic rollback"
  kubectl rollout undo deployment/$DEPLOYMENT -n $NAMESPACE
  kubectl rollout status deployment/$DEPLOYMENT -n $NAMESPACE --timeout=120s
  echo "Rollback complete. Notifying team..."
  exit 1
fi

echo "Rollout successful"

GitHub Actions with Auto-Rollback

- name: Deploy
  run: kubectl set image deployment/my-app app=my-app:${{ github.sha }} -n production

name: Wait for rollout
  id: rollout

  run: kubectl rollout status deployment/my-app -n production --timeout=300s
  continue-on-error: true

name: Auto-rollback on failure
  if: steps.rollout.outcome == 'failure'

  run: |
    kubectl rollout undo deployment/my-app -n production
    kubectl rollout status deployment/my-app -n production --timeout=120s

name: Notify on rollback
  if: steps.rollout.outcome == 'failure'

  uses: slackapi/slack-github-action@v1.26.0
  with:
    payload: |
      {"text": "Deployment of my-app was automatically rolled back. SHA: ${{ github.sha }}"}

Post-Rollback Checklist

# 1. Verify rollback is healthy
kubectl get pods -l app=my-app -n production
kubectl rollout status deployment/my-app -n production

# 2. Confirm correct image version
kubectl get deployment my-app -n production \
  -o jsonpath='{.spec.template.spec.containers[0].image}'

# 3. Verify user-facing health
curl -sf https://api.yourcompany.com/health

# 4. Check error rates returned to normal (Grafana/Prometheus)

# 5. Document the incident

Prevention: Deploy Safely

---

Frequently Asked Questions

How fast is a Kubernetes rollback?

A kubectl rollback is near-instant to initiate. Actual completion depends on your rolling update configuration and how quickly pods pass readiness probes. With maxUnavailable: 0 and fast health checks, expect 30-60 seconds for a full rollback.

Does rollback cause downtime?

No, if configured properly. With maxUnavailable: 0, Kubernetes ensures old (good) pods keep running until rollback pods are ready. Traffic shifts gradually from bad pods to good pods.

How many revisions should I keep?

Set revisionHistoryLimit to at least 5-10. Old ReplicaSets are lightweight (just metadata), so storage cost is minimal. More history gives more rollback options during incidents.

What's the difference between kubectl rollout undo and helm rollback?

kubectl rollout undo only reverts the Deployment (images, env vars). helm rollback reverts all resources in the release (ConfigMaps, Secrets, Services, Ingress). If your failure involves config changes beyond the Deployment, use Helm rollback.

How do I prevent bad deployments from reaching production?

Use progressive delivery: deploy to a small traffic percentage first, run automated health analysis, and auto-promote or auto-rollback based on metrics. Argo Rollouts with AnalysisTemplates is the standard approach.

---

Related Resources

• ArgoCD GitOps Complete Guide — GitOps deployment and sync strategies
• Zero Downtime Deployment Strategies — Blue-green, canary, and rolling updates
• Helm Charts Production Guide — Helm best practices
• Kubernetes Pod Troubleshooting — Debug failing pods after deploy