Backend interview prep

Start

Production Kubernetes Interview Questions (Top 100 Questions with Answers)

Master Production Kubernetes Interview Questions with practical explanations covering cluster architecture, high availability, autoscaling, observability, security, upgrades, disaster recovery, GitOps, resource management, networking, storage, deployment strategies, troubleshooting, and enterprise Kubernetes operations.

Module Navigation

Previous: Container Security QA | Parent: Containers Learning Path | Next: Serverless

Introduction

Creating a Kubernetes cluster is relatively easy.

Running Kubernetes reliably in production requires much more than deploying Pods and Services. Production Kubernetes teams must design for:

  • High availability
  • Security
  • Resource efficiency
  • Application resilience
  • Observability
  • Controlled deployments
  • Cluster upgrades
  • Disaster recovery
  • Cost optimization
  • Incident response
  • Governance
  • Multi-team operations

A typical production Kubernetes platform looks like this:

Developers
    │
    ▼
Git Repository
    │
    ▼
CI Pipeline
    │
    ▼
Container Registry
    │
    ▼
GitOps Controller
    │
    ▼
Kubernetes Cluster
    │
    ├── Ingress
    ├── Applications
    ├── Autoscaling
    ├── Security Policies
    ├── Observability
    └── Persistent Storage

This guide contains the Top 100 Production Kubernetes Interview Questions with real-world explanations, architecture diagrams, troubleshooting approaches, best practices, and senior-level interview follow-ups.

Production Kubernetes Learning Roadmap

Kubernetes Fundamentals
         │
         ▼
Highly Available Cluster
         │
         ▼
Workload Reliability
         │
         ▼
Networking and Storage
         │
         ▼
Security and Governance
         │
         ▼
Observability
         │
         ▼
Autoscaling
         │
         ▼
Deployment Automation
         │
         ▼
Upgrades and Maintenance
         │
         ▼
Disaster Recovery
         │
         ▼
Production Operations

Production Architecture Fundamentals

1. What makes a Kubernetes cluster production-ready?

A production-ready Kubernetes cluster should provide:

  • Highly available control plane
  • Multiple worker nodes
  • Multi-zone deployment
  • Secure API access
  • Workload isolation
  • Resource requests and limits
  • Health probes
  • Autoscaling
  • Centralized logs, metrics, and traces
  • Backup and recovery
  • Automated deployment
  • Upgrade strategy
  • Security policies
  • Incident alerting
  • Cost governance

Production readiness is not only about cluster availability. It also includes application reliability, security, operations, and recoverability.

2. What are the main layers of a production Kubernetes platform?

Application Layer
       │
Platform Services
       │
Kubernetes Workloads
       │
Cluster Control Plane
       │
Worker Nodes
       │
Cloud or Physical Infrastructure

The layers commonly include:

  1. Cloud or datacenter infrastructure
  2. Kubernetes control plane
  3. Worker nodes
  4. Networking and storage
  5. Platform services
  6. Application workloads
  7. Observability and security
  8. CI/CD and GitOps

3. Should development and production workloads run in the same cluster?

Usually, separate clusters are preferred for production and non-production.

Benefits include:

  • Stronger failure isolation
  • Reduced security risk
  • Independent upgrade schedules
  • Separate capacity management
  • Easier compliance
  • Reduced accidental production impact

Namespaces provide logical isolation, but they do not provide the same failure boundary as separate clusters.

4. When can multiple environments share one cluster?

Sharing may be acceptable when:

  • Workloads are low risk
  • Strong namespace isolation exists
  • Resource quotas are enforced
  • NetworkPolicies are configured
  • RBAC is properly separated
  • Cost is a major concern
  • Compliance does not require isolation

A common strategy is:

Development Cluster
   ├── Dev Namespace
   ├── Test Namespace
   └── Integration Namespace

Production Cluster
   ├── Application A
   ├── Application B
   └── Shared Platform Services

5. What is the shared responsibility model in managed Kubernetes?

In managed services such as EKS, AKS, and GKE:

Cloud provider commonly manages

  • Control-plane infrastructure
  • API Server availability
  • etcd platform maintenance
  • Control-plane patching
  • Some cluster integrations

Customer commonly manages

  • Worker nodes
  • Kubernetes versions
  • Workload security
  • RBAC
  • NetworkPolicies
  • Application availability
  • Container images
  • Secrets
  • Observability
  • Backup and recovery

Managed Kubernetes reduces operational work but does not eliminate platform responsibility.

High Availability

6. How do you design a highly available Kubernetes cluster?

Use:

  • Multiple control-plane nodes
  • Odd-numbered etcd cluster
  • Multiple worker nodes
  • Multiple Availability Zones
  • Redundant load balancers
  • Replicated DNS
  • Pod anti-affinity
  • Topology spread constraints
  • PodDisruptionBudgets
  • Highly available storage
  • Multiple Ingress Controller replicas
                  Load Balancer
                        │
        ┌───────────────┼───────────────┐
        ▼               ▼               ▼
 Control Plane 1  Control Plane 2  Control Plane 3
        │               │               │
        └───────────────┼───────────────┘
                        ▼
                    etcd Cluster

7. Why should etcd have an odd number of members?

etcd uses quorum-based consensus.

An odd number of members provides efficient fault tolerance.

etcd Members Quorum Failures Tolerated
1 1 0
3 2 1
5 3 2
7 4 3

Adding one member to an odd-sized cluster does not immediately increase failure tolerance.

8. Why should worker nodes span multiple Availability Zones?

Multi-zone worker nodes protect workloads from:

  • Rack failure
  • Power failure
  • Network failure
  • Datacenter failure
  • Zone-level maintenance

Pods should also be distributed across those zones.

9. What is Pod anti-affinity?

Pod anti-affinity prevents matching Pods from being scheduled together.

Example:

affinity:
  podAntiAffinity:
    preferredDuringSchedulingIgnoredDuringExecution:
      - weight: 100
        podAffinityTerm:
          topologyKey: kubernetes.io/hostname
          labelSelector:
            matchLabels:
              app: payment-api

This helps prevent all replicas from running on the same node.

10. What are topology spread constraints?

Topology spread constraints distribute Pods across failure domains such as nodes or zones.

topologySpreadConstraints:
  - maxSkew: 1
    topologyKey: topology.kubernetes.io/zone
    whenUnsatisfiable: DoNotSchedule
    labelSelector:
      matchLabels:
        app: payment-api

They provide more predictable distribution than simple anti-affinity.

PodDisruptionBudgets and Maintenance

11. What is a PodDisruptionBudget?

A PodDisruptionBudget limits how many application Pods may be voluntarily disrupted simultaneously.

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: payment-api
spec:
  minAvailable: 2
  selector:
    matchLabels:
      app: payment-api

It protects against voluntary disruptions such as:

  • Node drains
  • Cluster upgrades
  • Node autoscaling
  • Planned maintenance

12. Does a PodDisruptionBudget protect against node failure?

No.

It applies mainly to voluntary disruptions.

It cannot prevent disruption caused by:

  • Hardware failure
  • Kernel crash
  • Power outage
  • Node network failure
  • Forced Pod deletion

13. What is the difference between minAvailable and maxUnavailable?

minAvailable: 2

Means at least two matching Pods should remain available.

maxUnavailable: 1

Means no more than one matching Pod should be unavailable.

Use one approach, not both, in a single PodDisruptionBudget.

14. Why can a badly configured PDB block cluster upgrades?

If a PDB requires more available Pods than the cluster can maintain, node draining may fail.

Example:

Deployment Replicas: 2
PDB minAvailable: 2
Node Drain Requires Eviction
Result: Eviction Blocked

PDBs must align with replica counts and cluster capacity.

15. What is node draining?

Node draining safely evicts workloads before node maintenance.

kubectl drain worker-node-1 \
  --ignore-daemonsets \
  --delete-emptydir-data

After maintenance:

kubectl uncordon worker-node-1

Application Reliability

16. What are the minimum requirements for a reliable production Deployment?

A reliable Deployment should normally include:

  • Multiple replicas
  • Resource requests and limits
  • Readiness probe
  • Liveness probe
  • Startup probe when required
  • RollingUpdate strategy
  • Graceful shutdown
  • PodDisruptionBudget
  • Anti-affinity or topology spreading
  • Versioned image
  • SecurityContext
  • Monitoring labels

17. Why are readiness probes critical in production?

Readiness probes determine whether a Pod should receive traffic.

A failed readiness probe removes the Pod from Service endpoints.

This prevents traffic from reaching:

  • Starting applications
  • Unhealthy applications
  • Temporarily unavailable applications
  • Applications with unavailable required dependencies

18. Why are liveness probes dangerous when configured incorrectly?

An aggressive liveness probe can repeatedly restart an application before it has time to recover or start.

Application Starting
       │
Liveness Fails
       │
Container Restarted
       │
Application Starts Again
       │
Liveness Fails Again

Use startup probes for slow-starting applications.

19. What is graceful Pod termination?

Graceful termination allows applications to finish active work before shutdown.

Recommended flow:

Pod Deletion
     │
     ▼
Pod Removed from Ready Endpoints
     │
     ▼
preStop Hook
     │
     ▼
SIGTERM
     │
     ▼
Application Stops Accepting Requests
     │
     ▼
In-Flight Requests Complete
     │
     ▼
Container Exits

20. How do you configure graceful shutdown?

spec:
  terminationGracePeriodSeconds: 45

  containers:
    - name: payment-api

      lifecycle:
        preStop:
          exec:
            command:
              - sh
              - -c
              - sleep 10

The application should handle SIGTERM and stop accepting new requests.

Resource Management

21. Why are resource requests important?

Requests tell the Scheduler how much CPU and memory a container requires.

resources:
  requests:
    cpu: 250m
    memory: 256Mi

Without requests:

  • Scheduling becomes inaccurate.
  • Nodes may be overloaded.
  • Autoscaling becomes less reliable.
  • Workloads may compete unpredictably.

22. Why are resource limits important?

Limits prevent one container from consuming excessive resources.

resources:
  limits:
    cpu: "1"
    memory: 512Mi

However, limits must be chosen carefully to avoid:

  • CPU throttling
  • OOM kills
  • Poor latency
  • Underutilized nodes

23. What happens when a container exceeds its CPU limit?

The container is throttled.

The Linux scheduler restricts CPU consumption, which may increase application latency.

24. What happens when a container exceeds its memory limit?

The process may be terminated by the kernel and reported as:

OOMKilled

Kubernetes may restart the container based on the Pod restart policy.

25. What are Kubernetes Quality of Service classes?

The three QoS classes are:

  • Guaranteed
  • Burstable
  • BestEffort

Guaranteed

Every container has equal CPU and memory requests and limits.

Burstable

At least one request or limit exists, but the Guaranteed requirements are not met.

BestEffort

No CPU or memory requests or limits are defined.

During resource pressure, BestEffort Pods are usually evicted first.

26. What is a LimitRange?

A LimitRange defines default or minimum and maximum resource constraints within a namespace.

apiVersion: v1
kind: LimitRange
metadata:
  name: application-limits
spec:
  limits:
    - type: Container
      defaultRequest:
        cpu: 100m
        memory: 128Mi
      default:
        cpu: 500m
        memory: 512Mi

27. What is a ResourceQuota?

A ResourceQuota limits total namespace resource consumption.

apiVersion: v1
kind: ResourceQuota
metadata:
  name: payments-quota
spec:
  hard:
    requests.cpu: "20"
    requests.memory: 40Gi
    limits.cpu: "40"
    limits.memory: 80Gi
    pods: "100"

28. Why should namespace quotas be used?

They prevent one team or application from consuming all cluster resources.

They also support:

  • Cost allocation
  • Capacity planning
  • Multi-team governance
  • Resource fairness
  • Environment isolation

Autoscaling

29. What is the Horizontal Pod Autoscaler?

The Horizontal Pod Autoscaler changes the number of Pod replicas based on observed metrics.

Metrics
   │
   ▼
HPA Controller
   │
   ▼
Desired Replica Count
   │
   ▼
Deployment

30. Show an HPA example.

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: payment-api
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: payment-api

  minReplicas: 3
  maxReplicas: 20

  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70

31. Why does HPA depend on resource requests?

CPU utilization is commonly calculated relative to requested CPU.

If requests are missing or inaccurate, HPA behavior may become unreliable.

32. What is the Cluster Autoscaler?

Cluster Autoscaler changes the number of worker nodes.

It:

  • Adds nodes when Pods cannot be scheduled
  • Removes underutilized nodes when workloads can move elsewhere

33. HPA vs Cluster Autoscaler?

HPA Cluster Autoscaler
Scales Pods Scales nodes
Uses workload metrics Uses scheduling and node-utilization signals
Works at application level Works at infrastructure level
Changes replicas Changes machine count

They are commonly used together.

34. What is the Vertical Pod Autoscaler?

VPA recommends or applies updated CPU and memory requests based on historical usage.

It may require Pod recreation to apply changes.

35. Can HPA and VPA be used together?

Yes, but not normally for the same resource metric in conflicting ways.

A common approach is:

  • HPA scales replicas using CPU or business metrics.
  • VPA recommends memory and CPU requests.
  • VPA recommendation mode avoids unexpected restarts.

36. What is KEDA?

KEDA provides event-driven Kubernetes autoscaling.

It can scale workloads based on:

  • Kafka lag
  • Queue length
  • Service Bus messages
  • Prometheus metrics
  • Database queries
  • Cloud events
  • Scheduled triggers

37. Why can autoscaling cause instability?

Poor configuration may cause:

  • Rapid scale-out and scale-in
  • Unnecessary Pod creation
  • Database connection spikes
  • Cache pressure
  • Cost increases
  • Cold-start latency
  • Node provisioning delays

Use stabilization windows, sensible thresholds, and load testing.

38. What metrics should be used for autoscaling?

Use metrics that represent actual workload demand.

Examples:

Workload Useful Metric
HTTP API Requests per second or latency
Kafka consumer Consumer lag
Queue worker Queue depth
CPU-heavy service CPU utilization
Memory-heavy service Memory working set
Batch processor Pending jobs
GPU workload GPU utilization

Networking

39. What are the major production Kubernetes networking components?

  • CNI plugin
  • Pod network
  • Services
  • CoreDNS
  • NetworkPolicies
  • Ingress Controller
  • External load balancer
  • Service mesh where required
  • Private endpoints
  • Egress controls

40. Why should NetworkPolicies be used?

NetworkPolicies restrict Pod communication.

Without them, many clusters permit broad east-west traffic.

Recommended strategy:

Default Deny
     │
     ▼
Explicitly Allow Required Traffic

41. Show a namespace default-deny policy.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny
  namespace: payments
spec:
  podSelector: {}
  policyTypes:
    - Ingress
    - Egress

42. What should be considered before enabling default-deny egress?

Required dependencies must be identified, including:

  • DNS
  • Databases
  • External APIs
  • Cloud metadata endpoints
  • Secret managers
  • Monitoring collectors
  • Time synchronization
  • Container registries where needed

Incorrect egress restrictions can break application functionality.

43. What is CoreDNS?

CoreDNS provides Kubernetes service discovery and DNS resolution.

Example Service name:

payment-api.payments.svc.cluster.local

Production DNS monitoring should include:

  • Query latency
  • Error rate
  • Cache performance
  • Pod CPU and memory
  • Upstream DNS health

44. What causes intermittent DNS problems in Kubernetes?

Common causes include:

  • CoreDNS overload
  • Node DNS issues
  • CNI packet loss
  • Incorrect search domains
  • Excessive DNS queries
  • Upstream DNS latency
  • NetworkPolicy blocking DNS
  • Conntrack exhaustion

45. What is an Ingress Controller?

An Ingress Controller implements HTTP and HTTPS routing rules defined through Ingress resources.

For production, use:

  • Multiple replicas
  • PodDisruptionBudget
  • Anti-affinity
  • TLS automation
  • WAF where required
  • Rate limiting
  • Centralized logging
  • Separate public and private controllers

46. Why might a cluster use separate public and internal Ingress Controllers?

This allows:

  • Public applications to use internet-facing load balancers
  • Internal applications to use private load balancers
  • Separate security policies
  • Separate IngressClasses
  • Reduced accidental exposure

47. What is the Kubernetes Gateway API?

Gateway API is a newer traffic-management model supporting resources such as:

  • GatewayClass
  • Gateway
  • HTTPRoute
  • GRPCRoute
  • TCPRoute
  • TLSRoute

It provides more expressive and role-based networking than traditional Ingress.

48. What is egress control?

Egress control restricts outbound workload communication.

It helps prevent:

  • Data exfiltration
  • Unapproved external connections
  • Malware command-and-control traffic
  • Unexpected cloud-service access

Controls may include:

  • NetworkPolicies
  • Egress gateways
  • Firewalls
  • Service mesh
  • DNS filtering
  • Proxy enforcement

Storage and Stateful Workloads

49. Should databases run inside Kubernetes?

They can, but the decision depends on operational maturity.

Consider:

  • Backup support
  • Replication
  • Storage performance
  • Failure recovery
  • Operator maturity
  • Upgrade complexity
  • Compliance
  • Cloud-managed database alternatives

For many teams, managed cloud databases reduce operational risk.

50. What is a StatefulSet?

A StatefulSet manages stateful Pods with:

  • Stable names
  • Stable network identities
  • Ordered deployment
  • Ordered termination
  • Persistent storage association

Example Pod names:

database-0
database-1
database-2

51. StatefulSet vs Deployment?

Deployment StatefulSet
Stateless workloads Stateful workloads
Interchangeable Pod identities Stable Pod identities
Random Pod names Ordered Pod names
Shared or no persistent storage Per-Pod storage commonly used
Parallel lifecycle Ordered lifecycle by default

52. What is a StorageClass?

A StorageClass defines how dynamic persistent storage is provisioned.

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: production-ssd
provisioner: csi.example.com
reclaimPolicy: Retain
volumeBindingMode: WaitForFirstConsumer

53. What is WaitForFirstConsumer?

It delays volume provisioning until a Pod using the claim is scheduled.

This helps select storage in the correct zone.

54. What is a reclaim policy?

Common reclaim policies include:

  • Delete
  • Retain

Delete

Deletes the underlying storage after the PVC and PV are removed.

Retain

Preserves the underlying storage for manual recovery or reuse.

Production data often requires careful use of Retain.

55. Is a PersistentVolume snapshot a complete application backup?

Not necessarily.

Storage snapshots may not guarantee application consistency.

Databases may require:

  • Quiescing writes
  • Transaction-consistent backup
  • Database-native backup
  • Log backup
  • Recovery testing

56. How should stateful workloads be backed up?

Use a layered backup strategy:

  • Database-native backups
  • Persistent-volume snapshots
  • Kubernetes resource backup
  • Cross-region copy
  • Encryption
  • Retention policies
  • Regular restore testing

Observability

57. What are the three pillars of Kubernetes observability?

  • Metrics
  • Logs
  • Traces
Application and Cluster
          │
    ┌─────┼─────┐
    ▼     ▼     ▼
 Metrics Logs Traces

58. What are the four golden signals?

The four golden signals are:

  • Latency
  • Traffic
  • Errors
  • Saturation

They provide a strong starting point for application monitoring.

59. What cluster-level metrics should be monitored?

Monitor:

  • Node readiness
  • Node CPU and memory
  • Disk pressure
  • Memory pressure
  • PID pressure
  • Pod scheduling failures
  • API Server latency
  • etcd latency and storage
  • Control-plane errors
  • CoreDNS health
  • Persistent-volume usage
  • Network errors
  • Container restarts

60. What application-level metrics should be monitored?

Monitor:

  • Request latency
  • Request rate
  • Error rate
  • Availability
  • Queue length
  • Thread usage
  • Database connections
  • Cache hit rate
  • Business transaction success
  • Dependency latency

61. Why are business metrics important?

Infrastructure may appear healthy while the business process is failing.

Examples include:

  • Payment failures
  • Order-processing failures
  • Login failures
  • Message-processing backlog
  • File-processing errors
  • Customer-registration failures

Production monitoring should combine technical and business metrics.

62. What tools are commonly used for Kubernetes metrics?

Common options include:

  • Prometheus
  • Grafana
  • OpenTelemetry
  • CloudWatch Container Insights
  • Azure Monitor Container Insights
  • Google Cloud Monitoring
  • Datadog
  • Dynatrace
  • New Relic

63. How should container logs be collected?

Applications should normally write logs to:

  • stdout
  • stderr

A node-level collector forwards them to centralized storage.

Typical flow:

Container Logs
      │
      ▼
Node Log Collector
      │
      ▼
Central Log Platform
      │
      ▼
Search, Alerts, Retention

64. Why should logs include correlation IDs?

Correlation IDs connect events across:

  • API gateway
  • Ingress
  • Microservices
  • Databases
  • Messaging
  • External systems

They help trace a request across distributed systems.

65. What is distributed tracing?

Distributed tracing follows a request across multiple services.

It helps identify:

  • Slow dependencies
  • Failed downstream calls
  • Service bottlenecks
  • Retry storms
  • Network latency

OpenTelemetry provides a vendor-neutral instrumentation standard.

66. What should Kubernetes alerts focus on?

Alerts should identify actionable problems.

Examples:

  • Application error rate above threshold
  • No ready replicas
  • CrashLoopBackOff increase
  • Persistent volume almost full
  • Node unavailable
  • Certificate near expiration
  • HPA at maximum replicas
  • API Server high latency
  • etcd backup failure
  • Queue backlog increasing

Avoid alerting on every low-value event.

Security and Governance

67. What are the minimum security controls for production Kubernetes?

  • Private or restricted API Server
  • Strong identity integration
  • Least-privilege RBAC
  • Pod Security Admission
  • Non-root containers
  • Default-deny NetworkPolicies
  • External secret management
  • Image scanning
  • Signed images
  • Admission policies
  • etcd encryption
  • Audit logging
  • Node patching
  • Runtime threat detection
  • Namespace isolation

68. Why should cluster-admin be restricted?

cluster-admin can manage nearly every resource in the cluster.

Excessive use increases the risk of:

  • Accidental deletion
  • Unauthorized secret access
  • Policy bypass
  • Privileged workload creation
  • Full cluster compromise

Use role-specific permissions and just-in-time access.

69. What is Pod Security Admission?

Pod Security Admission enforces Pod Security Standards at namespace level.

Common production configuration:

metadata:
  labels:
    pod-security.kubernetes.io/enforce: restricted
    pod-security.kubernetes.io/audit: restricted
    pod-security.kubernetes.io/warn: restricted

70. Why should ServiceAccount tokens be disabled when unused?

Many applications do not need Kubernetes API access.

Disable automatic token mounting:

automountServiceAccountToken: false

This reduces credential exposure if the Pod is compromised.

71. How should application secrets be managed?

Prefer:

  • Cloud secret managers
  • HashiCorp Vault
  • External Secrets Operator
  • Secrets Store CSI Driver
  • Short-lived credentials
  • Workload identity
  • Automated rotation

Do not store secrets in:

  • Container images
  • Git
  • Plain-text Helm values
  • Source code
  • Logs

72. What is workload identity?

Workload identity allows Pods to access cloud services without static access keys.

Examples include:

  • IAM roles for service accounts
  • Azure workload identity
  • GKE workload identity
Pod ServiceAccount
        │
        ▼
Cloud Identity Mapping
        │
        ▼
Temporary Credentials
        │
        ▼
Cloud Service

73. What are admission policies?

Admission policies validate Kubernetes resources before they are stored.

They can enforce:

  • Approved registries
  • Non-root execution
  • Resource limits
  • Image signatures
  • Required labels
  • Blocked hostPath mounts
  • Restricted capabilities
  • Required NetworkPolicies

74. Which tools are used for Kubernetes policy enforcement?

Common tools include:

  • ValidatingAdmissionPolicy
  • Kyverno
  • Open Policy Agent Gatekeeper
  • Cloud-provider policy services
  • Security platform admission controls

75. What is Kubernetes audit logging?

Audit logging records requests made to the Kubernetes API.

It can capture:

  • User identity
  • Requested resource
  • Action performed
  • Source address
  • Request result
  • Timestamp

Audit logs are important for:

  • Security investigations
  • Compliance
  • Change tracking
  • Incident response

CI/CD and GitOps

76. What should a Kubernetes CI pipeline include?

Source Commit
      │
      ▼
Unit Tests
      │
      ▼
Static Analysis
      │
      ▼
Dependency Scan
      │
      ▼
Container Build
      │
      ▼
Image Scan
      │
      ▼
SBOM and Signing
      │
      ▼
Manifest Validation
      │
      ▼
Artifact Publication

Deployment can then occur through CD or GitOps.

77. What is GitOps?

GitOps uses Git as the source of truth for Kubernetes configuration.

A controller compares Git with cluster state and reconciles differences.

Common tools include:

  • Argo CD
  • Flux

78. What are the benefits of GitOps?

  • Declarative deployments
  • Audit trail
  • Pull-request approval
  • Drift detection
  • Easier rollback
  • Separation of CI and cluster credentials
  • Consistent environments
  • Automated reconciliation

79. Why should production deployments avoid kubectl apply from developer laptops?

Risks include:

  • Unreviewed changes
  • Missing audit trail
  • Local configuration differences
  • Excessive credentials
  • Configuration drift
  • Difficult rollback
  • Accidental environment selection

Use controlled CI/CD or GitOps workflows.

80. What is configuration drift?

Drift occurs when actual cluster state differs from declared configuration.

Examples:

  • Manual replica change
  • Image updated directly
  • Resource deleted manually
  • ConfigMap edited in the cluster
  • Security policy bypassed

GitOps tools detect and optionally correct drift.

Deployment Strategies

81. What is a rolling deployment?

A rolling deployment gradually replaces old Pods with new Pods.

strategy:
  type: RollingUpdate
  rollingUpdate:
    maxSurge: 1
    maxUnavailable: 0

82. What is a blue-green deployment?

Two complete environments are maintained:

Blue Environment  → Current Production
Green Environment → New Version

After validation, traffic switches from blue to green.

Benefits:

  • Fast rollback
  • Isolated validation

Trade-off:

  • Higher temporary infrastructure cost

83. What is a canary deployment?

A canary deployment sends a small percentage of traffic to a new version.

Example:

Version 1 → 95%
Version 2 → 5%

Traffic increases after validation.

84. Which tools support progressive delivery?

Common options include:

  • Argo Rollouts
  • Flagger
  • Service meshes
  • Ingress Controller traffic splitting
  • Gateway API implementations
  • Cloud load balancers

85. What metrics should determine canary success?

Use:

  • Error rate
  • Latency
  • Availability
  • Resource usage
  • Business success rate
  • Dependency failures
  • Customer-impact metrics

A canary should automatically stop or roll back when thresholds fail.

86. How should database migrations be handled during deployments?

Use backward-compatible migrations.

Recommended approach:

Expand Schema
      │
      ▼
Deploy Compatible Application
      │
      ▼
Migrate Data
      │
      ▼
Remove Old Schema Later

Avoid destructive schema changes before all application versions are upgraded.

Cluster Upgrades

87. Why are Kubernetes upgrades important?

Supported versions receive:

  • Security fixes
  • Bug fixes
  • Compatibility improvements
  • Cloud-provider support

Running unsupported versions increases operational and security risk.

88. What should be checked before a Kubernetes upgrade?

Check:

  • Deprecated APIs
  • Admission webhooks
  • CNI compatibility
  • CSI compatibility
  • Ingress compatibility
  • Monitoring stack
  • Autoscaler compatibility
  • Operators
  • CustomResourceDefinitions
  • Node operating systems
  • Application PodDisruptionBudgets
  • Backup status

89. What is the safest cluster upgrade sequence?

A typical sequence is:

Backup Cluster State
       │
       ▼
Test in Non-Production
       │
       ▼
Upgrade Control Plane
       │
       ▼
Upgrade Add-ons
       │
       ▼
Upgrade Worker Nodes Gradually
       │
       ▼
Validate Applications
       │
       ▼
Monitor Closely

90. How should worker nodes be upgraded?

Use rolling replacement:

  1. Add upgraded nodes.
  2. Cordon old node.
  3. Drain workloads.
  4. Validate rescheduling.
  5. Remove old node.
  6. Repeat gradually.

Managed node groups simplify this process.

91. What are deprecated Kubernetes APIs?

Deprecated APIs are older resource versions scheduled for removal.

Before upgrading, scan manifests and cluster resources for removed API versions.

Examples of affected resources over Kubernetes history include:

  • Ingress versions
  • PodSecurityPolicy
  • CronJob versions
  • HorizontalPodAutoscaler versions

92. What is a Kubernetes upgrade rollback plan?

A rollback plan should include:

  • Control-plane recovery approach
  • Node-image rollback
  • Application rollback
  • etcd backup
  • Add-on version rollback
  • Restore procedures
  • Maintenance communication
  • Tested emergency process

Some managed control-plane upgrades cannot simply be downgraded, making pre-upgrade validation essential.

Backup and Disaster Recovery

93. What should be backed up in Kubernetes?

Back up:

  • Kubernetes resource definitions
  • etcd where self-managed
  • Persistent application data
  • Database-native backups
  • Secrets where permitted
  • Certificates
  • CustomResourceDefinitions
  • GitOps configuration
  • Cluster configuration
  • Cloud load-balancer and DNS configuration

94. Is etcd backup enough for disaster recovery?

No.

etcd stores Kubernetes state but not necessarily:

  • Persistent-volume data
  • External databases
  • Object storage
  • External DNS
  • Cloud resources
  • Container registry artifacts
  • External secret-manager contents

A complete DR strategy must cover dependencies outside Kubernetes.

95. What are RPO and RTO?

Recovery Point Objective

Maximum acceptable data loss measured in time.

Example:

RPO = 15 minutes

Recovery Time Objective

Maximum acceptable service restoration time.

Example:

RTO = 1 hour

Backup frequency and DR architecture should be based on RPO and RTO.

96. What are common Kubernetes disaster-recovery strategies?

  • Backup and restore
  • Pilot-light cluster
  • Warm standby
  • Active-passive multi-region
  • Active-active multi-region

The correct choice depends on:

  • Business criticality
  • RPO
  • RTO
  • Cost
  • Data architecture
  • Operational complexity

97. Why must Kubernetes restores be tested?

A successful backup does not guarantee successful recovery.

Restore tests validate:

  • Backup completeness
  • Application startup
  • Storage recovery
  • Secret availability
  • DNS configuration
  • External dependencies
  • Recovery timing
  • Team procedures

Troubleshooting and Operations

98. What is a structured approach to Kubernetes production troubleshooting?

Use this sequence:

Confirm User Impact
       │
       ▼
Check Recent Changes
       │
       ▼
Review Application Metrics
       │
       ▼
Review Cluster Metrics
       │
       ▼
Inspect Kubernetes Events
       │
       ▼
Inspect Pod Status and Logs
       │
       ▼
Check Networking and Storage
       │
       ▼
Mitigate Impact
       │
       ▼
Identify Root Cause

Useful commands:

kubectl get pods -A
kubectl get events -A --sort-by=.lastTimestamp
kubectl describe pod <pod-name>
kubectl logs <pod-name> --previous
kubectl top nodes
kubectl top pods -A

99. What are common production Kubernetes failures?

Common failures include:

  • CrashLoopBackOff
  • ImagePullBackOff
  • Pending Pods
  • OOMKilled containers
  • CPU throttling
  • Failed readiness probes
  • DNS failures
  • CNI issues
  • Persistent-volume attachment failures
  • Certificate expiration
  • Node pressure
  • Incorrect autoscaling
  • Failed admission webhooks
  • NetworkPolicy errors
  • Control-plane API latency
  • Configuration drift

Use:

  • Managed or highly available control plane
  • Multi-zone worker nodes
  • Separate system and application node pools
  • Private cluster networking
  • Public and internal Ingress separation
  • Versioned, scanned, and signed images
  • GitOps deployment
  • Resource requests and limits
  • Health probes
  • HPA and Cluster Autoscaler
  • PodDisruptionBudgets
  • Topology spread constraints
  • Default-deny NetworkPolicies
  • Restricted Pod Security
  • Workload identity
  • External secret management
  • Centralized metrics, logs, and traces
  • Automated backups
  • Tested disaster recovery
  • Controlled cluster upgrades
  • Cost allocation and governance

Production Kubernetes Deployment Example

apiVersion: apps/v1
kind: Deployment
metadata:
  name: payment-api
  namespace: payments
  labels:
    app: payment-api
spec:
  replicas: 3
  revisionHistoryLimit: 5

  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0

  selector:
    matchLabels:
      app: payment-api

  template:
    metadata:
      labels:
        app: payment-api
        version: v1

    spec:
      serviceAccountName: payment-api
      automountServiceAccountToken: false
      terminationGracePeriodSeconds: 45

      securityContext:
        runAsNonRoot: true
        runAsUser: 10001
        runAsGroup: 10001
        seccompProfile:
          type: RuntimeDefault

      topologySpreadConstraints:
        - maxSkew: 1
          topologyKey: topology.kubernetes.io/zone
          whenUnsatisfiable: DoNotSchedule
          labelSelector:
            matchLabels:
              app: payment-api

        - maxSkew: 1
          topologyKey: kubernetes.io/hostname
          whenUnsatisfiable: ScheduleAnyway
          labelSelector:
            matchLabels:
              app: payment-api

      containers:
        - name: payment-api
          image: registry.example.com/payment-api:3.5.1
          imagePullPolicy: IfNotPresent

          ports:
            - name: http
              containerPort: 8080

          securityContext:
            allowPrivilegeEscalation: false
            privileged: false
            readOnlyRootFilesystem: true
            capabilities:
              drop:
                - ALL

          resources:
            requests:
              cpu: 250m
              memory: 256Mi
            limits:
              cpu: "1"
              memory: 512Mi

          startupProbe:
            httpGet:
              path: /actuator/health/liveness
              port: http
            failureThreshold: 30
            periodSeconds: 10

          readinessProbe:
            httpGet:
              path: /actuator/health/readiness
              port: http
            periodSeconds: 10
            failureThreshold: 3

          livenessProbe:
            httpGet:
              path: /actuator/health/liveness
              port: http
            periodSeconds: 10
            failureThreshold: 3

          lifecycle:
            preStop:
              exec:
                command:
                  - sh
                  - -c
                  - sleep 10

          volumeMounts:
            - name: temporary-data
              mountPath: /tmp

      volumes:
        - name: temporary-data
          emptyDir: {}

Production Service

apiVersion: v1
kind: Service
metadata:
  name: payment-api
  namespace: payments
spec:
  type: ClusterIP

  selector:
    app: payment-api

  ports:
    - name: http
      port: 80
      targetPort: http

Production HorizontalPodAutoscaler

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: payment-api
  namespace: payments
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: payment-api

  minReplicas: 3
  maxReplicas: 20

  behavior:
    scaleUp:
      stabilizationWindowSeconds: 0
      policies:
        - type: Percent
          value: 100
          periodSeconds: 60
        - type: Pods
          value: 4
          periodSeconds: 60
      selectPolicy: Max

    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
        - type: Percent
          value: 25
          periodSeconds: 60

  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70

Production PodDisruptionBudget

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: payment-api
  namespace: payments
spec:
  minAvailable: 2

  selector:
    matchLabels:
      app: payment-api

Production NetworkPolicy

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: payment-api
  namespace: payments
spec:
  podSelector:
    matchLabels:
      app: payment-api

  policyTypes:
    - Ingress
    - Egress

  ingress:
    - from:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: ingress-nginx

      ports:
        - protocol: TCP
          port: 8080

  egress:
    - to:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: database

      ports:
        - protocol: TCP
          port: 5432

    - to:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: kube-system

      ports:
        - protocol: UDP
          port: 53
        - protocol: TCP
          port: 53

Production Platform Architecture

                        Global Users
                             │
                             ▼
                   DNS / Global Load Balancer
                             │
                             ▼
                      CDN / WAF / DDoS
                             │
                             ▼
                  Regional Load Balancer
                             │
                             ▼
                    Ingress Controller
                             │
          ┌──────────────────┼──────────────────┐
          ▼                  ▼                  ▼
    Orders Service     Payments Service      User Service
          │                  │                  │
          ▼                  ▼                  ▼
       Pods + HPA         Pods + HPA         Pods + HPA
          │                  │                  │
          └──────────────────┼──────────────────┘
                             ▼
                    Internal Platform Services
              ┌──────────────┼──────────────┐
              ▼              ▼              ▼
          Messaging        Cache        Databases
                             │
                             ▼
                    Observability Platform
                  Metrics + Logs + Traces

GitOps Production Flow

Application Source
       │
       ▼
CI Pipeline
       │
 ┌─────┼─────────────┐
 ▼     ▼             ▼
Test  Scan       Build Image
       │             │
       ▼             ▼
Generate SBOM    Sign Image
       │             │
       └──────┬──────┘
              ▼
      Private Registry
              │
              ▼
 Update Environment Git Repository
              │
              ▼
        GitOps Controller
              │
              ▼
        Kubernetes Cluster
              │
              ▼
      Automated Reconciliation

Kubernetes Upgrade Flow

Review Release Notes
        │
        ▼
Scan Deprecated APIs
        │
        ▼
Backup Cluster and Data
        │
        ▼
Test Non-Production
        │
        ▼
Upgrade Control Plane
        │
        ▼
Upgrade Platform Add-ons
        │
        ▼
Roll Worker Nodes
        │
        ▼
Validate Applications
        │
        ▼
Monitor and Close Change

Disaster-Recovery Architecture

                     Global DNS
                         │
              ┌──────────┴──────────┐
              ▼                     ▼
       Primary Region         Recovery Region
              │                     │
       Kubernetes Cluster     Standby Cluster
              │                     │
       Primary Database  ───► Replicated Database
              │                     │
       Object Storage    ───► Replicated Storage
              │
              ▼
        Backup Repository

Production Kubernetes Checklist

✓ Highly Available Control Plane
✓ Multi-Zone Worker Nodes
✓ Separate System and Application Node Pools
✓ Multiple Application Replicas
✓ Resource Requests and Limits
✓ Startup, Readiness, and Liveness Probes
✓ Graceful Shutdown
✓ PodDisruptionBudgets
✓ Topology Spread Constraints
✓ Horizontal Pod Autoscaler
✓ Cluster Autoscaler
✓ ResourceQuotas and LimitRanges
✓ Private Cluster Networking
✓ Public and Internal Ingress Separation
✓ Default-Deny NetworkPolicies
✓ Restricted Pod Security
✓ Least-Privilege RBAC
✓ Dedicated ServiceAccounts
✓ Workload Identity
✓ External Secret Management
✓ Image Scanning and Signing
✓ Admission Policy Enforcement
✓ Centralized Metrics
✓ Centralized Logs
✓ Distributed Tracing
✓ Actionable Alerts
✓ GitOps Deployment
✓ Tested Rollback Strategy
✓ Automated Cluster Backups
✓ Persistent-Data Backups
✓ Tested Disaster Recovery
✓ Controlled Upgrade Process
✓ Cost Monitoring and Allocation

Quick Revision

Topic Key Point
High Availability Multi-zone control plane and worker nodes
PodDisruptionBudget Limits voluntary disruption
Topology Spread Distributes replicas across failure domains
Resource Request Scheduler reservation
Resource Limit Maximum container consumption
HPA Scales Pod replicas
Cluster Autoscaler Scales worker nodes
VPA Adjusts resource requests
KEDA Event-driven autoscaling
NetworkPolicy Restricts Pod communication
Pod Security Admission Enforces Pod security standards
Workload Identity Cloud access without static keys
GitOps Git-driven reconciliation
Rolling Deployment Gradual version replacement
Canary Deployment Limited new-version traffic
Observability Metrics, logs, and traces
RPO Acceptable data-loss window
RTO Acceptable recovery duration
etcd Backup Protects Kubernetes cluster state
Cluster Upgrade Controlled control-plane and node update

Interview Tips

During Production Kubernetes interviews:

  • Explain Kubernetes as an operating platform rather than only a container scheduler.
  • Always discuss high availability across nodes, zones, networking, storage, and applications.
  • Mention resource requests, limits, quotas, probes, PDBs, and topology spreading.
  • Clearly distinguish HPA, VPA, Cluster Autoscaler, and KEDA.
  • Explain observability using metrics, logs, traces, golden signals, and business metrics.
  • Discuss default-deny NetworkPolicies, restricted Pod Security, RBAC, workload identity, and external secrets.
  • Explain GitOps, drift detection, controlled promotion, and rollback.
  • Be prepared to discuss rolling, blue-green, and canary deployments.
  • Describe cluster upgrades, deprecated API checks, node replacement, and add-on compatibility.
  • Explain that etcd backup alone is not a complete disaster-recovery strategy.
  • Use RPO and RTO to justify backup and multi-region designs.
  • Follow a structured incident-troubleshooting process and identify recent changes first.
  • Support answers with practical examples from Amazon EKS, Azure AKS, Google GKE, OpenShift, or self-managed Kubernetes.

Summary

Production Kubernetes requires much more than creating Pods and Deployments.

A reliable platform must combine:

  • High availability
  • Application resilience
  • Resource governance
  • Autoscaling
  • Secure networking
  • Persistent storage
  • Observability
  • GitOps
  • Deployment automation
  • Cluster upgrades
  • Backup
  • Disaster recovery
  • Security governance
  • Operational troubleshooting

Mastering these 100 Production Kubernetes interview questions prepares you for Kubernetes, OpenShift, Amazon EKS, Azure AKS, Google GKE, DevOps Engineer, Platform Engineer, Cloud Engineer, Site Reliability Engineer, Kubernetes Administrator, Technical Lead, Principal Engineer, and Solution Architect interviews.