Backend interview prep

Start

Idempotency Interview Questions and Answers

Learn Idempotency with interview questions, Mermaid diagrams, Spring Boot examples, Kafka integration, duplicate handling strategies, and enterprise production best practices.

Idempotency - Interview Questions & Answers

One of the most important concepts in distributed systems is Idempotency.

Even if Kafka supports Exactly Once Semantics (EOS), applications can still receive duplicate requests because of:

  • Network Failures
  • Consumer Retries
  • Producer Retries
  • Replay Operations
  • Dead Letter Queue (DLQ) Reprocessing
  • Service Restarts

Without idempotency, duplicate requests can cause serious business issues such as:

  • Double Payments
  • Duplicate Orders
  • Multiple Reward Credits
  • Incorrect Inventory
  • Duplicate Emails

Idempotency ensures that processing the same request multiple times produces the same business result.

Idempotency Architecture

flowchart LR

Producer --> Kafka

Kafka --> Consumer

Consumer --> DuplicateCheck["Duplicate Check"]

DuplicateCheck["Duplicate Check"] --> BusinessLogic["Business Logic"]

BusinessLogic["Business Logic"] --> Database

Q1. What is Idempotency?

Answer

Idempotency means that executing the same operation multiple times results in the same final state.

Example:

Transfer ₹10,000

↓

Process Once

↓

Balance Updated

↓

Process Again

↓

No Additional Debit

The second execution has no business impact.

Workflow

flowchart TD

Request --> DuplicateCheck["Duplicate Check"]

DuplicateCheck["Duplicate Check"] --> AlreadyProcessed["Already Processed?"]

AlreadyProcessed["Already Processed?"] -- Yes --> Ignore

AlreadyProcessed["Already Processed?"] -- No --> Process

Q2. Why is Idempotency important?

Answer

Distributed systems cannot guarantee that every message is delivered only once.

Duplicate requests may occur because of:

  • Retry Policies
  • Consumer Restarts
  • Network Timeouts
  • DLQ Replay
  • Kafka Rebalancing

Without idempotency:

  • Duplicate Payments
  • Duplicate Orders
  • Duplicate Notifications
  • Incorrect Reports

Without Idempotency

flowchart LR

Message --> Consumer

Consumer --> Database

Database --> Retry

Retry --> DatabaseAgain["Database Again"]

With Idempotency

flowchart LR

Message --> DuplicateCheck["Duplicate Check"]

DuplicateCheck["Duplicate Check"] --> IgnoreDuplicate["Ignore Duplicate"]

Q3. How is Idempotency implemented?

Answer

A unique identifier is stored for every successfully processed request.

Examples:

  • Transaction ID
  • Order ID
  • Payment ID
  • Event ID
  • Correlation ID

Workflow:

  1. Receive message.
  2. Check if ID already exists.
  3. If yes → Ignore.
  4. If no → Process and save ID.

Processing Flow

flowchart TD

Message --> CheckRequestId["Check Request ID"]

CheckRequestId["Check Request ID"] --> Processed?

Processed? -- Yes --> Ignore

Processed? -- No --> Process

Process --> SaveRequestId["Save Request ID"]

Q4. What is an Idempotency Key?

Answer

An Idempotency Key is a unique identifier supplied by the client or generated by the system.

Example:

POST /payments

Idempotency-Key:

PAY-2026-00001

If the client retries the request with the same key, the server returns the original response instead of processing it again.

Idempotency Key

flowchart LR

Client --> IdempotencyKey["Idempotency Key"]

IdempotencyKey["Idempotency Key"] --> Server

Server --> DuplicateCheck["Duplicate Check"]

Best Practice

Use UUIDs or business transaction IDs as idempotency keys.

Q5. How does Kafka use Idempotency?

Answer

Kafka supports Idempotent Producers.

Benefits:

  • Prevent duplicate records caused by producer retries.
  • Preserve ordering within partitions.
  • Work with Kafka Transactions.

However, consumer-side idempotency is still required because duplicates may occur after consumption.

Kafka Flow

flowchart TD

Producer --> Kafka

Kafka --> Consumer

Consumer --> DuplicateCheck["Duplicate Check"]

DuplicateCheck["Duplicate Check"] --> Database

Interview Tip

Kafka producer idempotency alone does not guarantee end-to-end exactly-once business processing.

Q6. How does Spring Boot implement Idempotency?

Answer

Typical Spring Boot implementation:

  1. Receive request.
  2. Validate idempotency key.
  3. Check database/cache.
  4. If already processed → Return previous result.
  5. Otherwise → Execute business logic.
  6. Persist key.

Spring Boot Architecture

flowchart TD

RestApi["REST API"] --> SpringService["Spring Service"]

SpringService["Spring Service"] --> IdempotencyStore["Idempotency Store"]

IdempotencyStore["Idempotency Store"] --> AlreadyExists["Already Exists?"]

AlreadyExists["Already Exists?"] -- Yes --> ReturnResponse["Return Response"]

AlreadyExists["Already Exists?"] -- No --> BusinessLogic["Business Logic"]

BusinessLogic["Business Logic"] --> Database

Q7. Where should Idempotency Keys be stored?

Answer

Common storage options include:

  • Relational Database
  • Redis
  • DynamoDB
  • Cassandra
  • Dedicated Idempotency Table

Storage Options

mindmap
  root((Idempotency Store))
    Database
    Redis
    DynamoDB
    Cassandra
    Cache

Best Practice

Use Redis for short-lived requests and a database for long-term financial transactions.

Q8. What are common Idempotency mistakes?

Answer

Common mistakes include:

  • Using timestamps as identifiers
  • No duplicate check
  • Processing before validation
  • Short key expiration
  • No replay support
  • Missing transaction boundaries
  • Ignoring consumer retries

Wrong Design

Process Request

↓

Store ID ❌

Correct Design

Check ID

↓

Process

↓

Store ID ✅

Q9. What are the challenges of Idempotency?

Answer

Challenges include:

  • Key expiration
  • Storage growth
  • High concurrency
  • Distributed databases
  • Replay operations
  • Performance overhead

Challenges

mindmap
  root((Idempotency Challenges))
    Concurrency
    Replay
    Storage
    Expiration
    Performance
    Duplicate Detection

Best Practice

Expire old keys only after the business replay window has passed.

Q10. What are the enterprise best practices for Idempotency?

Answer

Follow these recommendations:

  • Generate unique request IDs.
  • Validate before processing.
  • Store processed IDs atomically.
  • Use database constraints where appropriate.
  • Make consumers idempotent.
  • Combine with Kafka transactions.
  • Use the Outbox Pattern.
  • Support replay safely.
  • Monitor duplicate requests.
  • Test retry scenarios.

Enterprise Architecture

flowchart TD

Client --> ApiGateway["API Gateway"]

ApiGateway["API Gateway"] --> SpringBoot["Spring Boot"]

SpringBoot["Spring Boot"] --> IdempotencyStore["Idempotency Store"]

SpringBoot["Spring Boot"] --> Kafka

Kafka --> Consumer

Consumer --> DuplicateCheck["Duplicate Check"]

DuplicateCheck["Duplicate Check"] --> BusinessDatabase["Business Database"]

Processing Pipeline

flowchart LR

Request --> IdempotencyCheck["Idempotency Check"]

IdempotencyCheck["Idempotency Check"] --> BusinessLogic["Business Logic"]

BusinessLogic["Business Logic"] --> Database

Database --> Response

Idempotency Overview

mindmap
  root((Idempotency))
    Request ID
    Duplicate Check
    Database
    Kafka
    Retry
    Replay
    Outbox
    Monitoring

Banking Example

A customer clicks the Pay Now button twice because of a slow network.

Without Idempotency:

Payment Request

↓

Processed Twice

↓

₹20,000 Debited ❌

With Idempotency:

Payment Request

↓

Duplicate Key Detected

↓

Second Request Ignored

↓

₹10,000 Debited Once ✅

REST API Example

Request

POST /payments

Idempotency-Key: PAY-100001

First Request

Payment Processed

HTTP 201 Created

Retry

Existing Key Found

Return Previous Response

HTTP 201 Created

The payment is not processed again.

Idempotency vs Transactions

Feature Idempotency Database Transaction
Prevents Duplicate Processing ✅ ❌
Ensures Atomicity ❌ ✅
Handles Retries ✅ ❌
Distributed Systems ✅ Limited
REST APIs ✅ ❌
Kafka Consumers ✅ Partial

Senior Interview Tip

Idempotency is one of the most important concepts in enterprise systems.

Almost every production messaging platform relies on it.

A production-ready architecture typically includes:

  • Spring Boot
  • Apache Kafka
  • Idempotent Producers
  • Idempotent Consumers
  • Idempotency Keys
  • Redis / Database
  • Outbox Pattern
  • Kafka Transactions
  • Dead Letter Queue
  • Retry Topics
  • Replay Services
  • Correlation IDs
  • Prometheus & Grafana
  • Distributed Tracing
  • Audit Logging

Remember:

  • Exactly Once Processing depends heavily on Idempotency.
  • Retries are safe only when consumers are idempotent.
  • Always check for duplicates before executing business logic.
  • Store processed request IDs atomically with the business operation whenever possible.

Quick Revision

  • Idempotency ensures repeated requests produce the same business outcome.
  • Duplicate requests are common in distributed systems.
  • Use unique request or transaction IDs for duplicate detection.
  • Validate idempotency before executing business logic.
  • Store processed IDs in a durable store.
  • Kafka producer idempotency does not replace consumer idempotency.
  • Combine idempotency with retries, DLQs, and replay strategies.
  • Monitor duplicate requests and processing failures.
  • Choose an appropriate storage mechanism based on retention needs.
  • Combine Spring Boot, Kafka, idempotency keys, Outbox Pattern, and monitoring for enterprise-grade exactly-once business processing.