---

Introduction

In production systems, failures are unavoidable.

Examples include

• Database connection failure
• Network timeout
• Invalid JSON
• Corrupted Kafka message
• Third-party API failure
• Business validation failure

A Kafka consumer must never simply crash when processing fails.

Instead, enterprise applications should

• Retry temporary failures
• Skip invalid messages
• Route failed records
• Preserve offsets correctly
• Prevent duplicate processing

Spring Kafka provides a comprehensive error-handling framework to build resilient event-driven systems.

---

Error Handling Architecture

flowchart LR

KafkaTopic --> KafkaListener

KafkaListener --> BusinessService

BusinessService --> Success

BusinessService --> DefaultErrorHandler

DefaultErrorHandler --> Retry

Retry --> Recovery

---

Q1. What is Error Handling in Spring Kafka?

Answer

Error handling is the process of detecting, managing, and recovering from failures while consuming Kafka records.

Objectives

• Prevent consumer crashes
• Retry transient failures
• Handle invalid records
• Maintain data consistency
• Recover failed events

Spring Kafka provides configurable error handlers for these scenarios.

---

Q2. What is DefaultErrorHandler?

DefaultErrorHandler is the primary error handler in modern Spring Kafka.

Responsibilities

• Retry failed records
• Seek offsets
• Skip records
• Recover failed messages

Example

@Bean
DefaultErrorHandler errorHandler() {

    return new DefaultErrorHandler();

}

It replaces the older SeekToCurrentErrorHandler.

---

Q3. How does DefaultErrorHandler work?

Workflow

1. Listener throws an exception.
2. Error handler intercepts the failure.
3. Retry policy is applied.
4. Recovery logic executes if retries fail.
5. Offset handling continues.

Error Flow

sequenceDiagram
Kafka->>Listener: Message
Listener->>BusinessService: Process
BusinessService-->>Listener: Exception
Listener->>DefaultErrorHandler: Failure
DefaultErrorHandler->>Retry: Retry
Retry-->>Recovery: Failed

---

Q4. What is BackOff?

BackOff controls the delay between retry attempts.

Spring provides

• FixedBackOff
• ExponentialBackOff

Example

new FixedBackOff(
    2000,
    3
);

Meaning

• Retry every 2 seconds
• Maximum 3 retries

---

Q5. What is Seek?

Seeking resets the consumer position to re-read records.

Example

Offset 120

↓

Failure

↓

Seek

↓

Offset 120 Again

Seek

flowchart LR

Offset120 --> Exception

Exception --> Seek

Seek --> Offset120

This allows Kafka to retry the same record.

---

Q6. What is ConsumerRecordRecoverer?

A ConsumerRecordRecoverer processes records after all retries fail.

Typical actions

• Log failure
• Store in database
• Send to Dead Letter Topic
• Notify operations team

Example

DeadLetterPublishingRecoverer

Recovery prevents permanent message loss.

---

Q7. What is ErrorHandlingDeserializer?

Sometimes deserialization itself fails.

Example

Invalid JSON

↓

Consumer cannot create object.

Spring provides

ErrorHandlingDeserializer

Benefits

• Prevent consumer crashes
• Handle malformed messages
• Continue processing valid records

---

Q8. Which exceptions should be retried?

Retry

• Network timeout
• Temporary database failure
• HTTP 503
• Broker unavailable

Do NOT retry

• Invalid payload
• Validation failure
• Duplicate event
• Business rule violation

Retrying permanent failures wastes resources.

---

Q9. Error Handler vs Retry Topic

Error Handler	Retry Topic
Retries in same consumer	Retries through Kafka topics
Simple configuration	Better scalability
Short-lived retries	Long-running retries
In-memory retry	Distributed retry

Retry Topics are covered separately in the next chapter.

---

Q10. Error Handling Best Practices

Retry Only Temporary Failures

Avoid retrying invalid data.

---

Configure BackOff

Prevent retry storms.

---

Use Dead Letter Topics

Never discard failed records.

---

Monitor Failures

Track retries and recoveries.

---

Make Consumers Idempotent

Prevent duplicate processing.

---

Banking Example

flowchart TD

PaymentTopic --> KafkaListener

KafkaListener --> PaymentService

PaymentService --> Success

PaymentService --> DefaultErrorHandler

DefaultErrorHandler --> Retry

Retry --> DeadLetterRecoverer

Temporary failures recover automatically while permanent failures are isolated.

---

Common Interview Questions

• What is Error Handling?
• What is DefaultErrorHandler?
• How does DefaultErrorHandler work?
• What is BackOff?
• What is Seek?
• What is ConsumerRecordRecoverer?
• What is ErrorHandlingDeserializer?
• Which exceptions should be retried?
• Error Handler vs Retry Topic?
• Error handling best practices?

---

Quick Revision

Topic	Summary
DefaultErrorHandler	Primary Spring Kafka error handler
BackOff	Delay between retries
FixedBackOff	Constant retry delay
ExponentialBackOff	Increasing retry delay
Seek	Re-read failed record
ConsumerRecordRecoverer	Recovery callback
ErrorHandlingDeserializer	Handle deserialization failures
Retry	Temporary failure recovery
Recovery	Final failure action
Dead Letter Topic	Store failed records

---

Error Handling Lifecycle

sequenceDiagram
Kafka->>Listener: Deliver Record
Listener->>BusinessService: Process
BusinessService-->>Listener: Exception
Listener->>DefaultErrorHandler: Failure
DefaultErrorHandler->>Retry: Retry
Retry-->>BusinessService: Retry Processing
BusinessService-->>DefaultErrorHandler: Failed
DefaultErrorHandler->>Recoverer: Recover Record

---

Production Example – Banking Payment Processing Failure

A banking platform consumes payment events.

Workflow

1. Consumer receives a PaymentInitiatedEvent.
2. Payment service attempts to update the database.
3. Database becomes temporarily unavailable.
4. DefaultErrorHandler catches the exception.
5. FixedBackOff retries three times with a two-second delay.
6. If the database becomes available, processing succeeds and the offset is committed.
7. If retries fail, the DeadLetterPublishingRecoverer sends the record to the Dead Letter Topic.
8. Operations teams investigate and replay the failed event later.

@Bean
DefaultErrorHandler errorHandler(
    DeadLetterPublishingRecoverer recoverer
) {

    return new DefaultErrorHandler(
        recoverer,
        new FixedBackOff(2000L, 3)
    );

}

flowchart LR

PaymentTopic --> KafkaListener

KafkaListener --> PaymentService

PaymentService --> Database

Database --> Failure

Failure --> DefaultErrorHandler

DefaultErrorHandler --> Retry1

Retry1 --> Retry2

Retry2 --> Retry3

Retry3 --> DeadLetterPublishingRecoverer

DeadLetterPublishingRecoverer --> PaymentDLT

Production Configuration Example

spring.kafka.listener.ack-mode=manual
spring.kafka.consumer.enable-auto-commit=false
spring.kafka.listener.type=single

Combined with DefaultErrorHandler, this configuration provides reliable processing and prevents message loss.

---

Exception Handling Strategy

Exception Type	Action
Network Timeout	Retry
Database Connection Failure	Retry
HTTP 503 Service Unavailable	Retry
Broker Timeout	Retry
Invalid JSON	Skip / DLT
Validation Failure	DLT
Duplicate Payment	Ignore or Log
Business Rule Violation	DLT
NullPointerException (Code Bug)	Investigate & Fix
Serialization Error	ErrorHandlingDeserializer

---

Key Takeaways

• DefaultErrorHandler is the recommended error handler for modern Spring Kafka applications.
• It supports configurable retries, offset seeking, recovery callbacks, and integration with Dead Letter Topics.
• BackOff strategies control retry timing, with FixedBackOff and ExponentialBackOff being the most common implementations.
• Seek allows consumers to reprocess failed records without losing data.
• ConsumerRecordRecoverer defines how permanently failed records are handled after retries are exhausted.
• ErrorHandlingDeserializer prevents malformed messages from crashing consumers during deserialization.
• Retry only transient failures; permanent business or validation failures should be routed to recovery mechanisms such as DLTs.
• Combining retries, recoverers, idempotent consumers, and proper monitoring results in resilient, production-ready Kafka consumer applications.