---

Introduction

Financial and enterprise applications cannot afford duplicate or lost events.

Examples

• Banking fund transfers
• Stock trading
• Payment processing
• Insurance claims
• Order processing

Consider this scenario:

1. Money is debited from the sender.
2. Database transaction commits.
3. Kafka event fails to publish.

Now downstream systems never receive the payment event.

Another scenario:

1. Kafka event is published.
2. Database transaction rolls back.

Consumers process an event for data that doesn't exist.

To solve these consistency problems, Kafka supports Transactions and Exactly-Once Semantics (EOS).

Spring Kafka integrates these features seamlessly with Spring's transaction management.

---

Kafka Transaction Architecture

flowchart LR

Application --> Transaction

Transaction --> Database

Transaction --> KafkaProducer

KafkaProducer --> KafkaBroker

KafkaBroker --> Consumer

---

Q1. What is a Kafka Transaction?

Answer

A Kafka transaction groups multiple Kafka operations into a single atomic unit.

Either

• All records are committed

OR

• None of them are visible to consumers.

Benefits

• Atomic publishing
• Data consistency
• Exactly-once processing
• Reliable event delivery

---

Q2. Why do we need Kafka Transactions?

Without Transactions

Database Commit

↓

Kafka Publish Fails

Result

Database and Kafka become inconsistent.

With Transactions

flowchart TD

BeginTransaction --> DatabaseUpdate

DatabaseUpdate --> KafkaPublish

KafkaPublish --> Commit

Commit --> Success

Commit --> Rollback

All operations succeed together or fail together.

---

Q3. What is Exactly-Once Semantics (EOS)?

Exactly-Once Semantics ensures that records are processed once and only once.

Without EOS

• Duplicate messages
• Lost updates
• Double payments

With EOS

• No duplicates
• No missing events
• Atomic visibility

Kafka introduced EOS in version 0.11.

---

Q4. How do you enable Kafka Transactions?

Configuration

spring.kafka.producer.transaction-id-prefix=payment-tx-

Spring automatically creates transactional producers.

Example

@Transactional
public void process(){

}

---

Q5. What is KafkaTransactionManager?

KafkaTransactionManager integrates Kafka transactions with Spring.

Responsibilities

• Begin transaction
• Commit transaction
• Rollback transaction
• Coordinate producer operations

Example

KafkaTransactionManager

It is the transaction manager for Kafka producers.

---

Q6. What is read_committed?

Consumers can choose whether to read uncommitted records.

Configuration

spring.kafka.consumer.isolation-level=read_committed

Benefits

• Ignore rolled-back messages
• Read only committed transactions
• Prevent inconsistent processing

Transaction Visibility

flowchart LR

ProducerTransaction --> Commit

Commit --> Consumer

ProducerTransaction --> Rollback

Rollback

-.Hidden.-> Consumer

---

Q7. Can Kafka and Database transactions work together?

Yes.

A typical workflow

1. Start transaction.
2. Update database.
3. Publish Kafka event.
4. Commit both.

Example

@Transactional

public void process(){

}

This ensures business consistency.

---

Q8. What is Transaction Rollback?

If any operation fails,

the entire transaction rolls back.

Example

Save Payment

↓

Kafka Publish Fails

↓

Rollback Database

No partial updates remain.

---

Q9. Transactions vs Idempotency

Transactions	Idempotency
Atomic operations	Duplicate protection
Producer-side consistency	Consumer-side safety
Commit/Rollback	Ignore duplicate events
Exactly-once publishing	Safe reprocessing

Enterprise systems usually implement both.

---

Q10. Kafka Transaction Best Practices

Enable EOS

Prevent duplicate events.

---

Use read_committed

Avoid processing rolled-back records.

---

Keep Transactions Short

Reduce lock duration.

---

Use Idempotent Consumers

Protect against replay scenarios.

---

Monitor Transaction Metrics

Track aborts and commit failures.

---

Banking Example

flowchart TD

PaymentService --> Database

PaymentService --> KafkaProducer

Database --> Commit

KafkaProducer --> Commit

Commit --> Consumer

Both database and Kafka remain consistent.

---

Common Interview Questions

• What is a Kafka Transaction?
• Why do we need Kafka Transactions?
• What is Exactly-Once Semantics?
• How do you enable Kafka Transactions?
• What is KafkaTransactionManager?
• What is read_committed?
• Kafka + Database transactions?
• What is Transaction Rollback?
• Transactions vs Idempotency?
• Kafka transaction best practices?

---

Quick Revision

Topic	Summary
Kafka Transaction	Atomic message publishing
EOS	Exactly-once processing
KafkaTransactionManager	Spring transaction manager
Transaction ID	Identifies producer transaction
read_committed	Read committed records only
Rollback	Undo failed transaction
Commit	Persist successful transaction
Idempotency	Prevent duplicates
Producer	Transaction initiator
Consumer	Reads committed events

---

Transaction Lifecycle

sequenceDiagram
Application->>TransactionManager: Begin Transaction
TransactionManager->>Database: Update Data
TransactionManager->>KafkaProducer: Publish Event
KafkaProducer->>Broker: Transaction
Broker-->>TransactionManager: Success
TransactionManager->>Database: Commit
TransactionManager->>KafkaProducer: Commit
KafkaProducer-->>Consumer: Visible

---

Production Example – Banking Fund Transfer

A banking platform processes fund transfers.

Requirements

• Debit sender account.
• Credit receiver account.
• Publish PaymentCompletedEvent.
• Ensure database and Kafka remain consistent.
• Prevent duplicate processing.

Workflow

1. Customer initiates a transfer.
2. Spring starts a transaction.
3. Sender account is debited.
4. Receiver account is credited.
5. Transaction history is stored.
6. Kafka publishes PaymentCompletedEvent.
7. Both the database transaction and Kafka transaction commit together.
8. Consumers configured with read_committed receive only committed events.

@Transactional
public void transfer(
    PaymentRequest request
) {

    accountService.debit(request);

    accountService.credit(request);

    kafkaTemplate.send(
        "payment-events",
        request
    );

}

flowchart LR

Customer --> PaymentService

PaymentService --> TransactionManager

TransactionManager --> PostgreSQL

TransactionManager --> KafkaProducer

KafkaProducer --> PaymentEventsTopic

PaymentEventsTopic --> PaymentConsumer

PaymentConsumer --> NotificationService

PaymentConsumer --> AuditService

Transaction Configuration

spring.kafka.producer.transaction-id-prefix=payment-tx-
spring.kafka.producer.properties.enable.idempotence=true
spring.kafka.consumer.isolation-level=read_committed

This configuration enables transactional producers, idempotent writes, and ensures consumers never process rolled-back events.

---

Kafka Transactions vs Traditional Database Transactions

Feature	Database Transaction	Kafka Transaction
Scope	Database Operations	Kafka Producer Operations
Commit	Database Commit	Kafka Transaction Commit
Rollback	Undo Database Changes	Abort Kafka Records
Isolation	Database Isolation Levels	read_committed / read_uncommitted
Atomicity	Database Rows	Kafka Messages
Typical Manager	DataSourceTransactionManager	KafkaTransactionManager

---

Key Takeaways

• Kafka Transactions provide atomic publishing of multiple Kafka records, ensuring all-or-nothing visibility.
• Exactly-Once Semantics (EOS) prevents duplicate event publication and ensures reliable event processing.
• Spring Kafka enables transactions through KafkaTransactionManager and a configured transaction-id-prefix.
• Consumers configured with read_committed only receive successfully committed transactional messages.
• Combining database updates and Kafka publishing within a Spring-managed transaction helps maintain business consistency.
• Transaction rollback prevents partial updates when failures occur during processing.
• Kafka transactions and idempotent consumers complement each other to build highly reliable distributed systems.
• Transactional messaging is essential for banking, financial services, and other enterprise applications where data consistency is critical.