Kafka Topics, Partitions and Offsets Interview Questions and Answers

Master Kafka Topics, Partitions, and Offsets with real-world interview questions, architecture diagrams, Spring Boot integration, message ordering, consumer offsets, and production best practices.

Kafka Topics, Partitions and Offsets Interview Questions and Answers

Topics, Partitions, and Offsets are the heart of Apache Kafka.

Almost every Kafka interview begins with questions like:

What is a Topic?
Why are Partitions needed?
What is an Offset?
How does Kafka guarantee ordering?
How does Kafka scale to millions of messages?

If you understand these three concepts well, you understand how Kafka stores and delivers data.

Kafka Storage Architecture

flowchart LR

Producer --> Topic

Topic --> Partition0["Partition 0"]

Topic --> Partition1["Partition 1"]

Topic --> Partition2["Partition 2"]

Partition0["Partition 0"] --> ConsumerA["Consumer A"]

Partition1["Partition 1"] --> ConsumerB["Consumer B"]

Partition2["Partition 2"] --> ConsumerC["Consumer C"]

Q1. What is a Kafka Topic?

Answer

A Topic is a logical category used to organize events.

Applications publish events to topics, and consumers subscribe to those topics.

Examples:

orders

payments

customers

notifications

audit-events

A topic itself does not store data directly.

Instead, it is divided into one or more partitions.

Topic Architecture

flowchart LR

OrdersTopic["Orders Topic"] --> Partition0["Partition 0"]

OrdersTopic["Orders Topic"] --> Partition1["Partition 1"]

OrdersTopic["Orders Topic"] --> Partition2["Partition 2"]

Real Example

Payment Service

↓

payments-topic

↓

Fraud Service

↓

Notification Service

↓

Analytics

Q2. Why are Topics divided into Partitions?

Answer

Partitions provide:

Horizontal Scalability
Parallel Processing
High Throughput
Fault Tolerance

Without partitions:

One Topic

↓

One Consumer

With partitions:

One Topic

↓

Multiple Consumers

Partitioning

flowchart LR

Topic --> Partition0["Partition 0"]

Topic --> Partition1["Partition 1"]

Topic --> Partition2["Partition 2"]

Partition0["Partition 0"] --> Consumer1["Consumer 1"]

Partition1["Partition 1"] --> Consumer2["Consumer 2"]

Partition2["Partition 2"] --> Consumer3["Consumer 3"]

Q3. What is a Kafka Partition?

Answer

A Partition is an append-only log that stores messages in the order they are received.

Characteristics:

Ordered
Immutable
Sequential Writes
Stored on Disk

Example:

Partition 0

Message A

Message B

Message C

Partition Storage

flowchart LR

Partition --> Offset0["Offset 0"]
Offset0["Offset 0"] --> Offset1["Offset 1"]

Offset1["Offset 1"] --> Offset2["Offset 2"]
Offset2["Offset 2"] --> Offset3["Offset 3"]

Q4. What is an Offset?

Answer

An Offset is a unique sequential number assigned to every message inside a partition.

Offsets identify the position of each message.

Example

Offset	Event
0	Order Created
1	Payment Started
2	Payment Completed
3	Notification Sent

Offset Progress

flowchart LR

Offset0["Offset 0"] --> Offset1["Offset 1"]
Offset1["Offset 1"] --> Offset2["Offset 2"]

Offset2["Offset 2"] --> Offset3["Offset 3"]

Interview Tip

Offsets are unique only within a partition, not across the entire topic.

Q5. How do Consumers use Offsets?

Answer

Consumers maintain their own offsets.

Kafka does not track message consumption automatically.

Consumer example:

Current Offset = 500

Next Read = 501

If the consumer restarts, it resumes from the last committed offset.

Consumer Offset

sequenceDiagram
participant Consumer
participant Partition
Consumer->>Partition: Read Offset 100
Consumer->>Partition: Process Message
Consumer->>Partition: Commit Offset 100
Consumer->>Partition: Read Offset 101

Q6. How does Kafka guarantee message ordering?

Answer

Kafka guarantees ordering only within a partition.

Example:

Partition 0

Order1

Order2

Order3

Ordering is preserved.

Across multiple partitions:

Partition 0 → Order1

Partition 1 → Order2

Partition 2 → Order3

Global ordering is not guaranteed.

Ordering

flowchart LR

Customer100["Customer 100"] --> Partition1["Partition 1"]

Partition1["Partition 1"] --> Consumer

Best Practice

Use a business key (such as customerId or accountId) as the partition key to preserve ordering for related events.

Q7. What is a Partition Key?

Answer

A Partition Key determines which partition receives a message.

Example:

customerId = 500

All events for customer 500 go to the same partition.

Partition Key

flowchart LR

Producer --> Hash(customerId)

Hash(customerId) --> Partition2["Partition 2"]

Benefits

Ordering
Balanced Distribution
Consistent Processing

Q8. What happens if there are more Consumers than Partitions?

Answer

Kafka assigns only one consumer per partition within the same consumer group.

Example:

3 Partitions

5 Consumers

Result:

Consumer 1 → Partition 0

Consumer 2 → Partition 1

Consumer 3 → Partition 2

Consumer 4 → Idle

Consumer 5 → Idle

Consumer Assignment

flowchart LR

Partition0["Partition 0"] --> ConsumerA["Consumer A"]

Partition1["Partition 1"] --> ConsumerB["Consumer B"]

Partition2["Partition 2"] --> ConsumerC["Consumer C"]

ConsumerD["Consumer D"] --> Idle

ConsumerE["Consumer E"] --> Idle

Interview Tip

Maximum parallelism inside one consumer group equals the number of partitions.

Q9. What happens if there are more Partitions than Consumers?

Answer

One consumer handles multiple partitions.

Example:

6 Partitions

3 Consumers

Assignment:

Consumer A → P0, P1

Consumer B → P2, P3

Consumer C → P4, P5

Distribution

flowchart LR

Partition0["Partition 0"] --> ConsumerA["Consumer A"]

Partition1["Partition 1"] --> ConsumerA["Consumer A"]

Partition2["Partition 2"] --> ConsumerB["Consumer B"]

Partition3["Partition 3"] --> ConsumerB["Consumer B"]

Partition4["Partition 4"] --> ConsumerC["Consumer C"]

Partition5["Partition 5"] --> ConsumerC["Consumer C"]

Q10. How do Topics, Partitions, and Offsets work together?

Answer

Complete flow:

Producer publishes an event.
Kafka chooses a partition.
Event receives an offset.
Broker stores the event.
Consumer reads the event.
Consumer commits the offset.

Complete Flow

flowchart TD

Producer --> Topic

Topic --> Partition

Partition --> Offset

Offset --> Consumer

Consumer --> CommitOffset["Commit Offset"]

Topic Lifecycle

sequenceDiagram
participant Producer
participant Topic
participant Partition
participant Consumer
Producer->>Topic: Publish Event
Topic->>Partition: Select Partition
Partition->>Partition: Assign Offset
Consumer->>Partition: Read Event
Consumer->>Partition: Commit Offset

Topic vs Partition vs Offset

Component	Purpose
Topic	Logical category of events
Partition	Physical storage and parallelism
Offset	Unique position inside a partition

Real Banking Example

A customer performs an online transfer.

Mobile Banking

↓

Transfer Service

↓

payments-topic

↓

Partition 2

↓

Offset 1050

↓

Fraud Detection

↓

Ledger Service

↓

Notification

All events for the same customer are routed to the same partition using customerId as the partition key, ensuring transaction ordering.

Production Best Practices

Choose partition count based on future scalability.
Use stable business keys as partition keys.
Avoid random partitioning when ordering matters.
Commit offsets only after successful processing.
Monitor partition skew.
Monitor consumer lag.
Avoid creating thousands of tiny topics.
Increase partitions only after evaluating the impact on ordering.
Balance partitions across brokers.
Use replication factor 3 for production.

Senior Interview Tips

Interviewers frequently ask:

What is a Topic?
What is a Partition?
Why are Partitions required?
What is an Offset?
Are Offsets globally unique?
How does Kafka guarantee ordering?
What is a Partition Key?
What happens when consumers exceed partitions?
What happens when partitions exceed consumers?
Can the number of partitions be increased?
What happens to ordering after increasing partitions?
How are offsets committed?
How does Kafka resume after a restart?

Remember:

Topic = Logical Category
Partition = Storage + Parallelism
Offset = Position inside a Partition
Ordering is guaranteed only within a partition.

Quick Revision

Topics organize related events.
Topics are divided into partitions for scalability and parallel processing.
Partitions are append-only logs that preserve message order.
Every message receives a unique offset within its partition.
Consumers track their own offsets and resume processing after restarts.
Partition keys determine which partition stores an event.
Ordering is guaranteed only within a single partition.
Consumer parallelism is limited by the number of partitions.
Monitor offsets, partition balance, and consumer lag in production.
Topics, Partitions, and Offsets form the foundation of Apache Kafka's distributed architecture.