---

Introduction

Traditional Kafka consumers process one event at a time.

However, modern enterprise applications often need continuous real-time data processing.

Examples

• Fraud detection
• Live stock market analysis
• Real-time dashboards
• Payment aggregation
• Customer analytics
• IoT sensor monitoring
• Recommendation engines

Instead of consuming records manually, Kafka Streams provides a powerful stream processing library that performs filtering, transformations, joins, aggregations, and windowing directly on Kafka topics.

Spring Boot integrates Kafka Streams using Spring for Apache Kafka Streams, making stream processing simple and production-ready.

---

Kafka Streams Architecture

flowchart LR

InputTopic --> KafkaStreams

KafkaStreams --> Processing

Processing --> OutputTopic

---

Q1. What is Kafka Streams?

Answer

Kafka Streams is a lightweight Java library for building real-time stream processing applications on top of Kafka.

Capabilities

• Filtering
• Mapping
• Joining
• Aggregation
• Windowing
• Stateful processing

Unlike Spark or Flink, Kafka Streams runs as a normal Java application.

---

Q2. Why use Kafka Streams?

Without Kafka Streams

Application manually

• Polls records
• Maintains state
• Aggregates data
• Writes results

With Kafka Streams

flowchart LR

Kafka --> KafkaStreams

KafkaStreams --> Output

Processing becomes declarative and highly scalable.

---

Q3. What is a KStream?

A KStream represents an unbounded stream of immutable events.

Examples

• Payment events
• Orders
• Clickstream
• Sensor data

Characteristics

• Every record is independent.
• Records are append-only.
• Suitable for event streams.

KStream

flowchart LR

Payment1 --> KStream

Payment2 --> KStream

Payment3 --> KStream

---

Q4. What is a KTable?

A KTable represents the latest value for each key.

Example

Customer

ID=101

Balance=100

Later update

ID=101

Balance=200

Only the latest value is retained.

KTable

flowchart LR

Updates --> KTable

KTable --> LatestState

Useful for maintaining current state.

---

Q5. What is a GlobalKTable?

A GlobalKTable replicates the complete table to every application instance.

Characteristics

• Entire dataset available locally
• No partition restrictions
• Faster joins

Use cases

• Product catalog
• Country codes
• Currency rates
• Reference data

---

Q6. What are Stateless Operations?

Stateless operations do not maintain state between records.

Examples

• filter()
• map()
• flatMap()
• peek()
• selectKey()

Example

stream.filter(...)

Stateless Flow

flowchart LR

Input --> Filter

Filter --> Map

Map --> Output

---

Q7. What are Stateful Operations?

Stateful operations maintain local state.

Examples

• count()
• aggregate()
• reduce()
• join()
• window()

Kafka Streams stores state in State Stores backed by changelog topics.

Stateful Processing

flowchart LR

Events --> StateStore

StateStore --> Aggregation

Aggregation --> Output

---

Q8. What is Windowing?

Windowing groups events by time.

Types

• Tumbling Window
• Hopping Window
• Sliding Window
• Session Window

Example

Payments

10:00–10:05

↓

Total Amount

Window

flowchart TD

Events --> TimeWindow

TimeWindow --> Aggregate

Useful for analytics and fraud detection.

---

Q9. What are Joins?

Kafka Streams supports

• KStream-KStream Join
• KStream-KTable Join
• KTable-KTable Join
• GlobalKTable Join

Example

Payments

Customer Information

↓

Enriched Payment

Stream Join

flowchart LR

PaymentStream --> Join

CustomerTable --> Join

Join --> EnrichedEvent

Joins enrich streaming events with reference data.

---

Q10. Kafka Streams Best Practices

Keep Processing Stateless When Possible

State increases complexity.

---

Use Appropriate Keys

Ensure correct partitioning.

---

Monitor State Stores

Track disk usage and restoration.

---

Use Windowing Carefully

Select window sizes based on business requirements.

---

Enable Exactly-Once Processing

Protect against duplicate results.

---

Banking Example

flowchart TD

PaymentTopic --> PaymentStream

PaymentStream --> FraudDetection

FraudDetection --> WindowAggregation

WindowAggregation --> AlertTopic

Payments are continuously analyzed for suspicious activity.

---

Common Interview Questions

• What is Kafka Streams?
• Why use Kafka Streams?
• What is a KStream?
• What is a KTable?
• What is a GlobalKTable?
• Stateless vs Stateful operations?
• What is Windowing?
• What are Joins?
• What are State Stores?
• Kafka Streams best practices?

---

Quick Revision

Topic	Summary
Kafka Streams	Real-time stream processing library
KStream	Continuous event stream
KTable	Latest value per key
GlobalKTable	Fully replicated table
Stateless Operation	No stored state
Stateful Operation	Uses State Store
Windowing	Time-based grouping
Join	Combine streams/tables
State Store	Local persistent state
Exactly-Once	Prevent duplicate results

---

Kafka Streams Processing Lifecycle

sequenceDiagram
Producer->>InputTopic: Publish Event
InputTopic->>KafkaStreams: Consume
KafkaStreams->>Filter: Filter Records
Filter->>Aggregation: Aggregate
Aggregation->>Join: Join Reference Data
Join->>OutputTopic: Publish Result
OutputTopic->>Consumer: Consume Result

---

Production Example – Banking Fraud Detection

A digital banking platform monitors payment transactions in real time.

Requirements

• Detect unusually high transaction volumes.
• Join payment events with customer profiles.
• Aggregate payments every five minutes.
• Generate fraud alerts immediately.
• Scale horizontally across multiple instances.

Workflow

1. Payment events arrive in the payment-events topic.
2. A KStream reads incoming payment events.
3. A GlobalKTable stores customer risk profiles.
4. A KStream-GlobalKTable Join enriches each payment with customer information.
5. Payments are grouped by customer ID.
6. A five-minute Tumbling Window calculates the total transaction amount.
7. If the total exceeds a threshold, a fraud alert is published to the fraud-alerts topic.

KStream<String, PaymentEvent> stream =
    builder.stream("payment-events");

flowchart LR

PaymentEventsTopic --> KStream

KStream --> Join

CustomerGlobalKTable --> Join

Join --> WindowAggregation

WindowAggregation --> FraudRules

FraudRules --> FraudAlertsTopic

FraudAlertsTopic --> FraudMonitoringDashboard

Production Configuration Example

spring.kafka.streams.application-id=fraud-detection-service
spring.kafka.streams.processing-guarantee=exactly_once_v2
spring.kafka.streams.state-dir=/var/lib/kafka-streams

This configuration enables exactly-once processing, persistent local state stores, and scalable stream processing.

---

Kafka Streams vs Traditional Kafka Consumer

Feature	Kafka Consumer	Kafka Streams
Processing Model	Manual	Declarative
State Management	Application Managed	Built-in State Stores
Aggregation	Manual	Built-in
Windowing	Manual	Built-in
Joins	Manual	Built-in
Scaling	Consumer Groups	Consumer Groups + State Stores
Exactly-Once Support	Supported	Native (exactly_once_v2)
Typical Use Case	Event Consumption	Real-time Analytics & Processing

---

Key Takeaways

• Kafka Streams is a lightweight Java library for real-time stream processing built directly on Apache Kafka.
• KStream represents an immutable stream of events, while KTable represents the latest value for each key.
• GlobalKTable replicates reference data to every application instance for efficient joins.
• Stateless operations such as filtering and mapping are lightweight, whereas stateful operations use persistent State Stores.
• Windowing enables time-based aggregations for analytics, monitoring, and fraud detection.
• Built-in joins simplify event enrichment by combining streams with reference data.
• Kafka Streams supports exactly-once processing, making it suitable for financial and enterprise workloads.
• Combining Kafka Streams with Spring Boot enables scalable, fault-tolerant, and production-ready real-time data processing applications.