Amazon MQ and Amazon MSK (Managed Kafka) with Spring Boot - Complete Guide

Learn the differences between Amazon MQ and Amazon MSK, when to use each messaging platform, and how to integrate them with Spring Boot for enterprise messaging and event streaming.

Introduction

Modern enterprise applications rely heavily on messaging systems to enable asynchronous communication, event-driven architectures, and real-time data processing.

AWS provides two managed messaging services:

Amazon MQ – Managed message broker supporting traditional messaging protocols like JMS, AMQP, MQTT, STOMP, and OpenWire.
Amazon MSK (Managed Streaming for Apache Kafka) – Fully managed Apache Kafka service designed for high-throughput event streaming.

Although both services move data between applications, they solve different problems. Understanding when to use Amazon MQ versus Amazon MSK is essential for designing scalable enterprise systems.

Why Messaging Matters

Imagine an online shopping platform.

When an order is placed:

Process payment
Update inventory
Send confirmation email
Notify warehouse
Update analytics
Trigger fraud detection

Executing all these tasks synchronously slows the application and tightly couples services.

Using a messaging platform:

The Order Service publishes an event.
Downstream services process the event independently.
Systems scale independently.
Temporary failures do not block the customer request.

High-Level Architecture

flowchart LR

USER[Customer]

API[Spring Boot Order Service]

MQ[Amazon MQ]

MSK[Amazon MSK]

PAYMENT[Payment Service]

EMAIL[Email Service]

ANALYTICS[Analytics]

WAREHOUSE[Warehouse Service]

USER --> API

API --> MQ

API --> MSK

MQ --> PAYMENT

MQ --> EMAIL

MSK --> ANALYTICS

MSK --> WAREHOUSE

Amazon MQ Overview

Amazon MQ is a managed message broker compatible with traditional enterprise messaging systems.

Supported brokers include:

ActiveMQ
RabbitMQ

Supported protocols:

JMS
AMQP
MQTT
STOMP
OpenWire

Organizations migrating legacy enterprise applications often choose Amazon MQ because existing applications require minimal changes.

Amazon MSK Overview

Amazon MSK is a fully managed Apache Kafka service.

Kafka is a distributed event streaming platform built for:

High throughput
Real-time streaming
Event sourcing
Data pipelines
Log aggregation

MSK removes the operational complexity of managing Kafka clusters while preserving the native Kafka APIs.

Messaging Models

Amazon MQ

Message Broker

Producer

↓

Broker

↓

Consumer

Messages are typically removed after successful processing.

Amazon MSK

Event Streaming Platform

Producer

↓

Kafka Topic

↓

Consumer Group

↓

Multiple Consumers

Events remain in the topic for a configurable retention period, allowing replay.

Core Components

Producer

Creates messages or events.

Examples:

Order Service
Payment Service
User Registration

Broker / Cluster

Stores and distributes messages.

Amazon MQ uses brokers.

Amazon MSK uses Kafka brokers organized into clusters.

Consumer

Processes incoming messages.

Examples:

Email Service
Shipping Service
Analytics Platform

Amazon MQ Architecture

flowchart LR

APP[Spring Boot]

BROKER[Amazon MQ Broker]

EMAIL[Email Service]

PAYMENT[Payment Service]

ERP[ERP Integration]

APP --> BROKER

BROKER --> EMAIL

BROKER --> PAYMENT

BROKER --> ERP

Amazon MSK Architecture

flowchart LR

ORDER[Order Service]

TOPIC[Kafka Topic]

PAYMENT[Payment Service]

INVENTORY[Inventory Service]

ANALYTICS[Analytics Service]

SEARCH[Search Index]

ORDER --> TOPIC

TOPIC --> PAYMENT

TOPIC --> INVENTORY

TOPIC --> ANALYTICS

TOPIC --> SEARCH

Spring Boot Integration

Spring Boot supports both platforms.

Amazon MQ

Typically uses:

Spring JMS
ActiveMQ
RabbitMQ client libraries

Common operations:

Send message
Receive message
Queue processing
Topic subscriptions

Amazon MSK

Uses the standard Kafka client libraries.

Common operations:

Publish events
Consume events
Consumer groups
Stream processing
Exactly-once workflows (when configured appropriately)

Queue vs Topic

Queue

One consumer processes a message.

Example:

Invoice Queue

↓

Invoice Service

Topic

Multiple consumers receive the same event.

Example:

Order Topic

↓

Inventory

↓

Shipping

↓

Analytics

Message Lifecycle

Amazon MQ

stateDiagram-v2

[*] --> Sent

Sent --> Queued

Queued --> Consumed

Consumed --> Deleted

Messages are removed after successful processing.

Amazon MSK

stateDiagram-v2

[*] --> Produced

Produced --> Stored

Stored --> Consumed

Consumed --> Retained

Retained --> Expired

Kafka retains events according to the configured retention policy, allowing consumers to replay them if needed.

Consumer Groups (Kafka)

Kafka allows multiple consumers to share workload.

flowchart TD

TOPIC[Kafka Topic]

TOPIC --> C1[Consumer 1]

TOPIC --> C2[Consumer 2]

TOPIC --> C3[Consumer 3]

Benefits:

Horizontal scalability
High throughput
Fault tolerance

Ordering

Amazon MQ

Queue ordering depends on broker configuration.

Suitable for request-response messaging.

Amazon MSK

Kafka guarantees ordering within a partition.

Applications should choose partition keys carefully to preserve ordering where required.

Message Replay

One of Kafka's biggest advantages.

Example:

Customer Registered Event

↓

Stored for 7 Days

↓

New Analytics Service Starts

↓

Replays Historical Events

Amazon MQ does not provide this event replay capability in the same way.

Enterprise Use Cases

Amazon MQ

Ideal for:

Legacy Java EE applications
JMS messaging
ERP integration
Financial middleware
Request-response systems
Traditional enterprise applications

Amazon MSK

Ideal for:

Event sourcing
Microservices
Clickstream analytics
IoT telemetry
Real-time dashboards
Streaming pipelines
Fraud detection

Security

Secure messaging using:

IAM integration (where supported)
TLS encryption
Authentication
Authorization
VPC networking
Security Groups
KMS encryption
Secrets management

Monitoring

Monitor messaging platforms using CloudWatch.

Important metrics include:

Amazon MQ

Queue depth
Consumer count
Message rate
Broker health
Connection count

Amazon MSK

Broker CPU
Partition count
Consumer lag
Request latency
Network throughput
Disk utilization

Enterprise Architecture

flowchart TD

USER[Users]

USER --> API[Spring Boot API]

API --> MQ[Amazon MQ]

API --> MSK[Amazon MSK]

MQ --> ERP[ERP]

MQ --> LEGACY[Legacy Applications]

MSK --> PAYMENT[Payment]

MSK --> INVENTORY[Inventory]

MSK --> ANALYTICS[Analytics]

MSK --> DATA[Data Lake]

PAYMENT --> CLOUDWATCH[CloudWatch]

ANALYTICS --> GRAFANA[Grafana]

Amazon MQ vs Amazon MSK

Feature	Amazon MQ	Amazon MSK
Technology	ActiveMQ / RabbitMQ	Apache Kafka
Messaging Style	Traditional Broker	Event Streaming
Primary Use	Enterprise Messaging	Real-Time Event Streaming
JMS Support	Yes	No
Event Replay	No	Yes
Consumer Groups	Limited	Native
Throughput	Moderate	Very High
Ordering	Broker-dependent	Guaranteed within a partition
Legacy Migration	Excellent	Not intended for legacy JMS migration
Microservices	Good	Excellent

Decision Guide

Choose Amazon MQ when:

Migrating JMS applications.
Integrating legacy enterprise systems.
Using ActiveMQ or RabbitMQ.
Request-response messaging is required.

Choose Amazon MSK when:

Building event-driven microservices.
Streaming millions of events.
Requiring event replay.
Processing real-time analytics.
Building modern cloud-native platforms.

Best Practices

Choose the messaging platform based on business requirements, not popularity.
Keep message payloads compact and versioned.
Use structured formats such as JSON or Avro when appropriate.
Implement idempotent consumers to handle duplicate deliveries.
Monitor queue depth, consumer lag, and broker health.
Secure brokers using least-privilege IAM, network controls, and encryption.
Use Dead Letter Queues (or equivalent retry strategies) for failed processing.
Partition Kafka topics thoughtfully to balance throughput and ordering.
Document message schemas and event contracts.
Test failure scenarios, retries, and recovery regularly.

Common Challenges

Challenge	Solution
Duplicate message processing	Build idempotent consumers
Growing queue backlog	Scale consumers or optimize processing
Kafka consumer lag	Increase consumer capacity or tune partitions
Legacy migration complexity	Use Amazon MQ to reduce application changes
Event schema evolution	Version messages and maintain compatibility

Typical Messaging Workflow

flowchart LR

REQUEST[Business Request]

REQUEST --> SPRING[Spring Boot]

SPRING --> MESSAGE[Message Broker / Event Stream]

MESSAGE --> SERVICE1[Service A]

MESSAGE --> SERVICE2[Service B]

MESSAGE --> SERVICE3[Analytics]

SERVICE1 --> DATABASE[(Database)]

Interview Questions

What is Amazon MQ?
What is Amazon MSK?
What is the difference between a message broker and an event streaming platform?
When would you choose Amazon MQ over Amazon MSK?
How does Kafka retain messages?
What are Kafka consumer groups?
How is message ordering handled in Kafka?
How would you design a high-throughput event-driven architecture using Spring Boot and Amazon MSK?

Summary

Amazon MQ and Amazon MSK address different messaging requirements.

Amazon MQ is best for traditional enterprise messaging, JMS applications, and legacy integration.
Amazon MSK is designed for modern event-driven architectures, high-throughput streaming, and real-time analytics.

By understanding their strengths and trade-offs, architects can select the right service for each workload. In many large enterprises, both services coexist—Amazon MQ supports legacy systems, while Amazon MSK powers new cloud-native platforms.

Loading likes...

Comments

Share a question, correction, or practical insight about this article.

Loading approved comments...