AWS CloudWatch Logs with Spring Boot

---

Introduction

In production, logs should not stay only inside the server.

If your Spring Boot application runs on AWS EC2, ECS, EKS, or Lambda, you need a centralized logging system to collect, search, monitor, and troubleshoot logs.

AWS CloudWatch Logs helps you:

• Store application logs centrally
• Search logs using Log Insights
• Create alerts from log patterns
• Debug production issues faster
• Monitor errors, exceptions, and API failures

---

Real-Time Use Case

Assume you have a Spring Boot order service running on EC2.

When a user places an order:

POST /orders

The application logs:

Order received
Payment validation started
Inventory updated
Order completed

Instead of checking logs manually inside EC2, we send logs to AWS CloudWatch Logs.

---

Architecture

flowchart LR
    U[User] --> ALB[Application Load Balancer]
    ALB --> EC2[EC2 Instance Running Spring Boot App]

    EC2 --> APPLOG[Application Log File]
    APPLOG --> AGENT[CloudWatch Agent]
    AGENT --> CWL[CloudWatch Logs]

    CWL --> LI[CloudWatch Log Insights]
    CWL --> ALARM[CloudWatch Alarms]
    ALARM --> SNS[SNS Notification]

---

Step 1: Create Spring Boot Application

Create a Spring Boot REST controller.

package com.codewithvenu.orders.controller;

import lombok.extern.slf4j.Slf4j;
import org.springframework.web.bind.annotation.*;

@RestController
@RequestMapping("/orders")
@Slf4j
public class OrderController {

    @PostMapping
    public String createOrder() {
        log.info("Order request received");

        try {
            log.info("Payment validation started");
            log.info("Inventory validation completed");
            log.info("Order created successfully");

            return "Order created successfully";
        } catch (Exception ex) {
            log.error("Order creation failed", ex);
            return "Order creation failed";
        }
    }

    @GetMapping("/{id}")
    public String getOrder(@PathVariable String id) {
        log.info("Fetching order details for orderId={}", id);
        return "Order details for " + id;
    }
}

---

Step 2: Configure Logback

Create this file:

src/main/resources/logback-spring.xml

<configuration>

    <property name="LOG_PATH" value="/opt/apps/order-service/logs"/>
    <property name="APP_NAME" value="order-service"/>

    <appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <pattern>%d{yyyy-MM-dd HH:mm:ss} %-5level [%thread] %logger{36} - %msg%n</pattern>
        </encoder>
    </appender>

    <appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <file>${LOG_PATH}/${APP_NAME}.log</file>

        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <fileNamePattern>${LOG_PATH}/${APP_NAME}.%d{yyyy-MM-dd}.log</fileNamePattern>
            <maxHistory>7</maxHistory>
        </rollingPolicy>

        <encoder>
            <pattern>%d{yyyy-MM-dd HH:mm:ss} %-5level [%thread] %logger{36} - %msg%n</pattern>
        </encoder>
    </appender>

    <root level="INFO">
        <appender-ref ref="CONSOLE"/>
        <appender-ref ref="FILE"/>
    </root>

</configuration>

---

Step 3: Build Spring Boot JAR

mvn clean package -DskipTests

Generated JAR:

target/order-service.jar

---

Step 4: Copy JAR to EC2

scp -i my-key.pem target/order-service.jar ec2-user@<EC2_PUBLIC_IP>:/home/ec2-user/

Login to EC2:

ssh -i my-key.pem ec2-user@<EC2_PUBLIC_IP>

Create application folder:

sudo mkdir -p /opt/apps/order-service/logs
sudo cp /home/ec2-user/order-service.jar /opt/apps/order-service/
sudo chown -R ec2-user:ec2-user /opt/apps/order-service

Run application:

java -jar /opt/apps/order-service/order-service.jar

Test API:

curl -X POST http://localhost:8080/orders

Check local log file:

cat /opt/apps/order-service/logs/order-service.log

---

Step 5: Create IAM Role for EC2

Create an IAM Role and attach this AWS managed policy:

CloudWatchAgentServerPolicy

Attach this role to your EC2 instance.

Required permission flow:

flowchart TD
    EC2[EC2 Instance] --> ROLE[IAM Role]
    ROLE --> POLICY[CloudWatchAgentServerPolicy]
    POLICY --> CWL[CloudWatch Logs Permission]

---

Step 6: Install CloudWatch Agent on EC2

For Amazon Linux:

sudo yum update -y
sudo yum install amazon-cloudwatch-agent -y

Verify installation:

amazon-cloudwatch-agent-ctl -h

---

Step 7: Create CloudWatch Agent Config

Create config file:

sudo vi /opt/aws/amazon-cloudwatch-agent/bin/config.json

Add this configuration:

{
  "logs": {
    "logs_collected": {
      "files": {
        "collect_list": [
          {
            "file_path": "/opt/apps/order-service/logs/order-service.log",
            "log_group_name": "/codewithvenu/springboot/order-service",
            "log_stream_name": "{instance_id}",
            "timezone": "UTC"
          }
        ]
      }
    }
  }
}

---

Step 8: Start CloudWatch Agent

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl \
-a fetch-config \
-m ec2 \
-c file:/opt/aws/amazon-cloudwatch-agent/bin/config.json \
-s

Check agent status:

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a status

---

Step 9: Verify Logs in AWS Console

Go to:

AWS Console → CloudWatch → Logs → Log groups

Find:

/codewithvenu/springboot/order-service

Open the log stream and check application logs.

---

Step 10: Generate Application Logs

Call the APIs:

curl -X POST http://localhost:8080/orders
curl http://localhost:8080/orders/101

Expected logs:

Order request received
Payment validation started
Inventory validation completed
Order created successfully
Fetching order details for orderId=101

---

Step 11: Query Logs Using CloudWatch Log Insights

Go to:

CloudWatch → Logs Insights

Select log group:

/codewithvenu/springboot/order-service

Query latest logs:

fields @timestamp, @message
| sort @timestamp desc
| limit 20

Find error logs:

fields @timestamp, @message
| filter @message like /ERROR/
| sort @timestamp desc
| limit 50

Find order-related logs:

fields @timestamp, @message
| filter @message like /Order/
| sort @timestamp desc
| limit 50

---

Step 12: Create Metric Filter for Errors

Create a metric filter to track application errors.

Pattern:

ERROR

Metric details:

Metric Namespace: CodeWithVenu/Application
Metric Name: OrderServiceErrorCount
Metric Value: 1

Flow:

flowchart LR
    LOGS[CloudWatch Logs] --> FILTER[Metric Filter: ERROR]
    FILTER --> METRIC[Custom Metric]
    METRIC --> ALARM[CloudWatch Alarm]
    ALARM --> SNS[SNS Email Alert]

---

Step 13: Create CloudWatch Alarm

Create alarm for error count.

Example condition:

If OrderServiceErrorCount >= 1 for 1 datapoint within 5 minutes

Send notification to SNS email topic.

---

Step 14: Recommended Production Log Format

For production systems, use structured logs.

Example JSON log:

{
  "timestamp": "2026-06-30T10:15:30Z",
  "level": "INFO",
  "service": "order-service",
  "traceId": "abc-123",
  "orderId": "101",
  "message": "Order created successfully"
}

This helps with:

• Faster searching
• Better debugging
• Traceability
• Alerting
• Audit logs

---

Step 15: Better Logback Pattern with Trace ID

If you use Spring Cloud Sleuth, Micrometer Tracing, or OpenTelemetry, include trace ID.

<pattern>
%d{yyyy-MM-dd HH:mm:ss} %-5level traceId=%X{traceId} spanId=%X{spanId} [%thread] %logger{36} - %msg%n
</pattern>

Example output:

2026-06-30 10:15:30 INFO traceId=abc123 spanId=xyz789 Order created successfully

---

Common Issues and Fixes

Issue	Reason	Fix
Logs not visible in CloudWatch	Agent not running	Check CloudWatch Agent status
Access denied	IAM role missing permission	Attach CloudWatchAgentServerPolicy
Log group not created	Wrong config or permission issue	Validate config file
Empty log stream	Wrong file path	Check application log path
Old logs missing	Log rotation issue	Configure proper log rotation
High log cost	Too many debug logs	Use INFO in production

---

Best Practices

Use these practices in enterprise applications:

1. Use INFO for business flow logs
2. Use ERROR only for real failures
3. Avoid logging passwords, tokens, PII, card numbers, or secrets
4. Add requestId, traceId, userId, orderId where required
5. Use structured JSON logs for large systems
6. Set log retention period
7. Create alarms for ERROR and critical failures
8. Avoid DEBUG logs in production
9. Use centralized dashboards
10. Connect logs with metrics and traces

---

Spring Boot Logging Flow

sequenceDiagram
    participant User
    participant API as Spring Boot API
    participant Logback
    participant File as Local Log File
    participant Agent as CloudWatch Agent
    participant CW as CloudWatch Logs

    User->>API: Call REST API
    API->>Logback: log.info / log.error
    Logback->>File: Write logs to file
    Agent->>File: Read log file
    Agent->>CW: Push logs to log group

---

Final Folder Structure

order-service
├── src
│   └── main
│       ├── java
│       │   └── com
│       │       └── codewithvenu
│       │           └── orders
│       │               └── controller
│       │                   └── OrderController.java
│       └── resources
│           ├── application.yml
│           └── logback-spring.xml
├── pom.xml
└── target
    └── order-service.jar

---

Final Summary

AWS CloudWatch Logs is one of the most important observability services for Spring Boot applications running on AWS.

In this implementation:

Spring Boot writes logs
CloudWatch Agent reads logs
CloudWatch Logs stores logs
Log Insights searches logs
Metric Filter tracks errors
CloudWatch Alarm sends alerts

This setup gives production teams a centralized logging system for debugging, monitoring, and alerting.

---