Logging Fundamentals

Learn Logging fundamentals including log levels, structured logging, centralized logging, correlation IDs, log rotation, JSON logging, logging architecture, and enterprise logging best practices.

Introduction

Logging is one of the most important aspects of modern software systems. Every enterprise application generates logs that help engineers understand application behavior, troubleshoot production issues, audit security events, and monitor business transactions.

Without proper logging, identifying production failures becomes extremely difficult. Modern organizations centralize logs from applications, databases, containers, Kubernetes clusters, and cloud services into unified logging platforms.

Popular enterprise logging platforms include:

ELK Stack
Loki
Splunk
Datadog Logs
CloudWatch Logs
Azure Monitor Logs
OpenTelemetry Logs

This guide introduces the logging concepts required for DevOps Engineers, Java Developers, Platform Engineers, SREs, Cloud Engineers, and Solution Architects.

Learning Objectives

After completing this guide, you'll understand

What is Logging?
Why Logging?
Types of Logs
Log Levels
Structured Logging
JSON Logging
Correlation ID
Log Rotation
Centralized Logging
Logging Architecture
Log Lifecycle
Common Logging Frameworks
Production Best Practices

What is Logging?

Logging is the process of recording application, system, or infrastructure events during execution.

Logs help answer questions such as:

What happened?
When did it happen?
Where did it happen?
Why did it happen?

Why Logging?

Without Logging

Application Failure

↓

No Information

↓

Long Investigation

↓

Unknown Root Cause

Problems

Difficult Debugging
Longer Downtime
Poor Visibility
Slow Incident Resolution

With Logging

flowchart LR

Application --> Logs

Logs --> CentralPlatform

CentralPlatform --> Engineer

Engineer --> RootCause

Benefits

Faster Debugging
Production Visibility
Security Auditing
Compliance
Root Cause Analysis

Types of Logs

Application Logs

Generated by applications.

Examples

User Login
Payment Success
Order Created
Exception

System Logs

Generated by operating systems.

Examples

Boot Events
CPU Events
Memory Events
Kernel Messages

Web Server Logs

Generated by

Apache
Nginx
IIS

Examples

HTTP Requests
Status Codes
Access Logs

Database Logs

Generated by databases.

Examples

SQL Queries
Slow Queries
Connections
Replication Events

Security Logs

Examples

Authentication
Authorization
Failed Login
Firewall Events
Audit Logs

Logging Architecture

flowchart LR

Application --> LogFramework

LogFramework --> LogFile

LogFile --> LogCollector

LogCollector --> CentralizedLogging

CentralizedLogging --> Dashboard

Log Levels

Most logging frameworks support the following levels.

Level	Purpose
TRACE	Detailed execution information
DEBUG	Debugging information
INFO	Normal application events
WARN	Potential problems
ERROR	Application failures
FATAL	Critical application failures

Log Level Hierarchy

TRACE

↓

DEBUG

↓

INFO

↓

WARN

↓

ERROR

↓

FATAL

Production Recommendation

Use

INFO
WARN
ERROR

Avoid enabling TRACE in production.

Logging Frameworks

Popular Java frameworks

SLF4J
Logback
Log4j2
java.util.logging

Other platforms

Python logging
Winston (Node.js)
Serilog (.NET)

Example Log

2026-07-13 10:15:22

INFO

PaymentService

Payment completed successfully

TransactionId=TX12345

Structured Logging

Instead of plain text

Payment completed.

Use structured logs

{
  "timestamp":"2026-07-13T10:15:22Z",
  "level":"INFO",
  "service":"payment-service",
  "transactionId":"TX12345",
  "status":"SUCCESS"
}

Advantages

Easy Search
Filtering
Analytics
Machine Readable

JSON Logging

JSON is the preferred format for cloud-native applications.

Benefits

Structured Data
Elasticsearch Friendly
Splunk Friendly
Loki Friendly

Correlation ID

Correlation IDs uniquely identify requests.

flowchart LR

Gateway --> UserService --> PaymentService --> Database

Every log generated for the request contains the same Correlation ID.

Example

CorrelationId

↓

API Gateway

↓

Order Service

↓

Payment Service

↓

Database

Benefits

End-to-End Tracing
Faster Debugging
Distributed Systems Support

MDC (Mapped Diagnostic Context)

MDC automatically attaches contextual information to every log.

Common fields

Correlation ID
User ID
Request ID
Session ID
Tenant ID

Example (Java)

MDC.put("correlationId", correlationId);

Log Rotation

Without rotation

application.log

↓

50 GB

↓

Disk Full

Log rotation creates new log files automatically.

Example

application.log

application.log.1

application.log.2

Benefits

Saves Disk Space
Easier Maintenance
Better Performance

Log Lifecycle

flowchart LR

Application --> GenerateLog --> Store --> Rotate --> Archive --> Delete

Centralized Logging

Instead of storing logs on every server

Server1

Server2

Server3

↓

Central Logging Platform

Benefits

Single Search
Faster Troubleshooting
Compliance
Long-Term Storage

Common Centralized Logging Platforms

Platform	Usage
ELK Stack	Enterprise Logging
Loki	Kubernetes Logging
Splunk	Enterprise Analytics
Datadog Logs	Cloud Monitoring
CloudWatch Logs	AWS
Azure Monitor Logs	Azure

Logging Workflow

flowchart LR

Application --> GenerateLog --> Collector --> Storage --> Search --> Dashboard

Log Retention

Organizations define retention policies.

Example

Application Logs

↓

30 Days

Audit Logs

↓

365 Days

Compliance requirements determine retention duration.

Sensitive Data

Never log

Passwords
Credit Card Numbers
API Keys
Access Tokens
Social Security Numbers
Encryption Keys

Always mask sensitive information.

Logging in Microservices

Every service generates logs.

flowchart LR

API --> UserService

UserService --> PaymentService

PaymentService --> NotificationService

NotificationService --> CentralLogs

Use Correlation IDs to connect logs across services.

Benefits of Logging

Production Troubleshooting
Security Auditing
Compliance
Root Cause Analysis
Capacity Planning
Business Analytics
Performance Analysis

Production Best Practices

Log Levels

INFO for Business Events
WARN for Recoverable Issues
ERROR for Failures
Avoid DEBUG in Production

Log Format

Use JSON
Structured Logging
Consistent Field Names

Security

Never Log Secrets
Mask Sensitive Data
Encrypt Logs
Control Access

Performance

Asynchronous Logging
Log Rotation
Centralized Storage
Retention Policies

Common Logging Frameworks

Framework	Platform
SLF4J	Java API
Logback	Java
Log4j2	Java
Winston	Node.js
Python Logging	Python
Serilog	.NET

Real-World Example

A Spring Boot application processes online payments.

A customer submits a payment request.
A Correlation ID is generated by the API Gateway.
Every microservice logs the request using structured JSON format.
Logs are written through Logback and forwarded to a centralized logging platform.
Security-sensitive fields such as credit card numbers are masked before logging.
Engineers search logs using the Correlation ID to trace the complete payment flow.
A failed database transaction is identified within minutes, reducing the Mean Time to Resolution (MTTR).

Interview Tips

Remember these keywords

Logging
Structured Logging
JSON Logging
Log Levels
TRACE
DEBUG
INFO
WARN
ERROR
FATAL
Correlation ID
MDC
Log Rotation
Centralized Logging
SLF4J
Logback
Log4j2

Summary

Logging provides the foundation for troubleshooting, auditing, and understanding application behavior in production environments. Modern systems rely on structured JSON logs, correlation IDs, centralized logging platforms, and appropriate log levels to make debugging faster and more efficient.

Mastering log levels, structured logging, MDC, log rotation, centralized logging, and secure logging practices prepares you for enterprise DevOps, Java, Cloud, Platform Engineering, SRE, and Solution Architect interviews.

In the next chapter, you'll explore Logging Advanced, covering ELK Stack, Loki, Fluent Bit, Fluentd, OpenTelemetry Logs, Kubernetes logging, cloud logging, indexing, performance optimization, and enterprise logging architectures.