Linux Advanced Concepts Interview Questions and Answers

Top Linux Advanced Concepts interview questions and answers for Java Developers, DevOps Engineers, Cloud Engineers, SREs, and Platform Engineers.

Introduction

Linux is the operating system behind most enterprise applications, cloud platforms, Kubernetes clusters, and Docker containers. Understanding Linux internals helps developers troubleshoot production issues, optimize applications, and perform well in technical interviews.

This guide covers the most commonly asked Linux Advanced interview questions with concise answers, diagrams, and practical commands.

1. What is the difference between User Space and Kernel Space?

Answer

Linux separates execution into two protected memory regions:

User Space — runs applications (Java, Python, Nginx, etc.) with restricted hardware access
Kernel Space — runs the OS core with full hardware access (CPU, memory, disk, network)

Applications cross the boundary only via System Calls. This separation prevents buggy or malicious applications from corrupting the kernel.

graph TD
    A[User Space<br/>Java / Python / Nginx / Shell] -->|System Call| B[Kernel Space<br/>Process Mgmt / Memory / I/O / Network]
    B -->|Hardware Abstraction| C[Hardware<br/>CPU / RAM / Disk / NIC]

Layer	Privilege	Examples
User Space	Unprivileged	Applications, libraries, shells
Kernel Space	Privileged	Kernel, device drivers, syscalls

2. What is a System Call?

Answer

A System Call (syscall) is the mechanism by which a user-space program requests a service from the Linux kernel. When a syscall is invoked, the CPU switches from user mode to kernel mode, executes the request, then switches back.

sequenceDiagram
    participant App as User Space App
    participant Lib as glibc (libc)
    participant Kernel as Linux Kernel
    App->>Lib: fopen() / read() / write()
    Lib->>Kernel: syscall instruction (INT 0x80 / SYSCALL)
    Kernel-->>Lib: return value / errno
    Lib-->>App: result

System Call	Number	Purpose
`read()`	0	Read from file/socket
`write()`	1	Write to file/socket
`open()`	2	Open a file
`fork()`	57	Create a child process
`exec()`	59	Execute a program
`exit()`	60	Terminate a process
`mmap()`	9	Map memory

# Trace all system calls made by a command
strace ls

# Summarise syscall counts
strace -c ls

# Attach to a running process
strace -p <PID>

3. Explain Linux Architecture.

Answer

Linux is structured into four layers, each building on top of the one below:

graph TD
    A[Applications<br/>Java · Nginx · Python · Databases] --> B[Shell<br/>bash · zsh · sh]
    B --> C[Kernel<br/>Process Mgmt · Memory Mgmt · VFS · Network Stack · Device Drivers]
    C --> D[Hardware<br/>CPU · RAM · Disk · NIC · GPU]

Layer	Role
Applications	End-user programs — communicate via POSIX APIs
Shell	Command interpreter, scripting engine, user interaction layer
Kernel	Core OS — manages CPU scheduling, memory, I/O, filesystems, IPC
Hardware	Physical resources that the kernel abstracts away

The kernel exposes hardware through device drivers and virtual filesystems like /proc and /sys.

4. What is a Process?

Answer

A Process is a running instance of a program. The kernel assigns it an isolated virtual address space, open file descriptors, and one or more execution threads.

Every process has:

Attribute	Description
PID	Unique Process ID
PPID	Parent Process ID
UID/GID	Owning user/group — controls privilege
Memory	Code (text), data, heap, stack segments
FDs	Open file descriptors (stdin=0, stdout=1, stderr=2)
Threads	One or more threads sharing the same address space

# List all processes with full detail
ps -ef

# Show process tree with PIDs
pstree -p

# Inspect a specific process
cat /proc/<PID>/status

# View memory map of a process
cat /proc/<PID>/maps

5. Explain Process Lifecycle.

Answer

A process transitions through several states from creation to termination:

graph LR
    A[New<br/>fork/exec called] --> B[Ready<br/>waiting for CPU]
    B --> C[Running<br/>executing on CPU]
    C --> B
    C --> D[Waiting<br/>blocked on I/O or event]
    D --> B
    C --> E[Terminated<br/>exit called]
    E --> F{Parent called wait?}
    F -->|Yes| G[Cleaned up]
    F -->|No| H[Zombie<br/>entry kept in process table]

Transition	Trigger
New → Ready	`fork()` / `exec()` completes
Ready → Running	Scheduler assigns CPU
Running → Waiting	I/O request, `sleep()`, `wait()`
Waiting → Ready	I/O completes, timer fires, signal received
Running → Terminated	`exit()` called or signal received

6. What are Process States?

Answer

State	Symbol	Description
Running	`R`	Actively using the CPU or in the run queue
Sleeping	`S`	Waiting for event (interruptible — can receive signals)
Disk Sleep	`D`	Waiting for I/O (uninterruptible — cannot be killed)
Stopped	`T`	Paused by `SIGSTOP` or debugger
Tracing Stop	`t`	Paused by debugger during tracing
Zombie	`Z`	Finished but parent has not called `wait()`
Idle	`I`	Idle kernel thread

# View all processes with state column
ps aux

# Show only zombie processes
ps aux | awk '$8 == "Z" {print}'

# Watch process states live
watch -n1 'ps -eo pid,stat,cmd | head -20'

7. What is a Zombie Process?

Answer

A Zombie process has completed execution but remains in the process table because the parent has not yet called wait() to collect its exit status. Zombies hold no memory or CPU but consume a process table slot.

graph TD
    A[Parent Process] -->|fork| B[Child Process]
    B -->|exit called| C[Zombie State Z<br/>entry stays in process table]
    C -->|parent calls wait| D[Entry removed - cleaned up]
    A -->|never calls wait| E[Zombie accumulates<br/>PID table exhaustion risk]

Risks:

Each zombie occupies one PID slot
If thousands accumulate, new processes cannot be created (fork() fails)

# Find zombie processes
ps aux | grep ' Z '

# Find parent of zombies (the responsible process)
ps -eo pid,ppid,stat,cmd | awk '$3 ~ /Z/'

# Force parent to reap by sending SIGCHLD
kill -SIGCHLD <PPID>

8. What is an Orphan Process?

Answer

An Orphan process is a child whose parent process has terminated before the child. Linux automatically re-parents all orphans to systemd (PID 1), which periodically calls wait() to collect their exit status — preventing them from becoming zombies.

graph TD
    A[Parent exits unexpectedly] --> B[Child becomes Orphan]
    B --> C[systemd PID 1 adopts orphan]
    C --> D[Child runs to completion]
    D --> E[systemd calls wait - no zombie]

Orphans are harmless — the key difference from zombies is that systemd actively manages them.

9. What is a Daemon Process?

Answer

A Daemon is a long-running background process that has no controlling terminal (TTY). Daemons are started at boot, run as system services, and typically end in d by convention.

graph LR
    A[systemd PID1] --> B[sshd<br/>SSH access]
    A --> C[nginx<br/>Web server]
    A --> D[crond<br/>Job scheduler]
    A --> E[dockerd<br/>Container runtime]
    A --> F[mysqld<br/>Database]
    A --> G[journald<br/>Logging]

Daemon	Purpose
`sshd`	SSH remote access
`nginx`	HTTP/HTTPS web server
`crond`	Scheduled job execution
`mysqld`	MySQL database server
`dockerd`	Docker container runtime
`kubelet`	Kubernetes node agent
`auditd`	Linux audit framework

# List all running services managed by systemd
systemctl list-units --type=service --state=running

# Check if a service is a daemon (no TTY)
cat /proc/<PID>/status | grep "^Tgid\|^TTY"

10. Difference between Process and Thread?

Answer

graph TD
    subgraph ProcessA[Process A - own address space]
        T1[Thread 1<br/>stack + registers]
        T2[Thread 2<br/>stack + registers]
        SH[Shared: Heap + Code + FDs + Globals]
    end
    subgraph ProcessB[Process B - separate address space]
        T3[Thread 3]
    end

Aspect	Process	Thread
Memory	Separate address space	Shared within the process
Weight	Heavyweight (slow to create)	Lightweight (fast to create)
Resources	Own file descriptors, signals	Shares parent's FDs, signals
Communication	IPC (pipes, sockets, shm)	Direct shared memory
Crash impact	Only the process dies	Can crash the entire process
Context switch	Expensive (TLB flush)	Cheaper (same address space)
Java relevance	JVM per process	JVM threads (platform/virtual)

In Java, virtual threads (JDK 21+) are lightweight user-space threads that drastically reduce thread-creation overhead.

11. What is Context Switching?

Answer

Context Switching is the kernel operation of saving the CPU state (registers, program counter, stack pointer, memory maps) of the current process/thread and restoring the state of the next scheduled one.

sequenceDiagram
    participant CPU
    participant ProcA as Process A
    participant Kernel as Kernel Scheduler
    participant ProcB as Process B
    ProcA->>CPU: executing
    Kernel->>CPU: timer interrupt fires
    CPU->>ProcA: save state (PCB)
    CPU->>ProcB: restore state (PCB)
    ProcB->>CPU: executing

Cost factors:

Saving/restoring CPU registers and FPU state
TLB flush (for process switches — expensive)
Cache warmup cost for the new process

# Monitor context switches in real time
vmstat 1 5
# cs column = context switches per second

# Per-process context switches
pidstat -w -p <PID> 1

# Voluntary vs involuntary switches
cat /proc/<PID>/status | grep ctxt

High involuntary context switches indicate CPU contention. High voluntary switches indicate the process is frequently waiting on I/O or locks.

12. What is Nice Value?

Answer

Nice Value adjusts the CPU scheduling priority hint for the Linux CFS (Completely Fair Scheduler).

Nice Value	Priority	Range
`-20`	Highest	Root only
`0`	Default	Any user
`+19`	Lowest	Any user

Lower nice = higher CPU priority. Non-root users can only increase nice values (lower priority). Root can set any value.

# Start a process with lower priority
nice -n 10 java -jar app.jar

# Change priority of a running process
renice -n 5 -p <PID>

# View nice values for all processes
ps -eo pid,ni,comm --sort=-ni | head -20

# View in top: NI column
top

Real-time priorities (chrt) bypass the nice system entirely and are used for latency-sensitive workloads:

# Set real-time FIFO priority
chrt -f 50 <command>

13. Difference between SIGTERM and SIGKILL?

Answer

Signal	Number	Catchable	Behaviour
`SIGHUP`	1	Yes	Terminal hangup — reload config in daemons
`SIGINT`	2	Yes	Keyboard Ctrl+C — graceful interrupt
`SIGTERM`	15	Yes	Graceful shutdown — process can clean up and exit
`SIGKILL`	9	No	Immediate forced termination — cannot be caught
`SIGSTOP`	19	No	Pause process — cannot be caught
`SIGCONT`	18	Yes	Resume a stopped process
`SIGCHLD`	17	Yes	Notify parent when child exits

# Graceful shutdown (always try first)
kill -15 <PID>
# or simply:
kill <PID>

# Force kill (last resort)
kill -9 <PID>

# Kill all processes matching a name
pkill -15 java
killall nginx

# Send SIGHUP to reload config (e.g. nginx)
kill -HUP <PID>

Best Practice: Always send SIGTERM first, wait 10–30 seconds for graceful exit, then send SIGKILL only if necessary. Kubernetes uses this pattern with terminationGracePeriodSeconds.

14. What is systemd?

Answer

systemd is the standard Linux init system (PID 1) used in all major distributions. It replaces the older SysV init and Upstart systems.

graph TD
    A[systemd PID 1] --> B[Service Units<br/>nginx.service, sshd.service]
    A --> C[Timer Units<br/>cron replacement]
    A --> D[Socket Units<br/>socket activation]
    A --> E[Mount Units<br/>filesystem mounts]
    A --> F[journald<br/>centralised logging]
    A --> G[logind<br/>session management]

Key responsibilities:

Function	Description
Parallel service startup	Starts services concurrently, reducing boot time
Dependency management	`After=`, `Requires=`, `Wants=` directives
Service supervision	Auto-restart on failure (`Restart=on-failure`)
Centralised logging	All service output captured by `journald`
Cgroup integration	Each service in its own cgroup for resource control

# Service lifecycle
systemctl status nginx
systemctl start / stop / restart / reload nginx

# Boot persistence
systemctl enable / disable nginx

# View unit file
systemctl cat nginx

# Show dependency tree
systemctl list-dependencies nginx

# Failed services
systemctl --failed

15. Explain Linux Boot Process.

Answer

graph TD
    A[Power On] --> B[BIOS / UEFI<br/>POST - hardware check, locate bootable device]
    B --> C[GRUB2 Bootloader<br/>loads kernel image + initramfs from disk]
    C --> D[Kernel Loads<br/>decompresses, initialises hardware, mounts initramfs]
    D --> E[initramfs tmpfs<br/>mounts real root filesystem, loads drivers]
    E --> F[systemd PID 1<br/>reads unit files, starts services in parallel]
    F --> G[Services<br/>network, storage, database, application]
    G --> H[Login Prompt / Display Manager]

Stage	Binary / File	Responsibility
BIOS/UEFI	Firmware	Hardware POST, find bootable partition
GRUB2	`/boot/grub/grub.cfg`	Presents boot menu, loads kernel + initramfs
Kernel	`/boot/vmlinuz-*`	Decompresses, init hardware, mounts tmpfs
initramfs	`/boot/initrd.img-*`	Early userspace — loads root FS drivers, decryption
systemd	`/sbin/init` → systemd	Starts all units per dependency graph

# View boot messages
journalctl -b

# Analyse boot timing
systemd-analyze
systemd-analyze blame      # slowest units
systemd-analyze critical-chain

# View GRUB config
cat /boot/grub/grub.cfg

16. What is journalctl?

Answer

journalctl queries logs stored by systemd-journald, which collects stdout/stderr from every systemd service, kernel messages, and audit events — all in a structured binary format.

# Stream all logs in real time
journalctl -f

# Logs for a specific service
journalctl -u nginx -f

# Logs since last boot
journalctl -b

# Logs from two boots ago
journalctl -b -2

# Time-bounded queries
journalctl --since "2024-01-15 10:00:00" --until "2024-01-15 11:00:00"
journalctl --since "1 hour ago"

# Kernel (dmesg) messages only
journalctl -k

# Priority filter (0=emerg, 3=err, 5=notice, 7=debug)
journalctl -p err

# JSON output for parsing
journalctl -u nginx -o json-pretty | head -40

# Disk usage of journal
journalctl --disk-usage

# Vacuum old logs
journalctl --vacuum-time=30d

17. What are Inodes?

Answer

An inode (index node) is a kernel data structure that stores file metadata — everything about a file except its name and data content.

graph LR
    A[Directory Entry<br/>filename → inode number] --> B[Inode<br/>UID/GID · permissions<br/>size · timestamps<br/>link count · data block ptrs]
    B --> C[Data Blocks<br/>actual file content]

Inode contents:

Attribute	Description
UID / GID	Owner user and group
Permissions	`rwxrwxrwx` + special bits
Size	File size in bytes
atime	Last access time
mtime	Last modification time (content)
ctime	Last change time (metadata or content)
Link count	Number of hard links pointing to this inode
Data pointers	Direct, indirect, double-indirect block pointers

The filename is stored in the directory, not the inode. This is why multiple filenames (hard links) can refer to the same inode.

# Show inode numbers
ls -li

# Show inode usage per filesystem
df -i

# Find files with a specific inode number
find / -inum <INODE_NUMBER>

# Stat a file (shows all inode metadata)
stat filename.txt

18. Difference between Hard Link and Soft Link?

Answer

graph TD
    subgraph Hardlink
        H1[file.txt - dir entry] --> IN[Inode 1234<br/>data blocks]
        H2[hardlink.txt - dir entry] --> IN
    end
    subgraph Softlink
        S1[original.txt - dir entry] --> IN2[Inode 5678]
        SL[symlink.txt - dir entry] --> IN3[Inode 9999<br/>stores path to original.txt]
        IN3 -.->|resolves to| IN2
    end

Aspect	Hard Link	Soft Link (Symlink)
Inode	Same inode as original	Different inode (stores path)
Cross-filesystem	❌ No	✅ Yes
Original deleted	File still accessible (link count > 0)	Link breaks (dangling symlink)
Target type	Files only	Files and directories
`ls -la` indicator	Number in 2nd column	`->` arrow
Common use	Backup without duplication	`/etc/alternatives`, versioned libs

# Create a hard link
ln file.txt hardlink.txt

# Create a symbolic (soft) link
ln -s /path/to/original.txt symlink.txt

# Verify: both hard links share same inode
ls -li file.txt hardlink.txt

# Find all hard links to an inode
find /home -inum $(stat -c '%i' file.txt)

# Find broken symlinks
find /etc -type l ! -e

19. What is a File Descriptor?

Answer

A File Descriptor (FD) is a non-negative integer that uniquely identifies an open file, socket, pipe, or device within a process. The kernel maintains an open file table per process.

graph LR
    A[Process FD Table] --> B[FD 0 stdin]
    A --> C[FD 1 stdout]
    A --> D[FD 2 stderr]
    A --> E[FD 3 - open socket]
    A --> F[FD 4 - open file]
    B --> G[Keyboard]
    C --> H[Terminal]
    D --> H
    E --> I[Network socket]
    F --> J[/var/log/app.log]

FD	Name	Default target	Redirect operator
0	`STDIN`	Keyboard	`< file`
1	`STDOUT`	Terminal	`> file`
2	`STDERR`	Terminal	`2> file`

# View all open FDs for a process
ls -l /proc/<PID>/fd

# Count open FDs (useful for FD leak detection)
ls /proc/<PID>/fd | wc -l

# System-wide open FD limit
cat /proc/sys/fs/file-max

# Per-process soft limit
ulimit -n

# Redirect STDERR to file
command 2> error.log

# Redirect both STDOUT and STDERR
command > output.log 2>&1

# /dev/null: discard output
command > /dev/null 2>&1

FD leak is a common Java production issue — a process opens files/sockets but never closes them, eventually hitting Too many open files (EMFILE).

20. Explain Linux File Permissions.

Answer

Linux uses a DAC (Discretionary Access Control) model. Every file has three permission sets and three operation types:

graph LR
    A[-rwxr-xr--] --> B[Owner rwx]
    A --> C[Group r-x]
    A --> D[Others r--]

- r w x | r - x | r - -
  ^ ^ ^   ^ ^ ^   ^ ^ ^
  |       |       |
  Owner   Group   Others

Permission	Octal	File meaning	Directory meaning
`r`	4	Read content	List directory entries
`w`	2	Write/modify	Create/delete files in dir
`x`	1	Execute	Enter / traverse directory

Numeric	Symbolic	Common usage
`755`	`rwxr-xr-x`	Executables, public directories
`644`	`rw-r--r--`	Config files, web assets
`600`	`rw-------`	Private keys, secrets
`700`	`rwx------`	Private scripts
`777`	`rwxrwxrwx`	Avoid — world writable

# Set permissions (numeric)
chmod 755 script.sh

# Set permissions (symbolic)
chmod u+x,g-w,o-r file.txt

# Recursive permission change
chmod -R 644 /var/www/html

# Change file ownership
chown user:group file.txt

# Recursive ownership change
chown -R www-data:www-data /var/www

21. What are SUID, SGID and Sticky Bit?

Answer

These are special permission bits that modify default execution or deletion behaviour.

Special Bit	Octal	Effect on Files	Effect on Directories
SUID	4000	File runs with owner's UID privileges	No effect
SGID	2000	File runs with group's GID privileges	New files inherit the directory's group
Sticky Bit	1000	No effect	Only the file owner can delete their files

Examples in the wild:

File/Dir	Bit set	Why
`/usr/bin/passwd`	SUID root	Needs to write `/etc/shadow` as root
`/usr/bin/newgrp`	SGID	Needs group access to change GID
`/tmp`	Sticky bit	Users can only delete their own files in shared directory
`/var/mail`	SGID	Shared group write access to mail spools

# Set SUID
chmod u+s /usr/bin/myapp
# or: chmod 4755 /usr/bin/myapp

# Set SGID on directory (shared team folder)
chmod g+s /shared/team
# or: chmod 2775 /shared/team

# Set Sticky Bit
chmod +t /tmp
# or: chmod 1777 /tmp

# Find all SUID files (security audit)
find / -perm /4000 -type f 2>/dev/null

# Find all SGID files
find / -perm /2000 -type f 2>/dev/null

Security note: SUID binaries on the root path are a common privilege escalation vector. Audit them regularly.

22. What is LVM?

Answer

LVM (Logical Volume Manager) provides an abstraction layer between physical disks and filesystems, enabling dynamic, flexible disk management.

graph TD
    A[Physical Disks<br/>sdb sdc sdd] --> B[Physical Volumes PV<br/>pvcreate]
    B --> C[Volume Group VG<br/>vgcreate - pool of storage]
    C --> D[Logical Volumes LV<br/>lvcreate - virtual partitions]
    D --> E[Filesystem<br/>ext4 xfs]
    E --> F[Mount Point<br/>/data /var /home]

LVM Layer	Command	Description
Physical Volume (PV)	`pvcreate`	Initialise a disk/partition for LVM
Volume Group (VG)	`vgcreate`	Pool of one or more PVs
Logical Volume (LV)	`lvcreate`	Flexible partition carved from VG

Key benefits:

Resize volumes without unmounting (ext4 online resize, xfs only grows)
Thin provisioning — allocate more space than physically available
Snapshots for point-in-time backups
Striping across multiple disks for performance

# Create physical volume
pvcreate /dev/sdb /dev/sdc

# Create volume group from two PVs
vgcreate datavg /dev/sdb /dev/sdc

# Create 20GB logical volume
lvcreate -L 20G -n datalv datavg

# Format and mount
mkfs.xfs /dev/datavg/datalv
mount /dev/datavg/datalv /mnt/data

# Extend logical volume by 10G (online)
lvextend -L +10G /dev/datavg/datalv
xfs_growfs /mnt/data         # XFS: online resize
# resize2fs /dev/datavg/datalv  # ext4: online resize

# Take a snapshot for backup
lvcreate -L 5G -s -n snap /dev/datavg/datalv

# Show LVM status
pvs; vgs; lvs

23. What is Swap Memory?

Answer

Swap is disk space the kernel uses as an overflow extension of RAM. When physical memory is exhausted, the kernel swaps out (pages out) inactive memory pages to the swap device to free up RAM.

graph LR
    A[Application<br/>requests more memory] --> B{RAM available?}
    B -->|Yes| C[Allocate from RAM]
    B -->|No| D[Page out cold pages to Swap]
    D --> E[Allocate freed RAM]
    E --> F[Application continues]
    D --> G{Swap also full?}
    G -->|Yes| H[OOM Killer fires]

Concept	Detail
`swappiness`	Kernel tendency to swap (0=avoid, 60=default, 100=aggressive)
Swap file	Regular file used as swap — flexible, easy to resize
Swap partition	Dedicated partition — slightly faster
`zswap`	Compressed swap cache in RAM — reduces disk I/O

# View RAM and swap usage
free -h

# Show active swap devices
swapon --show

# Tune swappiness (lower = prefer RAM)
sysctl vm.swappiness=10
echo "vm.swappiness=10" >> /etc/sysctl.conf

# Create and enable a 2GB swap file
dd if=/dev/zero of=/swapfile bs=1M count=2048
chmod 600 /swapfile
mkswap /swapfile
swapon /swapfile
echo '/swapfile none swap sw 0 0' >> /etc/fstab

24. What is the OOM Killer?

Answer

The OOM (Out of Memory) Killer is a Linux kernel subsystem that terminates one or more processes when the system exhausts both physical RAM and swap, to prevent a complete system freeze.

graph TD
    A[Memory allocation request fails] --> B[Kernel OOM path triggered]
    B --> C[Calculate oom_score for all processes<br/>0 = never kill, 1000 = kill first]
    C --> D[Select victim - highest oom_score]
    D --> E[Send SIGKILL to victim]
    E --> F[Memory freed - system continues]
    B --> G[Log to dmesg and journald]

oom_score factors:

Process memory consumption (larger = higher score)
Process age (newer = higher score)
Nice value
oom_score_adj manual adjustment (-1000 = fully protected, +1000 = always first)

# Check OOM score of a process
cat /proc/<PID>/oom_score

# Check adjustment value
cat /proc/<PID>/oom_score_adj

# Protect a critical process from OOM killer (root only)
echo -1000 > /proc/<PID>/oom_score_adj

# Make a process the first OOM victim
echo 1000 > /proc/<PID>/oom_score_adj

# Check if OOM killer has fired
dmesg | grep -i "oom\|killed process"
journalctl -k | grep -i "out of memory"

# Persist oom_score_adj in a systemd service
# Add to service unit file:
# OOMScoreAdjust=-500

Java note: JVM processes often have large oom_score because they pre-allocate heap. Use -XX:+ExitOnOutOfMemoryError and set OOMScoreAdjust=-500 in the systemd unit to protect critical Java services.

25. What is Load Average?

Answer

Load Average is a running average of the number of processes in a runnable (R) or uninterruptible sleep (D) state sampled over 1, 5, and 15 minutes.

# View load average
uptime
# output: 14:32 up 10 days, 2 users, load average: 1.20, 0.85, 0.60
#                                                    ^^^^  ^^^^  ^^^^
#                                                    1min  5min  15min

Interpretation:

Load vs CPU cores	Meaning
Load < n_cpus	System is underloaded — capacity available
Load ≈ n_cpus	System is at full utilisation — no spare capacity
Load > n_cpus	System is overloaded — processes queuing for CPU

# Number of logical CPU cores
nproc

# Detailed CPU info
lscpu

# Load per CPU in real time
mpstat -P ALL 1

# Historical load trend
sar -q 1 10

A high load average driven by D state processes usually indicates I/O bottleneck (disk or network), not CPU starvation. Use iostat to distinguish.

26. Difference between top and htop?

Answer

Feature	`top`	`htop`
Availability	Pre-installed on all Linux	Requires installation
Interface	Monochrome, less interactive	Colour-coded, fully interactive
Scroll	Limited	Full vertical and horizontal scroll
Process tree	No	Yes (`F5`)
Kill process	`k` key (type PID)	Arrow key to select → `F9`
Filter/Search	Basic	`F3` incremental search
Per-CPU bars	Single aggregate line	Individual bar per CPU core
Mouse support	No	Yes
Config persistence	No	Yes (`~/.config/htop/htoprc`)

# Standard process monitor
top

# Useful top shortcuts:
# P = sort by CPU, M = sort by memory
# k = kill, r = renice, 1 = toggle per-CPU

# Install htop
sudo apt install htop     # Debian/Ubuntu
sudo yum install htop     # RHEL/CentOS

htop

# alternatives for container environments
sudo apt install glances  # web UI + sensors

27. Difference between df and du?

Answer

Command	Scope	Source	Shows
`df`	Filesystem / mount point	Superblock metadata	Total / used / available
`du`	Directory or file tree	Walks filesystem recursively	Actual disk usage by path

df and du can disagree — deleted files still held open by processes consume space df sees but du misses.

# Filesystem usage (human-readable)
df -h

# Inode usage (detect inode exhaustion)
df -i

# Size of current directory
du -sh .

# Size of each immediate subdirectory
du -sh /*

# Top 10 largest directories under /var
du -ah /var 2>/dev/null | sort -rh | head -10

# Find deleted files still held open (space reclaim)
lsof | grep deleted

28. What are Linux Namespaces?

Answer

Namespaces partition global kernel resources so that each set of processes sees its own isolated instance. Namespaces are the primary technology that makes containers (Docker, Kubernetes) possible.

graph TD
    A[Linux Kernel] --> B[pid namespace<br/>isolated process IDs]
    A --> C[net namespace<br/>isolated network stack]
    A --> D[mnt namespace<br/>isolated filesystem mounts]
    A --> E[ipc namespace<br/>isolated IPC objects]
    A --> F[user namespace<br/>isolated UID/GID mapping]
    A --> G[uts namespace<br/>isolated hostname]
    A --> H[cgroup namespace<br/>isolated cgroup view]
    A --> I[time namespace<br/>isolated clock offsets]

Namespace	Flag	Isolates
`pid`	`CLONE_NEWPID`	Process IDs — PID 1 inside container
`net`	`CLONE_NEWNET`	Network interfaces, routes, iptables rules
`mnt`	`CLONE_NEWNS`	Mount points and filesystem view
`ipc`	`CLONE_NEWIPC`	SysV IPC, POSIX message queues
`user`	`CLONE_NEWUSER`	UID/GID mappings (rootless containers)
`uts`	`CLONE_NEWUTS`	Hostname and NIS domain name
`cgroup`	`CLONE_NEWCGROUP`	cgroup root directory view

# View namespaces of a running process
ls -la /proc/<PID>/ns

# View namespaces of a container
docker inspect --format '{{.State.Pid}}' <container>
ls -la /proc/<PID>/ns

# Enter a container's network namespace
nsenter --net=/proc/<PID>/ns/net ip addr

# Create and use a new network namespace
ip netns add myns
ip netns exec myns bash

29. What are cgroups?

Answer

Control Groups (cgroups) limit and account for resource usage (CPU, memory, I/O, network) of groups of processes. Together with namespaces, cgroups form the foundation of container isolation.

graph TD
    A[cgroup v2 hierarchy /sys/fs/cgroup] --> B[system.slice<br/>OS services]
    A --> C[user.slice<br/>user sessions]
    A --> D[docker<br/>containers]
    D --> E[container-abc<br/>cpu.max memory.max io.max]
    D --> F[container-xyz<br/>cpu.max memory.max io.max]

Resource	cgroup v2 file	Controls
CPU	`cpu.max`	CPU bandwidth quota (e.g. 50% of one core)
CPU weight	`cpu.weight`	Relative CPU shares (default 100)
Memory	`memory.max`	Hard memory limit — triggers OOM
Memory soft	`memory.high`	Soft limit — triggers reclaim before OOM
Disk I/O	`io.max`	Read/write IOPS and bandwidth throttle
PIDs	`pids.max`	Maximum number of PIDs in group

# View cgroup v2 hierarchy
ls /sys/fs/cgroup/

# Find cgroup of a running process
cat /proc/<PID>/cgroup

# Check memory limit of a Docker container
cat /sys/fs/cgroup/system.slice/docker-<ID>.scope/memory.max

# View CPU quota for a container
cat /sys/fs/cgroup/system.slice/docker-<ID>.scope/cpu.max
# format: quota period (e.g. 50000 100000 = 50% of 1 core)

# Resource limit via systemd
systemctl set-property nginx.service CPUQuota=50% MemoryMax=512M

Docker / Kubernetes mapping: --cpus=0.5 → cpu.max=50000 100000, --memory=512m → memory.max=536870912.

30. What is the /proc Filesystem?

Answer

/proc is a virtual filesystem (pseudofilesystem) that the Linux kernel exposes at runtime. It contains no files on disk — everything is generated on-the-fly by the kernel.

Path	Content
`/proc/<PID>/`	Per-process directory
`/proc/<PID>/status`	Process state, memory, UID, threads
`/proc/<PID>/cmdline`	Full command line
`/proc/<PID>/fd/`	Open file descriptors
`/proc/<PID>/maps`	Memory map (virtual address spaces)
`/proc/<PID>/net/`	Network statistics for the process's namespace
`/proc/cpuinfo`	CPU model, cores, features
`/proc/meminfo`	RAM and swap statistics
`/proc/loadavg`	Load averages + running process count
`/proc/net/tcp`	TCP connection table
`/proc/sys/`	Kernel tunables (writable via `sysctl`)

# Memory stats
cat /proc/meminfo

# CPU info
cat /proc/cpuinfo | grep 'model name' | uniq

# Kernel network tunables
cat /proc/sys/net/ipv4/ip_forward

# Enable IP forwarding immediately (required for K8s/Docker networking)
echo 1 > /proc/sys/net/ipv4/ip_forward
# or:
sysctl -w net.ipv4.ip_forward=1

# Check what a process is doing
cat /proc/<PID>/status
cat /proc/<PID>/cmdline | tr '\0' ' '

31. A Java application is consuming 100% CPU. How do you troubleshoot it?

Answer

Follow a systematic top-down approach:

graph TD
    A[Alert: Java process at 100% CPU] --> B[Identify the PID<br/>ps -ef grep java]
    B --> C[Find hot thread<br/>top -H -p PID]
    C --> D[Convert TID to hex<br/>printf '%x' TID]
    D --> E[Capture thread dump<br/>jstack PID]
    E --> F[Find thread by nid=0xHEX<br/>in thread dump]
    F --> G{Thread state?}
    G -->|RUNNABLE in app code| H[Infinite loop or hot algorithm]
    G -->|RUNNABLE in GC| I[Excessive GC - check jstat]
    G -->|BLOCKED| J[Lock contention - check monitors]
    G -->|Many RUNNABLE| K[High load / thread pool saturated]

Step-by-step commands:

# Step 1 — Identify the Java process
ps -ef | grep java
# Note the PID

# Step 2 — Confirm CPU usage
top
# 'P' key to sort by CPU — confirm PID

# Step 3 — Find the hot thread (thread-level CPU breakdown)
top -H -p <PID>
# Note the TID (thread ID) consuming most CPU

# Step 4 — Convert TID to hex (for jstack mapping)
printf '%x\n' <TID>
# Example: TID 12345 → 0x3039

# Step 5 — Capture a thread dump
jstack <PID> > /tmp/threaddump.txt
grep -A 30 'nid=0x<HEX_TID>' /tmp/threaddump.txt

# Step 6 — Check GC activity
jstat -gcutil <PID> 1000 10
# If FGC is high and GC time > 50%: memory leak or heap too small

# Step 7 — Check heap usage
jmap -heap <PID>

# Step 8 — Review logs
journalctl -u myapp -f
tail -f /var/log/myapp/application.log

# Step 9 — Check system-level resources
free -h        # available memory
df -h          # disk space (full disk can cause GC loops)
iostat -xz 1   # disk I/O utilisation

Common Root Causes

Root Cause	Indicator	Fix
Infinite loop	Single thread pinned at 100% in app code	Code review, add loop guards
Excessive GC (full GC)	`jstat` FGC high, EC/OC near 100%	Increase heap, fix memory leak
Lock contention	Many threads in `BLOCKED` state in thread dump	Reduce lock scope, use concurrent collections
High traffic spike	Many `RUNNABLE` threads in business logic	Scale out, add load balancer
Memory leak → OOM loop	GC overhead limit exceeded, OOM in logs	Heap dump + MAT analysis
Regex catastrophic backtrack	Single thread in `java.util.regex`	Rewrite regex, add timeout

Interview Summary

Topic	Key Concept
Linux Architecture	Hardware → Kernel → Shell → Applications
User / Kernel Space	System calls bridge the privilege boundary
System Calls	`strace` to trace, every I/O is a syscall
Processes	PID, PPID, states: R/S/D/T/Z
Process Lifecycle	New → Ready → Running → Waiting → Terminated
Zombie / Orphan	Zombie = parent didn't `wait()`; Orphan = adopted by PID 1
Signals	SIGTERM (graceful) vs SIGKILL (forced, uncatchable)
Process Scheduling	Nice values (-20 to +19), CFS scheduler
Context Switching	Save PCB of A, restore PCB of B — TLB flush cost
systemd	PID 1, parallel start, journald, unit dependencies
Boot Process	BIOS → GRUB → Kernel → initramfs → systemd
journalctl	Query structured logs with time/service/priority filters
Inodes	File metadata (not name/data); `df -i` for inode space
Hard vs Soft Links	Hard = same inode; Soft = path pointer, can break
File Descriptors	0=stdin, 1=stdout, 2=stderr; FD leaks → EMFILE
File Permissions	DAC: owner/group/others + SUID/SGID/Sticky
LVM	PV → VG → LV → Filesystem; online resize, snapshots
Swap	Disk overflow for RAM; `swappiness` controls aggressiveness
OOM Killer	Kills highest `oom_score` process; protect with `-1000`
Load Average	> n_cpus = overloaded; `D` state = I/O bottleneck
Namespaces	pid/net/mnt/ipc/user/uts — foundation of containers
cgroups	CPU/memory/I/O limits — Docker `--cpus` / `--memory`
/proc filesystem	Virtual kernel interface; `/proc/sys` = live tunables
Java 100% CPU debug	`top -H` → `jstack` → match `nid=0xHEX` → find root cause