All technical articles

Fix Linux Disk Full: Logs, Inodes and Deleted Files

A safe workflow for diagnosing Linux disk exhaustion, inode limits, oversized logs, and deleted files still held open.

Fix Linux Disk Full: Logs, Inodes and Deleted Files

“No space left on device” does not always mean the filesystem has used every gigabyte. Inodes may be exhausted, or a deleted file may still be held open by a process. Random deletion can damage databases and services, so measure first.

Check capacity and inodes separately

df -hT
df -i
sudo du -xhd1 / | sort -h
sudo du -xhd1 /var | sort -h

df reports filesystem allocation while du sees reachable files. A large mismatch often indicates an unlinked file that remains open.

Find large files and logs

sudo find /var -xdev -type f -size +500M -printf '%s %p
' | sort -n
sudo journalctl --disk-usage
sudo lsof +L1

Identify the owning service before changing anything. Set journal retention and correct logrotate rules. Never delete database data, WAL, or transaction logs without the product’s supported procedure.

Handle inode exhaustion

Millions of cache, session, or temporary files can consume all inodes. Compare suspicious trees with find /path -xdev -type f | wc -l, then remove only files whose owner and lifecycle are understood.

Recover and prevent recurrence

  1. Protect critical data with a snapshot or backup.
  2. Stop or throttle the process causing growth.
  3. Rotate, compress, or vacuum logs through supported commands.
  4. Recheck free space and inodes.
  5. Alert on usage percentage, growth rate, and inode pressure.

Finally test database writes, application health, and log creation. The incident is not closed until the growth source is fixed.

Short answer

“No space left on device” can mean exhausted blocks, exhausted inodes, or deleted files still held open by a process. Check all three before deleting data. Random cleanup may remove evidence or application files while failing to release the space actually consumed by an open descriptor.

Common symptoms

  • df reports a full filesystem
  • df -i shows inode exhaustion despite free gigabytes
  • Disk space remains used after a large log is deleted

Likely causes

  • Unbounded logs, backups, or temporary files
  • Millions of small cache or session files
  • A deleted log still open by a running process

Diagnostic commands and sample output

df -hT
df -i
du -xhd1 /var | sort -h
lsof +L1

Sample:

COMMAND PID USER FD TYPE DEVICE SIZE/OFF NLINK NAME
nginx  221 root 4w REG  8,1    8G       0 /var/log/nginx/access.log (deleted)

How to verify the fix

Confirm both block and inode use return below the alert threshold. Restart or reopen the owning service and verify lsof +L1 no longer lists the deleted large file.

Risks and rollback

Do not delete database, journal, or package files blindly. Archive logs, validate backups, and keep the command history. Roll back by restoring required files and reverting logrotate changes.

Anonymized field scenario

A server remained 100% full after an access log was removed. lsof +L1 showed Nginx still holding the deleted file; reopening logs released the space without deleting application data.

Official references

Last reviewed: 2026-10-01