19

The drive is constantly filling up. You've hunted down all the spare and random files you can. grep'd for coredump files and even removed some of the un-needed backups...

What would be your next move.

The actual server in question has 10GB of website files and the OS shouldn't take any more than 10GB so how do you track down what's filling a 50GB (virtual) drive?

HopelessN00b
  • 54,273
Gareth
  • 8,733

11 Answers11

26

Surely there are more elaborate ways, but the one I remember is

du --max-depth=1 -h /

Now take the directory that uses up most space (du --max-depth=1 -h /yourdir) and go deeper until you find your culprit.
If want your output sorted by size and don't care for the human-readable format, you could also do du --max-depth=1 /your_dir | sort -n

Marie Fischer
  • 2,023
12

I find ncdu (http://dev.yorhel.nl/ncdu) to be quite helpful for this.

Scott
  • 1,072
5

I use the Gnome program baobab. You can run this on your desktop and t can connect via SSH to the server. It shows an easy to read graphical map of disk space usage. It's installed under Gnome as "Disk Usage Analyzer"

Josh
  • 9,398
3

Give gt5 a try.

Dennis Williamson
  • 64,083
2

df -k shows which fs are the problem. Then cd to the top level dir for it and run du -xk | sort -n | tail -25 this will show the top 25 dir, sorted, for sun 9 or earlier, replace the x with a d.

Ronald Pottol
  • 1,743
1

Note that files can be deleted while still being written to, so they use diskspace while their creating process is running, but not have a filename.

This makes it unfindable with the usual tools - you can use lsof to investigate which processes have open files.

Thorbjørn Ravn Andersen
  • 1,384
0

Below is the much more usable command that lists top largest folders in /home. You can replace /home with / also.

sudo du -a --max-depth=2 --human-readable --time --exclude=.* /home 2>/dev/null | sort --human-numeric-sort --reverse | head -n 15

The great thing about above command is that it will only recurse upto 2 levels so you eliminate a lot of noise.

Shital Shah
  • 141
  • 3
0

If you can run software on the system, then xdiskusage will graphically show you which directories/files are eating your space. Extremely useful.

I believe KDE contains something similar.

If it's text-only and you cannot install extra software, then creative use of du will probably get you there as well.

sleske
  • 10,234
0

here's something I cobbled together to track down some rogue processes on our database servers: rabbitfinder

#!/bin/sh
tree -s -f > /tmp/out1 && sleep 5 && tree -s -f > /tmp/out2; diff /tmp/out1 /tmp/out2 | egrep "\|--" | awk -F[ '{print $2}' | awk -F] '{print $2 }' | sort | uniq | xargs fuser -f | xargs ps -lFp

it's kinda kludgey and not very robust, but it works thusly:

  1. generate a recursive tree list of the current directory
  2. wait 5 seconds
  3. generate another list
  4. compare the two outputs
  5. fuser the files that have changed size and

  6. ps -lFp will show the files what process owns them

    user@poseidon:~$ tree -s -f > /tmp/out1 && sleep 5 && tree -s -f > /tmp/out2; diff /tmp/out1 /tmp/out2 | egrep "\|--" | awk -F[ '{print $2}' | awk -F] '{print $2 }' | sort | uniq | xargs fuser -f | xargs ps -lFp
./tmp/output:       
F S UID        PID  PPID  C PRI  NI ADDR SZ WCHAN    RSS PSR STIME TTY          TIME CMD
0 R 1000     14310 14275 23  80   0 -  1072 -        748   1 22:19 pts/2    00:00:06 dd if /dev/zero of ./output bs 1024 count 10000000
kenorb
  • 7,125
Greeblesnort
  • 1,779
0
  1. cd to the web servers home directory (apache's home directory)
  2. run the command "du -a |head -30|sort -nr"
  3. it will give you 30 largest disk consuming files/directories
  4. you can find them and delete (if not usefull)
ASEveryWhere Technologies
  • 1
0

You can use the following commands to find what files or folders taking too much space.

E.g. to display the biggest top 20 directories in the current folder, use the following one-liner:

du -ah . | sort -rh | head -20

or:

du -a . | sort -rn | head -20

For the top 20 biggest files in the current directory (recursively):

ls -1Rs | sed -e "s/^ *//" | grep "^[0-9]" | sort -nr | head -n20

or with human readable sizes:

ls -1Rhs | sed -e "s/^ *//" | grep "^[0-9]" | sort -hr | head -n20

The second command to work on OSX/BSD properly (as sort doesn't have -h), you need to install sort from coreutils. Then add the bin folder to your PATH.

You can define these commands as aliases (e.g. add to your rc files such as .bash_profile):

alias big='du -ah . | sort -rh | head -20'
alias big-files='ls -1Rhs | sed -e "s/^ *//" | grep "^[0-9]" | sort -hr | head -n20'

Then run big or big-files inside the folders which you think take place (e.g. in /home).

kenorb
  • 7,125