12:49 a.m.
Hello all,
Can anyone help me with a disk usage problem? I have a disk
partition of 65GB in LVM; df says:
/dev/dm-0 65570580 60494828 1744888 98% /
However, if I use either du or Baobab they reckon the directories in it
add up to 30Gb or so. As it is my root directory, and there are various
others it is a little difficult to get the total, so I booted under
liveUSB and saw exactly the same - 30Gb used, but 98% full.
So something is stealing half my disk. If I try to write more it is
out of space.
Any ideas how I get my space back? fsck reports the disk is clean.
Thanks,
Bill
--
Bill Murray ---- ATLAS
STFC RAL at: Bat 161 1-14, CERN,1211 Meyrin, Geneve 23, Switzerland
Tel:- CERN +41 22 7673028 or RAL +44 (0)1235 446256
--
Scanned by iCritical.
7:11 a.m.
On 01/20/2010 07:49 AM, William John Murray wrote:
Hello all,
Can anyone help me with a disk usage problem? I have a disk
partition of 65GB in LVM; df says:
/dev/dm-0 65570580 60494828 1744888 98% /
However, if I use either du or Baobab they reckon the directories in it
add up to 30Gb or so.
How do you give the du command?
Any /.hidden folders in root?
Mogens
--
Mogens Kjaer, Carlsberg A/S, Computer Department
Gamle Carlsberg Vej 10, DK-2500 Valby, Denmark
Phone: +45 33 27 53 25, Mobile: +45 22 12 53 25
Email: mk(a)crc.dk Homepage: http://www.crc.dk
11:51 a.m.
On 01/19/2010 10:49 PM, William John Murray wrote:
Hello all,
Can anyone help me with a disk usage problem? I have a disk
partition of 65GB in LVM; df says:
/dev/dm-0 65570580 60494828 1744888 98% /
However, if I use either du or Baobab they reckon the directories in it
add up to 30Gb or so. As it is my root directory, and there are various
others it is a little difficult to get the total, so I booted under
liveUSB and saw exactly the same - 30Gb used, but 98% full.
So something is stealing half my disk. If I try to write more it is
out of space.
Any ideas how I get my space back? fsck reports the disk is clean.
As someone else said, have a look for ".*" files (files whose names
start with a dot). Also keep in mind that du and df look at "sparse"
files differently. df shows what's been allocated and du shows what's
actually used.
----------------------------------------------------------------------
- Rick Stevens, Systems Engineer ricks(a)nerd.com -
- AIM/Skype: therps2 ICQ: 22643734 Yahoo: origrps2 -
- -
- All generalizations are false. -
----------------------------------------------------------------------
12:49 p.m.
On 10-01-20 12:51:30, Rick Stevens wrote:
...
... Also keep in mind that du and df look at "sparse"
files differently. df shows what's been allocated and du shows what's
actually used.
Are you sure that those two things are different? See du's --apparant-
size option (not the default), and note that df reports the free blocks
on the disk (parse files don't have blocks allocated for their empty
parts).
The OP has some other problem.
--
____________________________________________________________________
TonyN.:' <mailto:tonynelson@georgeanelson.com>
' <http://www.georgeanelson.com/>
12:47 p.m.
On 10-01-20 01:49:37, William John Murray wrote:
Hello all,
Can anyone help me with a disk usage problem? I have a disk
partition of 65GB in LVM; df says:
/dev/dm-0 65570580 60494828 1744888 98% /
However, if I use either du or Baobab they reckon the directories in
it add up to 30Gb or so. As it is my root directory, and there are
various others it is a little difficult to get the total, so I booted
under liveUSB and saw exactly the same - 30Gb used, but 98% full.
So something is stealing half my disk. If I try to write more it
is out of space.
Any ideas how I get my space back? fsck reports the disk is clean.
If the filesystem seems clean you need to use the -f option to force
the check to actually happen.
If that doesn't help, you could copy everything to someplace else, make
a new filesystem, and copy back again, assuming you have a "someplace
else". cpio or tar would work, booted from liveUSB.
--
____________________________________________________________________
TonyN.:' <mailto:tonynelson@georgeanelson.com>
' <http://www.georgeanelson.com/>
7:01 a.m.
On Wed, 2010-01-20 at 07:49 +0100, William John Murray wrote:
Hello all,
Can anyone help me with a disk usage problem? I have a disk
partition of 65GB in LVM; df says:
/dev/dm-0 65570580 60494828 1744888 98% /
However, if I use either du or Baobab they reckon the directories in it
add up to 30Gb or so. As it is my root directory, and there are various
others it is a little difficult to get the total, so I booted under
liveUSB and saw exactly the same - 30Gb used, but 98% full.
So something is stealing half my disk. If I try to write more it is
out of space.
Any ideas how I get my space back? fsck reports the disk is clean.
Thanks,
Bill
Thanks for all the suggestions. The dead links (lsof | grep deleted) is
very interesting - I have 49 of them, which seems bad, mostly /tmp files
from a
"gnome-terminal -ssh XXX.YYY"
which are all 3Mb. However, this is only 150MB, I am hunting for 30GB.
Rebooting should remove all such, and makes no difference.
Chris Smart asked what 'du' I had tried. 'It was 'du -sh'. Note that
it
agreed well with baobob so I thought that made it trustworthy.
Any more ideas? I guess I could copy the filesystem contents to
another disk and back, I have the space for that, but it seems a little
over-the-top. And it may well come back...
Bill
--
Bill Murray ---- ATLAS
STFC RAL at: Bat 161 1-14, CERN, 1211 Meyrin, Geneve 23, Switzerland
Tel:- CERN +41 22 7673028 or RAL +44 (0)1235 446256
--
Scanned by iCritical.
7:23 a.m.
On 01/21/2010 02:01 PM, William John Murray wrote:
...
Any more ideas?
What happens if you read the whole file system:
cd /
tar c --one-file-system -f - .|dd of=/dev/null bs=1M
dd should tell you the size of the generated tar
archive.
It may take a while...
Mogens
--
Mogens Kjaer, Carlsberg A/S, Computer Department
Gamle Carlsberg Vej 10, DK-2500 Valby, Denmark
Phone: +45 33 27 53 25, Mobile: +45 22 12 53 25
Email: mk(a)crc.dk Homepage: http://www.crc.dk
8:33 a.m.
On 01/21/2010 02:23 PM, Mogens Kjaer wrote:
tar c --one-file-system -f - .|dd of=/dev/null bs=1M
Hm, I forgot the -S option:
# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg_mk-lv_root
222G 48G 163G 23% /
# tar c --one-file-system -S -f - .|dd of=/dev/null bs=1M
...
50122147840 bytes (50 GB) copied, 1737.65 s, 28.8 MB/s
Without the -S I get:
134616043520 bytes (135 GB) copied, 1394.3 s, 96.5 MB/s
as I have some large sparse files.
48G and 50G are pretty close...
Mogens
--
Mogens Kjaer, Carlsberg A/S, Computer Department
Gamle Carlsberg Vej 10, DK-2500 Valby, Denmark
Phone: +45 33 27 53 25, Mobile: +45 22 12 53 25
Email: mk(a)crc.dk Homepage: http://www.crc.dk
9:42 a.m.
William John Murray wrote:
On Wed, 2010-01-20 at 07:49 +0100, William John Murray wrote:
> Hello all,
> Can anyone help me with a disk usage problem? I have a disk
> partition of 65GB in LVM; df says:
>
> /dev/dm-0 65570580 60494828 1744888 98% /
>
> However, if I use either du or Baobab they reckon the directories in it
> add up to 30Gb or so. As it is my root directory, and there are various
> others it is a little difficult to get the total, so I booted under
> liveUSB and saw exactly the same - 30Gb used, but 98% full.
> So something is stealing half my disk. If I try to write more it is
> out of space.
> Any ideas how I get my space back? fsck reports the disk is clean.
> Thanks,
> Bill
>
Thanks for all the suggestions. The dead links (lsof | grep deleted) is
very interesting - I have 49 of them, which seems bad, mostly /tmp files
from a
"gnome-terminal -ssh XXX.YYY"
which are all 3Mb. However, this is only 150MB, I am hunting for 30GB.
Rebooting should remove all such, and makes no difference.
Chris Smart asked what 'du' I had tried. 'It was 'du -sh'. Note that
it
agreed well with baobob so I thought that made it trustworthy.
Any more ideas? I guess I could copy the filesystem contents to
another disk and back, I have the space for that, but it seems a little
over-the-top. And it may well come back...
Anything that is hidden below an active mount point in that file system
will not be seen by 'du'. If you boot with "init=/bin/sh" as a kernel
parameter you can look at the file system before anything (not even /dev
or /proc) has been mounted. Caution: You will be working in a very
restricted and unfamiliar environment. I've never tried dealing with
LVM that way. You might be better off booting from a rescue disk and
examining your file system that way.
--
Bob Nichols "NOSPAM" is really part of my email address.
Do NOT delete it.
6:26 a.m.
On Thu, Jan 21, 2010 at 5:01 AM, William John Murray
<bill.murray(a)stfc.ac.uk> wrote:
Any more ideas? I guess I could copy the filesystem contents to
another disk and back, I have the space for that, but it seems a little
over-the-top. And it may well come back...
That would have the added benefit of completely defragmenting your filesystem.
I know it's off-topic, but it's really the best way to defragment;
defragmentation tools are generally unable to defragment everything.
Don Quixote
--
Don Quixote de la Mancha
quixote(a)dulcineatech.com
http://www.dulcineatech.com
Dulcinea Technologies Corporation: Software of Elegance and Beauty.
12:55 p.m.
On 01/20/2010 07:49 AM, William John Murray wrote:
Hello all,
Can anyone help me with a disk usage problem? I have a disk
partition of 65GB in LVM; df says:
/dev/dm-0 65570580 60494828 1744888 98% /
However, if I use either du or Baobab they reckon the directories in it
add up to 30Gb or so. As it is my root directory, and there are various
others it is a little difficult to get the total, so I booted under
liveUSB and saw exactly the same - 30Gb used, but 98% full.
So something is stealing half my disk. If I try to write more it is
out of space.
Any ideas how I get my space back? fsck reports the disk is clean.
Thanks,
Bill
Hi Bill,
could it be that the "reserved Blocks percent" Option of an ext2/3/4 FS
is the matter for that behaviour?
Find out how it is set by using dumpe2fs /dev/ice
You can set it to zero by using tune2fs -m 0 /dev/ice.
Regards, Florian
3:35 a.m.
On Wed, 2010-01-20 at 07:49 +0100, William John Murray wrote:
Hello all,
Can anyone help me with a disk usage problem? I have a disk
partition of 65GB in LVM; df says:
/dev/dm-0 65570580 60494828 1744888 98% /
However, if I use either du or Baobab they reckon the directories in it
add up to 30Gb or so. As it is my root directory, and there are various
others it is a little difficult to get the total, so I booted under
liveUSB and saw exactly the same - 30Gb used, but 98% full.
So something is stealing half my disk. If I try to write more it is
out of space.
Any ideas how I get my space back? fsck reports the disk is clean.
Thanks,
Bill
Thanks to everyone,
I found it, using the idea of making a tar file - thanks
Mogens. [I don't know why booting of a USB system didn't have the same
affect. Maybe I was careless]
The tar file was 60Gb, so I actually made it and inspected it and
found that the problem was:
* I have a big USB disk
* I Make a nightly backup to this disk of /home /etc for 2 machines.
* As it is USB. gnome mounts it under /media WHEN I LOG IN
* If the machine reboots without my log in the USB is not mounted
* The autobackup rsync looks for and fails to find /media/backup
- so it makes a new one WHICH IS in /
* I did not regard and of the /media/XXXX directories as part of / so
missed it.
So now I better fix the mounting of USB drives with /etc/fstab,
Bill
--
Bill Murray ---- ATLAS
STFC RAL at: Bat 161 1-14, CERN, 1211 Meyrin, Geneve 23, Switzerland
Tel:- CERN +41 22 7673028 or RAL +44 (0)1235 446256
--
Scanned by iCritical.
9:13 a.m.
William John Murray wrote:
On Wed, 2010-01-20 at 07:49 +0100, William John Murray wrote:
Thanks to everyone,
I found it, using the idea of making a tar file - thanks
Mogens. [I don't know why booting of a USB system didn't have the same
affect. Maybe I was careless]
The tar file was 60Gb, so I actually made it and inspected it and
found that the problem was:
* I have a big USB disk
* I Make a nightly backup to this disk of /home /etc for 2 machines.
* As it is USB. gnome mounts it under /media WHEN I LOG IN
* If the machine reboots without my log in the USB is not mounted
* The autobackup rsync looks for and fails to find /media/backup
- so it makes a new one WHICH IS in /
The way I protect against that happening is to give my backup prog a
destination that is actually a symlink to a subdirectory on the
external drive. If the drive is not mounted the backup process
complains about the broken symlink and fails. Use of a subdirectory
on the external drive is to protect against the case where the drive
is not mounted but the mount point directory in /media somehow did
not get deleted.
--
Bob Nichols "NOSPAM" is really part of my email address.
Do NOT delete it.
2:17 p.m.
* I did not regard and of the /media/XXXX directories as part of /
so
missed it.
So now I better fix the mounting of USB drives with /etc/fstab,
Hi Bill,
I created a label for my backup device/partition. Then I wrote this into
my backup script:
mount -L Backup /home/data/backup
RETURN=`mount|grep /home/data/backup|wc -l`
if [ $RETURN -ne 1 ]
then
logger "Backup failed due to unmounted backup device."
exit 1
fi
---
Will Y.
--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.
10:02 a.m.
Thanks Will,
I have fixed this now to automount the USB drive, which is better
because it means the backup is made.
Thanks anyway,
Bill
On Wed, 2010-02-10 at 15:17 -0500, aragonx(a)dcsnow.com wrote:
> * I did not regard and of the /media/XXXX directories as part
of / so
> missed it.
>
> So now I better fix the mounting of USB drives with /etc/fstab,
Hi Bill,
I created a label for my backup device/partition. Then I wrote this into
my backup script:
mount -L Backup /home/data/backup
RETURN=`mount|grep /home/data/backup|wc -l`
if [ $RETURN -ne 1 ]
then
logger "Backup failed due to unmounted backup device."
exit 1
fi
---
Will Y.
--
--
Scanned by iCritical.
4:16 p.m.
Thanks Will,
I have fixed this now to automount the USB drive, which is better
because it means the backup is made.
Thanks anyway,
Bill
On Wed, 2010-02-10 at 15:17 -0500, aragonx(a)dcsnow.com wrote:
> > * I did not regard and of the /media/XXXX directories as part of /
> so
> > missed it.
> >
> > So now I better fix the mounting of USB drives with /etc/fstab,
>
> Hi Bill,
>
> I created a label for my backup device/partition. Then I wrote this
> into
> my backup script:
>
> mount -L Backup /home/data/backup
> RETURN=`mount|grep /home/data/backup|wc -l`
> if [ $RETURN -ne 1 ]
> then
> logger "Backup failed due to unmounted backup device."
> exit 1
> fi
Hi Bill,
Maybe I'm paranoid but I don't want to have my backup device mounted all
the time. So I only mount it when I need to write to it. In that way, if
I do something stupid like run an unlimited search or delete, the backup
device will not be affected.
My next thought was to put the drive to sleep right after I unmount it.
Has anyone done that with something like:
hdparm -Y /dev/sd{x}
I'm hoping to be a little green, reduce wear on the drive (extend the
life) and protect my data all at the same time.
Is this a good or bad idea?
---
Will Y.
--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.
2:29 p.m.
William John Murray wrote:
* The autobackup rsync looks for and fails to find /media/backup
- so it makes a new one WHICH IS in /
See the positive side.
You unknowingly had an extra backup for some time. :-)
(but, being on the same disk as the original data, it was a poor one)
--
Roberto Ragusa mail at robertoragusa.it
5181
days inactive
5210
days old
16 comments
9 participants
participants (9)
-
Don Quixote de la Manvha -
Florian Gerstenberger -
Mogens Kjaer -
Rick Stevens -
Robert Nichols -
Roberto Ragusa -
Tony Nelson -
Will Yonker -
William John Murray