Re: /var with capacity -1%
[EMAIL PROTECTED] said: 1) Softupdates can cause this. Run sync a few times, then wait 30 seconds or so; does it go away? Tried. And booted several times. No luck [EMAIL PROTECTED] said: 2) Files which are open (have active file descriptors associated with them) on /var before it filled may be causing this. fstat may help you here. But /var is not full. It is _more_ than empty. After the system has been running a couple of days (more files in /var/log, /var/db/locate.database, etc.) I now have an almost empty /var: ~#du -sk /var 15378 /var ~#df /var Filesystem 1K-blocks Used Avail Capacity Mounted on /dev/da0s1d 5077038 -21470 4692346-0%/var I guess I could be happy for this extra diskspace, but I am afraid that something is not quite right... --Ingeborg -- Ingeborg Østrem Hellemo -- [EMAIL PROTECTED] (Univ. of Tromsø, Norway) ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: /var with capacity -1%
On Wed, Apr 02, 2008 at 08:03:09AM +0200, Ingeborg Hellemo wrote: [EMAIL PROTECTED] said: 1) Softupdates can cause this. Run sync a few times, then wait 30 seconds or so; does it go away? Tried. And booted several times. No luck [EMAIL PROTECTED] said: 2) Files which are open (have active file descriptors associated with them) on /var before it filled may be causing this. fstat may help you here. But /var is not full. It is _more_ than empty. It is possible to have files that are open and held by processes on the filesystem that are no longer listed. If you kill the offending process the space will be freed up. -- Jonathan Chen [EMAIL PROTECTED] -- You can get farther with a kind word and a gun than you can with a kind word alone - Al Capone ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: /var with capacity -1%
On Wed, Apr 02, 2008 at 08:03:09AM +0200, Ingeborg Hellemo wrote: [EMAIL PROTECTED] said: 2) Files which are open (have active file descriptors associated with them) on /var before it filled may be causing this. fstat may help you here. But /var is not full. It is _more_ than empty. I'm sorry, I wrote my mail in haste. Yes, you're correct, but what you're seeing is commonly the result of softupdates. I've seen it happen many MANY times, and it's easily reproducable. However, it goes away after a few minutes. If yours is that way constantly, I can't really explain what's going on. I'm just pointing out that negative disk space used is something I've seen many times, and it's always gone away on its own fairly quickly. What you have might be some corrupted filesystem, but someone more familiar with UFS/FFS will have to comment. Output from ffsinfo(8) on that filesystem may be useful. -- | Jeremy Chadwickjdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: /var with capacity -1%
[EMAIL PROTECTED] said: It is possible to have files that are open and held by processes on the filesystem that are no longer listed. If you kill the offending L process the space will be freed up. lsof +aL1 file_system shows unlinked open files on the specified file system (quoting its man page). If you read my first mail you will see that this is not a case of overfull /var (101% used) and a need to free space. Something is wrong with either the filesystem or df(1) since it claims that I am using a negative amount of disk-blocks. There are no unlinked open files on /var, and I fail to understand how they could have explained the output of df(1) --Ingeborg -- Ingeborg Østrem Hellemo -- [EMAIL PROTECTED] (Univ. of Tromsø, Norway) ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: /var with capacity -1%
[EMAIL PROTECTED] said: What you have might be some corrupted filesystem, but someone more familiar with UFS/FFS will have to comment. Output from ffsinfo(8) on that filesystem may be useful. Any wishes for the '-l level' flag? The default gives me a 400Mb file. --Ingeborg -- Ingeborg Østrem Hellemo -- [EMAIL PROTECTED] (Univ. of Tromsø, Norway) ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: /var with capacity -1%
On Wed, Apr 02, 2008 at 07:41:16PM +1300, Jonathan Chen wrote: 2) Files which are open (have active file descriptors associated with them) on /var before it filled may be causing this. fstat may help you here. But /var is not full. It is _more_ than empty. It is possible to have files that are open and held by processes on the filesystem that are no longer listed. If you kill the offending process the space will be freed up. lsof +aL1 file_system shows unlinked open files on the specified file system (quoting its man page). Eugene Grosbein ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: /var with capacity -1%
Ingeborg Hellemo wrote: ProLiant DL380 G5, freshly installed FreeBSD 7.0-RELEASE amd64. I have done similar installations on 2 other servers, and suddenly: Any ideas? fsck -B /var -- Sphinx of black quartz judge my vow. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: /var with capacity -1%
On Wed, Apr 02, 2008 at 09:17:03AM +0200, Ingeborg Hellemo wrote: [EMAIL PROTECTED] said: It is possible to have files that are open and held by processes on the filesystem that are no longer listed. If you kill the offending L process the space will be freed up. lsof +aL1 file_system shows unlinked open files on the specified file system (quoting its man page). If you read my first mail you will see that this is not a case of overfull /var (101% used) and a need to free space. Something is wrong with either the filesystem or df(1) since it claims that I am using a negative amount of disk-blocks. There are no unlinked open files on /var, and I fail to understand how they could have explained the output of df(1) Did you perform the lsof or fstat commands recommended? We understand you fail to understand how open fds to files which were removed could cause what you're seeing -- you'll have to trust us. Can you do that? -- | Jeremy Chadwickjdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: /var with capacity -1%
On Wed, Apr 2, 2008 at 12:54 AM, Jeremy Chadwick [EMAIL PROTECTED] wrote: On Wed, Apr 02, 2008 at 09:17:03AM +0200, Ingeborg Hellemo wrote: [EMAIL PROTECTED] said: It is possible to have files that are open and held by processes on the filesystem that are no longer listed. If you kill the offending L process the space will be freed up. lsof +aL1 file_system shows unlinked open files on the specified file system (quoting its man page). If you read my first mail you will see that this is not a case of overfull /var (101% used) and a need to free space. Something is wrong with either the filesystem or df(1) since it claims that I am using a negative amount of disk-blocks. There are no unlinked open files on /var, and I fail to understand how they could have explained the output of df(1) Did you perform the lsof or fstat commands recommended? We understand you fail to understand how open fds to files which were removed could cause what you're seeing -- you'll have to trust us. Can you do that? Was the file system on a machine that was upgraded from 4.x by any chance? There were some header changes a while ago that could cause pervasive accounting problems. A fsck fixes that one. -- Peter Wemm - [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED] All of this is for nothing if we don't go to the stars - JMS/B5 If Java had true garbage collection, most programs would delete themselves upon execution. -- Robert Sewell **WANTED TO BUY: Garmin Streetpilot 2650 or 2660. Not later model! ** ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: /var with capacity -1%
On Wed, Apr 02, 2008 at 10:12:30AM +0200, Ingeborg Hellemo wrote: ~/#fsck /dev/da0s1d ** /dev/da0s1d (NO WRITE) ** Last Mounted on /var ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups FREE BLK COUNT(S) WRONG IN SUPERBLK SALVAGE? no 275 files, 7699 used, 2549244 free (596 frags, 318581 blocks, 0.0% fragmentation) After reboot to single mode and fsck -y we are back to a more normal state: ~/df /var Filesystem 1K-blocks Used Avail Capacity Mounted on /dev/da0s1d 5077038 14448 4656428 0%/var But shouldn't I have been notified of this error during previous reboots? Did you have any recent crashes, panics, or anything that could (even remotely) cause filesystem inconsistencies? If not, then the best explanation is then bit rot, or bad hardware causing corruption of data. Jeff Bonwick wrote a blog entry about this kind of problem: http://blogs.sun.com/bonwick/entry/zfs_end_to_end_data I suppose there could be a bug in FFS, but I would expect a significantly larger number of users to be reporting this... -- | Jeremy Chadwickjdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: /var with capacity -1%
[EMAIL PROTECTED] said: Did you have any recent crashes, panics, or anything that could (even remotely) cause filesystem inconsistencies? No, that was why I was asking in the first place. The system was freshly installed from 7.0-RELEASE-amd64-disc1.iso with no error messages during the install. Since nobody else has reported this I guess it is not a bug, but rather due to sunspot activity - or just my _negative_ attitude :-P --Ingeborg -- Ingeborg Østrem Hellemo -- [EMAIL PROTECTED] (Univ. of Tromsø, Norway) ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: /var with capacity -1%
On Wed, Apr 02, 2008 at 10:25:48AM +0200, Ingeborg Hellemo wrote: [EMAIL PROTECTED] said: Did you have any recent crashes, panics, or anything that could (even remotely) cause filesystem inconsistencies? No, that was why I was asking in the first place. The system was freshly installed from 7.0-RELEASE-amd64-disc1.iso with no error messages during the install. Since nobody else has reported this I guess it is not a bug, but rather due to sunspot activity - or just my _negative_ attitude :-P You are not alone. I've been thinking it was no big deal due to not seeing any issues with use of that partition. I don't know when it started or if we had a crash near that time. 11:02:34 Wed Apr 02 $ df Filesystem SizeUsed Avail Capacity Mounted on /dev/ad0s1a496M129M328M28%/ devfs 1.0K1.0K 0B 100%/dev /dev/ad0s1e989M377M534M41%/tmp /dev/ad0s1f 14G1.7G 11G13%/usr /dev/ad0s1d1.9G488M1.3G27%/var /dev/ad2s1d 72G -1.5G 68G-2%/dumps devfs 1.0K1.0K 0B 100%/var/named/dev 11:02:50 Wed Apr 02 $ sudo umount /dumps 11:03:23 Wed Apr 02 $ sudo fsck /dev/ad2s1d ** /dev/ad2s1d ** Last Mounted on /dumps ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups FREE BLK COUNT(S) WRONG IN SUPERBLK SALVAGE? [yn] y SUMMARY INFORMATION BAD SALVAGE? [yn] y BLK(S) MISSING IN BIT MAPS SALVAGE? [yn] y 4 files, 4 used, 37842672 free (32 frags, 4730330 blocks, 0.0% fragmentation) * FILE SYSTEM MARKED CLEAN * * FILE SYSTEM WAS MODIFIED * 11:04:06 Wed Apr 02 $ sudo mount /dumps 11:04:35 Wed Apr 02 $ df Filesystem SizeUsed Avail Capacity Mounted on /dev/ad0s1a496M129M328M28%/ devfs 1.0K1.0K 0B 100%/dev /dev/ad0s1e989M377M534M41%/tmp /dev/ad0s1f 14G1.7G 11G13%/usr /dev/ad0s1d1.9G488M1.3G27%/var devfs 1.0K1.0K 0B 100%/var/named/dev /dev/ad2s1d 72G8.0K 66G 0%/dumps -- Scott LambertKC5MLE Unix SysAdmin [EMAIL PROTECTED] ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: /var with capacity -1%
On Tue, Apr 01, 2008 at 02:57:17PM +0200, Ingeborg Hellemo wrote: ProLiant DL380 G5, freshly installed FreeBSD 7.0-RELEASE amd64. I have done similar installations on 2 other servers, and suddenly: ~/#df Filesystem1K-blocks UsedAvail Capacity Mounted on /dev/da0s1a 1012974 385846 54609241%/ devfs 110 100%/dev /dev/da0s1e 507630 12 467008 0%/tmp /dev/da0s1f58730650 1302436 52729762 2%/usr /dev/da0s1d 5077038 -33660 4704536-1%/var ~/#du -sk /var 3188/var I have not done anything with neither newfs nor tunefs. ~/#tunefs -p tunefs: ACLs: (-a) disabled tunefs: MAC multilabel: (-l) disabled tunefs: soft updates: (-n) enabled tunefs: gjournal: (-J) disabled tunefs: maximum blocks per file in a cylinder group: (-e) 2048 tunefs: average file size: (-f)16384 tunefs: average number of files in a directory: (-s) 64 tunefs: minimum percentage of free space: (-m) 8% tunefs: optimization preference: (-o) time tunefs: volume label: (-L)Fra /var/run/dmesg.boot: From /var/run/dmesg.boot: da0 at ciss0 bus 0 target 0 lun 0 da0: COMPAQ RAID 1 VOLUME OK Fixed Direct Access SCSI-0 device da0: 135.168MB/s transfers da0: 69973MB (143305920 512 byte sectors: 255H 32S/T 17562C) There were no complaints during slicing og installing. Any ideas? 1) Softupdates can cause this. Run sync a few times, then wait 30 seconds or so; does it go away? 2) Files which are open (have active file descriptors associated with them) on /var before it filled may be causing this. fstat may help you here. -- | Jeremy Chadwickjdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
