If a reboot fixes it, make sure that any jobs that truncate logfiles do so by cat /dev/null > file rather than rm file, to prevent zombies or open file handles from mucking with free space.
Can you create a new file on the file system with touch? Do you get "no more extents", or something different? -P On Mon, Jun 20, 2011 at 8:34 PM, Mark J. Bailey <m...@jobsoft.com> wrote: > That is weird. I like Brian idea of rebooting and seeing what you have > afterwards. > > -----Original Message----- > From: nlug-talk@googlegroups.com [mailto:nlug-talk@googlegroups.com] On > Behalf Of Bruce W. Martin > Sent: Monday, June 20, 2011 7:28 PM > To: nlug-talk@googlegroups.com > Subject: Re: [nlug] df says partition is full but du numbers don't add up > > > On Jun 20, 2011, at 6:33 PM, Mark J. Bailey wrote: > > > Bruce, if you have the 'lsof' (list of open files) util on that > > system, you might be able to do some analysis from that perspective > > that might help narrow it down. Also, if it is lots of small files, > > what is the default block size (and/or minimum allocation) for a file. > > I think it is 4k. > > > What filesystem? > > ext3 > > > Could the overhead for the filesystem structure with LOTS of files be > > eating up space in the filesystem/inode datastructure? > > I don't think so. > > > Wild > > guess, but overhead for such a largely allocated filesystem might > > account for some dark matter. > > There was about 500GB available on Friday with the only directory (3T) > that is seeing any significant activity was using 1.5TB. Today the > partition is 100% full but the 3T directory is now only using 1.1TB with > all other directories unchanged since Friday. I don't know what the users > have been doing over the weekend. > > Bruce > > > > > -----Original Message----- > > From: nlug-talk@googlegroups.com [mailto:nlug-talk@googlegroups.com] > > On Behalf Of Bruce W. Martin > > Sent: Monday, June 20, 2011 6:25 PM > > To: nlug-talk@googlegroups.com > > Subject: Re: [nlug] df says partition is full but du numbers don't add > > up > > > > > > On Jun 20, 2011, at 4:00 PM, Mark J. Bailey wrote: > > > >> IS there some sort of reserve/overhead with the array or the > >> filesystem > > on it? > > > > No. > > > >> Is it being replicated somewhere? > > > > NO. > > > >> And, do you have a whole lot of small files or a lesser number of > >> huge > > files? > > > > It is mostly small files. > > > > Paul, Thanks for that thought but there are plenty of inodes available > > on that partition. I deleted a large number of files last week but the > > space showed free right afterwards. I ran lsof but nothing on that > > partition showed up in the list. > > > > Brian, I have a sneaking suspicion that you may be correct about the > > running or zombie processes, I will have to look into that further. > > This partition is served up to about a dozen machines so I suspect > > that i will have to look at processes on each of the other machines to > > find if there is a culprit down this path. > > > > Thanks for the suggestions. Any other thoughts? > > > > Bruce > > > > > >> > >> From: nlug-talk@googlegroups.com [mailto:nlug-talk@googlegroups.com] > >> On Behalf Of Brian H. Ward > >> Sent: Monday, June 20, 2011 3:46 PM > >> To: nlug-talk@googlegroups.com > >> Subject: Re: [nlug] df says partition is full but du numbers don't > >> add up > >> > >> unlinked (deleted) files which are still held open by running (or > >> even > > zombie) processes will keep disk space from being returned to df, but > > won't show up in du. > >> > >> On Mon, Jun 20, 2011 at 4:34 PM, Bruce W. Martin <marti...@gmail.com> > > wrote: > >> I have a 4.6TB partition on a RAID array that is reporting as 100% > >> full > > but running du on the partition only shows 3.5TB worth of files. What > > gives? Is there some way to fix this short of rebooting? I am not sure > > of the right question to pose to do a proper google search for this. > >> > >> > >> Bruce > >> -- > >> Bruce W. Martin, KQ4TV > >> Trustee for NT4UX > >> Nashville Linux User Group - Amateur Radio- Special Interest Group > >> NLUG-AR-SIG > >> > >> > >> -- > >> Our visit to this planet is short, so we should use our time > >> meaningfully, which we can do by helping others wherever possible. > >> And if we cannot help others, at least we should try not to create > >> pain and suffering for them. - Dalai Lama > > -- > You received this message because you are subscribed to the Google Groups > "NLUG" group. > To post to this group, send email to nlug-talk@googlegroups.com To > unsubscribe from this group, send email to > nlug-talk+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/nlug-talk?hl=en > > -- > You received this message because you are subscribed to the Google Groups > "NLUG" group. > To post to this group, send email to nlug-talk@googlegroups.com > To unsubscribe from this group, send email to > nlug-talk+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/nlug-talk?hl=en > -- You received this message because you are subscribed to the Google Groups "NLUG" group. To post to this group, send email to nlug-talk@googlegroups.com To unsubscribe from this group, send email to nlug-talk+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/nlug-talk?hl=en
