RE: filesystem information
On Behalf Of Jim >> Just a thought, but in normal circumstances files *are* written to, >> even when they are just being read: the access time is updated (unless >> you mount the fs with the noatime flag). >> > > quite true, but isn't that file metadata and not the actual file? I > thought most filesystems had a file-entry section, with all the > metadata, permissions, etc, and a file data section, which contains > the information of the actual file. > > I guess I wouldn't be surprised if the metadata being edited were > corrupted, but to corrupt the file data/location seems odd to me. > Especially with soft updates; unless I'm mistaken, that's what it is > supposed to minimise. > -Jim But if the power failure interrupts an update, all files with data in the sector(s) it was writing are at risk. A corrupted sector can contain multiple file entries, and any or all of their entries may be lost. It is up to fsck and friends to determine which of them can still be safely accessed and restore those entries. It is not only possible, but likely that some will be lost each time this occurs. On the other hand, I thought this was one of the problems that journal led file systems were invented to solve. Bob McConnell ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: filesystem information
> Just a thought, but in normal circumstances files *are* written to, > even when they are just being read: the access time is updated (unless > you mount the fs with the noatime flag). > quite true, but isn't that file metadata and not the actual file? I thought most filesystems had a file-entry section, with all the metadata, permissions, etc, and a file data section, which contains the information of the actual file. I guess I wouldn't be surprised if the metadata being edited were corrupted, but to corrupt the file data/location seems odd to me. Especially with soft updates; unless I'm mistaken, that's what it is supposed to minimise. -Jim ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: filesystem information
On Mon, Jun 30, 2008 at 03:12:59PM -0400, Jim typed: > > I'm aware of nothing but a UPS can completely protect me from an > outage. I was just wondering why that ONE file system was misbehaving, > and the rest are prefectly fine - which seemed odd. Additionally, why > were files that are read, but not written, being lost? I can > understand losing files that are being written, but if there's a file > that has bene written several restarts ago, not written to thereafter, > and has been fine ever since, why is it being lost now? Just a thought, but in normal circumstances files *are* written to, even when they are just being read: the access time is updated (unless you mount the fs with the noatime flag). Ruben ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: filesystem information
In response to Jim <[EMAIL PROTECTED]>: > > If the files themselves are disappearing, then it could be the directory > > entry that's getting corrupted. > > The files are there, but their content is corrupted. Well ... that seems to contradict my theory ... > > Even if you're > > not doing it directly, is your mp3 software writing temp or other > > status files to that directory? If you're curious, you could run your > > mp3 software under ktrace and then grep the output for file creation > > and removal syscalls. > > OK. The files are actually FLAC, and I use XMMS. I assume I trace XMMS > and not the FLAC library? I'll try when I get home. Thanks Not familiar with the XMMS/FLAC software architecture, so I can't be sure ... but my guess would be that tracing XMMS is going to catch any oddities in file creation. -- Bill Moran http://www.potentialtech.com ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: filesystem information
> If the files themselves are disappearing, then it could be the directory > entry that's getting corrupted. The files are there, but their content is corrupted. > Even if you're > not doing it directly, is your mp3 software writing temp or other > status files to that directory? If you're curious, you could run your > mp3 software under ktrace and then grep the output for file creation > and removal syscalls. OK. The files are actually FLAC, and I use XMMS. I assume I trace XMMS and not the FLAC library? I'll try when I get home. Thanks -Jim Stapleton ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: filesystem information
In response to Jim <[EMAIL PROTECTED]>: > > I'm aware of nothing but a UPS can completely protect me from an > outage. I was just wondering why that ONE file system was misbehaving, > and the rest are prefectly fine - which seemed odd. Additionally, why > were files that are read, but not written, being lost? I can > understand losing files that are being written, but if there's a file > that has bene written several restarts ago, not written to thereafter, > and has been fine ever since, why is it being lost now? If the files themselves are disappearing, then it could be the directory entry that's getting corrupted. You mentioned mp3s earlier ... if I had to guess, I'd say you frequently add and rename files in that directory. If the power goes out during an update to the directory entry, it's anybody's guess as to what filenames could disappear. Even if you're not doing it directly, is your mp3 software writing temp or other status files to that directory? If you're curious, you could run your mp3 software under ktrace and then grep the output for file creation and removal syscalls. Just speculation. -- Bill Moran http://www.potentialtech.com ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: filesystem information
> In case of frequent power outages, I guess the right answer is "get a > UPS". :) Aye, I just got one. But for the longest time, it was a bit out of my price range due to other priorities. Actually, the whole model line was defective, so they are sending me a new one, and I have to wait for it to arrive. > Without a UPS nothing can protect you against power outages. Even when > running the filesystem with the sync flag and setting ATA devices to > write-through the cache cannot guarantee you won't lose data. If the > power fails when a write is in progress, you're screwed. I'm aware of nothing but a UPS can completely protect me from an outage. I was just wondering why that ONE file system was misbehaving, and the rest are prefectly fine - which seemed odd. Additionally, why were files that are read, but not written, being lost? I can understand losing files that are being written, but if there's a file that has bene written several restarts ago, not written to thereafter, and has been fine ever since, why is it being lost now? ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: filesystem information
On Mon, Jun 30, 2008 at 07:05:51PM +0200, Wojciech Puchar wrote: > > Ans set 'hw.ata.wc="0"' in /boot/loader.conf to stop the drives from > > caching writes. > > it will GREATLY reduce write performance. not just a bit, but many times. Of course. And mounting filesystems with sync will also reduce performance. But if you have frequent outages and without a UPS they will at least help shorten fsck times and lessen potential data loss. Roland -- R.F.Smith http://www.xs4all.nl/~rsmith/ [plain text _non-HTML_ PGP/GnuPG encrypted/signed email much appreciated] pgp: 1A2B 477F 9970 BA3C 2914 B7CE 1277 EFB0 C321 A725 (KeyID: C321A725) pgpjJuylc9Yt6.pgp Description: PGP signature
Re: filesystem information
Ans set 'hw.ata.wc="0"' in /boot/loader.conf to stop the drives from caching writes. it will GREATLY reduce write performance. not just a bit, but many times. WRT softupdates/gjournal, see below. In case of frequent power outages, I guess the right answer is "get a UPS". :) it is definitely the only solution. and not expensive today ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: filesystem information
On Mon, Jun 30, 2008 at 12:30:38PM -0400, Jim wrote: > On Mon, Jun 30, 2008 at 7:30 AM, Bill Moran <[EMAIL PROTECTED]> wrote: > > In response to Jim <[EMAIL PROTECTED]>: > > > >> I have a computer that is in a situation where it is losing power > >> occasionally. All but one of the filesystems are going along fine. > >> Once file system seems to lose data on a power outage. Even if it only > >> reads a file, and doesn't write it, it may still lose a file (ex, > >> about half the audio files on my xmms playlist, a couple data files in > >> my wine directory that, to my knowledge, are unlikely to be written > >> after they are first installed). > >> > >> What I'd like to do is get an output of the flags and options on my > >> filesystems to see what is different between that filesystem and the > >> others. Any suggestion on how to do that? This particular FS has > >> lasted through several rebuilds since it doesn't hold OS critical > >> stuff, just data files. > > > > tunefs -p and/or dumpfs -m > > > >> Any suggestions? > > > > Sounds like you're on the right track with hunting this down. Perhaps > > turn softupdates off and mount the filesystem sync if you're seeing > > lots of power outages. Ans set 'hw.ata.wc="0"' in /boot/loader.conf to stop the drives from caching writes. > Thanks, it looks like the 'good' filesystems have softupdates off > (except one), and the one the broke has it off. I thought softupdates > were supposed to fix this? Is gjournal a better solution? Is 'just use > neither' a better solution? WRT softupdates/gjournal, see below. In case of frequent power outages, I guess the right answer is "get a UPS". :) Without a UPS nothing can protect you against power outages. Even when running the filesystem with the sync flag and setting ATA devices to write-through the cache cannot guarantee you won't lose data. If the power fails when a write is in progress, you're screwed. A proper UPS with monitoring software will give your system time to shut down properly (finishing writes, unmounting etc) before its battery runs out. > Any reference material on the subject See http://en.wikipedia.org/wiki/Soft_updates : Instead of duplicating metadata writes in a journal, soft updates work by properly ordering the metadata writes to guarantee consistency after a crash. Like journaling, soft updates do not guarantee that no data will be lost, but do make sure the filesystem is consistent In FreeBSD softupdates have a longer track record than journaling. Roland -- R.F.Smith http://www.xs4all.nl/~rsmith/ [plain text _non-HTML_ PGP/GnuPG encrypted/signed email much appreciated] pgp: 1A2B 477F 9970 BA3C 2914 B7CE 1277 EFB0 C321 A725 (KeyID: C321A725) pgpSsHWWBWGIU.pgp Description: PGP signature
Re: filesystem information
On Mon, Jun 30, 2008 at 7:30 AM, Bill Moran <[EMAIL PROTECTED]> wrote: > In response to Jim <[EMAIL PROTECTED]>: > >> I have a computer that is in a situation where it is losing power >> occasionally. All but one of the filesystems are going along fine. >> Once file system seems to lose data on a power outage. Even if it only >> reads a file, and doesn't write it, it may still lose a file (ex, >> about half the audio files on my xmms playlist, a couple data files in >> my wine directory that, to my knowledge, are unlikely to be written >> after they are first installed). >> >> What I'd like to do is get an output of the flags and options on my >> filesystems to see what is different between that filesystem and the >> others. Any suggestion on how to do that? This particular FS has >> lasted through several rebuilds since it doesn't hold OS critical >> stuff, just data files. > > tunefs -p and/or dumpfs -m > >> Any suggestions? > > Sounds like you're on the right track with hunting this down. Perhaps > turn softupdates off and mount the filesystem sync if you're seeing > lots of power outages. > > -- > Bill Moran > http://www.potentialtech.com > Thanks, it looks like the 'good' filesystems have softupdates off (except one), and the one the broke has it off. I thought softupdates were supposed to fix this? Is gjournal a better solution? Is 'just use neither' a better solution? Any reference material on the subject would be appreciated (I'm about to use google now). Thanks, -Jim Stapleton ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: filesystem information
In response to Jim <[EMAIL PROTECTED]>: > I have a computer that is in a situation where it is losing power > occasionally. All but one of the filesystems are going along fine. > Once file system seems to lose data on a power outage. Even if it only > reads a file, and doesn't write it, it may still lose a file (ex, > about half the audio files on my xmms playlist, a couple data files in > my wine directory that, to my knowledge, are unlikely to be written > after they are first installed). > > What I'd like to do is get an output of the flags and options on my > filesystems to see what is different between that filesystem and the > others. Any suggestion on how to do that? This particular FS has > lasted through several rebuilds since it doesn't hold OS critical > stuff, just data files. tunefs -p and/or dumpfs -m > Any suggestions? Sounds like you're on the right track with hunting this down. Perhaps turn softupdates off and mount the filesystem sync if you're seeing lots of power outages. -- Bill Moran http://www.potentialtech.com ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
