Re: Problems unmounting/fssyncking extern UFS filesystem
Erik Trulsson wrote: Oliver Fromme wrote: Christopher Sean Hilton wrote: Please understand that this is an honest question but isn't that true only of IDE heritage drives. E.g. If a SCSI drive with Tagged Queuing says that the the bits have been written doesn't it mean that the bits have really been written? Right, unless the SCSI drive has a label saying Quantum. In other words: Yes, assuming the SCSI drive works as it should. Not all SCSI drives work as they should. Right. However, exactly the same is also true for IDE/ATA disks: assuming that they work as they should ... But it seems that for (server-grade) SCSI drives the chances are better than for (consumer-grade) IDE/ATA drives. PS: Quantum is today owned by maxtor, isn't it? I've lost track of HD manufacturers when Seagate bought Conner ... Quantum and Maxtor merged and became Maxtor. Maxtor was recently bought by Seagate, so it is Seagate that owns Quantum these days. Uh, so now Seagate practically owns Quantum, Maxtor _and_ Conner? They got pretty fat then, it seems. Best regards Oliver -- Oliver Fromme, secnetix GmbH Co. KG, Marktplatz 29, 85567 Grafing Dienstleistungen mit Schwerpunkt FreeBSD: http://www.secnetix.de/bsd Any opinions expressed in this message may be personal to the author and may not necessarily reflect the opinions of secnetix in any way. I made up the term 'object-oriented', and I can tell you I didn't have C++ in mind. -- Alan Kay, OOPSLA '97 ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Problems unmounting/fssyncking extern UFS filesystem
On Thu, Nov 30, 2006 at 01:42:35PM +0100, Oliver Fromme wrote: Erik Trulsson wrote: Oliver Fromme wrote: Christopher Sean Hilton wrote: Please understand that this is an honest question but isn't that true only of IDE heritage drives. E.g. If a SCSI drive with Tagged Queuing says that the the bits have been written doesn't it mean that the bits have really been written? Right, unless the SCSI drive has a label saying Quantum. In other words: Yes, assuming the SCSI drive works as it should. Not all SCSI drives work as they should. Right. However, exactly the same is also true for IDE/ATA disks: assuming that they work as they should ... But it seems that for (server-grade) SCSI drives the chances are better than for (consumer-grade) IDE/ATA drives. Yes, my impression is that most SCSI drives do 'the right thing', while most ATA drives don't. PS: Quantum is today owned by maxtor, isn't it? I've lost track of HD manufacturers when Seagate bought Conner ... Quantum and Maxtor merged and became Maxtor. Maxtor was recently bought by Seagate, so it is Seagate that owns Quantum these days. Uh, so now Seagate practically owns Quantum, Maxtor _and_ Conner? They got pretty fat then, it seems. Yes. I think Seagate had managed to become the world's largest harddisk manufacturer even before they bought Maxtor, and that acquistion made them even larger so they have indeed got 'pretty fat'. There are only a few HDD manufacturers left these days. As far as I can tell the following is a *complete* list of companies that are manufacturing harddisks today: Seagate Hitachi Western Digital Samsung Fujitsu Toshiba ExcelStor -- Insert your favourite quote here. Erik Trulsson [EMAIL PROTECTED] ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Problems unmounting/fssyncking extern UFS filesystem
Christopher Sean Hilton wrote: Kevin Oberman wrote: [ ... ] The magic phrase is buffer cache has been flushed. In the real world of discs with cache there is no way to be certain when the data is REALLY on the disc. That is why things get clobbered if the power is cycled to the disc too soon after syncing and why a sleep (or typing sync three times) is still a good idea and probably always will be. Please understand that this is an honest question but isn't that true only of IDE heritage drives. E.g. If a SCSI drive with Tagged Queuing says that the the bits have been written doesn't it mean that the bits have really been written? Right, unless the SCSI drive has a label saying Quantum. Best regards Oliver PS: Quantum is today owned by maxtor, isn't it? I've lost track of HD manufacturers when Seagate bought Conner ... -- Oliver Fromme, secnetix GmbH Co. KG, Marktplatz 29, 85567 Grafing Dienstleistungen mit Schwerpunkt FreeBSD: http://www.secnetix.de/bsd Any opinions expressed in this message may be personal to the author and may not necessarily reflect the opinions of secnetix in any way. [...] one observation we can make here is that Python makes an excellent pseudocoding language, with the wonderful attribute that it can actually be executed. -- Bruce Eckel ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Problems unmounting/fssyncking extern UFS filesystem
On Wed, Nov 29, 2006 at 05:07:01PM +0100, Oliver Fromme wrote: Christopher Sean Hilton wrote: Kevin Oberman wrote: [ ... ] The magic phrase is buffer cache has been flushed. In the real world of discs with cache there is no way to be certain when the data is REALLY on the disc. That is why things get clobbered if the power is cycled to the disc too soon after syncing and why a sleep (or typing sync three times) is still a good idea and probably always will be. Please understand that this is an honest question but isn't that true only of IDE heritage drives. E.g. If a SCSI drive with Tagged Queuing says that the the bits have been written doesn't it mean that the bits have really been written? Right, unless the SCSI drive has a label saying Quantum. In other words: Yes, assuming the SCSI drive works as it should. Not all SCSI drives work as they should. (I am not familiar with the Quantum disks in particular, but I would be very surprised if there were no SCSI disks that were broken in this regard.) Best regards Oliver PS: Quantum is today owned by maxtor, isn't it? I've lost track of HD manufacturers when Seagate bought Conner ... Quantum and Maxtor merged and became Maxtor. Maxtor was recently bought by Seagate, so it is Seagate that owns Quantum these days. -- Insert your favourite quote here. Erik Trulsson [EMAIL PROTECTED] ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Problems unmounting/fssyncking extern UFS filesystem
On Tue, Nov 28, 2006 at 12:18:03AM +0100 I heard the voice of O. Hartmann, and lo! it spake thus: Ronald Klop wrote: IMHO: Please discuss this on [EMAIL PROTECTED] And read the handbook (http://www.freebsd.org/handbook) about releases/versions/branches. -CURRENT is known to have bugs. I am, in fact, quite familiar with the meaning of -CURRENT, and I have mentioned the issue on that list a few times in the past. I offer it here merely as an example of why syncing and waiting is _not_, in fact, quite obsolete. One of the fellows herein told me this discussion is subject to STABLE! He was yelling at me, not you ;) -- Matthew Fuller (MF4839) | [EMAIL PROTECTED] Systems/Network Administrator | http://www.over-yonder.net/~fullermd/ On the Internet, nobody can hear you scream. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Problems unmounting/fssyncking extern UFS filesystem
On Nov 27, 2006, at 8:41 AM, Kevin Oberman wrote: As far as I know, that's not different from calling sync just once. It might make more sense to put a little sleep between the sync calls, though. The traditional mantra was sync sync sync and not sync;sync;sync. The reason was timing. By entering the sync command three times as fast as anyone could type, the sync could reliably complete. Agreed. Although I've heard rumors that some systems treated 3 syncs as some sort of special case, but I've never seen anything in code to support the notion. That mantra is about 25 years old, so its validity on modern hardware is questionable, but the need for a delay is very real. I would suggest something like: sync sleep 5 The other choice would be to make sync [or the sync(2) system call, more precisely] blocking, so that it does not return until the buffer cache has been flushed and all dirty pages in VM have been written to disk. -- -Chuck ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Problems unmounting/fssyncking extern UFS filesystem
From: Chuck Swiger [EMAIL PROTECTED] Date: Tue, 28 Nov 2006 11:36:52 -0800 On Nov 27, 2006, at 8:41 AM, Kevin Oberman wrote: As far as I know, that's not different from calling sync just once. It might make more sense to put a little sleep between the sync calls, though. The traditional mantra was sync sync sync and not sync;sync;sync. The reason was timing. By entering the sync command three times as fast as anyone could type, the sync could reliably complete. Agreed. Although I've heard rumors that some systems treated 3 syncs as some sort of special case, but I've never seen anything in code to support the notion. That mantra is about 25 years old, so its validity on modern hardware is questionable, but the need for a delay is very real. I would suggest something like: sync sleep 5 The other choice would be to make sync [or the sync(2) system call, more precisely] blocking, so that it does not return until the buffer cache has been flushed and all dirty pages in VM have been written to disk. The magic phrase is buffer cache has been flushed. In the real world of discs with cache there is no way to be certain when the data is REALLY on the disc. That is why things get clobbered if the power is cycled to the disc too soon after syncing and why a sleep (or typing sync three times) is still a good idea and probably always will be. Of course, some day all discs may stop lying about when data is written, but I'll be too busy ducking the flying pig. -- R. Kevin Oberman, Network Engineer Energy Sciences Network (ESnet) Ernest O. Lawrence Berkeley National Laboratory (Berkeley Lab) E-mail: [EMAIL PROTECTED] Phone: +1 510 486-8634 Key fingerprint:059B 2DDF 031C 9BA3 14A4 EADA 927D EBB3 987B 3751 pgp1JeCskGyG7.pgp Description: PGP signature
Re: Problems unmounting/fssyncking extern UFS filesystem
On Nov 28, 2006, at 2:36 PM, Chuck Swiger wrote: The other choice would be to make sync [or the sync(2) system call, more precisely] blocking, so that it does not return until the buffer cache has been flushed and all dirty pages in VM have been written to disk. I would love a flag to sync(1) that would also make it wait until all softupdate pending actions are complete as well and force them to happen immediately.
Re: Problems unmounting/fssyncking extern UFS filesystem
On Tue, 2006-11-28 at 11:51 -0800, Kevin Oberman wrote: [ ... ] The magic phrase is buffer cache has been flushed. In the real world of discs with cache there is no way to be certain when the data is REALLY on the disc. That is why things get clobbered if the power is cycled to the disc too soon after syncing and why a sleep (or typing sync three times) is still a good idea and probably always will be. Please understand that this is an honest question but isn't that true only of IDE heritage drives. E.g. If a SCSI drive with Tagged Queuing says that the the bits have been written doesn't it mean that the bits have really been written? -- Chris ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Problems unmounting/fssyncking extern UFS filesystem
O. Hartmann wrote: Maybe amd() dismounts to early ... Don't know. Maybe the magic 'sync;sync;sync' before dismounting will help, I'll try it. As far as I know, that's not different from calling sync just once. It might make more sense to put a little sleep between the sync calls, though. Best regards Oliver -- Oliver Fromme, secnetix GmbH Co. KG, Marktplatz 29, 85567 Grafing Dienstleistungen mit Schwerpunkt FreeBSD: http://www.secnetix.de/bsd Any opinions expressed in this message may be personal to the author and may not necessarily reflect the opinions of secnetix in any way. Emacs ist für mich kein Editor. Für mich ist das genau das gleiche, als wenn ich nach einem Fahrrad (für die Sonntagbrötchen) frage und einen pangalaktischen Raumkreuzer mit 10 km Gesamtlänge bekomme. Ich weiß nicht, was ich damit soll. -- Frank Klemm, de.comp.os.unix.discussion ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Problems unmounting/fssyncking extern UFS filesystem
Date: Mon, 27 Nov 2006 14:53:06 +0100 (CET) From: Oliver Fromme [EMAIL PROTECTED] Sender: [EMAIL PROTECTED] O. Hartmann wrote: Maybe amd() dismounts to early ... Don't know. Maybe the magic 'sync;sync;sync' before dismounting will help, I'll try it. As far as I know, that's not different from calling sync just once. It might make more sense to put a little sleep between the sync calls, though. The traditional mantra was sync sync sync and not sync;sync;sync. The reason was timing. By entering the sync command three times as fast as anyone could type, the sync could reliably complete. That mantra is about 25 years old, so its validity on modern hardware is questionable, but the need for a delay is very real. I would suggest something like: sync sleep 5 -- R. Kevin Oberman, Network Engineer Energy Sciences Network (ESnet) Ernest O. Lawrence Berkeley National Laboratory (Berkeley Lab) E-mail: [EMAIL PROTECTED] Phone: +1 510 486-8634 Key fingerprint:059B 2DDF 031C 9BA3 14A4 EADA 927D EBB3 987B 3751 pgpYYS68ors84.pgp Description: PGP signature
Re: Problems unmounting/fssyncking extern UFS filesystem
I realise the original posting was related to amd(8) and NFS is not a normal filesystem but in the interest of trying to stamp out this myth... On Mon, 2006-Nov-27 08:41:19 -0800, Kevin Oberman wrote: The traditional mantra was sync sync sync ... That mantra is about 25 years old, so its validity on modern hardware is questionable, but the need for a delay is very real. For any modern Un*x, it is totally unnecessary. All current Un*x filesystems will automatically flush all buffers as part of the unmount process - specifically, any FS with a 'CLEAN' flag can be guaranteed to do so. I would suggest something like: sync sleep 5 In the specific case of softupdates, this is not adequate to flush all outstanding writes. The sync will flush one level of dependencies but can still leave outstanding writes. 'sleep 5' may or may not be adequate, depending on the amount of dirty cached data. As an experiment, I suggest creating or deleting a FS tree on an otherwise idle system and looking at the 'dirtybuf' value reported by 'systat -v 1'. See how many sync's and how long it takes to get it to blank (0). -- Peter Jeremy pgpxS6pQQXEBP.pgp Description: PGP signature
Re: Problems unmounting/fssyncking extern UFS filesystem
On Tue, Nov 28, 2006 at 05:37:58AM +1100 I heard the voice of Peter Jeremy, and lo! it spake thus: All current Un*x filesystems will automatically flush all buffers as part of the unmount process That Depends(tm), partly on what you mean by 'unmount'. With my Nov05 and Jun06 -CURRENT's, I had to take great care to sync and sync and wait and sync and sync filesystems before mount -u -o ro'ing them, because otherwise they'd end up NOT flushing everything, leaving unreferenced stuff around that fsck had to clean up, but only if I ran it manually because mount DID mark the filesystem as clean. I just tried to reproduce it on my last-week -CURRENT, and it no longer does that. Instead, it locked itself into a softdep_waitidle: Failed to flush worklist loop and won't LET me remount r/o (or unmount) the filesystems. Obviously, I should have kept up my now-established habit of sync'ing and waiting a while before un/remounting... -- Matthew Fuller (MF4839) | [EMAIL PROTECTED] Systems/Network Administrator | http://www.over-yonder.net/~fullermd/ On the Internet, nobody can hear you scream. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Problems unmounting/fssyncking extern UFS filesystem
On Mon, 27 Nov 2006 21:19:40 +0100, Matthew D. Fuller [EMAIL PROTECTED] wrote: On Tue, Nov 28, 2006 at 05:37:58AM +1100 I heard the voice of Peter Jeremy, and lo! it spake thus: All current Un*x filesystems will automatically flush all buffers as part of the unmount process That Depends(tm), partly on what you mean by 'unmount'. With my Nov05 and Jun06 -CURRENT's, I had to take great care to sync and sync and wait and sync and sync filesystems before mount -u -o ro'ing them, because otherwise they'd end up NOT flushing everything, leaving unreferenced stuff around that fsck had to clean up, but only if I ran it manually because mount DID mark the filesystem as clean. I just tried to reproduce it on my last-week -CURRENT, and it no longer does that. Instead, it locked itself into a softdep_waitidle: Failed to flush worklist loop and won't LET me remount r/o (or unmount) the filesystems. Obviously, I should have kept up my now-established habit of sync'ing and waiting a while before un/remounting... IMHO: Please discuss this on [EMAIL PROTECTED] And read the handbook (http://www.freebsd.org/handbook) about releases/versions/branches. -CURRENT is known to have bugs. -- Ronald Klop Amsterdam, The Netherlands ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Problems unmounting/fssyncking extern UFS filesystem
Ronald Klop wrote: On Mon, 27 Nov 2006 21:19:40 +0100, Matthew D. Fuller [EMAIL PROTECTED] wrote: On Tue, Nov 28, 2006 at 05:37:58AM +1100 I heard the voice of Peter Jeremy, and lo! it spake thus: All current Un*x filesystems will automatically flush all buffers as part of the unmount process That Depends(tm), partly on what you mean by 'unmount'. With my Nov05 and Jun06 -CURRENT's, I had to take great care to sync and sync and wait and sync and sync filesystems before mount -u -o ro'ing them, because otherwise they'd end up NOT flushing everything, leaving unreferenced stuff around that fsck had to clean up, but only if I ran it manually because mount DID mark the filesystem as clean. I just tried to reproduce it on my last-week -CURRENT, and it no longer does that. Instead, it locked itself into a softdep_waitidle: Failed to flush worklist loop and won't LET me remount r/o (or unmount) the filesystems. Obviously, I should have kept up my now-established habit of sync'ing and waiting a while before un/remounting... IMHO: Please discuss this on [EMAIL PROTECTED] And read the handbook (http://www.freebsd.org/handbook) about releases/versions/branches. -CURRENT is known to have bugs. -- Ronald Klop Amsterdam, The Netherlands One of the fellows herein told me this discussion is subject to STABLE! Regards, Oliver ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Problems unmounting/fssyncking extern UFS filesystem
On Sat, Nov 25, 2006 at 05:08:03PM +0100, O. Hartmann wrote: A while ago since now I receive kernel messages like this in FreeBSD 6.2-PRE/AMD64: fsync: giving up on dirty 0xff000362c7c0: tag devfs, type VCHR usecount 1, writecount 0, refcount 325 mountedhere 0xff00504d8400 flags () v_object 0xff00013c80e0 ref 0 pages 1286 lock type devfs: EXCL (count 1) by thread 0xff0050287260 (pid 14109) dev ufs/BACKUP Filesystem is an external USB attached SATA HD, ohci() driven (due to ehci() is not working stable and properly on FreeBSD 6.2). Filesystem is mounted via amd() and there via the'script' option in amd() due to problems of the amd() mounting process recognizing UFS filesystems. After 30 seconds of inactivity the filesystems gets dismounted. This worked quite well in the past, but now I see this kernel error messages. Before doing a PR, I would like to serious ask whether this is an issue or not. It's not a serious issue, AFAIK. Kris pgpUE3sLPh2y8.pgp Description: PGP signature
Problems unmounting/fssyncking extern UFS filesystem
A while ago since now I receive kernel messages like this in FreeBSD 6.2-PRE/AMD64: fsync: giving up on dirty 0xff000362c7c0: tag devfs, type VCHR usecount 1, writecount 0, refcount 325 mountedhere 0xff00504d8400 flags () v_object 0xff00013c80e0 ref 0 pages 1286 lock type devfs: EXCL (count 1) by thread 0xff0050287260 (pid 14109) dev ufs/BACKUP Filesystem is an external USB attached SATA HD, ohci() driven (due to ehci() is not working stable and properly on FreeBSD 6.2). Filesystem is mounted via amd() and there via the'script' option in amd() due to problems of the amd() mounting process recognizing UFS filesystems. After 30 seconds of inactivity the filesystems gets dismounted. This worked quite well in the past, but now I see this kernel error messages. Before doing a PR, I would like to serious ask whether this is an issue or not. Regards, Oliver ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Problems unmounting/fssyncking extern UFS filesystem
On Sat, Nov 25, 2006 at 05:08:03PM +0100, O. Hartmann wrote: A while ago since now I receive kernel messages like this in FreeBSD 6.2-PRE/AMD64: fsync: giving up on dirty 0xff000362c7c0: tag devfs, type VCHR usecount 1, writecount 0, refcount 325 mountedhere 0xff00504d8400 flags () v_object 0xff00013c80e0 ref 0 pages 1286 lock type devfs: EXCL (count 1) by thread 0xff0050287260 (pid 14109) dev ufs/BACKUP Filesystem is an external USB attached SATA HD, ohci() driven (due to ehci() is not working stable and properly on FreeBSD 6.2). The external USB harddisk I'm using works fine with ehci (VIA VT6202 USB 2.0 controller) on 6.2-PRERELEASE amd64: Controller /dev/usb4: addr 1: high speed, self powered, config 1, EHCI root hub(0x), VIA(0x), rev 1.00 port 1 powered port 2 addr 2: high speed, power 100 mA, config 1, Mass Storage Device(0x3507), Prolific Technology Inc.(0x067b), rev 1.00 Filesystem is mounted via amd() and there via the'script' option in amd() due to problems of the amd() mounting process recognizing UFS filesystems. After 30 seconds of inactivity the filesystems gets dismounted. This worked quite well in the past, but now I see this kernel error messages. The only problems I ever had wer with the firewire interface, not USB. But I don't use amd, and I'm using GEOM_ELI encyption. If amd doesn't work well with ufs, would using glabel be a workaround? Roland -- R.F.Smith http://www.xs4all.nl/~rsmith/ [plain text _non-HTML_ PGP/GnuPG encrypted/signed email much appreciated] pgp: 1A2B 477F 9970 BA3C 2914 B7CE 1277 EFB0 C321 A725 (KeyID: C321A725) pgpxtu85cyukL.pgp Description: PGP signature
