Re: Journaling: Surviving or allowing unclean shutdown?
On Mon, Jan 08, 2001 at 12:02:49PM +, Stephen C. Tweedie wrote: > Right. There are two distinct meanings: > > 1) Do not write to this medium, ever (physical readonly); and > > 2) Do not allow modifications to the filesystem (logical readonly). > > The fact is that the kernel confuses the two, but that just isn't >[snip] > We just don't have a way of specifying these two things independently. Is this call for a new mount option?, or should we just clutter /dev even further with devices with ro permissions as the marker. TTFN -- Roger Think of the mess on the carpet. Sensible people do all their demon-summoning in the garage, which you can just hose down afterwards. -- [EMAIL PROTECTED] - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
In article <[EMAIL PROTECTED]> you wrote: > Also thing about cases where powerplant fails, or when electricity in > the house fails. I've seen places where electricity failed 5 times a > day, because someone put 10A fuse and we were using just about 2kW... Especially evil is a power failure, and then a second failure while fsck is running... So i hope Journaling FSs will have the power of normal FSs soon, so we can have them as a default for consumers. Otherwise: /dev/scsi/host0/bus0/target6/lun0/part1 on / type ext2 (ro) proc on /proc type proc (rw) devpts on /dev/pts type devpts (rw) /dev/sda5 on /usr type ext2 (ro) /dev/sda6 on /var type ext2 (r,noexec) /dev/sda7 on /home type ext2 (rw,nosuid) /dev/scd0 on /cdrom type iso9660 (ro) some read only filesystems :) Greetings Bernd - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
Hi! > 1. setup the power switch so it doesn't actually turn things off (it > issues the shutdown command instead) Evil. Devices that are powered off should stay powered off, and there should be big mechanical switch to do that, so that no EMI or power glitch can make them power up. Also thing about cases where powerplant fails, or when electricity in the house fails. I've seen places where electricity failed 5 times a day, because someone put 10A fuse and we were using just about 2kW... Pavel -- I'm [EMAIL PROTECTED] "In my country we have almost anarchy and I don't care." Panos Katsaloulis describing me w.r.t. patents at [EMAIL PROTECTED] - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
Hi! > > > Being able to shut down by hitting the power switch is a little luxury > > > for which I've been willing to invest more than a year of my life to > > > attain. Clueless newbies don't know why it should be any other way, and > > > it's essential for embedded devices. > > > > Clueless newbies (and slightly less clueless less newbie) type people > > don't think that they should HAVE to. My two interests are coping with > > particularly pedantic people who don't want there computer to hastle them > > about what they should or shouldn't do, and slightly embedded systems > > (e.g. set top box/web browsery thing that you want to be able to turn off > > like a TV but it should still be able to have a writeable disc for config > > and stuff you download/cache etc). > > Nothing wrong with a filesystem (or apps) that can handle being powered > down. > But I prefer to handle this kind of users with a power switch that > merely > acts as a "shutdown button" instead of actually killing power. > The os will then run the equivalent of "shutdown -h now" > You may not have this luxury on an ordinary pc, but you do if you design > embedded > devices. You will then be free to use existing GPL software that > not necessarily handle a power failure well. Heck, I hate hardware like that. Power button should kill the power, immediately. It should not sync, it should not tell system, it should just cut off the power. There _might_ be other button telling system to shutdown. Why is power button important? I already had 2 situations where softpower could be pretty fatal, and one where it _was_ fatal. 1. on times of 486, badly inserted ISA card led to explosion of one of low-integration chips. I've actually seen blue light. You can see that soft-power-off is just not appropriate for such situation. [But I probably could have survived soft-power off. Chip exploded and no more damage seemed to happen.] 2. on the same 486, one wire got short-circuited. Smoke started comming out of the box. By powering it down I even saved the wire (its isolation started boiling). 3. few days ago, our hot-wave owen commited suicide. Device with soft power controlled by CPU. During one night, CPU got crazy (power glitch? EMI?) and heating turned on itself. Fan did not. Owen destroyed itself. Pavel -- I'm [EMAIL PROTECTED] "In my country we have almost anarchy and I don't care." Panos Katsaloulis describing me w.r.t. patents at [EMAIL PROTECTED] - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
Hi, On Sat, Jan 06, 2001 at 08:57:26PM +0100, Marc Lehmann wrote: > On Fri, Jan 05, 2001 at 11:58:56AM +, David Woodhouse <[EMAIL PROTECTED]> >wrote: > > You mount it read-only, recover as much as possible from it, and bin it. > > > > You _don't_ want the fs code to ignore your explicit instructions not to > > write to the medium, and to destroy whatever data were left. > > The problem is: where did you give the explicit instruction? Just that you > define "read-only" as "the medium should not be written" does not mean > everybody else thinks the same. > > actually, I regard "ro" mainly as a "hey kernel, I won't handle writes > now, so please don't try it", like for cd-roms or other non-writeale > media, and please filesystem stay in a clean state. Right. There are two distinct meanings: 1) Do not write to this medium, ever (physical readonly); and 2) Do not allow modifications to the filesystem (logical readonly). The fact is that the kernel confuses the two, but that just isn't always legitimate. Filesystems may want to do things like garbage collection or defragmentation in the background even when there is no appplication write activity. We just don't have a way of specifying these two things independently. A journaled filesystem can only be dirty if it was in the middle of write activity, so mounting an unclean filesystem physically readonly is not much different from suddenly making the underlying media readonly during active service: of _course_ the filesystem is going to get upset! > That ro means "the medium is never written" is an assumption that does not > hold for most disks anyway and is, in the case of journlaing filesystems, > often impossible to implement. You simply can't salvage data without a log > reply. Sure, you can do virtual log replays, but for example the reiserfs > log is currently 32mb. Pinning down that much memory for a virtual log > reply is not possible on low-memory machines. You don't have to: it is theoretically possible to scan the journal and create a mapping from physical blocks on the rest of the filesystem to journal entries. The filesystem can then read from the log whenever a replayable block is accessed. However, this imposes a lookup cost on all accesses in the filesystem even when we aren't in this virtual translating mode, so it's not a cost I'd want to add unnecessarily. The bottom line is that journaling and recovery both require write access for now. I might implement an emergency readonly mount mode, but I'm not going to make it particularly streamlined at the cost of slowing down normal mounts. Cheers, Stephen - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
On Sat, Jan 06, 2001 at 03:35:02PM -0500, Chris Mason <[EMAIL PROTECTED]> wrote: > > Nobody with working brain would read it completely into memory. Instead everybody with a working brain would introduce another hashing layer for every block access? I don't think the reiserfs code (e.g.) would cope with yte another compliation in the code ;) -- -==- | ==-- _ | ---==---(_)__ __ __ Marc Lehmann +-- --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] |e| -=/_/_//_/\_,_/ /_/\_\ XX11-RIPE --+ The choice of a GNU generation | | - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
On Saturday, January 06, 2001 09:09:51 PM +0100 Stefan Traby <[EMAIL PROTECTED]> wrote: > On Sat, Jan 06, 2001 at 08:57:26PM +0100, Marc Lehmann wrote: > >> reply. Sure, you can do virtual log replays, but for example the reiserfs >> log is currently 32mb. Pinning down that much memory for a virtual log >> reply is not possible on low-memory machines. > > Nobody with working brain would read it completely into memory. > One just put the block-# that are in the journal into a hash-table > and read the block out of the journal when it's requested. > Because there may be multiple copies of the same block in the > journal, one should take the newest one that can be found in > the last commited transaction. > > IMHO Chris Mason already wrote such code, at least he talked about > it... > Talked about it, but never wrote it. However, I know there was code to do this for grub, I'm not sure if it ever made it into their releases. -chris - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
On Sat, Jan 06, 2001 at 08:57:26PM +0100, Marc Lehmann wrote: > reply. Sure, you can do virtual log replays, but for example the reiserfs > log is currently 32mb. Pinning down that much memory for a virtual log > reply is not possible on low-memory machines. Nobody with working brain would read it completely into memory. One just put the block-# that are in the journal into a hash-table and read the block out of the journal when it's requested. Because there may be multiple copies of the same block in the journal, one should take the newest one that can be found in the last commited transaction. IMHO Chris Mason already wrote such code, at least he talked about it... -- ciao - Stefan " ( cd /lib ; ln -s libBrokenLocale-2.2.so libNiedersachsen.so ) " Stefan TrabyLinux/ia32 fax: +43-3133-6107-9 Mitterlasznitzstr. 13 Linux/alphaphone: +43-3133-6107-2 8302 Nestelbach Linux/sparc http://www.hello-penguin.com Austriamailto:[EMAIL PROTECTED] Europe mailto:[EMAIL PROTECTED] - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
On Fri, Jan 05, 2001 at 11:58:56AM +, David Woodhouse <[EMAIL PROTECTED]> wrote: > You mount it read-only, recover as much as possible from it, and bin it. > > You _don't_ want the fs code to ignore your explicit instructions not to > write to the medium, and to destroy whatever data were left. The problem is: where did you give the explicit instruction? Just that you define "read-only" as "the medium should not be written" does not mean everybody else thinks the same. actually, I regard "ro" mainly as a "hey kernel, I won't handle writes now, so please don't try it", like for cd-roms or other non-writeale media, and please filesystem stay in a clean state. That ro means "the medium is never written" is an assumption that does not hold for most disks anyway and is, in the case of journlaing filesystems, often impossible to implement. You simply can't salvage data without a log reply. Sure, you can do virtual log replays, but for example the reiserfs log is currently 32mb. Pinning down that much memory for a virtual log reply is not possible on low-memory machines. So the first thing would be to precisely define the meaning of the "ro" flag. Before this has happened it is ansolutely senseless to argue about what it means, as it doesn't mean anything at the moment, except (man mount): ro Mount the file system read-only. Which it does even with journaling filesystems... -- -==- | ==-- _ | ---==---(_)__ __ __ Marc Lehmann +-- --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] |e| -=/_/_//_/\_,_/ /_/\_\ XX11-RIPE --+ The choice of a GNU generation | | - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
[EMAIL PROTECTED] said: > Many newer cell phones, even low spec ones, will have a software > power switch (usually with a hardware override after about 5 seconds > of continuous press). There are many other concessions that need to > be made to power efficiency, like the ability to toggle power to even > very minor peripherals and chips (not only each CPU and DSP, but the > bus controllers and UARTs connecting things together). These things > sell in the millions, so their designers can easily budget the custom > logic. Even if it only costs pennies - why on earth would someone want to add logic to a board when they could just fix the software? It soon adds up when you ship a million units. -- dwmw2 - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
On Thu, Jan 04, 2001 at 06:11:11PM +, Alan Cox wrote: > > in an enbedded device you can > > 1. setup the power switch so it doesn't actually turn things off (it > > issues the shutdown command instead) > > Costs too much money Many newer cell phones, even low spec ones, will have a software power switch (usually with a hardware override after about 5 seconds of continuous press). There are many other concessions that need to be made to power efficiency, like the ability to toggle power to even very minor peripherals and chips (not only each CPU and DSP, but the bus controllers and UARTs connecting things together). These things sell in the millions, so their designers can easily budget the custom logic. miket - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
Hi, On Fri, Jan 05, 2001 at 12:46:19PM +, Alan Cox wrote: > > recovery. Because the ext3 journal is just a series of data blocks to > > be copied into the filesystem (rather than "actions" to be done), it > > doesn't matter how many times it is done. The recovery flags are not > > reset until after the journal replay is completed. > > Which means an ext3 volume cannot be recovered on a hard disk error. Depends on the error. If the disk has gone hard-readonly, then we need to recover in core, and that's something which is not yet implemented but is a known todo item. Otherwise, it's not worse than an error on ext2: you don't have a guaranteed safe state to return to so you fall back to recovering what you can from the journal and then running an e2fsck pass. e2fsck groks the journal already. And yes, a badly faulty drive can mess this up, but it can mess it for ext2 just as badly. Cheers, Stephen - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
> Unless Stephen says otherwise, my understanding is that a crash during > journal recovery will just mean the journal is replayed again at the next > recovery. Because the ext3 journal is just a series of data blocks to > be copied into the filesystem (rather than "actions" to be done), it > doesn't matter how many times it is done. The recovery flags are not > reset until after the journal replay is completed. Which means an ext3 volume cannot be recovered on a hard disk error. Alan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
Hi, On Fri, Jan 05, 2001 at 02:01:37AM +0100, Stefan Traby wrote: > > Please tell me how to specify "noreplay" for the initial "/" mount > :) You don't have to: the filesystem knows when a root mount is happening, and can do the extra work then to make sure that the mount isn't failed on a readonly media. --Stephen - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
[EMAIL PROTECTED] said: > Powering down a VCR whilst recording can damage the tape or even > worse have the tap get jammed in the video. I have also had a TV die > because it was unpowered from the mains without being switched off > first. > Sure, these things don't always happen -- but they sometimes do. I > would argue things like VCRs and TVs are just more tolerant than more > complex systems -- not immune. The reasoning here seems to be that because many other systems break under these circumstances, we shouldn't bother to make Linux reliable. I don't quite understand that way of thinking. I will continue to test the boards I work on with random power cycles and to consider the filesystem broken if it doesn't like that treatment. -- dwmw2 - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
[EMAIL PROTECTED] said: > In what way? A root fs readonly mount is usually designed to prevent ^^^ > the filesystem from being stomped on during the initial boot so that > fsck can run without the filesystem being volatile. That's the only > reason for the readonly mount: to allow recovery before we enable > writes. With ext3, that recovery is done in the kernel, so doing that > recovery during mount makes perfect sense even if the user is mounting > root readonly. Alternative reasons for readonly mount include "my hard drive is dying and I don't want _anything_ to write to it because it'll explode". You mount it read-only, recover as much as possible from it, and bin it. You _don't_ want the fs code to ignore your explicit instructions not to write to the medium, and to destroy whatever data were left. -- dwmw2 - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
Hi, On Fri, Jan 05, 2001 at 01:31:12AM +0100, Daniel Phillips wrote: > "Stephen C. Tweedie" wrote: > > > Yes, and so long as your journal is not on another partition/disk things > will eventually be set right. The combination of a partially updated > filesystem and its journal is in some sense a complete, consistent > filesystem. Right. > I'm curious - how does ext3 handle the possibility of a crash during > journal recovery? Recovery comes in two parts: replay of the log, and deletion of orphaned inodes. The log replay is fully idempotent: it just consists of writing the appropriate log blocks to their home locations on disk. It doesn't matter if you do those writes once, twice or a hundred times, you still get the same result, so crashing mid-recovery and starting again from scratch is perfectly safe. Once that recovery is done, the filesystem is marked, atomically, to be clean. Secondly, we do the deletion of orphaned inodes. By this time, the journal itself has been recovered so we are journaling our operations, so each orphaned inode delete is being logged. A crash will recover to a consistent state via the normal journaling transaction methods, and will restart orphan recovery from where it left off. Cheers, Stephen - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
Stefan, you write: > [Re: read-only filesystem vs. read-only device] > Anyway, it is "especially" critical on the root filesystem because the > authors of filesystems can't support two ro states on boot. > > Reiserfs allowed -oro,noreplay. > > Please tell me how to specify "noreplay" for the initial "/" mount :) Actually, for ext3 Stephen added a kernel option "rootflags" so that you can pass mount options to the root filesystem. This was previously needed to add a journal to an existing ext2 root filesystem. I hope he keeps it around anyways, and submits it as a regular kernel patch, because it is useful for many other things. > But this has nothing to do with forcing a write on "ro" mounts, which > I interpret as design bug. (ro,noreplay is also a kind of design bug, > everything except a virtual replay under physical ro conditions looks > like a design bug to me because it breaks user expectations either > by writing on "ro" or by giving an invalid view by "noreplay") If the VM subsystem can tolerate keeping dirty pages around for a read-only device then virtual replay can be made to work, for no more memory than might be pinned by having a journal in the first place. The only problem is that normal journal pages will eventually be freed, whereas virtual journal replay pages would not (until the filesystem is unmounted or mounted read-write). This _may_ be OK in some cases, but I think most people with journalled filesystem have more journal space than RAM, so you will likely get into bad situations very quickly, all for a technical nit. Currently ext3 just refuses to load on a read-only device if the journal is dirty. However, changes are upcoming to allow LVM snapshots on ext3 filesystems, so there is little legitimate need for a dirty journal on a read-only device. Cheers, Andreas -- Andreas Dilger \ "If a man ate a pound of pasta and a pound of antipasto, \ would they cancel out, leaving him still hungry?" http://www-mddsp.enel.ucalgary.ca/People/adilger/ -- Dogbert - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
Daniel writes: > Yes, and so long as your journal is not on another partition/disk things > will eventually be set right. The combination of a partially updated > filesystem and its journal is in some sense a complete, consistent > filesystem. > > I'm curious - how does ext3 handle the possibility of a crash during > journal recovery? Unless Stephen says otherwise, my understanding is that a crash during journal recovery will just mean the journal is replayed again at the next recovery. Because the ext3 journal is just a series of data blocks to be copied into the filesystem (rather than "actions" to be done), it doesn't matter how many times it is done. The recovery flags are not reset until after the journal replay is completed. Cheers, Andreas -- Andreas Dilger \ "If a man ate a pound of pasta and a pound of antipasto, \ would they cancel out, leaving him still hungry?" http://www-mddsp.enel.ucalgary.ca/People/adilger/ -- Dogbert - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
> I have an old IBM Aptiva 486 SX that actually DOES something like this; what > it does is, it suspends to disk when you hit the power button (you have to > turn that option on though). > Point being, if it was possible back then (6 years ago), why on earth would > it be too expensive now? I'd guess the aptiva new was a bit over $100 ? - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
On Thu, 4 Jan 2001, Alan Cox wrote: > From: Alan Cox <[EMAIL PROTECTED]> > Subject: Re: Journaling: Surviving or allowing unclean shutdown? > > > in an enbedded device you can > > 1. setup the power switch so it doesn't actually turn things off (it > > issues the shutdown command instead) > > Costs too much money I have an old IBM Aptiva 486 SX that actually DOES something like this; what it does is, it suspends to disk when you hit the power button (you have to turn that option on though). I once actually had a situation where a badly behaved application prevented this suspend-to-disk, and I had to pull the plug. Point being, if it was possible back then (6 years ago), why on earth would it be too expensive now? Greetings, Chipzz AKA Jan Van Buggenhout -- -- UNIX isn't dead - It just smells funny [EMAIL PROTECTED] -- - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
On Thu, Jan 04, 2001 at 10:49:46PM +, Stephen C. Tweedie wrote: > > I think any other action (only replaying on rw mount and presenting > > a broken filesystem on ro) is quite fatal, at least if I think of > > a replay on -remount,rw :) > > Correct. > > > Also, an unconditional hidden replay even if "ro" is specified is not nice. > > > This is especially critical on root filesystem > > In what way? A root fs readonly mount is usually designed to prevent > fsck can run without the filesystem being volatile. That's the only There are people out that say that readonly mount was initially designed to behave as physically read only. fsck was a wanted side-effect... And trust me, most people think so before dialing with a journaled filesystem. This was discussed to death on the reiserFS list. Clearly the definition of "ro" is weak, does it mean media or filesystem view ? It's even weak on lower levels: There are also disks out there that write even if physical write-protection is enabled (for example auto-remapping after an ECC-recovery read). Anyway, it is "especially" critical on the root filesystem because the authors of filesystems can't support two ro states on boot. Reiserfs allowed -oro,noreplay. Please tell me how to specify "noreplay" for the initial "/" mount :) Yes, I think that the journal is an integral part of the filesystem. But this has nothing to do with forcing a write on "ro" mounts, which I interpret as design bug. (ro,noreplay is also a kind of design bug, everything except a virtual replay under physical ro conditions looks like a design bug to me because it breaks user expectations either by writing on "ro" or by giving an invalid view by "noreplay") -- ciao - Stefan " ( cd /lib ; ln -s libBrokenLocale-2.2.so libNiedersachsen.so ) " Stefan TrabyLinux/ia32 fax: +43-3133-6107-9 Mitterlasznitzstr. 13 Linux/alphaphone: +43-3133-6107-2 8302 Nestelbach Linux/sparc http://www.hello-penguin.com Austriamailto:[EMAIL PROTECTED] Europe mailto:[EMAIL PROTECTED] - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
"Stephen C. Tweedie" wrote: > > On Wed, Jan 03, 2001 at 05:27:25PM +0100, Daniel Phillips wrote: > > > > Tux2 is explicitly designed to legitimize pulling the plug as a valid > > way of shutting down. Metadata-only journalling filesystems are not > > designed to be used this way, and even with full-data journalling you > > should bear in mind that your on-disk filesystem image remains in an > > invalid state until the journal recovery program has run successfully. > > ext3 does the recovery automatically during mount(8), so user space > will never see an unrecovered filesystem. (There are filesystem flags > set by the journal code to make sure that an unrecovered filesystem > never gets mounted by a kernel which doesn't know how to do the > appropriate recovery.) Hi Stephen. Yes, and so long as your journal is not on another partition/disk things will eventually be set right. The combination of a partially updated filesystem and its journal is in some sense a complete, consistent filesystem. I'm curious - how does ext3 handle the possibility of a crash during journal recovery? -- Daniel - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
Hi, On Thu, Jan 04, 2001 at 10:08:21PM +0100, Stefan Traby wrote: > On Thu, Jan 04, 2001 at 07:21:04PM +, Stephen C. Tweedie wrote: > > > ext3 does the recovery automatically during mount(8), so user space > > will never see an unrecovered filesystem. (There are filesystem flags > > set by the journal code to make sure that an unrecovered filesystem > > never gets mounted by a kernel which doesn't know how to do the > > appropriate recovery.) > > I did not follow the ext3 development recently, how did you solve > the "read-only mount(2) (optionally on write protected media)" issue ? > > Does the mount fail, or does the code virtually replays (without writing) > only ? The code currently checks if the underlying media is write-protected. If it is, it fails the mount; if not, it does the replay (so that mounting a root fs readonly works correctly). I will be adding support for virtual replay for root filesystems to act as a last-chance way of recovering if you really cannot write to the root, but journaling filesystems really do expect to be able to write to the media so I am not sure whether it makes sense to support this on non-root filesystems too. > I think any other action (only replaying on rw mount and presenting > a broken filesystem on ro) is quite fatal, at least if I think of > a replay on -remount,rw :) Correct. > Also, an unconditional hidden replay even if "ro" is specified is not nice. > This is especially critical on root filesystem In what way? A root fs readonly mount is usually designed to prevent the filesystem from being stomped on during the initial boot so that fsck can run without the filesystem being volatile. That's the only reason for the readonly mount: to allow recovery before we enable writes. With ext3, that recovery is done in the kernel, so doing that recovery during mount makes perfect sense even if the user is mounting root readonly. Cheers, Stephen - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
On Thu, Jan 04, 2001 at 03:59:40PM -0500, Richard B. Johnson wrote: > Well they do! It's just not allowed for us (the users) to know that they > __didn't__ run completely out of power! If the thing is so dead > that it won't recharge, it still has 'power' (enough to keep static RAM > alive). Just remove the battery, wait about 120 seconds for a capacitor > to discharge, and, zap, no more stored phone numbers. Static RAM with > an electrolytic capacitor, isolated with a diode, takes so little power > that you can normally change defective batteries if you don't take > too long. Not over here in Europe where we use GSM phones. The phone numbers, phone settings, and SMS (Small Message Services) messages are all stored in the flash memory on the SIM (Subscriber Identity Module, a 1x2.5 cm chip card). Erik -- J.A.K. (Erik) Mouw, Information and Communication Theory Group, Department of Electrical Engineering, Faculty of Information Technology and Systems, Delft University of Technology, PO BOX 5031, 2600 GA Delft, The Netherlands Phone: +31-15-2783635 Fax: +31-15-2781843 Email: [EMAIL PROTECTED] WWW: http://www-ict.its.tudelft.nl/~erik/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
> > Being able to shut down by hitting the power switch is a little luxury > > for which I've been willing to invest more than a year of my life to > > attain. Clueless newbies don't know why it should be any other way, and > > it's essential for embedded devices. Just some food for thought - hitting the power switch on my old Indy actually performs the equivalent of "shutdown -r now"; the system only cuts the power when it's done cleaning up (sometimes several minutes later). I suspect most workstation-class systems do similar things. Of course this creates a confusing distinction between "pulling the plug" and "hitting the power switch." Uninformed users might even be more bewildered by the flurry of disk activity after performing the latter; heck, I wouldn't blame someone who freaks out and pull the plug to make it stop =). Also, such a system obviously has little benefit in the event of an AC power failure. Dan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
On Thu, Jan 04, 2001 at 07:21:04PM +, Stephen C. Tweedie wrote: > ext3 does the recovery automatically during mount(8), so user space > will never see an unrecovered filesystem. (There are filesystem flags > set by the journal code to make sure that an unrecovered filesystem > never gets mounted by a kernel which doesn't know how to do the > appropriate recovery.) I did not follow the ext3 development recently, how did you solve the "read-only mount(2) (optionally on write protected media)" issue ? Does the mount fail, or does the code virtually replays (without writing) only ? I think any other action (only replaying on rw mount and presenting a broken filesystem on ro) is quite fatal, at least if I think of a replay on -remount,rw :) Also, an unconditional hidden replay even if "ro" is specified is not nice. This is especially critical on root filesystem, because there is IMHO no way to specify mount arguments to the "/" mount, except ro/rw. -- ciao - Stefan " ( cd /lib ; ln -s libBrokenLocale-2.2.so libNiedersachsen.so ) " Stefan TrabyLinux/ia32 fax: +43-3133-6107-9 Mitterlasznitzstr. 13 Linux/alphaphone: +43-3133-6107-2 8302 Nestelbach Linux/sparc http://www.hello-penguin.com Austriamailto:[EMAIL PROTECTED] Europe mailto:[EMAIL PROTECTED] - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
On 4 Jan, Richard B. Johnson wrote: > Well they do! It's just not allowed for us (the users) to know that > they > __didn't__ run completely out of power! If the thing is so dead > that it won't recharge, it still has 'power' (enough to keep static > RAM alive). Just remove the battery, wait about 120 seconds for a > capacitor to discharge, and, zap, no more stored phone numbers. > Static RAM with an electrolytic capacitor, isolated with a diode, > takes so little power that you can normally change defective batteries > if you don't take too long. I've several Nokias and Siemens phones here. All of them will survive for weeks without any battery connected; and not only the phone numbers on the SIM card. I guess they all have FlashRAM, although I haven't disassembled one recently. -- Servus, Daniel - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
Well. How does tivo handle this situation? Certainly, there are instances when your power will fail in your home and your Tivo will be without juice. When it powers back on what does it do to check the fs? Does it need to check the fs? Brett G. Person 415-358-2656 [EMAIL PROTECTED] Penguin Computing - The World's Most Reliable Linux Systems - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
On Thu, 4 Jan 2001 [EMAIL PROTECTED] wrote: > On 4 Jan, Richard B. Johnson wrote: > > > A mobile-phone that runs out of battery power will also lose all the > > phone numbers you have stored, etc. The same is true for most all > > embedded systems that save data. > > In your world maybe. I would be quite pissed if my mobile phones lost > the stored numbers every time they run out of power. Nearly all embedded > devices nowadays keep their settings without power; be it a satellite > receiver, a PBX, a fax machine or a coffee brewer. Well they do! It's just not allowed for us (the users) to know that they __didn't__ run completely out of power! If the thing is so dead that it won't recharge, it still has 'power' (enough to keep static RAM alive). Just remove the battery, wait about 120 seconds for a capacitor to discharge, and, zap, no more stored phone numbers. Static RAM with an electrolytic capacitor, isolated with a diode, takes so little power that you can normally change defective batteries if you don't take too long. > > > This means that the data-base > > software has to roll-back and redo the temporarily-lost updates > > when it restarts. It uses the journal to accomplish this. As > > N-seconds gets smaller, the overhead necessary to maintain data > > consistency gets greater, so there are trade-offs. > > And depending on the application they may really be worth it. > > > A journaling file-system also needs some number to show the > > order of operations so that roll-backs and restarts work. > > Unfortunately, the systems that I have seen all use time for > > the number! You don't want time to reconstruct 'order'. You > > need a number that represents 'order'. Time is not your friend. > > Since the metadata has to be sync anyway what about using a > normal transaction counter? > Correct! Cheers, Dick Johnson Penguin : Linux version 2.4.0 on an i686 machine (799.54 BogoMips). "Memory is like gasoline. You use it up when you are running. Of course you get it all back when you reboot..."; Actual explanation obtained from the Micro$oft help desk. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
On 4 Jan, Richard B. Johnson wrote: > A mobile-phone that runs out of battery power will also lose all the > phone numbers you have stored, etc. The same is true for most all > embedded systems that save data. In your world maybe. I would be quite pissed if my mobile phones lost the stored numbers every time they run out of power. Nearly all embedded devices nowadays keep their settings without power; be it a satellite receiver, a PBX, a fax machine or a coffee brewer. > This means that the data-base > software has to roll-back and redo the temporarily-lost updates > when it restarts. It uses the journal to accomplish this. As > N-seconds gets smaller, the overhead necessary to maintain data > consistency gets greater, so there are trade-offs. And depending on the application they may really be worth it. > A journaling file-system also needs some number to show the > order of operations so that roll-backs and restarts work. > Unfortunately, the systems that I have seen all use time for > the number! You don't want time to reconstruct 'order'. You > need a number that represents 'order'. Time is not your friend. Since the metadata has to be sync anyway what about using a normal transaction counter? -- Servus, Daniel - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
On Thu, 4 Jan 2001, Mo McKinlay wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Today, David Lang ([EMAIL PROTECTED]) wrote: > > > On Thu, 4 Jan 2001, Mo McKinlay wrote: > > > > > > The off button need not and _does not_ remove power instantly (if at > > > > all) on many appliances. > > > > > > Indeed - but unplugging your VCR from the wall won't harm it. Everyone > > > knows the power button on a TV/VCR/etc doesn't actually kill the power, > > > just reduce consumption (i.e., standby mode). But unplugging it at the > > > wall doesn't have any detrimental effects - doing that to a PC will. > > > > if you change that statement to "usually won't harm it" I agree with you > > (I have had a VCR eat a tape when this was done) > > Crikey. Most people would consider that a fault in the VCR. > > Just goes to show how far removed from appliances PCs currently are. > A nice "brand-name" Network-centric printer (with an IP address) requires that you "save" the parameters in NVRAM. If you pull the plug during this operation, it comes up with its default IP and you have to do it again. A mobile-phone that runs out of battery power will also lose all the phone numbers you have stored, etc. The same is true for most all embedded systems that save data. A file-system is a bit more sophisticated. You can design it so that no "known" corruption occurs. A simple, but not fast method would be to never actually modify an existing file. Instead, when an existing file is opened for write, a copy is created. All new work is done on the copy (which is probably invisible at the time). Once the file is closed, and all buffers written, the new file replaces the old. This has to be done in a manner in which a power-failure at any time will leave you with either file, but not both. This requires a journal to show how far you've gotten into the 'rename' operation. A corrupted journal just leaves you with the old file. This same idea can be modularized into file-sections that are much smaller than an entire file. In this manner, the working file that is actually accessed will always seem to be perfect, although your last N-seconds of updates may have been lost to the power failure. This means that the data-base software has to roll-back and redo the temporarily-lost updates when it restarts. It uses the journal to accomplish this. As N-seconds gets smaller, the overhead necessary to maintain data consistency gets greater, so there are trade-offs. In the case of NVRAM data, the printer software could have been designed to survive a power failure even though a cursory look at the way NVRAM operates may make it seem impossible. To write NVRAM, you erase if first. If you have a power-failure at this time, all is lost --not. You just need to erase sectors (pages), or have two NVRAM devices. Part of the data saved, is a number that increases during each save operation. The CPU software gets its parameters from the latest (highest number). A journaling file-system also needs some number to show the order of operations so that roll-backs and restarts work. Unfortunately, the systems that I have seen all use time for the number! You don't want time to reconstruct 'order'. You need a number that represents 'order'. Time is not your friend. Cheers, Dick Johnson Penguin : Linux version 2.4.0 on an i686 machine (799.54 BogoMips). "Memory is like gasoline. You use it up when you are running. Of course you get it all back when you reboot..."; Actual explanation obtained from the Micro$oft help desk. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
Hi, On Wed, Jan 03, 2001 at 05:27:25PM +0100, Daniel Phillips wrote: > > Tux2 is explicitly designed to legitimize pulling the plug as a valid > way of shutting down. Metadata-only journalling filesystems are not > designed to be used this way, and even with full-data journalling you > should bear in mind that your on-disk filesystem image remains in an > invalid state until the journal recovery program has run successfully. ext3 does the recovery automatically during mount(8), so user space will never see an unrecovered filesystem. (There are filesystem flags set by the journal code to make sure that an unrecovered filesystem never gets mounted by a kernel which doesn't know how to do the appropriate recovery.) --Stephen - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Today, David Lang ([EMAIL PROTECTED]) wrote: > On Thu, 4 Jan 2001, Mo McKinlay wrote: > > > > The off button need not and _does not_ remove power instantly (if at > > > all) on many appliances. > > > > Indeed - but unplugging your VCR from the wall won't harm it. Everyone > > knows the power button on a TV/VCR/etc doesn't actually kill the power, > > just reduce consumption (i.e., standby mode). But unplugging it at the > > wall doesn't have any detrimental effects - doing that to a PC will. > > if you change that statement to "usually won't harm it" I agree with you > (I have had a VCR eat a tape when this was done) Crikey. Most people would consider that a fault in the VCR. Just goes to show how far removed from appliances PCs currently are. - -- Mo McKinlay [EMAIL PROTECTED] - - GnuPG/PGP Key: pub 1024D/76A275F9 2000-07-22 -BEGIN PGP SIGNATURE- Version: GnuPG v1.0.4 (GNU/Linux) Comment: For info see http://www.gnupg.org iEYEARECAAYFAjpUvwcACgkQRcGgB3aidfkR3wCfVTY9NJY8irZ5BNxgQ1jrQWsP 3jIAnjVpIdJtOb66Q1wK451QPH00Q7wH =90Eb -END PGP SIGNATURE- - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
On Thu, 4 Jan 2001, Alan Cox wrote: > > for crying out loud, even windows tells the users they need to shutdown > > first and gripes at them if they pull the plug. what users are you trying > > to protect, ones to clueless to even run windows? > > Clueless ? Hardly. Every other appliance in the home you turn it off and it > goes off. You turn it on and it comes on. You get confused you turn it off and > on. Its the definitive model of how home appliances works and its how people > expect them to work. I can see both sides of the argument - consider your video - users can switch that off without thinking; but really its off switch is soft and actually unloads the tape from the heads first before switching off. Having said that look at something like test equipments; you can just pull the power on them - I don't think large scopes/logic analysers with discs in have anything clever hardware wise to to a careful switch off. (Some of these are running Win98 and some HP-UX these days with journalling stuff in). Dave -- /--\ | Dr. David Alan Gilbert | Work:[EMAIL PROTECTED] +44-161-286-2000 Ex258| | G7FHJ | | | Home: [EMAIL PROTECTED]http://www.treblig.org | \--/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
On Thu, 4 Jan 2001, Mo McKinlay wrote: > > The off button need not and _does not_ remove power instantly (if at > > all) on many appliances. > > Indeed - but unplugging your VCR from the wall won't harm it. Everyone > knows the power button on a TV/VCR/etc doesn't actually kill the power, > just reduce consumption (i.e., standby mode). But unplugging it at the > wall doesn't have any detrimental effects - doing that to a PC will. if you change that statement to "usually won't harm it" I agree with you (I have had a VCR eat a tape when this was done) David Lang > Being able to change what the power button does is cool, but it does mask > the real issue - what happens when you pull the plug, and how do you make > it so that it's acceptable? > > -- > Mo McKinlay > [EMAIL PROTECTED] > - > GnuPG/PGP Key: pub 1024D/76A275F9 2000-07-22 > > > > > - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 > The off button need not and _does not_ remove power instantly (if at > all) on many appliances. Indeed - but unplugging your VCR from the wall won't harm it. Everyone knows the power button on a TV/VCR/etc doesn't actually kill the power, just reduce consumption (i.e., standby mode). But unplugging it at the wall doesn't have any detrimental effects - doing that to a PC will. Being able to change what the power button does is cool, but it does mask the real issue - what happens when you pull the plug, and how do you make it so that it's acceptable? - -- Mo McKinlay [EMAIL PROTECTED] - - GnuPG/PGP Key: pub 1024D/76A275F9 2000-07-22 -BEGIN PGP SIGNATURE- Version: GnuPG v1.0.4 (GNU/Linux) Comment: For info see http://www.gnupg.org iEYEARECAAYFAjpUvcEACgkQRcGgB3aidflzEwCgpYdN7Tp7e1S3HGoTA6JKBS40 +GUAn20lCCVeqJbPzJ5k+qJd1OHsZjqu =YQ4B -END PGP SIGNATURE- - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 > for crying out loud, even windows tells the users they need to shutdown > first and gripes at them if they pull the plug. what users are you trying > to protect, ones to clueless to even run windows? > > David Lang > > > > > it's essential for embedded devices. Answer your question? - -- Mo McKinlay [EMAIL PROTECTED] - - GnuPG/PGP Key: pub 1024D/76A275F9 2000-07-22 -BEGIN PGP SIGNATURE- Version: GnuPG v1.0.4 (GNU/Linux) Comment: For info see http://www.gnupg.org iEYEARECAAYFAjpUvNYACgkQRcGgB3aidfmyAgCfRdJT1cIUN5eu28Op6BZrytb5 QEoAnReB9nUaWFIEemV7QJsSkMES32Vi =vGm7 -END PGP SIGNATURE- - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
> in an enbedded device you can > 1. setup the power switch so it doesn't actually turn things off (it > issues the shutdown command instead) Costs too much money > 2. run from read-only media almost exclusivly so that power event's don't > bother you much Depends on the device > 3. you can add extra power inside the device so that if someone does pull > the plug, you have a few seconds of power to do the clean shutdown Incredibly expensive > 4. you can run out of ram and force the user to do an extra step to save > any changes to non-volitile storage (and if they power off during the save > the results are undefined) Frowned upon because you keep getting dead units back > an improved filesystem that tolorates bad shutdowns reasonably well will > be welcomed for other reasons, but should not be viewed as a fix for > people pulling the plug on you. If it doesnt fix the pulling the plug case (at least as far as 'after fsync returned this data is safe') then its not working. A journalling file system is always consistent. fsync/sync/msync define write barrier operations. Applications do have to deal with more things (partial dbase write for example) but competently written databases and transaction managers handle this. Ext3 has been tested on the random live switchoffs on a real box because ftp.linux.org.uk kept crashing during backups and it took us a while to figure out that the network backup tool was backing up in /proc/bus/pci and stopping the box dead when it backed up the pci config space of the ACPI controller. Alan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
in an enbedded device you can 1. setup the power switch so it doesn't actually turn things off (it issues the shutdown command instead) 2. run from read-only media almost exclusivly so that power event's don't bother you much 3. you can add extra power inside the device so that if someone does pull the plug, you have a few seconds of power to do the clean shutdown 4. you can run out of ram and force the user to do an extra step to save any changes to non-volitile storage (and if they power off during the save the results are undefined) I have seen all of these approaches used in different devices (that are not running linux). This is not a new problem and the people working in this space have a bunch of answers. an improved filesystem that tolorates bad shutdowns reasonably well will be welcomed for other reasons, but should not be viewed as a fix for people pulling the plug on you. David Lang On Thu, 4 Jan 2001, David Woodhouse wrote: > Date: Thu, 04 Jan 2001 17:52:25 + > From: David Woodhouse <[EMAIL PROTECTED]> > To: David Lang <[EMAIL PROTECTED]> > Cc: Daniel Phillips <[EMAIL PROTECTED]>, > Helge Hafting <[EMAIL PROTECTED]>, [EMAIL PROTECTED] > Subject: Re: Journaling: Surviving or allowing unclean shutdown? > > > [EMAIL PROTECTED] said: > > for crying out loud, even windows tells the users they need to > > shutdown first and gripes at them if they pull the plug. what users > > are you trying to protect, ones to clueless to even run windows? > > Precisely. Users of embedded devices don't expect to have to treat them > like computers. > > -- > dwmw2 > > - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
> for crying out loud, even windows tells the users they need to shutdown > first and gripes at them if they pull the plug. what users are you trying > to protect, ones to clueless to even run windows? Clueless ? Hardly. Every other appliance in the home you turn it off and it goes off. You turn it on and it comes on. You get confused you turn it off and on. Its the definitive model of how home appliances works and its how people expect them to work. In the embedded world you will regularly see adherence to that model in the specification. Firstly because the users do it, secondly because power cuts ensure it happens anyway Alan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
[EMAIL PROTECTED] said: > for crying out loud, even windows tells the users they need to > shutdown first and gripes at them if they pull the plug. what users > are you trying to protect, ones to clueless to even run windows? Precisely. Users of embedded devices don't expect to have to treat them like computers. -- dwmw2 - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
for crying out loud, even windows tells the users they need to shutdown first and gripes at them if they pull the plug. what users are you trying to protect, ones to clueless to even run windows? David Lang On Thu, 4 Jan 2001, Daniel Phillips wrote: > Date: Thu, 04 Jan 2001 11:14:53 +0100 > From: Daniel Phillips <[EMAIL PROTECTED]> > To: Helge Hafting <[EMAIL PROTECTED]>, [EMAIL PROTECTED] > Subject: Re: Journaling: Surviving or allowing unclean shutdown? > > Helge Hafting wrote: > > > > [...] > > > > Being able to shut down by hitting the power switch is a little luxury > > > > for which I've been willing to invest more than a year of my life to > > > > attain. Clueless newbies don't know why it should be any other way, and > > > > it's essential for embedded devices. > > > > > > Clueless newbies (and slightly less clueless less newbie) type people > > > don't think that they should HAVE to. My two interests are coping with > > > particularly pedantic people who don't want there computer to hastle them > > > about what they should or shouldn't do, and slightly embedded systems > > > (e.g. set top box/web browsery thing that you want to be able to turn off > > > like a TV but it should still be able to have a writeable disc for config > > > and stuff you download/cache etc). > > > > Nothing wrong with a filesystem (or apps) that can handle being powered > > down. > > But I prefer to handle this kind of users with a power switch that > > merely > > acts as a "shutdown button" instead of actually killing power. > > The os will then run the equivalent of "shutdown -h now" > > And you give your customer clear instructions that they are not under > any circumstances to unplug the device without turning it off first? > And when they do it anyway you void their warranty? > > -- > Daniel > - > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to [EMAIL PROTECTED] > Please read the FAQ at http://www.tux.org/lkml/ > - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
On Thu, 4 Jan 2001, Daniel Phillips wrote: > > A lot of applications always rely on their file i/o being done in some > > manner that has atomic (from the application's point of view) operations > > other than system calls -- heck, even make(1) does that. > > Nobody is forcing you to hit the power switch in the middle of a build. > But now that you mention it, you've provided a good example of a broken > application. Make with its reliance on timestamps for determining build > status is both painfully slow and unreliable. Actually I mean its reliance on files being deleted if the problem or SIGTERM happened in the middle of build ing them. > What happens if you > adjust your system clock? Don't adjust the system clock in the middle of the build. Adjusting clock backward for more than a second is much more rare operation than a shutdown. > That said, Tux2 can preserve the per-write > atomicity quite easily, or better, make could take advantage of the new > journal-oriented transaction api that's being cooked up and specify its > requirement for atomicity in a precise way. I have already said that programs don't use syscalls as the only atomic operations on files -- yes, it may be a good idea to add transactions API on the top of this (and it will have a lot of uses), but then it should be made in a way that its use will be easy to add to existing applications. > Do you have any other examples of programs that would be hurt by sudden > termination? Certainly we'd consider a desktop gui broken if it failed > to come up again just because you bailed out with the power switch > instead of logging out nicely. Any application that writes multiple times over the same files and has any data consistency requirements beyond the piece of data in the chunk sent in one write(). -- Alex -- Excellent.. now give users the option to cut your hair you hippie! -- Anonymous Coward - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
On Thu, 4 Jan 2001, Daniel Phillips wrote: > > Nothing wrong with a filesystem (or apps) that can handle being > > powered down. But I prefer to handle this kind of users with a > > power switch that merely acts as a "shutdown button" instead of > > actually killing power. The os will then run the equivalent of > > "shutdown -h now" > > And you give your customer clear instructions that they are not under > any circumstances to unplug the device without turning it off first? > And when they do it anyway you void their warranty? s/unplug the device/remove the battery/ makes it _slightly_ more sensible but not a lot. I spent a week with an X10 controller and a device with JFFS root filesystem running dbench from init, power cycling it every five minutes. If JFFS at least doesn't survive unclean shutdowns, I want to know about it. -- dwmw2 - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
Helge Hafting wrote: > > [...] > > > Being able to shut down by hitting the power switch is a little luxury > > > for which I've been willing to invest more than a year of my life to > > > attain. Clueless newbies don't know why it should be any other way, and > > > it's essential for embedded devices. > > > > Clueless newbies (and slightly less clueless less newbie) type people > > don't think that they should HAVE to. My two interests are coping with > > particularly pedantic people who don't want there computer to hastle them > > about what they should or shouldn't do, and slightly embedded systems > > (e.g. set top box/web browsery thing that you want to be able to turn off > > like a TV but it should still be able to have a writeable disc for config > > and stuff you download/cache etc). > > Nothing wrong with a filesystem (or apps) that can handle being powered > down. > But I prefer to handle this kind of users with a power switch that > merely > acts as a "shutdown button" instead of actually killing power. > The os will then run the equivalent of "shutdown -h now" And you give your customer clear instructions that they are not under any circumstances to unplug the device without turning it off first? And when they do it anyway you void their warranty? -- Daniel - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
[...] > > Being able to shut down by hitting the power switch is a little luxury > > for which I've been willing to invest more than a year of my life to > > attain. Clueless newbies don't know why it should be any other way, and > > it's essential for embedded devices. > > Clueless newbies (and slightly less clueless less newbie) type people > don't think that they should HAVE to. My two interests are coping with > particularly pedantic people who don't want there computer to hastle them > about what they should or shouldn't do, and slightly embedded systems > (e.g. set top box/web browsery thing that you want to be able to turn off > like a TV but it should still be able to have a writeable disc for config > and stuff you download/cache etc). Nothing wrong with a filesystem (or apps) that can handle being powered down. But I prefer to handle this kind of users with a power switch that merely acts as a "shutdown button" instead of actually killing power. The os will then run the equivalent of "shutdown -h now" You may not have this luxury on an ordinary pc, but you do if you design embedded devices. You will then be free to use existing GPL software that not necessarily handle a power failure well. Helge Hafting - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
Alex Belits wrote: > > On Wed, 3 Jan 2001, Daniel Phillips wrote: > > > I don't doubt that if the 'power switch' method of shutdown becomes > > popular we will discover some applications that have windows where they > > can be hurt by sudden shutdown, even will full filesystem data state > > being preserved. Such applications are arguably broken because they > > will behave badly in the event of accidental shutdown anyway, and we > > should fix them. Well-designed applications are explicitly 'serially > > reuseable', in other words, you can interrupt at any point and start > > again from the beginning with valid and expected results. > > I strongly disagree. All valid ways to shut down the system involve > sending SIGTERM to running applications -- only broken ones would > live long enough after that to be killed by subsequent SIGKILL. > > A lot of applications always rely on their file i/o being done in some > manner that has atomic (from the application's point of view) operations > other than system calls -- heck, even make(1) does that. Nobody is forcing you to hit the power switch in the middle of a build. But now that you mention it, you've provided a good example of a broken application. Make with its reliance on timestamps for determining build status is both painfully slow and unreliable. What happens if you adjust your system clock? That said, Tux2 can preserve the per-write atomicity quite easily, or better, make could take advantage of the new journal-oriented transaction api that's being cooked up and specify its requirement for atomicity in a precise way. Do you have any other examples of programs that would be hurt by sudden termination? Certainly we'd consider a desktop gui broken if it failed to come up again just because you bailed out with the power switch instead of logging out nicely. -- Daniel - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
On Wed, 3 Jan 2001, Daniel Phillips wrote: > Tux2 is explicitly designed to legitimize pulling the plug as a valid > way of shutting down. Hmm - that IMHO is a good thing; I'll have to look at Tux2. > Metadata-only journalling filesystems are not > designed to be used this way, and even with full-data journalling you > should bear in mind that your on-disk filesystem image remains in an > invalid state until the journal recovery program has run successfully. > You would not want to upgrade your OS with your filesystem in this > state, nor would you want to remove a disk drive that didn't have the > journal file on it. Sure. > Being able to shut down by hitting the power switch is a little luxury > for which I've been willing to invest more than a year of my life to > attain. Clueless newbies don't know why it should be any other way, and > it's essential for embedded devices. Clueless newbies (and slightly less clueless less newbie) type people don't think that they should HAVE to. My two interests are coping with particularly pedantic people who don't want there computer to hastle them about what they should or shouldn't do, and slightly embedded systems (e.g. set top box/web browsery thing that you want to be able to turn off like a TV but it should still be able to have a writeable disc for config and stuff you download/cache etc). > I don't doubt that if the 'power switch' method of shutdown becomes > popular we will discover some applications that have windows where they > can be hurt by sudden shutdown, even will full filesystem data state > being preserved. Such applications are arguably broken because they > will behave badly in the event of accidental shutdown anyway, and we > should fix them. Well-designed applications are explicitly 'serially > reuseable', in other words, you can interrupt at any point and start > again from the beginning with valid and expected results. My own opinion here is that I'm only talking about not hurting the system if you repeatedly pull the plug. Applications in general should cope with death gently (in particular never destroying a saved file if they are killed at an annoying time). > > -- > Daniel > - > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to [EMAIL PROTECTED] > Please read the FAQ at http://www.tux.org/lkml/ > > -- /--\ | Dr. David Alan Gilbert | Work:[EMAIL PROTECTED] +44-161-286-2000 Ex258| | G7FHJ | | | Home: [EMAIL PROTECTED]http://www.treblig.org | \--/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
On Wed, 3 Jan 2001, Daniel Phillips wrote: > I don't doubt that if the 'power switch' method of shutdown becomes > popular we will discover some applications that have windows where they > can be hurt by sudden shutdown, even will full filesystem data state > being preserved. Such applications are arguably broken because they > will behave badly in the event of accidental shutdown anyway, and we > should fix them. Well-designed applications are explicitly 'serially > reuseable', in other words, you can interrupt at any point and start > again from the beginning with valid and expected results. I strongly disagree. All valid ways to shut down the system involve sending SIGTERM to running applications -- only broken ones would live long enough after that to be killed by subsequent SIGKILL. A lot of applications always rely on their file i/o being done in some manner that has atomic (from the application's point of view) operations other than system calls -- heck, even make(1) does that. -- Alex -- Excellent.. now give users the option to cut your hair you hippie! -- Anonymous Coward - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
Rik van Riel wrote: > > On Wed, 3 Jan 2001, Dr. David Gilbert wrote: > > > I got wondering as to whether the various journaling file > > system activities were designed to survive the occasional > > unclean shutdown or were designed to allow the user to just pull > > the plug as a regular means of shutting down. > > 1. a journaling filesystem is designed to be "consistent" >(or rather, easily recoverable) all of the time > 2. there's no difference between the "2 situations" you >describe above Welll... crashes tend to produce different effects from sudden power interruptions. In the first case parts of the system keep running, and bizarre results are possible. An even bigger difference is the matter of intent. Tux2 is explicitly designed to legitimize pulling the plug as a valid way of shutting down. Metadata-only journalling filesystems are not designed to be used this way, and even with full-data journalling you should bear in mind that your on-disk filesystem image remains in an invalid state until the journal recovery program has run successfully. You would not want to upgrade your OS with your filesystem in this state, nor would you want to remove a disk drive that didn't have the journal file on it. Being able to shut down by hitting the power switch is a little luxury for which I've been willing to invest more than a year of my life to attain. Clueless newbies don't know why it should be any other way, and it's essential for embedded devices. I don't doubt that if the 'power switch' method of shutdown becomes popular we will discover some applications that have windows where they can be hurt by sudden shutdown, even will full filesystem data state being preserved. Such applications are arguably broken because they will behave badly in the event of accidental shutdown anyway, and we should fix them. Well-designed applications are explicitly 'serially reuseable', in other words, you can interrupt at any point and start again from the beginning with valid and expected results. -- Daniel - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
On Wed, Jan 03, 2001 at 01:26:18PM -0200, Rik van Riel wrote: > On Wed, 3 Jan 2001, Dr. David Gilbert wrote: > > > I got wondering as to whether the various journaling file > > system activities were designed to survive the occasional > > unclean shutdown or were designed to allow the user to just pull > > the plug as a regular means of shutting down. > > > Thoughts? > > 1. a journaling filesystem is designed to be "consistent" >(or rather, easily recoverable) all of the time The file system metadata is, but it has no idea if the application's data is consistent or not. Also when you listen to tytso's tales of certain cheap IDE disks writing crap when they and the system are powered off during a write you should probably better not turn off the system running when not absolutely needed. If the system was designed for situation (2) you would probably only use applications that do their own log (or some other disk consistency technique). That does not seem to be the case currently. Also when unclean shutdown was very common you also be probably better of with a complete log structured fs that does not need to replay on mount. -Andi - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
> On Wed, 3 Jan 2001, Dr. David Gilbert wrote: > > > I got wondering as to whether the various journaling file > > system activities were designed to survive the occasional > > unclean shutdown or were designed to allow the user to just pull > > the plug as a regular means of shutting down. > > > Thoughts? Journaling filesystems only guarantee consistancy of filesystem metadata. Data that has not been flushed from buffers will be lost, and applications not given a chance to shut themselves down may do bad things if you just unplug the box. Journaling mostly means not having to run FSCK. -M - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
Re: Journaling: Surviving or allowing unclean shutdown?
On Wed, 3 Jan 2001, Dr. David Gilbert wrote: > I got wondering as to whether the various journaling file > system activities were designed to survive the occasional > unclean shutdown or were designed to allow the user to just pull > the plug as a regular means of shutting down. > Thoughts? 1. a journaling filesystem is designed to be "consistent" (or rather, easily recoverable) all of the time 2. there's no difference between the "2 situations" you describe above regards, Rik -- Hollywood goes for world dumbination, Trailer at 11. http://www.surriel.com/ http://www.conectiva.com/ http://distro.conectiva.com.br/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] Please read the FAQ at http://www.tux.org/lkml/
