Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-05-18 Thread Stephen C. Tweedie
Hi, On Fri, May 11, 2001 at 04:54:44PM +0200, Daniel Phillips wrote: > The only reasonable way I can think of getting a block-coherent view > underneath a mounted fs is to have a reverse map, and update it each > time we map block into the page cache or unmap it. It's called the "buffer cache

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-05-11 Thread Alexander Viro
On Mon, 7 May 2001, Pavel Machek wrote: > OTOH with current way if you make mistake in kernel, fsck will not > automatically inherit it; therefore fsck is likely to work even if > kernel ext2 is b0rken [and that's fairly important] ... and by the same logics you should make fsck implement its o

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-05-11 Thread Daniel Phillips
On Monday 07 May 2001 20:42, Pavel Machek wrote: > > > It's not exactly "kernel-based fsck". What I've been talking > > > about is secondary filesystem providing coherent access to > > > primary fs metadata. I.e. mount -t ext2meta -o master=/usr none > > > /mnt and then access through /mnt/su

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-05-10 Thread Pavel Machek
gHi! > > It's not exactly "kernel-based fsck". What I've been talking about > > is secondary filesystem providing coherent access to primary fs > > metadata. I.e. mount -t ext2meta -o master=/usr none /mnt and > > then access through /mnt/super, /mnt/block_bitmap, etc. > > > > C

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-05-09 Thread Helge Hafting
[EMAIL PROTECTED] wrote: > > I have tried this approach too a couple of years ago. I came to the idea > that I want some kind of "event reporting" mechanism to know when > application faults and when other events (like I/O) occurs. Booting is > just the tip of the iceberg. MOST big apps are seeki

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-05-08 Thread volodya
I have tried this approach too a couple of years ago. I came to the idea that I want some kind of "event reporting" mechanism to know when application faults and when other events (like I/O) occurs. Booting is just the tip of the iceberg. MOST big apps are seeking on startup because a) their c

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-05-06 Thread Andreas Dilger
Alan writes: > > Actually, the EVMS project does exactly this. All I/O is done on a full > > disk basis, and essentially does block remapping for each partition. This > > also solves the problem of cache inconsistency if accessing the parent > > device vs. accessing the partition. > > Interesti

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-05-06 Thread Alan Cox
> Actually, the EVMS project does exactly this. All I/O is done on a full > disk basis, and essentially does block remapping for each partition. This > also solves the problem of cache inconsistency if accessing the parent > device vs. accessing the partition. Interesting. Can EVMS handle the p

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-05-06 Thread Andreas Dilger
Alan writes: > > an interesting task when your root lives on /dev/sda1. Ditto for destroying > > a single partition (not mounted/used by swap/etc.) while you have some > > other partition in use. IWBNI we had a decent API for handling partition > > tables... > > Partitions are just very crude log

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-05-06 Thread Alan Cox
> an interesting task when your root lives on /dev/sda1. Ditto for destroying > a single partition (not mounted/used by swap/etc.) while you have some > other partition in use. IWBNI we had a decent API for handling partition > tables... Partitions are just very crude logical volumes, and ultimia

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-05-05 Thread Alexander Viro
On Sun, 6 May 2001, Chris Wedgwood wrote: > It's not exactly "kernel-based fsck". What I've been talking about > is secondary filesystem providing coherent access to primary fs > metadata. I.e. mount -t ext2meta -o master=/usr none /mnt and > then access through /mnt/super, /mn

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-05-05 Thread Andrea Arcangeli
On Sun, May 06, 2001 at 03:00:58PM +1200, Chris Wedgwood wrote: > On Sun, May 06, 2001 at 04:50:01AM +0200, Andrea Arcangeli wrote: > > Moving e2fsck into the kernel is a completly different matter > than caching the blockdevice accesses with pagecache instead of > buffercache. > > N

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-05-05 Thread Andreas Dilger
Chris Wedgewood writes: > As I said, I'm not takling about kernel based fsck, although for > _VERY_ large filesystems even with journalling I suspect it will be > required one day (so it can run in the background and do consistency > checking when the machine is idle). Actually, I was talking wit

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-05-05 Thread Alexander Viro
On Sun, 6 May 2001, Chris Wedgwood wrote: > On Sun, May 06, 2001 at 04:50:01AM +0200, Andrea Arcangeli wrote: > About a kernel based fsck Alexander told me he likes it, I > personally don't care about it that much because I believe... > > As I said, I'm not takling about kernel based

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-05-05 Thread Andrea Arcangeli
On Sun, May 06, 2001 at 02:14:37PM +1200, Chris Wedgwood wrote: > You don't need block device for fsck, in fact some OS require you use > character devices (e.g. Solaris). Moving e2fsck into the kernel is a completly different matter than caching the blockdevice accesses with pagecache instead of

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-05-05 Thread Andrea Arcangeli
On Sat, May 05, 2001 at 03:18:08PM +1200, Chris Wedgwood wrote: > On Fri, May 04, 2001 at 05:29:40PM +0200, Andrea Arcangeli wrote: > > once block_dev is in pagecache there will obviously be no-way to > share cache between the block device and the filesystem, because > all the caches

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-05-05 Thread Richard Gooch
Rogier Wolff writes: > Richard Gooch wrote: > > > > - next boot, init(8) checks the file, and if it exists, opens the root > > FS block device and reads in each block listed in the file. The > > effect is to warm the buffer cache extremely quickly. The head will > > move in one direction, g

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-05-05 Thread Alexander Viro
On Sat, 5 May 2001, Albert D. Cahalan wrote: > case P_SWAP: > sprintf(tmp, "%4.4s ", > scale_k(((task->size - task->resident) << CL_pg_shift), 4, 1)); > break; Albert, you can't be serious. The system had demand-loading for almost ten years. ->size - ->resident can be huge with

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-05-05 Thread Rogier Wolff
Richard Gooch wrote: > > - next boot, init(8) checks the file, and if it exists, opens the root > FS block device and reads in each block listed in the file. The > effect is to warm the buffer cache extremely quickly. The head will > move in one direction, grabbing data as it flys by. I exp

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-05-05 Thread Albert D. Cahalan
Alexander Viro writes: >> On Fri, 4 May 2001, Alexander Viro wrote: >>> Ehh... There _is_ a way to deal with that, but it's deeply Albertesque: > ^^^ Ah, you learn from the master. > ObProcfs: I don't think that walking the pa

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-05-04 Thread Linus Torvalds
On Fri, 4 May 2001, Alan Cox wrote: > > iso9660 alas doesn't allow you to do that. You can speed it up by reading > the entire file into memory rather than paging it in (or reading it in and > then executing it). iso9660 layout is pretty constrained and designed for > linear file reads Note tha

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-05-04 Thread Alan Cox
> Now, if you want to speed up accesses, there are things you can do. You > can lay out the filesystem in the access order - trace the IO accesses at > bootup ("which file, which offset, which metadata block?") and lay out the > blocks of the files in exactly the right order. Then you will get lin

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-05-04 Thread Richard Gooch
Alexander Viro writes: > > > On Fri, 4 May 2001, Richard Gooch wrote: > > > I don't bother splitting /usr off /. I gave up doing that when disc > > became cheap. There's no point anymore. And since I have a lightweight > > Yes, there is. Locality. Resistance to fs fuckups. Resistance to > disk

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-05-04 Thread Alexander Viro
On Fri, 4 May 2001, Richard Gooch wrote: > I don't bother splitting /usr off /. I gave up doing that when disc > became cheap. There's no point anymore. And since I have a lightweight Yes, there is. Locality. Resistance to fs fuckups. Resistance to disk fuckups. Easier to restore from tape. Di

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-05-04 Thread Richard Gooch
Alexander Viro writes: > > > On Fri, 4 May 2001, Richard Gooch wrote: > > > > Two of them: use less bloated shell (and link it statically) and > > > clean your rc scripts. > > > > No, because I'm not using the latest bloated version of bash, and I'm > > Umm... Last version of bash I could cal

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-05-04 Thread Alexander Viro
On Fri, 4 May 2001, Richard Gooch wrote: > > Two of them: use less bloated shell (and link it statically) and > > clean your rc scripts. > > No, because I'm not using the latest bloated version of bash, and I'm Umm... Last version of bash I could call not bloated was _long_ time ago. Somethin

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-05-04 Thread Linus Torvalds
On Fri, 4 May 2001, Alexander Viro wrote: > > ObProcfs: I don't think that walking the page tables is a good way to > compute RSS, especially since VM maintains the thing. Well, the VM didn't always use to maintain the stuff it does now, so I bet that most of the code is just old code that stil

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-05-04 Thread Jens Axboe
On Fri, May 04 2001, Richard Gooch wrote: > The idea I had (motivated by the desire to eliminate random disc > seeks, which is the limiting factor in how fast my boxes boot) was: > > - init(8) issues an ioctl(2) on the root FS block device which turns > on recording of block reads (it records b

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-05-04 Thread Richard Gooch
Alexander Viro writes: > > > On Fri, 4 May 2001, Richard Gooch wrote: > > > However, doing an ioctl(2) on the block device won't help. So the > > question is, where to add the hook? One possibility is the FS, and > > record inum,bnum pairs. But of course we don't have a way of accessing > > via

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-05-04 Thread Alexander Viro
On Fri, 4 May 2001, Richard Gooch wrote: > However, doing an ioctl(2) on the block device won't help. So the > question is, where to add the hook? One possibility is the FS, and > record inum,bnum pairs. But of course we don't have a way of accessing > via inum in user-space. So that's no good.

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-05-04 Thread Alexander Viro
On Fri, 4 May 2001, Linus Torvalds wrote: > > On Fri, 4 May 2001, Alexander Viro wrote: > > > > Ehh... There _is_ a way to deal with that, but it's deeply Albertesque: ^^^ > > * add pagecache access for block device >

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-05-04 Thread Richard Gooch
Linus Torvalds writes: > Now, if you want to speed up accesses, there are things you can > do. You can lay out the filesystem in the access order - trace the > IO accesses at bootup ("which file, which offset, which metadata > block?") and lay out the blocks of the files in exactly the right > ord

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-05-04 Thread Linus Torvalds
On Fri, 4 May 2001, Alexander Viro wrote: > > Ehh... There _is_ a way to deal with that, but it's deeply Albertesque: > * add pagecache access for block device > * put your "real" root on /dev/loop0 (setup from initrd) > * dd You're one sick puppy. Now, the above is basically

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-05-04 Thread Alexander Viro
On Fri, 4 May 2001, Linus Torvalds wrote: > Now, if you want to speed up accesses, there are things you can do. You > can lay out the filesystem in the access order - trace the IO accesses at > bootup ("which file, which offset, which metadata block?") and lay out the > blocks of the files in e

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-05-04 Thread Linus Torvalds
On Fri, 4 May 2001, Andrea Arcangeli wrote: > On Fri, May 04, 2001 at 01:56:14PM +0200, Jens Axboe wrote: > > Or you can rewrite block_read/write to use the page cache, in which case > > you'd have more luck doing the above. > > once block_dev is in pagecache there will obviously be no-way to s

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-05-04 Thread Linus Torvalds
On Fri, 4 May 2001, Rogier Wolff wrote: > > Linus Torvalds wrote: > > > > Ehh. Doing that would be extremely stupid, and would slow down your boot > > and nothing more. > > Ehhh, Linus, Linearly reading my harddisk goes at 26Mb per second. You obviously didn't read my explanation of _why_ it i

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-05-04 Thread Andrea Arcangeli
On Fri, May 04, 2001 at 01:56:14PM +0200, Jens Axboe wrote: > Or you can rewrite block_read/write to use the page cache, in which case > you'd have more luck doing the above. once block_dev is in pagecache there will obviously be no-way to share cache between the block device and the filesystem,

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-05-04 Thread Marc SCHAEFER
Rogier Wolff <[EMAIL PROTECTED]> wrote: > during boot. I can then reshuffle my disk to have that 50M of data at > the beginning and reading all that into 50M of cache, I can save Wasn't that one of the goals of the LVM project, along snapshots and block-level HSM ? - To unsubscribe from this lis

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-05-04 Thread Jens Axboe
On Fri, May 04 2001, Rogier Wolff wrote: > > On Thu, 3 May 2001, Alan Cox wrote: > > > Ditto for some CD based stuff. You burn the important binaries to the front > > > of the CD, then at boot dd 64Mb to /dev/null to prime the libraries and > > > avoid a lot of seeking during boot up from the CD-R

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-05-04 Thread Rogier Wolff
Linus Torvalds wrote: > > On Thu, 3 May 2001, Alan Cox wrote: > > Ditto for some CD based stuff. You burn the important binaries to the front > > of the CD, then at boot dd 64Mb to /dev/null to prime the libraries and > > avoid a lot of seeking during boot up from the CD-ROM. > > > > However I c

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-05-03 Thread Linus Torvalds
On Thu, 3 May 2001, Alan Cox wrote: > > > > discussion in itself), and there really are no valid uses for opening a > > > block device that is already mounted. More importantly, I don't think > > > anybody actually does. > > > > Actually I did. I might do it again :) The point was to get the ker

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-05-03 Thread Alan Cox
> > discussion in itself), and there really are no valid uses for opening a > > block device that is already mounted. More importantly, I don't think > > anybody actually does. > > Actually I did. I might do it again :) The point was to get the kernel to > cache certain blocks in the RAM. Ditto

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-05-02 Thread volodya
On Thu, 26 Apr 2001, Linus Torvalds wrote: > > > On Thu, 26 Apr 2001, Alexander Viro wrote: > > On Thu, 26 Apr 2001, Andrea Arcangeli wrote: > > > > > > how can the read in progress see a branch that we didn't spliced yet? We > > > > fd = open("/dev/hda1", O_RDONLY); > > read(fd, buf, sizeof(

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-04-30 Thread Neil Conway
Hiya. Linus Torvalds wrote: > So anybody who depends on "dump" getting backups right is already playing > russian rulette with their backups. It's not at all guaranteed to get the > right results - you may end up having stale data in the buffer cache that > ends up being "backed up". > > Dump w

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-04-28 Thread Olaf Titz
> or such. tar/cpio and friends don't deal properly with > a. holes inside of files. > b. hardlinks between files. GNU tar handles both of these. (Not particularly efficiently in the case of sparse files, but that's a minor issue in this case.) See -S flag. Perhaps more importantly, for a _robus

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-04-28 Thread Jens Axboe
On Sat, Apr 28 2001, Albert D. Cahalan wrote: > Linus Torvalds writes: > > > The buffer cache is "virtual" in the sense that /dev/hda is a > > completely separate name-space from /dev/hda1, even if there > > is some physical overlap. > > So the aliasing problems and elevator algorithm confusion

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-04-28 Thread Matthias Urlichs
Martin Dalecki : > tar/cpio and friends don't deal properly with > > a. holes inside of files. > b. hardlinks between files. > ??? GNU tar does both. The only thing it currently cannot handle is Not Changing Anything: either atime or ctime _will_ be updated. - To unsubscribe from this list: send

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-04-27 Thread Albert D. Cahalan
Linus Torvalds writes: > The buffer cache is "virtual" in the sense that /dev/hda is a > completely separate name-space from /dev/hda1, even if there > is some physical overlap. So the aliasing problems and elevator algorithm confusion remain? Is this ever likely to change, and what is with the

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-04-27 Thread Shane Wegner
On Fri, Apr 27, 2001 at 09:52:19AM -0700, Linus Torvalds wrote: > > On Fri, 27 Apr 2001, Vojtech Pavlik wrote: > > > > Actually this is done quite often, even on mounted fs's: > > > > hdparm -t /dev/hda > > Note that this one happens to be ok. > > The buffer cache is "virtual" in the sense th

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-04-27 Thread dek_ml
On Fri, Apr 27, 2001 at 11:02:17AM -0700, LA Walsh wrote: > Andrzej Krzysztofowicz wrote: > > > I know a few people that often do: > > > > dd if=/dev/hda1 of=/dev/hdc1 > > e2fsck /dev/hdc1 > > > > to make an "exact" copy of a currently working system. > > --- > Presumably this isn't a proble

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-04-27 Thread LA Walsh
Andrzej Krzysztofowicz wrote: > I know a few people that often do: > > dd if=/dev/hda1 of=/dev/hdc1 > e2fsck /dev/hdc1 > > to make an "exact" copy of a currently working system. --- Presumably this isn't a problem is the source disks are either unmounted or mounted 'read-only' ? -- The ab

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-04-27 Thread Jeff Garzik
Linus Torvalds wrote: > On Fri, 27 Apr 2001, Neil Conway wrote: > > > > I'm surprised that dump is deprecated (by you at least ;-)). What to > > use instead for backups on machines that can't umount disks regularly? > > Note that dump simply won't work reliably at all even in 2.4.x: the buffer >

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-04-27 Thread Martin Dalecki
Linus Torvalds wrote: > Dump was a stupid program in the first place. Leave it behind. Not quite Linus - dump/restore are nice tools to create for example automatic over network installation servers, i.e. efficient system images or such. tar/cpio and friends don't deal properly with a. holes in

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-04-27 Thread Linus Torvalds
[ linux-kernel added back as a cc ] On Fri, 27 Apr 2001, Neil Conway wrote: > > I'm surprised that dump is deprecated (by you at least ;-)). What to > use instead for backups on machines that can't umount disks regularly? Note that dump simply won't work reliably at all even in 2.4.x: the bu

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-04-27 Thread Linus Torvalds
On Fri, 27 Apr 2001, Vojtech Pavlik wrote: > > Actually this is done quite often, even on mounted fs's: > > hdparm -t /dev/hda Note that this one happens to be ok. The buffer cache is "virtual" in the sense that /dev/hda is a completely separate name-space from /dev/hda1, even if there is som

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-04-27 Thread Ville Herva
On Fri, Apr 27, 2001 at 09:23:57AM -0400, you [Alexander Viro] claimed: > > > On Fri, 27 Apr 2001, Vojtech Pavlik wrote: > > > Actually this is done quite often, even on mounted fs's: > > > > hdparm -t /dev/hda > > You would need either hdparm -t /dev/hda or mounting the > whole /dev/hda. >

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-04-27 Thread Andi Kleen
On Thu, Apr 26, 2001 at 01:08:25PM -0700, Linus Torvalds wrote: > Note that I think all these arguments are fairly bogus. Doing things like > "dump" on a live filesystem is stupid and dangerous (in my opinion it is > stupid and dangerous to use "dump" at _all_, but that's a whole 'nother > discus

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-04-27 Thread Alexander Viro
On Fri, 27 Apr 2001, Vojtech Pavlik wrote: > Actually this is done quite often, even on mounted fs's: > > hdparm -t /dev/hda You would need either hdparm -t /dev/hda or mounting the whole /dev/hda. Buffer cache for the disk is unrelated to buffer cache for parititions. - To unsubscribe from

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-04-27 Thread Vojtech Pavlik
On Thu, Apr 26, 2001 at 01:08:25PM -0700, Linus Torvalds wrote: > On Thu, 26 Apr 2001, Alexander Viro wrote: > > On Thu, 26 Apr 2001, Andrea Arcangeli wrote: > > > > > > how can the read in progress see a branch that we didn't spliced yet? We > > > > fd = open("/dev/hda1", O_RDONLY); > > read(fd,

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-04-27 Thread Richard B. Johnson
On Thu, 26 Apr 2001, Alexander Viro wrote: > > > On Thu, 26 Apr 2001, Richard B. Johnson wrote: > > > The disk image, raw.bin, does NOT contain the image of the floppy. > > Most of boot stuff added by lilo is missing. It will eventually > > get there, but it's not there now, even though the fl

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-04-26 Thread Andrea Arcangeli
On Thu, Apr 26, 2001 at 05:19:26PM -0700, Linus Torvalds wrote: > Detail nit: don't do this. Use "current_text_addr()" instead. Simpler to > read, and gcc will actually do the right thing wrt inlining etc. Agreed, thanks for the info. Andrea - To unsubscribe from this list: send the line "unsubs

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-04-26 Thread Linus Torvalds
On Fri, 27 Apr 2001, Andrea Arcangeli wrote: > + __label__ here; > + here: > + printk(KERN_ERR "IO error or racy use of wait_on_buffer() from %p\n", >&&here); Detail nit: don't do this. Use "current_text_addr()" instead. Simpler to read, and gcc will actually do the

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-04-26 Thread Andrea Arcangeli
On Thu, Apr 26, 2001 at 07:25:23PM -0400, Alexander Viro wrote: > > > On Thu, 26 Apr 2001, Andrea Arcangeli wrote: > > > > How about adding > > > if (!buffer_uptodate(bh)) { > > > printk(KERN_ERR "IO error or racy use of wait_on_buffer()"); > > > show_task(current); > > >

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-04-26 Thread Alexander Viro
On Thu, 26 Apr 2001, Andrea Arcangeli wrote: > > How about adding > > if (!buffer_uptodate(bh)) { > > printk(KERN_ERR "IO error or racy use of wait_on_buffer()"); > > show_task(current); > > } > > in the end of wait_on_buffer() for a while? > > At the _top_ of w

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-04-26 Thread Andrzej Krzysztofowicz
> > Note that I think all these arguments are fairly bogus. Doing things like > "dump" on a live filesystem is stupid and dangerous (in my opinion it is > stupid and dangerous to use "dump" at _all_, but that's a whole 'nother > discussion in itself), and there really are no valid uses for openi

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-04-26 Thread Andrea Arcangeli
On Thu, Apr 26, 2001 at 01:26:15PM -0700, Linus Torvalds wrote: > > > On Thu, 26 Apr 2001, Andrea Arcangeli wrote: > > > > What I'm saying above is that even without the wait_on_buffer ext2 can > > screwup itself because the splice happens after the buffer are just all > > uptodate so any "reade

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-04-26 Thread Andrea Arcangeli
On Thu, Apr 26, 2001 at 04:49:20PM -0400, Alexander Viro wrote: > getblk(); if (!buffer_uptodate) wait_on_buffer(); > is not in that class. It _is_ OK on UP as long as we don't block, but on > SMP it doesn't guarantee anything - buffer_head can be in any state > when we are done. IMO all suc

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-04-26 Thread Andrea Arcangeli
On Thu, Apr 26, 2001 at 10:11:09PM +0200, Andrea Arcangeli wrote: > On Thu, Apr 26, 2001 at 03:55:19PM -0400, Alexander Viro wrote: > > > > > > On Thu, 26 Apr 2001, Andrea Arcangeli wrote: > > > > > On Thu, Apr 26, 2001 at 03:34:00PM -0400, Alexander Viro wrote: > > > > Same scenario, but with

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-04-26 Thread Andrea Arcangeli
On Thu, Apr 26, 2001 at 01:08:25PM -0700, Linus Torvalds wrote: > But the fact is that nobody should ever do the thing that could cause > problems. dump in 2.4 also gets uncoherent view of the data which make things even worse than in 2.2 (to change that we should hash in the buffer hashtable all

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-04-26 Thread Andrea Arcangeli
On Thu, Apr 26, 2001 at 03:55:19PM -0400, Alexander Viro wrote: > > > On Thu, 26 Apr 2001, Andrea Arcangeli wrote: > > > On Thu, Apr 26, 2001 at 03:34:00PM -0400, Alexander Viro wrote: > > > Same scenario, but with read-in-progress started before we do getblk(). BTW, > > > > how can the read i

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-04-26 Thread Alexander Viro
On Thu, 26 Apr 2001, Richard B. Johnson wrote: > The disk image, raw.bin, does NOT contain the image of the floppy. > Most of boot stuff added by lilo is missing. It will eventually > get there, but it's not there now, even though the floppy was > un-mounted! I doubt that you can reproduce tha

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-04-26 Thread Alexander Viro
On Thu, 26 Apr 2001, Linus Torvalds wrote: > Note that I think all these arguments are fairly bogus. Doing things like > "dump" on a live filesystem is stupid and dangerous (in my opinion it is > stupid and dangerous to use "dump" at _all_, but that's a whole 'nother > discussion in itself), a

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-04-26 Thread Richard B. Johnson
On Thu, 26 Apr 2001, Alexander Viro wrote: > > > On Thu, 26 Apr 2001, Andrea Arcangeli wrote: > > > > the wait-on-buffer is not strictly necessary: it's probably there to make > > > > maybe not but I need to check some more bit to be sure. > > Same scenario, but with read-in-progress started

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-04-26 Thread Linus Torvalds
On Thu, 26 Apr 2001, Andrea Arcangeli wrote: > > What I'm saying above is that even without the wait_on_buffer ext2 can > screwup itself because the splice happens after the buffer are just all > uptodate so any "reader" (I mean any reader through ext2 not through > block_dev) will never try to

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-04-26 Thread Andrea Arcangeli
On Thu, Apr 26, 2001 at 03:17:54PM -0400, Alexander Viro wrote: > > > On Thu, 26 Apr 2001, I wrote: > > > On Thu, 26 Apr 2001, Linus Torvalds wrote: > > > > > I see the race, but I don't see how you can actually trigger it. > > > > > > Exactly _who_ does the "read from device" part? Somebody

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-04-26 Thread Linus Torvalds
On Thu, 26 Apr 2001, Alexander Viro wrote: > On Thu, 26 Apr 2001, Andrea Arcangeli wrote: > > > > how can the read in progress see a branch that we didn't spliced yet? We > > fd = open("/dev/hda1", O_RDONLY); > read(fd, buf, sizeof(buf)); Note that I think all these arguments are fairly bogus.

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-04-26 Thread Alexander Viro
On Thu, 26 Apr 2001, Andrea Arcangeli wrote: > On Thu, Apr 26, 2001 at 03:34:00PM -0400, Alexander Viro wrote: > > Same scenario, but with read-in-progress started before we do getblk(). BTW, > > how can the read in progress see a branch that we didn't spliced yet? We fd = open("/dev/hda1", O

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-04-26 Thread Andrea Arcangeli
On Thu, Apr 26, 2001 at 03:34:00PM -0400, Alexander Viro wrote: > Same scenario, but with read-in-progress started before we do getblk(). BTW, how can the read in progress see a branch that we didn't spliced yet? We clear and mark uptodate the new part of the branch before it's visible to any rea

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-04-26 Thread Andrea Arcangeli
On Thu, Apr 26, 2001 at 09:15:57PM +0200, Andrea Arcangeli wrote: > maybe not but I need to check some more bit to be sure. yes we probably don't need it for fs against fs in 2.4 because we make the new metadata block visible to a reader (splice) only after they're all uptodate and all directory

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-04-26 Thread Alexander Viro
On Thu, 26 Apr 2001, Andrea Arcangeli wrote: > > the wait-on-buffer is not strictly necessary: it's probably there to make > > maybe not but I need to check some more bit to be sure. Same scenario, but with read-in-progress started before we do getblk(). BTW, old writeback is harmless - we wi

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-04-26 Thread Andrea Arcangeli
On Thu, Apr 26, 2001 at 11:49:14AM -0700, Linus Torvalds wrote: > > On Thu, Apr 26, 2001 at 11:45:47AM -0400, Alexander Viro wrote: > > > > Ext2 does getblk+wait_on_buffer for new metadata blocks before > > filling them with zeroes. While that is enough for single-processor, > > on SMP we hav

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-04-26 Thread Alexander Viro
On Thu, 26 Apr 2001, I wrote: > On Thu, 26 Apr 2001, Linus Torvalds wrote: > > > I see the race, but I don't see how you can actually trigger it. > > > > Exactly _who_ does the "read from device" part? Somebody doing a > > "fsck" while the filesystem is mounted read-write and actively writte

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-04-26 Thread Alexander Viro
On Thu, 26 Apr 2001, Linus Torvalds wrote: > I see the race, but I don't see how you can actually trigger it. > > Exactly _who_ does the "read from device" part? Somebody doing a > "fsck" while the filesystem is mounted read-write and actively written > to? Yeah, you'd get disk corruption tha

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-04-26 Thread Chris Mason
On Thursday, April 26, 2001 02:24:26 PM -0400 Alexander Viro <[EMAIL PROTECTED]> wrote: > > > On Thu, 26 Apr 2001, Andrea Arcangeli wrote: > >> correct. I bet other fs are affected as well btw. > > If only... block_read() vs. block_write() has the same race. I'm going > through the list of

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-04-26 Thread Linus Torvalds
On Thu, Apr 26, 2001 at 11:45:47AM -0400, Alexander Viro wrote: > > Ext2 does getblk+wait_on_buffer for new metadata blocks before > filling them with zeroes. While that is enough for single-processor, > on SMP we have the following race: > > getblk gives us unlocked, non-uptodate bh > wai

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-04-26 Thread Alexander Viro
On Thu, 26 Apr 2001, Andrea Arcangeli wrote: > correct. I bet other fs are affected as well btw. If only... block_read() vs. block_write() has the same race. I'm going through the list of all wait_on_buffer() users right now. - To unsubscribe from this list: send the line "unsubscribe linux-k

Re: [PATCH] SMP race in ext2 - metadata corruption.

2001-04-26 Thread Andrea Arcangeli
On Thu, Apr 26, 2001 at 11:45:47AM -0400, Alexander Viro wrote: > Ext2 does getblk+wait_on_buffer for new metadata blocks before > filling them with zeroes. While that is enough for single-processor, > on SMP we have the following race: > > getblk gives us unlocked, non-uptodate bh > wait_o

[PATCH] SMP race in ext2 - metadata corruption.

2001-04-26 Thread Alexander Viro
Ext2 does getblk+wait_on_buffer for new metadata blocks before filling them with zeroes. While that is enough for single-processor, on SMP we have the following race: getblk gives us unlocked, non-uptodate bh wait_on_buffer() does nothing read from