Re: Break 2.4 VM in five easy steps
Miles Lane wrote: > > So please, if you have new facts that you want to offer that > will help us characterize and understand these VM issues better > or discover new problems, feel free to share them. But if you > just want to rant, I, for one, would rather you didn't. *sigh* Not to prolong an already pointless thread, but that really was the intent of my original message. I had figured out a specific way, with easy-to-follow steps, to make the VM misbehave under very certain conditions. I even offered to help figure out a solution in any way I could, considering I'm not familiar with kernel code. However, I guess this whole "too much swap" issue has a lot of people on edge and immediately assumed I was talking about this subject, without actually reading my original message. -- -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- #!/usr/bin/perl -w $_='while(read+STDIN,$_,2048){$a=29;$b=73;$c=142;$t=255;@t=map {$_%16or$t^=$c^=($m=(11,10,116,100,11,122,20,100)[$_/16%8])&110; $t^=(72,@z=(64,72,$a^=12*($_%16-2?0:$m&17)),$b^=$_%64?12:0,@z) [$_%8]}(16..271);if((@a=unx"C*",$_)[20]&48){$h=5;$_=unxb24,join "",@b=map{xB8,unxb8,chr($_^$a[--$h+84])}@ARGV;s/...$/1$&/;$d= unxV,xb25,$_;$e=256|(ord$b[4])<<9|ord$b[3];$d=$d>>8^($f=$t&($d >>12^$d>>4^$d^$d/8))<<17,$e=$e>>8^($t&($g=($q=$e>>14&7^$e)^$q* 8^$q<<6))<<9,$_=$t[$_]^(($h>>=8)+=$f+(~$g&$t))for@a[128..$#a]} print+x"C*",@a}';s/x/pack+/g;eval usage: qrpff 153 2 8 105 225 < /mnt/dvd/VOB_FILENAME \ | extract_mpeg2 | mpeg2dec - http://www.eff.org/http://www.opendvd.org/ http://www.cs.cmu.edu/~dst/DeCSS/Gallery/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Break 2.4 VM in five easy steps
Mike Galbraith wrote: > > Can you try the patch below to see if it helps? If you watch > with vmstat, you should see swap shrinking after your test. > Let is shrink a while and then see how long swapoff takes. > Under a normal load, it'll munch a handfull of them at least > once a second and keep them from getting annoying. (theory;) Hi Mike, I'll give that patch a spin this evening after work when I have time to patch and recompile the kernel. -- -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Break 2.4 VM in five easy steps
"Eric W. Biederman" wrote: > > Derek Glidden <[EMAIL PROTECTED]> writes: > > > The problem I reported is not that 2.4 uses huge amounts of swap but > > that trying to recover that swap off of disk under 2.4 can leave the > > machine in an entirely unresponsive state, while 2.2 handles identical > > situations gracefully. > > > > The interesting thing from other reports is that it appears to be kswapd > using up CPU resources. Not the swapout code at all. So it appears > to be a fundamental VM issue. And calling swapoff is just a good way > to trigger it. > > If you could confirm this by calling swapoff sometime other than at > reboot time. That might help. Say by running top on the console. That's exactly what my original test was doing. I think it was Jeffrey Baker complaining about "swapoff" at reboot. See my original post that started this thread and follow the "five easy steps." :) I'm sucking down a lot of swap, although not all that's available which is something I am specifically trying to avoid - I wanted to stress the VM/swap recovery procedure, not "out of RAM and swap" memory pressure - and then running 'swapoff' from an xterm or a console. The problem with being able to see what's eating up CPU resources is that the whole machine stops responding for me to tell. consoles stop updating, the X display freezes, keyboard input is locked out, etc. As far as anyone can tell, for several minutes, the whole machine is locked up. (except, strangely enough, the machine will still respond to ping) I've tried running 'top' to see what task is taking up all the CPU time, but the system hangs before it shows anything meaningful. I have been able to tell that it hits 100% "system" utilization very quickly though. I did notice that the first thing sys_swapoff() does is call lock_kernel() ... so if sys_swapoff() takes a long time, I imagine things will get very unresponsive quickly. (But I'm not intimately familiar with the various kernel locks, so I don't know what granularity/atomicity/whatever lock_kernel() enforces.) -- -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- #!/usr/bin/perl -w $_='while(read+STDIN,$_,2048){$a=29;$b=73;$c=142;$t=255;@t=map {$_%16or$t^=$c^=($m=(11,10,116,100,11,122,20,100)[$_/16%8])&110; $t^=(72,@z=(64,72,$a^=12*($_%16-2?0:$m&17)),$b^=$_%64?12:0,@z) [$_%8]}(16..271);if((@a=unx"C*",$_)[20]&48){$h=5;$_=unxb24,join "",@b=map{xB8,unxb8,chr($_^$a[--$h+84])}@ARGV;s/...$/1$&/;$d= unxV,xb25,$_;$e=256|(ord$b[4])<<9|ord$b[3];$d=$d>>8^($f=$t&($d >>12^$d>>4^$d^$d/8))<<17,$e=$e>>8^($t&($g=($q=$e>>14&7^$e)^$q* 8^$q<<6))<<9,$_=$t[$_]^(($h>>=8)+=$f+(~$g&$t))for@a[128..$#a]} print+x"C*",@a}';s/x/pack+/g;eval usage: qrpff 153 2 8 105 225 < /mnt/dvd/VOB_FILENAME \ | extract_mpeg2 | mpeg2dec - http://www.eff.org/http://www.opendvd.org/ http://www.cs.cmu.edu/~dst/DeCSS/Gallery/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Break 2.4 VM in five easy steps
"Eric W. Biederman" wrote: > > > Or are you saying that if someone is unhappy with a particular > > situation, they should just keep their mouth shut and accept it? > > It's worth complaining about. It is also worth digging into and find > out what the real problem is. I have a hunch that this hole > conversation on swap sizes being irritating is hiding the real > problem. I totally agree with this, and want to reiterate that the original problem I posted has /nothing/ to do with the "swap == 2*RAM" issue. The problem I reported is not that 2.4 uses huge amounts of swap but that trying to recover that swap off of disk under 2.4 can leave the machine in an entirely unresponsive state, while 2.2 handles identical situations gracefully. I'm annoyed by 2.4's "requirement" of too much swap, but I consider that less a bug and more a severe design flaw. -- -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- #!/usr/bin/perl -w $_='while(read+STDIN,$_,2048){$a=29;$b=73;$c=142;$t=255;@t=map {$_%16or$t^=$c^=($m=(11,10,116,100,11,122,20,100)[$_/16%8])&110; $t^=(72,@z=(64,72,$a^=12*($_%16-2?0:$m&17)),$b^=$_%64?12:0,@z) [$_%8]}(16..271);if((@a=unx"C*",$_)[20]&48){$h=5;$_=unxb24,join "",@b=map{xB8,unxb8,chr($_^$a[--$h+84])}@ARGV;s/...$/1$&/;$d= unxV,xb25,$_;$e=256|(ord$b[4])<<9|ord$b[3];$d=$d>>8^($f=$t&($d >>12^$d>>4^$d^$d/8))<<17,$e=$e>>8^($t&($g=($q=$e>>14&7^$e)^$q* 8^$q<<6))<<9,$_=$t[$_]^(($h>>=8)+=$f+(~$g&$t))for@a[128..$#a]} print+x"C*",@a}';s/x/pack+/g;eval usage: qrpff 153 2 8 105 225 < /mnt/dvd/VOB_FILENAME \ | extract_mpeg2 | mpeg2dec - http://www.eff.org/http://www.opendvd.org/ http://www.cs.cmu.edu/~dst/DeCSS/Gallery/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Break 2.4 VM in five easy steps
John Alvord wrote: > > On Wed, 06 Jun 2001 11:31:28 -0400, Derek Glidden > <[EMAIL PROTECTED]> wrote: > > > > >I'm beginning to be amazed at the Linux VM hackers' attitudes regarding > >this problem. I expect this sort of behaviour from academics - ignoring > >real actual problems being reported by real actual people really and > >actually experiencing and reporting them because "technically" or > >"theoretically" they "shouldn't be an issue" or because "the "literature > >[documentation] says otherwise - but not from this group. > > There have been multiple comments that a fix for the problem is > forthcoming. Is there some reason you have to keep talking about it? Because there have been many more comments that "The rule for 2.4 is 'swap == 2*RAM' and that's the way it is" and "disk space is cheap - just add more" than there have been "this is going to be fixed" which is extremely discouraging and doesn't instill me with all sorts of confidence that this problem is being taken seriously. Or are you saying that if someone is unhappy with a particular situation, they should just keep their mouth shut and accept it? -- -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- #!/usr/bin/perl -w $_='while(read+STDIN,$_,2048){$a=29;$b=73;$c=142;$t=255;@t=map {$_%16or$t^=$c^=($m=(11,10,116,100,11,122,20,100)[$_/16%8])&110; $t^=(72,@z=(64,72,$a^=12*($_%16-2?0:$m&17)),$b^=$_%64?12:0,@z) [$_%8]}(16..271);if((@a=unx"C*",$_)[20]&48){$h=5;$_=unxb24,join "",@b=map{xB8,unxb8,chr($_^$a[--$h+84])}@ARGV;s/...$/1$&/;$d= unxV,xb25,$_;$e=256|(ord$b[4])<<9|ord$b[3];$d=$d>>8^($f=$t&($d >>12^$d>>4^$d^$d/8))<<17,$e=$e>>8^($t&($g=($q=$e>>14&7^$e)^$q* 8^$q<<6))<<9,$_=$t[$_]^(($h>>=8)+=$f+(~$g&$t))for@a[128..$#a]} print+x"C*",@a}';s/x/pack+/g;eval usage: qrpff 153 2 8 105 225 < /mnt/dvd/VOB_FILENAME \ | extract_mpeg2 | mpeg2dec - http://www.eff.org/http://www.opendvd.org/ http://www.cs.cmu.edu/~dst/DeCSS/Gallery/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Break 2.4 VM in five easy steps
> Funny. I can count many ways in which 4.3BSD, SunOS{3,4} and post-4.4 BSD > systems I've used were broken, but I've never thought that swap==2*RAM rule > was one of them. Yes, but Linux isn't 4.3BSD, SunOS or post-4.4 BSD. Not to mention, all other OS's I've had experience using *don't* break severely if you don't follow the "swap==2*RAM" rule. Except Linux 2.4. > Not that being more kind on swap would be a bad thing, but that rule for > amount of swap is pretty common. ISTR similar for (very old) SCO, so it's > not just BSD world. How are modern Missed'em'V variants in that respect, BTW? Yes, but that has traditionally been one of the big BENEFITS of Linux, and other UNIXes. As Sean Hunter said, "Virtual memory is one of the killer features of unix." Linux has *never* in the past REQUIRED me to follow that rule. Which is a big reason I use it in so many places. Take an example mentioned by someone on the list already: a laptop. I have two laptops that run Linux. One has a 4GB disk, one has a 12GB disk. Both disks are VERY full of data and both machines get pretty heavy use. It's a fact that I just bumped one laptop (with 256MB of swap configured) from 128MB to 256MB of RAM. Does this mean that if I want to upgrade to the 2.4 kernel on that machine I now have to back up all that data, repartition the drive and restore everything just so I can fastidiously follow the "swap == 2*RAM" rule else the 2.4 VM subsystem will break? Bollocks, to quote yet another participant in this silly discussion. I'm beginning to be amazed at the Linux VM hackers' attitudes regarding this problem. I expect this sort of behaviour from academics - ignoring real actual problems being reported by real actual people really and actually experiencing and reporting them because "technically" or "theoretically" they "shouldn't be an issue" or because "the "literature [documentation] says otherwise - but not from this group. -- -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Break 2.4 VM in five easy steps
Helge Hafting wrote: > > The drive is inactive because it isn't needed, the machine is > running loops on data in memory. And it is unresponsive because > nothing else is scheduled, maybe "swapoff" is easier to implement I don't quite get what you're saying. If the system becomes unresponsive because the VM swap recovery parts of the kernel are interfering with the kernel scheduler then that's also bad because there absolutely *are* other processes that should be getting time, like the console windows/shells at which I'm logged in. If they aren't getting it specifically because the VM is preventing them from receiving execution time, then that's another bug. > when processes cannot try to allocate more or touch pages > while it runs. "swapoff" isn't something you normally do often, > so it don't have to be nice. I'm not familiar enough with the swapping bits of the kernel code, so I could be totally wrong, but turning off a swap file/partition should just call the same parts of the VM subsystem that would normally try to recover swap space under memory pressure. Using "swapoff" to force this behaviour should just force it to happen manually rather than when memory pressure is high enough. Which means that if that's the normal behaviour of the VM subsystem when memory pressure gets high and it needs to recover unused pages from swap - i.e. the machine stops running - then that's still very broken behaviour, no matter what instigated the occurance. > Still, I find it strange that swapoff should take much more time, > even if you can get 2.2 to have the same amount in swap. So do I. Hence the original report. -- -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- #!/usr/bin/perl -w $_='while(read+STDIN,$_,2048){$a=29;$b=73;$c=142;$t=255;@t=map {$_%16or$t^=$c^=($m=(11,10,116,100,11,122,20,100)[$_/16%8])&110; $t^=(72,@z=(64,72,$a^=12*($_%16-2?0:$m&17)),$b^=$_%64?12:0,@z) [$_%8]}(16..271);if((@a=unx"C*",$_)[20]&48){$h=5;$_=unxb24,join "",@b=map{xB8,unxb8,chr($_^$a[--$h+84])}@ARGV;s/...$/1$&/;$d= unxV,xb25,$_;$e=256|(ord$b[4])<<9|ord$b[3];$d=$d>>8^($f=$t&($d >>12^$d>>4^$d^$d/8))<<17,$e=$e>>8^($t&($g=($q=$e>>14&7^$e)^$q* 8^$q<<6))<<9,$_=$t[$_]^(($h>>=8)+=$f+(~$g&$t))for@a[128..$#a]} print+x"C*",@a}';s/x/pack+/g;eval usage: qrpff 153 2 8 105 225 < /mnt/dvd/VOB_FILENAME \ | extract_mpeg2 | mpeg2dec - http://www.eff.org/http://www.opendvd.org/ http://www.cs.cmu.edu/~dst/DeCSS/Gallery/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Break 2.4 VM in five easy steps
Xavier Bestel wrote: > > Did you try to put twice as much swap as you have RAM ? (e.g. add a 512M > swapfile to your box) > This is what Linus recommended for 2.4 (swap = 2 * RAM), saying that > anything less won't do any good: 2.4 overallocates swap even if it > doesn't use it all. So in your case you just have enough swap to map > your RAM, and nothing to really swap your apps. Yes, the example given is against the machine at work, which is configured 512/512. My machine at home is configured 512/1024 and has the same problems. Further, this machine *used* to have only 256MB of RAM, and I could still cause the misbehaviour. -- -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- #!/usr/bin/perl -w $_='while(read+STDIN,$_,2048){$a=29;$b=73;$c=142;$t=255;@t=map {$_%16or$t^=$c^=($m=(11,10,116,100,11,122,20,100)[$_/16%8])&110; $t^=(72,@z=(64,72,$a^=12*($_%16-2?0:$m&17)),$b^=$_%64?12:0,@z) [$_%8]}(16..271);if((@a=unx"C*",$_)[20]&48){$h=5;$_=unxb24,join "",@b=map{xB8,unxb8,chr($_^$a[--$h+84])}@ARGV;s/...$/1$&/;$d= unxV,xb25,$_;$e=256|(ord$b[4])<<9|ord$b[3];$d=$d>>8^($f=$t&($d >>12^$d>>4^$d^$d/8))<<17,$e=$e>>8^($t&($g=($q=$e>>14&7^$e)^$q* 8^$q<<6))<<9,$_=$t[$_]^(($h>>=8)+=$f+(~$g&$t))for@a[128..$#a]} print+x"C*",@a}';s/x/pack+/g;eval usage: qrpff 153 2 8 105 225 < /mnt/dvd/VOB_FILENAME \ | extract_mpeg2 | mpeg2dec - http://www.eff.org/http://www.opendvd.org/ http://www.cs.cmu.edu/~dst/DeCSS/Gallery/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Break 2.4 VM in five easy steps
Bill Pringlemeir wrote: > > [snip] > Derek> overwhelmed. On the system I'm using to write this, with > Derek> 512MB of RAM and 512MB of swap, I run two copies of this > > Please see the following message on the kernel mailing list, > > 3086:Linus 2.4.0 notes are quite clear that you need at least twice RAM of swap > Message-Id: <[EMAIL PROTECTED]> Yes, I'm aware of this. However, I still believe that my original problem report is a BUG. No matter how much swap I have, or don't have, and how much is or isn't being used, running "swapoff" and forcing the VM subsystem to reclaim unused swap should NOT cause my machine to feign death for several minutes. I can easily take 256MB out of this machine, and then I *will* have twice as much swap as RAM and I can still cause the exact same behaviour. It's a bug, and no number of times saying "You need twice as much swap as RAM" will change that fact. -- -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Break 2.4 VM in five easy steps
On Wed, Jun 06, 2001 at 12:16:30PM +1000, Andrew Morton wrote: > "Jeffrey W. Baker" wrote: > > > > Because the 2.4 VM is so broken, and > > because my machines are frequently deeply swapped, > > The swapoff algorithms in 2.2 and 2.4 are basically identical. > The problem *appears* worse in 2.4 because it uses lots > more swap. I disagree with the terminology you're using. It *is* worse in 2.4, period. If it only *appears* worse, then if I encounter a situation where a 2.2 box has utilized as much swap as a 2.4 box, I should see the same results. Yet this happens not to be the case. A 2.2 box that's a hundred or more megs into swap, even when that swap space is "actual programs" rather than just mysteriously allocated swap (as with 2.4), it only takes the time to page all that off disk back into RAM, making the machine a little sluggish while it's happening and definitely not making the machine entirely unresponsive. On the other hand, a 2.4 box that's a hundred or more megs into swap, even when there's nothing actually running to take up that swap space, i.e. it's just "mysteriously allocated for some reason" swap, will take several minutes, while the CPU is pegged, the drive is inactive, and the entire machine is completely unresponsive to anything - for all appearances locked up tight. I have been unable to make a box running the 2.2 kernel behave the same way as 2.4 does, even if I put the 2.2 kernel under severe memory pressure and the 2.4 kernel is entirely idle with no tasks but the most basic system processes running. So from my perspective, which really is a real-world perspective because I can cause this same behaviour during normal day-to-day desktop operation of my machine, the behaviour of 2.4 *is* different from the behaviour of 2.2 when it comes to memory management. > > they can sometimes take over 30 minutes to shutdown. > > Yes. The sys_swapoff() system call can take many minutes > of CPU time. It basically does: >[...] > It's interesting that you've found a case where this > actually has an operational impact. I can't tell if this is humour or not. I hope it is, because I can sure show you actual operational impact of this mis-behaviour all day long. :) -- -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Break 2.4 VM in five easy steps
After reading the messages to this list for the last couple of weeks and playing around on my machine, I'm convinced that the VM system in 2.4 is still severely broken. This isn't trying to test extreme low-memory pressure, just how the system handles recovering from going somewhat into swap, which is a real day-to-day problem for me, because I often run a couple of apps that most of the time live in RAM, but during heavy computation runs, can go a couple hundred megs into swap for a few minutes at a time. Whenever that happens, my machine always starts acting up afterwards, so I started investigating and found some really strange stuff going on. To demonstrate this to a co-worker, I cooked up this really simple, really stupid, very effective test. (Note that this all is probably specific to IA32, which is the platform on which I'm running.) -- How to Break your 2.4 kernel VM in 5 easy steps 1) compile the following code: #include void main(void) { /* allocate a buttload of memory and try to touch it all */ void *ptr = (void *)calloc(1, sizeof(int)) ; /* sleep for a bit to let the system quiesce */ sleep(20); /* let it all go away now */ free(ptr); } 2) depending on the amount of RAM/swap available in your machine, you might need to adjust the calloc to allocate a different amount. This allocates about 400MB. 3) Run the program, or more than one copy at once. You want to put your machine somewhat into swap, but not totally overwhelmed. On the system I'm using to write this, with 512MB of RAM and 512MB of swap, I run two copies of this program simultaneously and it puts me a couple hundred megs into swap. 4) Let the program exit, run "free" or cat /proc/memstat or something to make sure your machine has paged a bunch of stuff out into swap. 5) try to "swapoff" your swap partition and watch the machine become completely and entirely unresponsive for several minutes. -- If I do this on my machine, which is a K7-700 on an ASUS K7M motherboard with 512MB each of swap and RAM where I'm writing this (but I can make any machine running 2.4 behave the same way, and any version I've tried it with from 2.4.2 on up through most of the -ac kernels too), the machine will become _entirely_ unresponsive for several minutes. The HD comes on for a few seconds at the very start of the "swapoff", CPU utilization immediately pegs up to 100% system time, and then for a few minutes after, as far as anyone can tell, the machine is TOTALLY locked up. No console response, no response from anything on the machine. However, after a few minutes of TOTAL catatonia, it will mysteriously come back to life, having finally released all its swap. Now, this is a VERY contrived test, but there are a couple of things about doing this against 2.4 compared with 2.2 that seem VERY BROKEN to me. 1) Running this against a machine running a 2.2-series kernel does nothing out of the ordinary. You hit a bunch of swap, exit the "allocate" program, swapoff, and everything is fine after a few seconds of disk activity as it pages everything back into RAM. Least surprise. Under 2.4, when you "swapoff" it appears as far as anyone can tell that the machine has locked up completely. Very surprising. In fact, the first time it happened to me, I hit the Big Red Switch thinking the machine _had_ locked up. It wasn't until I started playing around with memory allocation a bit more and read some of the problems on LKML that I started to realize it wasn't locked up - just spinning. 2) Under 2.2, when the "allocate" programs exit, the amount of mem and swap that show up in the "used" column are quite small - about what you'd expect from all the apps that are actually running. No surprise there. Under 2.4, after running the "allocate" program, "free" shows about 200MB each under mem and swap as "used". A lot of memory shows up in the "cached" column, so that explains the mem usage, (although not what's cached, unless it's caching swap activity, which is odd) but what the heck is in that swap space? Very surprising. Now, I'm sure some of the response will be "Don't run 2.4. If you want to run a stable kernel run 2.2." That may be a reasonable, but there are a couple of features and a couple of drivers that make the 2.4 very appealing, and somewhat necessary, to me. Also, I want to help FIX these problems. I don't know if my hokey test is an indication of something for real, but hopefully it's something that's simple enough that a lot of people can run it and see if they experience similar things. And, AFAIC, a truly stable kernel (like 2.2) should be able to go deep into swap, and once the applications taking up the memory have exited, be able to turn off that swap and not have something utterly surprising, like the machine becoming comatose for several minutes, happen. If it does, that's an indication to me that there is something severely wrong. Now, with that being said, is there anything I