Re: 2.4.5 VM
> On a side question: does Linux support swap-files in addition to > sawp-partitions? Even if that has a performance penalty, when the system > is swapping performance is dead anyway. Yes. A possible solution could be: > dd if=/dev/zero of=/swap bs=1M count= > mkswap /swap > swapon /swap Works fine for me. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: 2.4.5 VM
> On a side question: does Linux support swap-files in addition to >sawp-partitions? Even if that has a performance penalty, when the system >is swapping performance is dead anyway. Yes. Simply use mkswap and swapon/off on a regular file instead of a partition device. I don't notice any significant performance penalty (a swapfile on a SCSI disk is faster than a swap-partition on an IDE disk), although you'd be advised to attempt to keep the file unfragmented. -- from: Jonathan "Chromatix" Morton mail: [EMAIL PROTECTED] (not for attachments) big-mail: [EMAIL PROTECTED] uni-mail: [EMAIL PROTECTED] The key to knowledge is not to rely on people to teach you it. Get VNC Server for Macintosh from http://www.chromatix.uklinux.net/vnc/ -BEGIN GEEK CODE BLOCK- Version 3.12 GCS$/E/S dpu(!) s:- a20 C+++ UL++ P L+++ E W+ N- o? K? w--- O-- M++$ V? PS PE- Y+ PGP++ t- 5- X- R !tv b++ DI+++ D G e+ h+ r++ y+(*) -END GEEK CODE BLOCK- - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: 2.4.5 VM
> On a side question: does Linux support swap-files in addition to > sawp-partitions? Even if that has a performance penalty, when the system since before 1.0 I believe 8) - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: 2.4.5 VM
On a side question: does Linux support swap-files in addition to sawp-partitions? Even if that has a performance penalty, when the system is swapping performance is dead anyway. Yes. Simply use mkswap and swapon/off on a regular file instead of a partition device. I don't notice any significant performance penalty (a swapfile on a SCSI disk is faster than a swap-partition on an IDE disk), although you'd be advised to attempt to keep the file unfragmented. -- from: Jonathan Chromatix Morton mail: [EMAIL PROTECTED] (not for attachments) big-mail: [EMAIL PROTECTED] uni-mail: [EMAIL PROTECTED] The key to knowledge is not to rely on people to teach you it. Get VNC Server for Macintosh from http://www.chromatix.uklinux.net/vnc/ -BEGIN GEEK CODE BLOCK- Version 3.12 GCS$/E/S dpu(!) s:- a20 C+++ UL++ P L+++ E W+ N- o? K? w--- O-- M++$ V? PS PE- Y+ PGP++ t- 5- X- R !tv b++ DI+++ D G e+ h+ r++ y+(*) -END GEEK CODE BLOCK- - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: 2.4.5 VM
On a side question: does Linux support swap-files in addition to sawp-partitions? Even if that has a performance penalty, when the system is swapping performance is dead anyway. Yes. A possible solution could be: dd if=/dev/zero of=/swap bs=1M count=whatever you like in MB mkswap /swap swapon /swap Works fine for me. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: 2.4.5 VM
On a side question: does Linux support swap-files in addition to sawp-partitions? Even if that has a performance penalty, when the system since before 1.0 I believe 8) - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: 2.4.5 VM
Hi, first of all, I am not complaining, or calling things buggy. I know that what I am running is "work in progress" and that one gets what one deserves :-) 2.4.x has been stable for me and given me no severe problem besides the changed pcmcia/cardbus support somewhere in 2.4.4-acx Just let me add my observation. The VM behaviour of 2.4.5 (started with some 2.4.4-ac kernel) is definitely less than an improvement for *my* setup. I am running a Thinkpad570 with 128 MB memory and about the same amount of swap (I know, against reccomendation). Under the new VM behaviour, I easily get in a situation where the system feels very sluggish. At that point in time, about 70-80% of memory are Cache, the rest is Used and some small amount of free. Swap is usually less than half filled and paging activity is about zero (some sporadic page out). Typical case is a kernel build plus a Netscape session. No unusal behaviour showing up in "top" - just sluggish system response. My gut feeling is that the Cache is pressing to hard against process memory. This may be great for some setups, but it is not good for others (like mine). What would be great (maybe someone is already working on it) are some tuning measures to tweak the cacheing behaviour. Just my 2 (Euro-)cents. Martin -- -- Martin Knoblauch |email: [EMAIL PROTECTED] TeraPort GmbH|Phone: +49-89-510857-309 C+ITS|Fax:+49-89-510857-111 http://www.teraport.de |Mobile: +49-170-4904759 - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: 2.4.5 VM
Hi, first of all, I am not complaining, or calling things buggy. I know that what I am running is work in progress and that one gets what one deserves :-) 2.4.x has been stable for me and given me no severe problem besides the changed pcmcia/cardbus support somewhere in 2.4.4-acx Just let me add my observation. The VM behaviour of 2.4.5 (started with some 2.4.4-ac kernel) is definitely less than an improvement for *my* setup. I am running a Thinkpad570 with 128 MB memory and about the same amount of swap (I know, against reccomendation). Under the new VM behaviour, I easily get in a situation where the system feels very sluggish. At that point in time, about 70-80% of memory are Cache, the rest is Used and some small amount of free. Swap is usually less than half filled and paging activity is about zero (some sporadic page out). Typical case is a kernel build plus a Netscape session. No unusal behaviour showing up in top - just sluggish system response. My gut feeling is that the Cache is pressing to hard against process memory. This may be great for some setups, but it is not good for others (like mine). What would be great (maybe someone is already working on it) are some tuning measures to tweak the cacheing behaviour. Just my 2 (Euro-)cents. Martin -- -- Martin Knoblauch |email: [EMAIL PROTECTED] TeraPort GmbH|Phone: +49-89-510857-309 C+ITS|Fax:+49-89-510857-111 http://www.teraport.de |Mobile: +49-170-4904759 - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
More data on 2.4.5 VM issues
This is 2.4.5 with Andrea Arcangeli's aa1 patch compiled with himem: Why is kswapd using so much CPU? If you reboot the machine and run the same user process kswapd CPU usage is almost 0% and none of the swap is used. This machine was upgraded from 2.2 and we did not have the luxury of re-partitioning it support the "new" 2.4 swap size requirements. After running for a few days with relatively constant memory usage: vmstat: procs memoryswap io system cpu r b w swpd free buff cache si sobibo incs us sy id 2 0 1 136512 5408504 209744 0 0 0 2 1949 10 26 64 top: 5:38pm up 3 days, 19:44, 2 users, load average: 2.08, 2.13, 2.15 34 processes: 32 sleeping, 2 running, 0 zombie, 0 stopped CPU0 states: 16.0% user, 56.4% system, 16.2% nice, 26.3% idle CPU1 states: 11.1% user, 57.0% system, 11.0% nice, 31.3% idle Mem: 1028804K av, 1023744K used,5060K free, 0K shrd, 504K buff Swap: 136512K av, 136512K used, 0K free 209876K cached PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND 28442 root 18 10 898M 812M 36188 R N 56.0 80.9 296:12 gateway.smart.5 28438 root 16 10 898M 811M 35084 S N 43.7 80.7 291:03 gateway.smart.5 5 root 9 0 00 0 SW 37.6 0.0 164:58 kswapd 2509 root 18 0 492 492 300 R 2.5 0.0 0:00 top 1 root 9 0680 0 SW0.0 0.0 0:08 init 2 root 9 0 00 0 SW0.0 0.0 0:00 keventd 3 root 19 19 00 0 SWN 0.0 0.0 1:11 ksoftirqd_CPU0 4 root 19 19 00 0 SWN 0.0 0.0 1:04 ksoftirqd_CPU1 6 root 9 0 00 0 SW0.0 0.0 0:00 kreclaimd 7 root 9 0 00 0 SW0.0 0.0 0:00 bdflush 8 root 9 0 00 0 SW0.0 0.0 0:07 kupdated 11 root 9 0 00 0 SW0.0 0.0 0:00 scsi_eh_0 315 root 9 0 1000 0 SW0.0 0.0 0:00 syslogd Hope this helps - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: 2.4.5 VM
I'd be forced to agree. I have 2.4.x in limited production, and with the exception of the HP/APIC fatal issues that have a "noapic" work-around, I have had no problem at all with any of the 2.4.x kernels I've used. Open software by definition will never reach the kind of monolithic stability that years of code freeze requires. Linux (especially 2.4.x) offers too much in return, and I can always run a 2.2.x kernel. I would say that the stability of the kernel has been *above* my expectations, frankly, considering all that's changed. It's definitely our responsibility as admins to test these kernels. I was running 2.4.0-test1 the second it was released, and the one problem I've found has been reported and investigated (it's apparently a tough one). As far as VM, I've never had the severe issues that some are reporting. This doesn't mean it's not a problem, but it definitely indicates that it's not a global showstopper. For VM-intense applications, I roll out a 2.2.19 kernel as a preventative measure while I wait for the VM code to be tweaked. I guess I would have expected these complaints during the -test phase. Not to mention that the distributions seem to have rolled out 2.4.x just fine. To wit: box1 1 ~> uptime 10:27am up 168 days, 2:43, 3 users, load average: 2.45, 2.30, 2.32 box1 2 ~> uname -a Linux box1.mynetwork.tld 2.4.0-test6 #1 SMP Sat Aug 19 04:26:58 PDT 2000 i686 unknown Not true production, but totally representative of my experiences FWIW. IMHO, YMMV, etc. -- Ken. On Friday, June 1, 2001, at 10:15 AM, Miquel Colom Piza wrote: > This is my first email to the list. I'm not subscribed but I've read it > for years. > > I don't agree with those claiming that 2.4.xx is bad or still beta. > > We the administrators have the responsability to test early kernels and > send good bug reports so the developers can solve the bugs. That's the > way we can contribute to the community. > > But it's really risky to use these kernels on MAIN 24x7 production > servers. > > This has been true for 1.2.x 2.0.x (I think that was the best linux > kernel series) 2.2.x and 2.4.x and will be for 2.6.x also > > Given we know that the support from open source developers is clearly > better than commercial contract supports, I don't see the reason to > complain about the work of those wonderfull hackers spending their spare > time coding for all of us. > > (I'm not subscribed to the list, Please CC me). > > - > To unsubscribe from this list: send the line "unsubscribe linux-kernel" > in > the body of a message to [EMAIL PROTECTED] > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: 2.4.5 VM
This is my first email to the list. I'm not subscribed but I've read it for years. I don't agree with those claiming that 2.4.xx is bad or still beta. We the administrators have the responsability to test early kernels and send good bug reports so the developers can solve the bugs. That's the way we can contribute to the community. But it's really risky to use these kernels on MAIN 24x7 production servers. This has been true for 1.2.x 2.0.x (I think that was the best linux kernel series) 2.2.x and 2.4.x and will be for 2.6.x also Given we know that the support from open source developers is clearly better than commercial contract supports, I don't see the reason to complain about the work of those wonderfull hackers spending their spare time coding for all of us. (I'm not subscribed to the list, Please CC me). - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: 2.4.5 VM
On Fri, 1 Jun 2001, Marcin Kowalski wrote: > Relating to Huge Dcache and InodeCache Entries and < 2xMem Swap. > I have a server with > 1.1gig of RAM which I have limited to 1gig (due to > stability - BUG at HIGHMEM.c: 155 crashes)... > > The size of the Swap space is 620mb... my memory usage is below : > > Mem:98 892748 7260 0 14796 171796 > -/+ buffers/cache: 706156 193852 > Swap: 641016 1792 639224 > > The extract out of slabinfo is: > inode_cache 903876 930840480 116355 1163551 : 124 62 > bdev_cache 1160 1357 64 23 231 : 252 126 > sigqueue 261261132991 : 252 126 > dentry_cache 775520 934560128 31152 311521 : 252 126 > > As you can see the usage is crazybearing in mind a number of things. > The system is running hardware raid5, reiserfs and massive amount of files > > 50 in >2000 directories.. > > It is a 2x933mhz PIII + 1.1gig of ram NEtservers > > What I really want to know is DOES THIS REALLY Matter. IE If memory is > needed by and application are these entries simply purged then.. In which > case there is no problem Could you check if they are purged heavily enough for you case when you start a big app? - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: 2.4.5 VM
I have a 2.4.5-ac3 box with 1G RAM and 2.6G Swapfirst time developers hit apache/php/zendcache after reboot and it swapped to a stop. I stop/restarted apache and it seems very happy...can't goto production like this tho :( Alan Cox wrote: > > My system has 128 Meg of Swap and RAM. > > Linus 2.4.0 notes are quite clear that you need at least twice RAM of swap > with 2.4. > > Marcelo is working to change that but right now you are running something > explicitly explained as not going to work as you want > > - > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to [EMAIL PROTECTED] > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ -- --- Russell Leighton[EMAIL PROTECTED] Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the Universe trying to produce bigger and better idiots. So far, the Universe is winning. - Rich Cook --- - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: 2.4.5 VM
I don't know myself, (it sounds like other bigmem problems), but setting up a 2GB swap file is easy enough to test. :-) -Dave On Fri, Jun 01, 2001 at 10:29:39AM +0200, Marcin Kowalski wrote: > > I found this post of interest. I have 1.1 Gig of RAM but only 800mb of > Swap as I expect NOT to use that much memory... Could this be the cause of > the machines VERY erratic behaviour??? Kernel Panics, HUGE INOde and > Dcache ?? > > On Thu, 31 May 2001 [EMAIL PROTECTED] wrote: > > > > My system has 128 Meg of Swap and RAM. > > > > Linus 2.4.0 notes are quite clear that you need at least twice RAM of swap > > with 2.4. > > > > Marcelo is working to change that but right now you are running something > > explicitly explained as not going to work as you want - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: 2.4.5 VM
I don't know myself, (it sounds like other bigmem problems), but setting up a 2GB swap file is easy enough to test. :-) -Dave On Fri, Jun 01, 2001 at 10:29:39AM +0200, Marcin Kowalski wrote: I found this post of interest. I have 1.1 Gig of RAM but only 800mb of Swap as I expect NOT to use that much memory... Could this be the cause of the machines VERY erratic behaviour??? Kernel Panics, HUGE INOde and Dcache ?? On Thu, 31 May 2001 [EMAIL PROTECTED] wrote: My system has 128 Meg of Swap and RAM. Linus 2.4.0 notes are quite clear that you need at least twice RAM of swap with 2.4. Marcelo is working to change that but right now you are running something explicitly explained as not going to work as you want - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: 2.4.5 VM
This is my first email to the list. I'm not subscribed but I've read it for years. I don't agree with those claiming that 2.4.xx is bad or still beta. We the administrators have the responsability to test early kernels and send good bug reports so the developers can solve the bugs. That's the way we can contribute to the community. But it's really risky to use these kernels on MAIN 24x7 production servers. This has been true for 1.2.x 2.0.x (I think that was the best linux kernel series) 2.2.x and 2.4.x and will be for 2.6.x also Given we know that the support from open source developers is clearly better than commercial contract supports, I don't see the reason to complain about the work of those wonderfull hackers spending their spare time coding for all of us. (I'm not subscribed to the list, Please CC me). - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: 2.4.5 VM
I'd be forced to agree. I have 2.4.x in limited production, and with the exception of the HP/APIC fatal issues that have a noapic work-around, I have had no problem at all with any of the 2.4.x kernels I've used. Open software by definition will never reach the kind of monolithic stability that years of code freeze requires. Linux (especially 2.4.x) offers too much in return, and I can always run a 2.2.x kernel. I would say that the stability of the kernel has been *above* my expectations, frankly, considering all that's changed. It's definitely our responsibility as admins to test these kernels. I was running 2.4.0-test1 the second it was released, and the one problem I've found has been reported and investigated (it's apparently a tough one). As far as VM, I've never had the severe issues that some are reporting. This doesn't mean it's not a problem, but it definitely indicates that it's not a global showstopper. For VM-intense applications, I roll out a 2.2.19 kernel as a preventative measure while I wait for the VM code to be tweaked. I guess I would have expected these complaints during the -test phase. Not to mention that the distributions seem to have rolled out 2.4.x just fine. To wit: box1 1 ~ uptime 10:27am up 168 days, 2:43, 3 users, load average: 2.45, 2.30, 2.32 box1 2 ~ uname -a Linux box1.mynetwork.tld 2.4.0-test6 #1 SMP Sat Aug 19 04:26:58 PDT 2000 i686 unknown Not true production, but totally representative of my experiences FWIW. IMHO, YMMV, etc. -- Ken. On Friday, June 1, 2001, at 10:15 AM, Miquel Colom Piza wrote: This is my first email to the list. I'm not subscribed but I've read it for years. I don't agree with those claiming that 2.4.xx is bad or still beta. We the administrators have the responsability to test early kernels and send good bug reports so the developers can solve the bugs. That's the way we can contribute to the community. But it's really risky to use these kernels on MAIN 24x7 production servers. This has been true for 1.2.x 2.0.x (I think that was the best linux kernel series) 2.2.x and 2.4.x and will be for 2.6.x also Given we know that the support from open source developers is clearly better than commercial contract supports, I don't see the reason to complain about the work of those wonderfull hackers spending their spare time coding for all of us. (I'm not subscribed to the list, Please CC me). - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
More data on 2.4.5 VM issues
This is 2.4.5 with Andrea Arcangeli's aa1 patch compiled with himem: Why is kswapd using so much CPU? If you reboot the machine and run the same user process kswapd CPU usage is almost 0% and none of the swap is used. This machine was upgraded from 2.2 and we did not have the luxury of re-partitioning it support the new 2.4 swap size requirements. After running for a few days with relatively constant memory usage: vmstat: procs memoryswap io system cpu r b w swpd free buff cache si sobibo incs us sy id 2 0 1 136512 5408504 209744 0 0 0 2 1949 10 26 64 top: 5:38pm up 3 days, 19:44, 2 users, load average: 2.08, 2.13, 2.15 34 processes: 32 sleeping, 2 running, 0 zombie, 0 stopped CPU0 states: 16.0% user, 56.4% system, 16.2% nice, 26.3% idle CPU1 states: 11.1% user, 57.0% system, 11.0% nice, 31.3% idle Mem: 1028804K av, 1023744K used,5060K free, 0K shrd, 504K buff Swap: 136512K av, 136512K used, 0K free 209876K cached PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND 28442 root 18 10 898M 812M 36188 R N 56.0 80.9 296:12 gateway.smart.5 28438 root 16 10 898M 811M 35084 S N 43.7 80.7 291:03 gateway.smart.5 5 root 9 0 00 0 SW 37.6 0.0 164:58 kswapd 2509 root 18 0 492 492 300 R 2.5 0.0 0:00 top 1 root 9 0680 0 SW0.0 0.0 0:08 init 2 root 9 0 00 0 SW0.0 0.0 0:00 keventd 3 root 19 19 00 0 SWN 0.0 0.0 1:11 ksoftirqd_CPU0 4 root 19 19 00 0 SWN 0.0 0.0 1:04 ksoftirqd_CPU1 6 root 9 0 00 0 SW0.0 0.0 0:00 kreclaimd 7 root 9 0 00 0 SW0.0 0.0 0:00 bdflush 8 root 9 0 00 0 SW0.0 0.0 0:07 kupdated 11 root 9 0 00 0 SW0.0 0.0 0:00 scsi_eh_0 315 root 9 0 1000 0 SW0.0 0.0 0:00 syslogd Hope this helps - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: 2.4.5 VM
I have a 2.4.5-ac3 box with 1G RAM and 2.6G Swapfirst time developers hit apache/php/zendcache after reboot and it swapped to a stop. I stop/restarted apache and it seems very happy...can't goto production like this tho :( Alan Cox wrote: My system has 128 Meg of Swap and RAM. Linus 2.4.0 notes are quite clear that you need at least twice RAM of swap with 2.4. Marcelo is working to change that but right now you are running something explicitly explained as not going to work as you want - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ -- --- Russell Leighton[EMAIL PROTECTED] Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the Universe trying to produce bigger and better idiots. So far, the Universe is winning. - Rich Cook --- - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: 2.4.5 VM
On Fri, 1 Jun 2001, Marcin Kowalski wrote: Relating to Huge Dcache and InodeCache Entries and 2xMem Swap. I have a server with 1.1gig of RAM which I have limited to 1gig (due to stability - BUG at HIGHMEM.c: 155 crashes)... The size of the Swap space is 620mb... my memory usage is below : Mem:98 892748 7260 0 14796 171796 -/+ buffers/cache: 706156 193852 Swap: 641016 1792 639224 The extract out of slabinfo is: inode_cache 903876 930840480 116355 1163551 : 124 62 bdev_cache 1160 1357 64 23 231 : 252 126 sigqueue 261261132991 : 252 126 dentry_cache 775520 934560128 31152 311521 : 252 126 As you can see the usage is crazybearing in mind a number of things. The system is running hardware raid5, reiserfs and massive amount of files 50 in 2000 directories.. It is a 2x933mhz PIII + 1.1gig of ram NEtservers What I really want to know is DOES THIS REALLY Matter. IE If memory is needed by and application are these entries simply purged then.. In which case there is no problem Could you check if they are purged heavily enough for you case when you start a big app? - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[Fwd: Plain 2.4.5 VM... (and 2.4.5-ac3)]
Here is a message from Steve Kieu that he couldn't get through... -- Einstein did not prove that everything is relative. Einstein explained how the speed of light could be constant. Benjamin Redelings I <>< http://www.bol.ucla.edu/~bredelin/ Just add my experience here... I use up to 2.4.4 and it is fine ; swap usage increase much but only in 2.4.5-acx IMHO it is because Alan did some changes? test with staroffice 5.1 in 35Mb RAM 32M swap 100Mhz just start staroffice and check, then exit check kernel usage usage when exit time (sec) 2.4.4-pre4 2.5M2.5M 48 2.4.4 2.0M2.0M 48 2.4.5 3.5M3.5M 49 2.4.5-ac1 7.9M7.9M 49 In 2.4.4 and 2.4.4-pre4 I noticed the kernel DID free swap when it needs. For example when I ran netscape for a while typing email in a web mail form (that way netscape will make memory leak) swap usage sometimes 17M. Quit netscape it reduced to about 12M; of course leave a lot free memory). Start netscape again SWAP REDUCED TO about 2M , just a bit bigger at the fisrt time I start netscape. Steve --- Benjamin Redelings I <[EMAIL PROTECTED]> wrote: > Vincent Stemen wrote: > > The problem is, that's not true. These problems > are not slipping > > through because of lack of testers. > Just to add some sanity to this thread, I have been > using the 2.4.x > kernels ever since they came out, on my personal > workstation and on some > workstations that I administrate for fellow students > in my department > here at UCLA. They have basically worked fine for > me. They are not > perfect, but many of the 2.4.x releases have been a > big improvement over > the 2.2.x releases. For one, 2.4.x actually can > tell which pages are > not used, and swap out unused daemons, which helps a > lot on a 64Mb box :) > > -BenR > -- > Einstein did not prove that everything is relative. > Einstein explained how the speed of light could be > constant. > Benjamin Redelings I <>< > http://www.bol.ucla.edu/~bredelin/ > > - > To unsubscribe from this list: send the line > "unsubscribe linux-kernel" in > the body of a message to [EMAIL PROTECTED] > More majordomo info at > http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ = S.KIEU _ http://messenger.yahoo.com.au - Yahoo! Messenger - Voice chat, mail alerts, stock quotes and favourite news and lots more!
Re: 2.4.5 VM
Actually I take everything back. I've been testing on linux-2.4.5-xfs and seen major improvements. -zim Christopher Zimmerman wrote: > Christopher Zimmerman wrote: > > > "Trever L. Adams" wrote: > > > > > In my opinion 2.4.x is NOT ready for primetime. The VM has been getting > > > worse since 2.4.0, I believe. Definitely since and including 2.4.3. I > > > cannot even edit a few images in gimp where the entire working set used > > > to fit entirely in memory. The system now locks in some loop (SAK still > > > works). > > > > > > FILE CACHING IS BROKEN. I don't care who says what, by the time swap is > > > half filled, it is time to start throwing away simple caches. Not wait > > > until there is no more memory free and then lock in an infinite loop. > > > > > > My system has 128 Meg of Swap and RAM. > > > > > > Trever Adams > > > > > > - > > > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > > > the body of a message to [EMAIL PROTECTED] > > > More majordomo info at http://vger.kernel.org/majordomo-info.html > > > Please read the FAQ at http://www.tux.org/lkml/ > > > > I've found that with the latest kernel release (2.4.5) VM performance has > > been greatly improved. kswapd and bdflush no longer use 200% of my cpu > > cycles when simply doing a dd bs=1024 count=8388608 if=/dev/zero > > of=test.file. All of my test systems remain responsive with about 180% cpu > > available. These systems are running software RAID and 3ware IDE raid with > > 2GB of memory and 4GB swap. Have you tried 2.4.5? > > > > -zim > > > > Christopher Zimmerman > > AltaVista Company > > > > - > > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > > the body of a message to [EMAIL PROTECTED] > > More majordomo info at http://vger.kernel.org/majordomo-info.html > > Please read the FAQ at http://www.tux.org/lkml/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: 2.4.5 VM
Christopher Zimmerman wrote: > "Trever L. Adams" wrote: > > > In my opinion 2.4.x is NOT ready for primetime. The VM has been getting > > worse since 2.4.0, I believe. Definitely since and including 2.4.3. I > > cannot even edit a few images in gimp where the entire working set used > > to fit entirely in memory. The system now locks in some loop (SAK still > > works). > > > > FILE CACHING IS BROKEN. I don't care who says what, by the time swap is > > half filled, it is time to start throwing away simple caches. Not wait > > until there is no more memory free and then lock in an infinite loop. > > > > My system has 128 Meg of Swap and RAM. > > > > Trever Adams > > > > - > > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > > the body of a message to [EMAIL PROTECTED] > > More majordomo info at http://vger.kernel.org/majordomo-info.html > > Please read the FAQ at http://www.tux.org/lkml/ > > I've found that with the latest kernel release (2.4.5) VM performance has > been greatly improved. kswapd and bdflush no longer use 200% of my cpu > cycles when simply doing a dd bs=1024 count=8388608 if=/dev/zero > of=test.file. All of my test systems remain responsive with about 180% cpu > available. These systems are running software RAID and 3ware IDE raid with > 2GB of memory and 4GB swap. Have you tried 2.4.5? > > -zim > > Christopher Zimmerman > AltaVista Company > > - > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to [EMAIL PROTECTED] > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: 2.4.5 VM
Christopher Zimmerman wrote: > > I've found that with the latest kernel release (2.4.5) VM performance has > been greatly improved. kswapd and bdflush no longer use 200% of my cpu > cycles when simply doing a dd bs=1024 count=8388608 if=/dev/zero > of=test.file. All of my test systems remain responsive with about 180% cpu > available. These systems are running software RAID and 3ware IDE raid with > 2GB of memory and 4GB swap. Have you tried 2.4.5? > > -zim > > Christopher Zimmerman > AltaVista Company > I have found that 2.4.5 is great, until it decides to stop freeing unused pages (simple file cache). Then it goes to hell in a handbasket at the speed of light. Yes, I have tried it, that is what I wrote about. Trever - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: 2.4.5 VM
"Trever L. Adams" wrote: > In my opinion 2.4.x is NOT ready for primetime. The VM has been getting > worse since 2.4.0, I believe. Definitely since and including 2.4.3. I > cannot even edit a few images in gimp where the entire working set used > to fit entirely in memory. The system now locks in some loop (SAK still > works). > > FILE CACHING IS BROKEN. I don't care who says what, by the time swap is > half filled, it is time to start throwing away simple caches. Not wait > until there is no more memory free and then lock in an infinite loop. > > My system has 128 Meg of Swap and RAM. > > Trever Adams > > - > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to [EMAIL PROTECTED] > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ I've found that with the latest kernel release (2.4.5) VM performance has been greatly improved. kswapd and bdflush no longer use 200% of my cpu cycles when simply doing a dd bs=1024 count=8388608 if=/dev/zero of=test.file. All of my test systems remain responsive with about 180% cpu available. These systems are running software RAID and 3ware IDE raid with 2GB of memory and 4GB swap. Have you tried 2.4.5? -zim Christopher Zimmerman AltaVista Company - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: 2.4.5 VM
> Actually I have tried 1x,2x,3x. In 2.4.0 to 2.4.3 I had some issues but > never a system freeze of any kind. With 2.4.4 I had more problems, but > I was ok. 2.4.5 I now have these freezes. Maybe I should go back to > 2x, but I still find this behavior crazy. > This still doesn't negate the point of freeing simple caches. The caches are in part shared. Remember page cache memory and read only application pages are the same thing - so its not that simple. I found 2.4.5 pretty bad. 2.4.5-ac seems to be better on the whole but I know its definitely not right yet. Marcelo and Rik are working on that more and more. Marcelo has a test patch to fix the (documented but annoying) 2x memory swap rule stuff. The balancing problem is harder but being worked on. If you can give Rik a summary of your config/what apps run/ps data then it may be valuable as he can duplicate your exact setup for testing his vm changes and add it to the test sets. Alan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: 2.4.5 VM
> My system has 128 Meg of Swap and RAM. Linus 2.4.0 notes are quite clear that you need at least twice RAM of swap with 2.4. Marcelo is working to change that but right now you are running something explicitly explained as not going to work as you want - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
2.4.5 VM
In my opinion 2.4.x is NOT ready for primetime. The VM has been getting worse since 2.4.0, I believe. Definitely since and including 2.4.3. I cannot even edit a few images in gimp where the entire working set used to fit entirely in memory. The system now locks in some loop (SAK still works). FILE CACHING IS BROKEN. I don't care who says what, by the time swap is half filled, it is time to start throwing away simple caches. Not wait until there is no more memory free and then lock in an infinite loop. My system has 128 Meg of Swap and RAM. Trever Adams - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM... (and 2.4.5-ac3)
Vincent Stemen wrote: > The problem is, that's not true. These problems are not slipping > through because of lack of testers. Just to add some sanity to this thread, I have been using the 2.4.x kernels ever since they came out, on my personal workstation and on some workstations that I administrate for fellow students in my department here at UCLA. They have basically worked fine for me. They are not perfect, but many of the 2.4.x releases have been a big improvement over the 2.2.x releases. For one, 2.4.x actually can tell which pages are not used, and swap out unused daemons, which helps a lot on a 64Mb box :) -BenR -- Einstein did not prove that everything is relative. Einstein explained how the speed of light could be constant. Benjamin Redelings I <>< http://www.bol.ucla.edu/~bredelin/ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
2.4.5 VM
In my opinion 2.4.x is NOT ready for primetime. The VM has been getting worse since 2.4.0, I believe. Definitely since and including 2.4.3. I cannot even edit a few images in gimp where the entire working set used to fit entirely in memory. The system now locks in some loop (SAK still works). FILE CACHING IS BROKEN. I don't care who says what, by the time swap is half filled, it is time to start throwing away simple caches. Not wait until there is no more memory free and then lock in an infinite loop. My system has 128 Meg of Swap and RAM. Trever Adams - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: 2.4.5 VM
Trever L. Adams wrote: In my opinion 2.4.x is NOT ready for primetime. The VM has been getting worse since 2.4.0, I believe. Definitely since and including 2.4.3. I cannot even edit a few images in gimp where the entire working set used to fit entirely in memory. The system now locks in some loop (SAK still works). FILE CACHING IS BROKEN. I don't care who says what, by the time swap is half filled, it is time to start throwing away simple caches. Not wait until there is no more memory free and then lock in an infinite loop. My system has 128 Meg of Swap and RAM. Trever Adams - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ I've found that with the latest kernel release (2.4.5) VM performance has been greatly improved. kswapd and bdflush no longer use 200% of my cpu cycles when simply doing a dd bs=1024 count=8388608 if=/dev/zero of=test.file. All of my test systems remain responsive with about 180% cpu available. These systems are running software RAID and 3ware IDE raid with 2GB of memory and 4GB swap. Have you tried 2.4.5? -zim Christopher Zimmerman AltaVista Company - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: 2.4.5 VM
Christopher Zimmerman wrote: I've found that with the latest kernel release (2.4.5) VM performance has been greatly improved. kswapd and bdflush no longer use 200% of my cpu cycles when simply doing a dd bs=1024 count=8388608 if=/dev/zero of=test.file. All of my test systems remain responsive with about 180% cpu available. These systems are running software RAID and 3ware IDE raid with 2GB of memory and 4GB swap. Have you tried 2.4.5? -zim Christopher Zimmerman AltaVista Company I have found that 2.4.5 is great, until it decides to stop freeing unused pages (simple file cache). Then it goes to hell in a handbasket at the speed of light. Yes, I have tried it, that is what I wrote about. Trever - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: 2.4.5 VM
Christopher Zimmerman wrote: Trever L. Adams wrote: In my opinion 2.4.x is NOT ready for primetime. The VM has been getting worse since 2.4.0, I believe. Definitely since and including 2.4.3. I cannot even edit a few images in gimp where the entire working set used to fit entirely in memory. The system now locks in some loop (SAK still works). FILE CACHING IS BROKEN. I don't care who says what, by the time swap is half filled, it is time to start throwing away simple caches. Not wait until there is no more memory free and then lock in an infinite loop. My system has 128 Meg of Swap and RAM. Trever Adams - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ I've found that with the latest kernel release (2.4.5) VM performance has been greatly improved. kswapd and bdflush no longer use 200% of my cpu cycles when simply doing a dd bs=1024 count=8388608 if=/dev/zero of=test.file. All of my test systems remain responsive with about 180% cpu available. These systems are running software RAID and 3ware IDE raid with 2GB of memory and 4GB swap. Have you tried 2.4.5? -zim Christopher Zimmerman AltaVista Company - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: 2.4.5 VM
Actually I take everything back. I've been testing on linux-2.4.5-xfs and seen major improvements. -zim Christopher Zimmerman wrote: Christopher Zimmerman wrote: Trever L. Adams wrote: In my opinion 2.4.x is NOT ready for primetime. The VM has been getting worse since 2.4.0, I believe. Definitely since and including 2.4.3. I cannot even edit a few images in gimp where the entire working set used to fit entirely in memory. The system now locks in some loop (SAK still works). FILE CACHING IS BROKEN. I don't care who says what, by the time swap is half filled, it is time to start throwing away simple caches. Not wait until there is no more memory free and then lock in an infinite loop. My system has 128 Meg of Swap and RAM. Trever Adams - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ I've found that with the latest kernel release (2.4.5) VM performance has been greatly improved. kswapd and bdflush no longer use 200% of my cpu cycles when simply doing a dd bs=1024 count=8388608 if=/dev/zero of=test.file. All of my test systems remain responsive with about 180% cpu available. These systems are running software RAID and 3ware IDE raid with 2GB of memory and 4GB swap. Have you tried 2.4.5? -zim Christopher Zimmerman AltaVista Company - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[Fwd: Plain 2.4.5 VM... (and 2.4.5-ac3)]
Here is a message from Steve Kieu that he couldn't get through... -- Einstein did not prove that everything is relative. Einstein explained how the speed of light could be constant. Benjamin Redelings I http://www.bol.ucla.edu/~bredelin/ Just add my experience here... I use up to 2.4.4 and it is fine ; swap usage increase much but only in 2.4.5-acx IMHO it is because Alan did some changes? test with staroffice 5.1 in 35Mb RAM 32M swap 100Mhz just start staroffice and check, then exit check kernel usage usage when exit time (sec) 2.4.4-pre4 2.5M2.5M 48 2.4.4 2.0M2.0M 48 2.4.5 3.5M3.5M 49 2.4.5-ac1 7.9M7.9M 49 In 2.4.4 and 2.4.4-pre4 I noticed the kernel DID free swap when it needs. For example when I ran netscape for a while typing email in a web mail form (that way netscape will make memory leak) swap usage sometimes 17M. Quit netscape it reduced to about 12M; of course leave a lot free memory). Start netscape again SWAP REDUCED TO about 2M , just a bit bigger at the fisrt time I start netscape. Steve --- Benjamin Redelings I [EMAIL PROTECTED] wrote: Vincent Stemen wrote: The problem is, that's not true. These problems are not slipping through because of lack of testers. Just to add some sanity to this thread, I have been using the 2.4.x kernels ever since they came out, on my personal workstation and on some workstations that I administrate for fellow students in my department here at UCLA. They have basically worked fine for me. They are not perfect, but many of the 2.4.x releases have been a big improvement over the 2.2.x releases. For one, 2.4.x actually can tell which pages are not used, and swap out unused daemons, which helps a lot on a 64Mb box :) -BenR -- Einstein did not prove that everything is relative. Einstein explained how the speed of light could be constant. Benjamin Redelings I http://www.bol.ucla.edu/~bredelin/ - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ = S.KIEU _ http://messenger.yahoo.com.au - Yahoo! Messenger - Voice chat, mail alerts, stock quotes and favourite news and lots more!
Re: 2.4.5 VM
My system has 128 Meg of Swap and RAM. Linus 2.4.0 notes are quite clear that you need at least twice RAM of swap with 2.4. Marcelo is working to change that but right now you are running something explicitly explained as not going to work as you want - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: 2.4.5 VM
Actually I have tried 1x,2x,3x. In 2.4.0 to 2.4.3 I had some issues but never a system freeze of any kind. With 2.4.4 I had more problems, but I was ok. 2.4.5 I now have these freezes. Maybe I should go back to 2x, but I still find this behavior crazy. This still doesn't negate the point of freeing simple caches. The caches are in part shared. Remember page cache memory and read only application pages are the same thing - so its not that simple. I found 2.4.5 pretty bad. 2.4.5-ac seems to be better on the whole but I know its definitely not right yet. Marcelo and Rik are working on that more and more. Marcelo has a test patch to fix the (documented but annoying) 2x memory swap rule stuff. The balancing problem is harder but being worked on. If you can give Rik a summary of your config/what apps run/ps data then it may be valuable as he can duplicate your exact setup for testing his vm changes and add it to the test sets. Alan - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM... (and 2.4.5-ac3)
On Wed, 30 May 2001, Vincent Stemen wrote: > On Wednesday 30 May 2001 15:17, Mike Galbraith wrote: > > On Wed, 30 May 2001, Vincent Stemen wrote: > > > On Wednesday 30 May 2001 01:02, Mike Galbraith wrote: > > > > On Tue, 29 May 2001, Vincent Stemen wrote: > > > > > On Tuesday 29 May 2001 15:16, Alan Cox wrote: > > > > > > > a reasonably stable release until 2.2.12. I do not understand > > > > > > > why code with such serious reproducible problems is being > > > > > > > introduced into the even numbered kernels. What happened to > > > > > > > the plan to use only the > > > > > > > > > > > > Who said it was introduced ?? It was more 'lurking' than > > > > > > introduced. And unfortunately nobody really pinned it down in > > > > > > 2.4test. > > > > > > > > > > I fail to see the distinction. First of all, why was it ever > > > > > released as 2.4-test? That question should probably be directed at > > > > > Linus. If it is not fully tested, then it should be released it as > > > > > an odd number. If it already existed in the odd numbered > > > > > development kernel and was known, then it should have never been > > > > > released as a production kernel until it was resolved. Otherwise, > > > > > it completely defeats the purpose of having the even/odd numbering > > > > > system. > > > > > > > > > > I do not expect bugs to never slip through to production kernels, > > > > > but known bugs that are not trivial should not, and serious bugs > > > > > like these VM problems especially should not. > > > > > > > > And you can help prevent them from slipping through by signing up as > > > > a shake and bake tester. Indeed, you can make your expectations > > > > reality absolutely free of charge, and or compensation > > > > what a bargain! > > > > > > > > X ___ ;-) > > > > > > > > -Mike > > > > > > The problem is, that's not true. These problems are not slipping > > > through because of lack of testers. As Alan said, the VM problem has > > > > Sorry, that's a copout. You (we) had many chances to notice. Don't > > push the problems back onto developers.. it's our problem. > > > > How is that a copout? The problem was noticed. I am only suggesting > that we not be in such a hurry to put code in the production kernels > until we are pretty sure it works well enough, and that we release > major production versions more often so that they do not contain 2 or > 3 years worth of new code making it so hard to debug. We probably > should have had 2 or 3 code freezes and production releases since > 2.2.x. As I mentioned in a previous posting, this way we do not have > to run a 2 or 3 year old kernel in order to have reasonable stability. I don't think you or I can do a better job of release management than Linus and friends, so there's no point in us discussing it. If you want to tell Linus, Alan et al how to do it 'right', you go do that. -Mike - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
On Wed, 30 May 2001, Marcelo Tosatti wrote: > On Wed, 30 May 2001, Mike Galbraith wrote: > > > On Wed, 30 May 2001, Rik van Riel wrote: > > > > > On Wed, 30 May 2001, Marcelo Tosatti wrote: > > > > > > > The problem is that we allow _every_ task to age pages on the system > > > > at the same time --- this is one of the things which is fucking up. > > > > > > This should not have any effect on the ratio of cache > > > reclaiming vs. swapout use, though... > > > > It shouldn't.. but when many tasks are aging, it does. > > What Rik means is that they are independant problems. Ok. > > > Excluding these guys certainly seems to make a difference. > > Sure, those guys are going to "help" kswapd to unmap pte's and allocate > swap space. > > Now even if only kswapd does this job (meaning a sane amount of cache > reclaims/swapouts), you still have to deal with the reclaim/swapout > tradeoff. > > See? Yes. -Mike - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM... (and 2.4.5-ac3)
On Wednesday 30 May 2001 15:17, Mike Galbraith wrote: > On Wed, 30 May 2001, Vincent Stemen wrote: > > On Wednesday 30 May 2001 01:02, Mike Galbraith wrote: > > > On Tue, 29 May 2001, Vincent Stemen wrote: > > > > On Tuesday 29 May 2001 15:16, Alan Cox wrote: > > > > > > a reasonably stable release until 2.2.12. I do not understand > > > > > > why code with such serious reproducible problems is being > > > > > > introduced into the even numbered kernels. What happened to > > > > > > the plan to use only the > > > > > > > > > > Who said it was introduced ?? It was more 'lurking' than > > > > > introduced. And unfortunately nobody really pinned it down in > > > > > 2.4test. > > > > > > > > I fail to see the distinction. First of all, why was it ever > > > > released as 2.4-test? That question should probably be directed at > > > > Linus. If it is not fully tested, then it should be released it as > > > > an odd number. If it already existed in the odd numbered > > > > development kernel and was known, then it should have never been > > > > released as a production kernel until it was resolved. Otherwise, > > > > it completely defeats the purpose of having the even/odd numbering > > > > system. > > > > > > > > I do not expect bugs to never slip through to production kernels, > > > > but known bugs that are not trivial should not, and serious bugs > > > > like these VM problems especially should not. > > > > > > And you can help prevent them from slipping through by signing up as > > > a shake and bake tester. Indeed, you can make your expectations > > > reality absolutely free of charge, and or compensation > > > what a bargain! > > > > > > X ___ ;-) > > > > > > -Mike > > > > The problem is, that's not true. These problems are not slipping > > through because of lack of testers. As Alan said, the VM problem has > > Sorry, that's a copout. You (we) had many chances to notice. Don't > push the problems back onto developers.. it's our problem. > How is that a copout? The problem was noticed. I am only suggesting that we not be in such a hurry to put code in the production kernels until we are pretty sure it works well enough, and that we release major production versions more often so that they do not contain 2 or 3 years worth of new code making it so hard to debug. We probably should have had 2 or 3 code freezes and production releases since 2.2.x. As I mentioned in a previous posting, this way we do not have to run a 2 or 3 year old kernel in order to have reasonable stability. > > Here are some of the problems I see: > > > > There was far to long of a stretch with to much code dumped into both > > the 2.2 and 2.4 kernels before release. There needs to be a smaller > > number changes between major releases so that they can be more > > thoroughly tested and debugged. In the race to get it out there they > > are making the same mistakes as Microsoft, releasing production > > kernels with known serious bugs because it is taking to long and they > > want to move on forward. I enjoy criticizing Microsoft so much for > > the same thing that I do not want to have to stop in order to not > > sound hypocritical :-). The Linux community has built a lot of it's > > reputation on not making these mistakes. Please lets try not to > > destroy that. > > > > They are disregarding the even/odd versioning system. > > For example: > > There was a new 8139too driver added to the the 2.4.5 (I think) kernel > > which Alan Cox took back out and reverted to the old one in his > > 2.4.5-ac? versions because it is apparently causing lockups. > > Shouldn't this new driver have been released in a 2.5.x development > > kernel and proven there before replacing the one in the production > > kernel? I haven't even seen a 2.5.x kernel released yet. > > > > Based on Linus's original very good plan for even/odd numbers, there > > should not have been 2.4.0-test? kernels either. This was another > > example of the rush to increment to 2.4 long before it was ready. > > There was a long stretch of test kernels and and now we are all the > > way to 2.4.5 and it is still not stable. We are repeating the 2.2.x > > process all over again. It should have been 2.3.x until the > > production release was ready. If they needed to distinguish a code > > freeze for final testing, it could be done with a 4th version > > component (2.3.xx.xx), where the 4 component is incremented for final > > bug fixes. > > Sorry, I disagree with every last bit. Either you accept a situation > or you try to do something about it. > > -Mike I am spending a lot of time testing new kernels, reporting bugs and offering suggestions that I think may improve on the stability of production kernels. Is this not considered doing something about it? It is necessary to point out where one sees a problem in order to offer possible solutions for improvement. - Vincent - To unsubscribe from this list: send the line
Re: Plain 2.4.5 VM... (and 2.4.5-ac3)
On Wednesday 30 May 2001 15:30, Rik van Riel wrote: > On Wed, 30 May 2001, Vincent Stemen wrote: > > The problem is, that's not true. These problems are not slipping > > through because of lack of testers. As Alan said, the VM problem has > > been lurking, which means that it was known already. > > Fully agreed, it went through because of a lack of hours > per day and the fact that the priority of developers was > elsewhere. > > For me, for example, the priorities have mostly been with > bugs that bothered me or that bothered Conectiva's customers. > > If you _really_ feel this strongly about the bug, you could > either try to increase the number of hours a day for all of I sure wish I could :-). > us or you could talk to my boss about hiring me as a consultant > to fix the problem for you on an emergency basis :) > The other two alternatives would be either waiting until > somebody gets around to fixing the bug or sending in a patch > yourself. > > Trying to piss off developers has adverse effect on all four > of the methods above :) > Why should my comments piss anybody off? I am just trying to point out a problem, as I see it, an offer suggestions for improvement. Other developers will either agree with me or they wont. Contributions are not made only through writing code. I contribute through code, bug reports, ideas, and suggestions. I would love to dive in and try to help fix some of the kernel problems but my hands are just to full right now. My comments are not meant to rush anybody and I am not criticizing how long it is taking. I know everybody is doing everything they can just like I am, and they are doing a terrific job. I am just suggesting a modification to the way the kernels are distributed that is more like the early versions that I hoped would allow us to maintain a stable kernel for distributions and production machines. - Vincent Stemen - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM... (and 2.4.5-ac3)
Ronald Bultje writes: > On 30 May 2001 14:58:57 -0500, Vincent Stemen wrote: > > There was a new 8139too driver added to the the 2.4.5 (I think) kernel > > which Alan Cox took back out and reverted to the old one in his > > 2.4.5-ac? versions because it is apparently causing lockups. > > Shouldn't this new driver have been released in a 2.5.x development > > kernel and proven there before replacing the one in the production > > kernel? I haven't even seen a 2.5.x kernel released yet. > > If every driver has to go thorugh the complete development cycle (of 2+ > years), I'm sure very little driver writers will be as motivated as they > are now - it takes ages before they see their efforts "rewarded" with a > place in the kernel. > The ideal case is that odd-numbered kernels are "for testing" and > even-numbered kernels are stable. However, this is only theory. In > practice, you can't rule out all bugs. And you can't test all things for > all cases and every test case, the linux community doesn't have the > manpower for that. And to prevent a complete driver development cycle > taking 2+ years, you have to compromise. > > If you would take 2+ years for a single driver development cycle, nobody > would be interested in linux since the new devices would only be > supported by a stable kernel two years after their release. See the > point? To prevent that, you need to compromise. and thus, sometimes, you > have some crashes. I agree with everything you say up till this point, but you are arguing against a point I never made. First of all, bugs like the 8139too lockup was found within the first day or two of release in the 2.4.3 kernel. Also, most show stopper bugs such as the VM problems are found fairly quickly. Even if it takes a long time to figure out how to fix them, I do not think they should be pushed on through into production kernels until they are until they are fixed. I already said that I do not expect minor bugs not to slip through. However, if they are minor, they can usually be fixed quickly once they are discovered and it is no big deal if they make it into a production kernel. > That's why there's still 2.2.x - that's purely stable > and won't crash as fast as 2.4.x, but misses the "newest > cutting-edge-technology device support" and "newest technology" (like > new SMP handling , ReiserFS, etc... But it *is* stable. > The reason I suggested more frequent major production releases is so that you don't have to go back to a 2 or 3 year old kernel and loose out on years worth of new features to have any stability. One show stopper bug like the VM problems would not be as much of a problem if there was a stable production kernel that we could run that was only 4 or 6 months old. > > Based on Linus's original very good plan for even/odd numbers, there > > should not have been 2.4.0-test? kernels either. This was another > > example of the rush to increment to 2.4 long before it was ready. > > There was a long stretch of test kernels and and now we are all the > > way to 2.4.5 and it is still not stable. We are repeating the 2.2.x > > process all over again. > > Wrong again. > 2.3.x is for development, adding new things, testing, adding, testing, > changing, testing, etc. Which is the same point I made. > 2.4-test is for testing only, it's some sort of feature freeze. Agreed. My only point here was that it suggests that there are only minor bugs left to be solved before the production release by setting the version to 2.4-test. That is one of the reasons I made the suggestion to keep it in the 2.3 range, since there were actually serious VM problems still upon the production 2.4 release. > 2.4.x is for final/stable 2.4. > It's a standard *nix development cycle. That's how everyone does it. My point exactly. > > Regards, > > Ronald Bultje - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
On Wed, 30 May 2001, Mike Galbraith wrote: > On Wed, 30 May 2001, Rik van Riel wrote: > > > On Wed, 30 May 2001, Marcelo Tosatti wrote: > > > > > The problem is that we allow _every_ task to age pages on the system > > > at the same time --- this is one of the things which is fucking up. > > > > This should not have any effect on the ratio of cache > > reclaiming vs. swapout use, though... > > It shouldn't.. but when many tasks are aging, it does. What Rik means is that they are independant problems. > Excluding these guys certainly seems to make a difference. Sure, those guys are going to "help" kswapd to unmap pte's and allocate swap space. Now even if only kswapd does this job (meaning a sane amount of cache reclaims/swapouts), you still have to deal with the reclaim/swapout tradeoff. See? - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
On Wed, 30 May 2001, Rik van Riel wrote: > On Wed, 30 May 2001, Marcelo Tosatti wrote: > > > The problem is that we allow _every_ task to age pages on the system > > at the same time --- this is one of the things which is fucking up. > > This should not have any effect on the ratio of cache > reclaiming vs. swapout use, though... It shouldn't.. but when many tasks are aging, it does. Excluding these guys certainly seems to make a difference. (could be seeing something else and interpreting it wrong...) -Mike - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM... (and 2.4.5-ac3)
On Wed, 30 May 2001, Vincent Stemen wrote: > The problem is, that's not true. These problems are not slipping > through because of lack of testers. As Alan said, the VM problem has > been lurking, which means that it was known already. Fully agreed, it went through because of a lack of hours per day and the fact that the priority of developers was elsewhere. For me, for example, the priorities have mostly been with bugs that bothered me or that bothered Conectiva's customers. If you _really_ feel this strongly about the bug, you could either try to increase the number of hours a day for all of us or you could talk to my boss about hiring me as a consultant to fix the problem for you on an emergency basis :) The other two alternatives would be either waiting until somebody gets around to fixing the bug or sending in a patch yourself. Trying to piss off developers has adverse effect on all four of the methods above :) Rik -- Virtual memory is like a game you can't win; However, without VM there's truly nothing to lose... http://www.surriel.com/ http://distro.conectiva.com/ Send all your spam to [EMAIL PROTECTED] (spam digging piggy) - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM... (and 2.4.5-ac3)
> There was a new 8139too driver added to the the 2.4.5 (I think) kernel > which Alan Cox took back out and reverted to the old one in his > 2.4.5-ac? versions because it is apparently causing lockups. > Shouldn't this new driver have been released in a 2.5.x development > kernel and proven there before replacing the one in the production > kernel? I haven't even seen a 2.5.x kernel released yet. Nope. The 2.4.3 one is buggy too - but differently (and it turns out a little less) buggy. Welcome to software. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM... (and 2.4.5-ac3)
On Wednesday 30 May 2001 01:02, Mike Galbraith wrote: > On Tue, 29 May 2001, Vincent Stemen wrote: > > On Tuesday 29 May 2001 15:16, Alan Cox wrote: > > > > a reasonably stable release until 2.2.12. I do not understand why > > > > code with such serious reproducible problems is being introduced > > > > into the even numbered kernels. What happened to the plan to use > > > > only the > > > > > > Who said it was introduced ?? It was more 'lurking' than introduced. > > > And unfortunately nobody really pinned it down in 2.4test. > > > > I fail to see the distinction. First of all, why was it ever released > > as 2.4-test? That question should probably be directed at Linus. If > > it is not fully tested, then it should be released it as an odd > > number. If it already existed in the odd numbered development kernel > > and was known, then it should have never been released as a production > > kernel until it was resolved. Otherwise, it completely defeats the > > purpose of having the even/odd numbering system. > > > > I do not expect bugs to never slip through to production kernels, but > > known bugs that are not trivial should not, and serious bugs like > > these VM problems especially should not. > > And you can help prevent them from slipping through by signing up as a > shake and bake tester. Indeed, you can make your expectations reality > absolutely free of charge, and or compensation > what a bargain! > > X ___ ;-) > > -Mike The problem is, that's not true. These problems are not slipping through because of lack of testers. As Alan said, the VM problem has been lurking, which means that it was known already. We currently have no development/production kernel distinction and I have not been able to find even one stable 2.4.x version to run on our main machines. Reverting back to 2.2.x is a real pain because of all the surrounding changes which will affect our initscripts and other system configuration issues, such as Unix98 pty's, proc filesystem differences, device numbering, etc. I have the greatest respect and appreciation for Linus, Alan, and all of the other kernel developers. My comments are not meant to criticize, but rather to point out some the problems I see that are making it so difficult to stabilize the kernel and encourage them to steer back on track. Here are some of the problems I see: There was far to long of a stretch with to much code dumped into both the 2.2 and 2.4 kernels before release. There needs to be a smaller number changes between major releases so that they can be more thoroughly tested and debugged. In the race to get it out there they are making the same mistakes as Microsoft, releasing production kernels with known serious bugs because it is taking to long and they want to move on forward. I enjoy criticizing Microsoft so much for the same thing that I do not want to have to stop in order to not sound hypocritical :-). The Linux community has built a lot of it's reputation on not making these mistakes. Please lets try not to destroy that. They are disregarding the even/odd versioning system. For example: There was a new 8139too driver added to the the 2.4.5 (I think) kernel which Alan Cox took back out and reverted to the old one in his 2.4.5-ac? versions because it is apparently causing lockups. Shouldn't this new driver have been released in a 2.5.x development kernel and proven there before replacing the one in the production kernel? I haven't even seen a 2.5.x kernel released yet. Based on Linus's original very good plan for even/odd numbers, there should not have been 2.4.0-test? kernels either. This was another example of the rush to increment to 2.4 long before it was ready. There was a long stretch of test kernels and and now we are all the way to 2.4.5 and it is still not stable. We are repeating the 2.2.x process all over again. It should have been 2.3.x until the production release was ready. If they needed to distinguish a code freeze for final testing, it could be done with a 4th version component (2.3.xx.xx), where the 4 component is incremented for final bug fixes. - Vincent Stemen - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
On Wed, 30 May 2001, Mike Galbraith wrote: > I wouldn't go so far as to say totally broken (mostly because I've > tried like _hell_ to find a better way, and [despite minor successes] > I've not been able to come up with something which covers all cases > that even _I_ [hw tech] can think of well). The "easy way out" seems to be physical -> virtual page reverse mappings, these make it trivial to apply balanced pressure on all pages. The downside of this measure is that it costs additional overhead and can up to double the amount of memory we take in with page tables. Of course, this amount is only prohibitive if the amount of page table memory was also prohibitively large in the first place, but ... :) regards, Rik -- Virtual memory is like a game you can't win; However, without VM there's truly nothing to lose... http://www.surriel.com/ http://distro.conectiva.com/ Send all your spam to [EMAIL PROTECTED] (spam digging piggy) - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
On Wed, 30 May 2001, Rik van Riel wrote: > On Wed, 30 May 2001, Marcelo Tosatti wrote: > > > The problem is that we allow _every_ task to age pages on the system > > at the same time --- this is one of the things which is fucking up. > > This should not have any effect on the ratio of cache > reclaiming vs. swapout use, though... Sure, who said that ? :) The current discussion between Mike/Jonathan and me is about the aging issue. > > > The another problem is that don't limit the writeout in the VM. > > This is a big problem too, but also unrelated to the > impossibility of balancing cache vs. swap in the current > scheme. ... - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
On Wed, 30 May 2001, Marcelo Tosatti wrote: > The problem is that we allow _every_ task to age pages on the system > at the same time --- this is one of the things which is fucking up. This should not have any effect on the ratio of cache reclaiming vs. swapout use, though... > The another problem is that don't limit the writeout in the VM. This is a big problem too, but also unrelated to the impossibility of balancing cache vs. swap in the current scheme. regards, Rik -- Virtual memory is like a game you can't win; However, without VM there's truly nothing to lose... http://www.surriel.com/ http://distro.conectiva.com/ Send all your spam to [EMAIL PROTECTED] (spam digging piggy) - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
On Wed, 30 May 2001, Marcelo Tosatti wrote: > On Wed, 30 May 2001, Jonathan Morton wrote: > > > >The page aging logic does seems fragile as heck. You never know how > > >many folks are aging pages or at what rate. If aging happens too fast, > > >it defeats the garbage identification logic and you rape your cache. If > > >aging happens too slowly.. sigh. > > > > Then it sounds like the current algorithm is totally broken and needs > > replacement. If it's impossible to make a system stable by choosing the > > right numbers, the system needs changing, not the numbers. I think that's > > pretty much what we're being taught in Control Engineering. :) > > The problem is that we allow _every_ task to age pages on the system at > the same time --- this is one of the things which is fucking up. Yes. (I've been muttering/mumbling about this for... ever. look at the last patch I posted in this light.. make -j30 load:) > The another problem is that don't limit the writeout in the VM. And sometimes we don't start writing out soon enough. > We (me and Rik) are going to work on this later --- right now I'm busy > with the distribution release and Rik is travelling. Cool. -Mike - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
On Wed, 30 May 2001, Jonathan Morton wrote: > >The page aging logic does seems fragile as heck. You never know how > >many folks are aging pages or at what rate. If aging happens too fast, > >it defeats the garbage identification logic and you rape your cache. If > >aging happens too slowly.. sigh. > > Then it sounds like the current algorithm is totally broken and needs > replacement. If it's impossible to make a system stable by choosing the > right numbers, the system needs changing, not the numbers. I think that's > pretty much what we're being taught in Control Engineering. :) I wouldn't go so far as to say totally broken (mostly because I've tried like _hell_ to find a better way, and [despite minor successes] I've not been able to come up with something which covers all cases that even _I_ [hw tech] can think of well). Hard numbers are just plain bad, see below. I _will_ say that it is entirely too easy to break for comfort ;-) > Not having studied the code too closely, it sounds as though there are half > a dozen different "clocks" running for different types of memory, and each > one runs at a different speed and is updated at a different time. > Meanwhile, the paging-out is done on the assumption that all the clocks are > (at least roughly) in sync. Makes sense, right? (not!) No, I don't think that's the case at all. The individual zone balancing logic (or individual page [content] type logic) hasn't been inplimented yet (content type handling I don't think we want), but doesn't look like it'll be any trouble to do with the structures in place. That's just a fleshing out thing. The variations in aging rate is the most difficult problem I can see. IMHO, it needs to be either decoupled and done by an impartial bystander (tried that, ran into info flow troubles because of scheduling) or integrated tightly into the allocator proper (tried that.. interesting results but has problems of it's own wrt the massive changes in general strategy needed to make it work.. approaches rewrite) > I think it's worthwhile to think of the page/buffer caches as having a > working set of their own - if they are being heavily used, they should get > more memory than if they are only lightly used. The important point to get > right is to ensure that the "clocks" used for each memory area remain in > sync - they don't have to measure real time, just be consistent and fine > granularity. IMHO, the only thing of interest you can do with clocks is set your state sample rate. If state is changing rapidly, you must sample rapidly. As far as corrections go, you can only insert a corrective vector into the mix and then see if the sum induced the desired change in direction. The correct magnitude of this vector is not even possible to know.. that's what makes it so darn hard [defining numerical goals is utterly bogus]. > I'm working on some relatively small changes to vmscan.c which should help > improve the behaviour without upsetting the balance too much. Watch this > space... With much interest :) -Mike - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
On Wed, 30 May 2001, Jonathan Morton wrote: > >The page aging logic does seems fragile as heck. You never know how > >many folks are aging pages or at what rate. If aging happens too fast, > >it defeats the garbage identification logic and you rape your cache. If > >aging happens too slowly.. sigh. > > Then it sounds like the current algorithm is totally broken and needs > replacement. If it's impossible to make a system stable by choosing the > right numbers, the system needs changing, not the numbers. I think that's > pretty much what we're being taught in Control Engineering. :) The problem is that we allow _every_ task to age pages on the system at the same time --- this is one of the things which is fucking up. The another problem is that don't limit the writeout in the VM. We (me and Rik) are going to work on this later --- right now I'm busy with the distribution release and Rik is travelling. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
>The page aging logic does seems fragile as heck. You never know how >many folks are aging pages or at what rate. If aging happens too fast, >it defeats the garbage identification logic and you rape your cache. If >aging happens too slowly.. sigh. Then it sounds like the current algorithm is totally broken and needs replacement. If it's impossible to make a system stable by choosing the right numbers, the system needs changing, not the numbers. I think that's pretty much what we're being taught in Control Engineering. :) Not having studied the code too closely, it sounds as though there are half a dozen different "clocks" running for different types of memory, and each one runs at a different speed and is updated at a different time. Meanwhile, the paging-out is done on the assumption that all the clocks are (at least roughly) in sync. Makes sense, right? (not!) I think it's worthwhile to think of the page/buffer caches as having a working set of their own - if they are being heavily used, they should get more memory than if they are only lightly used. The important point to get right is to ensure that the "clocks" used for each memory area remain in sync - they don't have to measure real time, just be consistent and fine granularity. I'm working on some relatively small changes to vmscan.c which should help improve the behaviour without upsetting the balance too much. Watch this space... -- from: Jonathan "Chromatix" Morton mail: [EMAIL PROTECTED] (not for attachments) big-mail: [EMAIL PROTECTED] uni-mail: [EMAIL PROTECTED] The key to knowledge is not to rely on people to teach you it. Get VNC Server for Macintosh from http://www.chromatix.uklinux.net/vnc/ -BEGIN GEEK CODE BLOCK- Version 3.12 GCS$/E/S dpu(!) s:- a20 C+++ UL++ P L+++ E W+ N- o? K? w--- O-- M++$ V? PS PE- Y+ PGP++ t- 5- X- R !tv b++ DI+++ D G e+ h+ r++ y+(*) -END GEEK CODE BLOCK- - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
On Tue, 29 May 2001, Craig Kulesa wrote: > Mike Galbraith ([EMAIL PROTECTED]) wrote: > > > > Emphatic yes. We went from cache collapse to cache bloat. > > Rik, I think Mike deserves his beer. ;) :) ... > So is there an ideal VM balance for everyone? I have found that low-RAM (I seriously doubt it) > systems seem to benefit from being on the "cache-collapse" side of the > curve (so I prefer the pre-2.4.5 balance more than Mike probably does) and I hate both bad behaviors equally. "cache bloat" hurts more people than "cache collapse" does though because it shows under light load. > those low-RAM systems are the first hit when, as now, we're favoring > "cache bloat". Should balance behaviors could be altered by the user > (via sysctl's maybe? Yeah, I hear the cringes)? Or better, is it > possible to dynamically choose where the watermarks in balancing should > lie, and alter them automatically? 2.5 stuff there, no doubt. Balancing > seems so *fragile* (to me). The page aging logic does seems fragile as heck. You never know how many folks are aging pages or at what rate. If aging happens too fast, it defeats the garbage identification logic and you rape your cache. If aging happens too slowly.. sigh. -Mike - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
On Tue, 29 May 2001, Craig Kulesa wrote: Mike Galbraith ([EMAIL PROTECTED]) wrote: Emphatic yes. We went from cache collapse to cache bloat. Rik, I think Mike deserves his beer. ;) :) ... So is there an ideal VM balance for everyone? I have found that low-RAM (I seriously doubt it) systems seem to benefit from being on the cache-collapse side of the curve (so I prefer the pre-2.4.5 balance more than Mike probably does) and I hate both bad behaviors equally. cache bloat hurts more people than cache collapse does though because it shows under light load. those low-RAM systems are the first hit when, as now, we're favoring cache bloat. Should balance behaviors could be altered by the user (via sysctl's maybe? Yeah, I hear the cringes)? Or better, is it possible to dynamically choose where the watermarks in balancing should lie, and alter them automatically? 2.5 stuff there, no doubt. Balancing seems so *fragile* (to me). The page aging logic does seems fragile as heck. You never know how many folks are aging pages or at what rate. If aging happens too fast, it defeats the garbage identification logic and you rape your cache. If aging happens too slowly.. sigh. -Mike - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
On Wed, 30 May 2001, Jonathan Morton wrote: The page aging logic does seems fragile as heck. You never know how many folks are aging pages or at what rate. If aging happens too fast, it defeats the garbage identification logic and you rape your cache. If aging happens too slowly.. sigh. Then it sounds like the current algorithm is totally broken and needs replacement. If it's impossible to make a system stable by choosing the right numbers, the system needs changing, not the numbers. I think that's pretty much what we're being taught in Control Engineering. :) The problem is that we allow _every_ task to age pages on the system at the same time --- this is one of the things which is fucking up. The another problem is that don't limit the writeout in the VM. We (me and Rik) are going to work on this later --- right now I'm busy with the distribution release and Rik is travelling. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
On Wed, 30 May 2001, Jonathan Morton wrote: The page aging logic does seems fragile as heck. You never know how many folks are aging pages or at what rate. If aging happens too fast, it defeats the garbage identification logic and you rape your cache. If aging happens too slowly.. sigh. Then it sounds like the current algorithm is totally broken and needs replacement. If it's impossible to make a system stable by choosing the right numbers, the system needs changing, not the numbers. I think that's pretty much what we're being taught in Control Engineering. :) I wouldn't go so far as to say totally broken (mostly because I've tried like _hell_ to find a better way, and [despite minor successes] I've not been able to come up with something which covers all cases that even _I_ [hw tech] can think of well). Hard numbers are just plain bad, see below. I _will_ say that it is entirely too easy to break for comfort ;-) Not having studied the code too closely, it sounds as though there are half a dozen different clocks running for different types of memory, and each one runs at a different speed and is updated at a different time. Meanwhile, the paging-out is done on the assumption that all the clocks are (at least roughly) in sync. Makes sense, right? (not!) No, I don't think that's the case at all. The individual zone balancing logic (or individual page [content] type logic) hasn't been inplimented yet (content type handling I don't think we want), but doesn't look like it'll be any trouble to do with the structures in place. That's just a fleshing out thing. The variations in aging rate is the most difficult problem I can see. IMHO, it needs to be either decoupled and done by an impartial bystander (tried that, ran into info flow troubles because of scheduling) or integrated tightly into the allocator proper (tried that.. interesting results but has problems of it's own wrt the massive changes in general strategy needed to make it work.. approaches rewrite) I think it's worthwhile to think of the page/buffer caches as having a working set of their own - if they are being heavily used, they should get more memory than if they are only lightly used. The important point to get right is to ensure that the clocks used for each memory area remain in sync - they don't have to measure real time, just be consistent and fine granularity. IMHO, the only thing of interest you can do with clocks is set your state sample rate. If state is changing rapidly, you must sample rapidly. As far as corrections go, you can only insert a corrective vector into the mix and then see if the sum induced the desired change in direction. The correct magnitude of this vector is not even possible to know.. that's what makes it so darn hard [defining numerical goals is utterly bogus]. I'm working on some relatively small changes to vmscan.c which should help improve the behaviour without upsetting the balance too much. Watch this space... With much interest :) -Mike - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
On Wed, 30 May 2001, Marcelo Tosatti wrote: On Wed, 30 May 2001, Jonathan Morton wrote: The page aging logic does seems fragile as heck. You never know how many folks are aging pages or at what rate. If aging happens too fast, it defeats the garbage identification logic and you rape your cache. If aging happens too slowly.. sigh. Then it sounds like the current algorithm is totally broken and needs replacement. If it's impossible to make a system stable by choosing the right numbers, the system needs changing, not the numbers. I think that's pretty much what we're being taught in Control Engineering. :) The problem is that we allow _every_ task to age pages on the system at the same time --- this is one of the things which is fucking up. Yes. (I've been muttering/mumbling about this for... ever. look at the last patch I posted in this light.. make -j30 load:) The another problem is that don't limit the writeout in the VM. And sometimes we don't start writing out soon enough. We (me and Rik) are going to work on this later --- right now I'm busy with the distribution release and Rik is travelling. Cool. -Mike - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
On Wed, 30 May 2001, Marcelo Tosatti wrote: The problem is that we allow _every_ task to age pages on the system at the same time --- this is one of the things which is fucking up. This should not have any effect on the ratio of cache reclaiming vs. swapout use, though... The another problem is that don't limit the writeout in the VM. This is a big problem too, but also unrelated to the impossibility of balancing cache vs. swap in the current scheme. regards, Rik -- Virtual memory is like a game you can't win; However, without VM there's truly nothing to lose... http://www.surriel.com/ http://distro.conectiva.com/ Send all your spam to [EMAIL PROTECTED] (spam digging piggy) - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
On Wed, 30 May 2001, Rik van Riel wrote: On Wed, 30 May 2001, Marcelo Tosatti wrote: The problem is that we allow _every_ task to age pages on the system at the same time --- this is one of the things which is fucking up. This should not have any effect on the ratio of cache reclaiming vs. swapout use, though... Sure, who said that ? :) The current discussion between Mike/Jonathan and me is about the aging issue. The another problem is that don't limit the writeout in the VM. This is a big problem too, but also unrelated to the impossibility of balancing cache vs. swap in the current scheme. ... - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
On Wed, 30 May 2001, Mike Galbraith wrote: I wouldn't go so far as to say totally broken (mostly because I've tried like _hell_ to find a better way, and [despite minor successes] I've not been able to come up with something which covers all cases that even _I_ [hw tech] can think of well). The easy way out seems to be physical - virtual page reverse mappings, these make it trivial to apply balanced pressure on all pages. The downside of this measure is that it costs additional overhead and can up to double the amount of memory we take in with page tables. Of course, this amount is only prohibitive if the amount of page table memory was also prohibitively large in the first place, but ... :) regards, Rik -- Virtual memory is like a game you can't win; However, without VM there's truly nothing to lose... http://www.surriel.com/ http://distro.conectiva.com/ Send all your spam to [EMAIL PROTECTED] (spam digging piggy) - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
On Wed, 30 May 2001, Rik van Riel wrote: On Wed, 30 May 2001, Marcelo Tosatti wrote: The problem is that we allow _every_ task to age pages on the system at the same time --- this is one of the things which is fucking up. This should not have any effect on the ratio of cache reclaiming vs. swapout use, though... It shouldn't.. but when many tasks are aging, it does. Excluding these guys certainly seems to make a difference. (could be seeing something else and interpreting it wrong...) -Mike - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
On Wed, 30 May 2001, Mike Galbraith wrote: On Wed, 30 May 2001, Rik van Riel wrote: On Wed, 30 May 2001, Marcelo Tosatti wrote: The problem is that we allow _every_ task to age pages on the system at the same time --- this is one of the things which is fucking up. This should not have any effect on the ratio of cache reclaiming vs. swapout use, though... It shouldn't.. but when many tasks are aging, it does. What Rik means is that they are independant problems. Excluding these guys certainly seems to make a difference. Sure, those guys are going to help kswapd to unmap pte's and allocate swap space. Now even if only kswapd does this job (meaning a sane amount of cache reclaims/swapouts), you still have to deal with the reclaim/swapout tradeoff. See? - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM... (and 2.4.5-ac3)
Ronald Bultje writes: On 30 May 2001 14:58:57 -0500, Vincent Stemen wrote: There was a new 8139too driver added to the the 2.4.5 (I think) kernel which Alan Cox took back out and reverted to the old one in his 2.4.5-ac? versions because it is apparently causing lockups. Shouldn't this new driver have been released in a 2.5.x development kernel and proven there before replacing the one in the production kernel? I haven't even seen a 2.5.x kernel released yet. If every driver has to go thorugh the complete development cycle (of 2+ years), I'm sure very little driver writers will be as motivated as they are now - it takes ages before they see their efforts rewarded with a place in the kernel. The ideal case is that odd-numbered kernels are for testing and even-numbered kernels are stable. However, this is only theory. In practice, you can't rule out all bugs. And you can't test all things for all cases and every test case, the linux community doesn't have the manpower for that. And to prevent a complete driver development cycle taking 2+ years, you have to compromise. If you would take 2+ years for a single driver development cycle, nobody would be interested in linux since the new devices would only be supported by a stable kernel two years after their release. See the point? To prevent that, you need to compromise. and thus, sometimes, you have some crashes. I agree with everything you say up till this point, but you are arguing against a point I never made. First of all, bugs like the 8139too lockup was found within the first day or two of release in the 2.4.3 kernel. Also, most show stopper bugs such as the VM problems are found fairly quickly. Even if it takes a long time to figure out how to fix them, I do not think they should be pushed on through into production kernels until they are until they are fixed. I already said that I do not expect minor bugs not to slip through. However, if they are minor, they can usually be fixed quickly once they are discovered and it is no big deal if they make it into a production kernel. That's why there's still 2.2.x - that's purely stable and won't crash as fast as 2.4.x, but misses the newest cutting-edge-technology device support and newest technology (like new SMP handling , ReiserFS, etc... But it *is* stable. The reason I suggested more frequent major production releases is so that you don't have to go back to a 2 or 3 year old kernel and loose out on years worth of new features to have any stability. One show stopper bug like the VM problems would not be as much of a problem if there was a stable production kernel that we could run that was only 4 or 6 months old. Based on Linus's original very good plan for even/odd numbers, there should not have been 2.4.0-test? kernels either. This was another example of the rush to increment to 2.4 long before it was ready. There was a long stretch of test kernels and and now we are all the way to 2.4.5 and it is still not stable. We are repeating the 2.2.x process all over again. Wrong again. 2.3.x is for development, adding new things, testing, adding, testing, changing, testing, etc. Which is the same point I made. 2.4-test is for testing only, it's some sort of feature freeze. Agreed. My only point here was that it suggests that there are only minor bugs left to be solved before the production release by setting the version to 2.4-test. That is one of the reasons I made the suggestion to keep it in the 2.3 range, since there were actually serious VM problems still upon the production 2.4 release. 2.4.x is for final/stable 2.4. It's a standard *nix development cycle. That's how everyone does it. My point exactly. Regards, Ronald Bultje - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM... (and 2.4.5-ac3)
On Wednesday 30 May 2001 15:30, Rik van Riel wrote: On Wed, 30 May 2001, Vincent Stemen wrote: The problem is, that's not true. These problems are not slipping through because of lack of testers. As Alan said, the VM problem has been lurking, which means that it was known already. Fully agreed, it went through because of a lack of hours per day and the fact that the priority of developers was elsewhere. For me, for example, the priorities have mostly been with bugs that bothered me or that bothered Conectiva's customers. If you _really_ feel this strongly about the bug, you could either try to increase the number of hours a day for all of I sure wish I could :-). us or you could talk to my boss about hiring me as a consultant to fix the problem for you on an emergency basis :) The other two alternatives would be either waiting until somebody gets around to fixing the bug or sending in a patch yourself. Trying to piss off developers has adverse effect on all four of the methods above :) Why should my comments piss anybody off? I am just trying to point out a problem, as I see it, an offer suggestions for improvement. Other developers will either agree with me or they wont. Contributions are not made only through writing code. I contribute through code, bug reports, ideas, and suggestions. I would love to dive in and try to help fix some of the kernel problems but my hands are just to full right now. My comments are not meant to rush anybody and I am not criticizing how long it is taking. I know everybody is doing everything they can just like I am, and they are doing a terrific job. I am just suggesting a modification to the way the kernels are distributed that is more like the early versions that I hoped would allow us to maintain a stable kernel for distributions and production machines. - Vincent Stemen - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
On Wed, 30 May 2001, Marcelo Tosatti wrote: On Wed, 30 May 2001, Mike Galbraith wrote: On Wed, 30 May 2001, Rik van Riel wrote: On Wed, 30 May 2001, Marcelo Tosatti wrote: The problem is that we allow _every_ task to age pages on the system at the same time --- this is one of the things which is fucking up. This should not have any effect on the ratio of cache reclaiming vs. swapout use, though... It shouldn't.. but when many tasks are aging, it does. What Rik means is that they are independant problems. Ok. Excluding these guys certainly seems to make a difference. Sure, those guys are going to help kswapd to unmap pte's and allocate swap space. Now even if only kswapd does this job (meaning a sane amount of cache reclaims/swapouts), you still have to deal with the reclaim/swapout tradeoff. See? Yes. -Mike - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM... (and 2.4.5-ac3)
On Wednesday 30 May 2001 15:17, Mike Galbraith wrote: On Wed, 30 May 2001, Vincent Stemen wrote: On Wednesday 30 May 2001 01:02, Mike Galbraith wrote: On Tue, 29 May 2001, Vincent Stemen wrote: On Tuesday 29 May 2001 15:16, Alan Cox wrote: a reasonably stable release until 2.2.12. I do not understand why code with such serious reproducible problems is being introduced into the even numbered kernels. What happened to the plan to use only the Who said it was introduced ?? It was more 'lurking' than introduced. And unfortunately nobody really pinned it down in 2.4test. I fail to see the distinction. First of all, why was it ever released as 2.4-test? That question should probably be directed at Linus. If it is not fully tested, then it should be released it as an odd number. If it already existed in the odd numbered development kernel and was known, then it should have never been released as a production kernel until it was resolved. Otherwise, it completely defeats the purpose of having the even/odd numbering system. I do not expect bugs to never slip through to production kernels, but known bugs that are not trivial should not, and serious bugs like these VM problems especially should not. And you can help prevent them from slipping through by signing up as a shake and bake tester. Indeed, you can make your expectations reality absolutely free of charge, microfont and or compensation /microfont what a bargain! X ___ ;-) -Mike The problem is, that's not true. These problems are not slipping through because of lack of testers. As Alan said, the VM problem has Sorry, that's a copout. You (we) had many chances to notice. Don't push the problems back onto developers.. it's our problem. How is that a copout? The problem was noticed. I am only suggesting that we not be in such a hurry to put code in the production kernels until we are pretty sure it works well enough, and that we release major production versions more often so that they do not contain 2 or 3 years worth of new code making it so hard to debug. We probably should have had 2 or 3 code freezes and production releases since 2.2.x. As I mentioned in a previous posting, this way we do not have to run a 2 or 3 year old kernel in order to have reasonable stability. Here are some of the problems I see: There was far to long of a stretch with to much code dumped into both the 2.2 and 2.4 kernels before release. There needs to be a smaller number changes between major releases so that they can be more thoroughly tested and debugged. In the race to get it out there they are making the same mistakes as Microsoft, releasing production kernels with known serious bugs because it is taking to long and they want to move on forward. I enjoy criticizing Microsoft so much for the same thing that I do not want to have to stop in order to not sound hypocritical :-). The Linux community has built a lot of it's reputation on not making these mistakes. Please lets try not to destroy that. They are disregarding the even/odd versioning system. For example: There was a new 8139too driver added to the the 2.4.5 (I think) kernel which Alan Cox took back out and reverted to the old one in his 2.4.5-ac? versions because it is apparently causing lockups. Shouldn't this new driver have been released in a 2.5.x development kernel and proven there before replacing the one in the production kernel? I haven't even seen a 2.5.x kernel released yet. Based on Linus's original very good plan for even/odd numbers, there should not have been 2.4.0-test? kernels either. This was another example of the rush to increment to 2.4 long before it was ready. There was a long stretch of test kernels and and now we are all the way to 2.4.5 and it is still not stable. We are repeating the 2.2.x process all over again. It should have been 2.3.x until the production release was ready. If they needed to distinguish a code freeze for final testing, it could be done with a 4th version component (2.3.xx.xx), where the 4 component is incremented for final bug fixes. Sorry, I disagree with every last bit. Either you accept a situation or you try to do something about it. -Mike I am spending a lot of time testing new kernels, reporting bugs and offering suggestions that I think may improve on the stability of production kernels. Is this not considered doing something about it? It is necessary to point out where one sees a problem in order to offer possible solutions for improvement. - Vincent - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM... (and 2.4.5-ac3)
On Wed, 30 May 2001, Vincent Stemen wrote: The problem is, that's not true. These problems are not slipping through because of lack of testers. As Alan said, the VM problem has been lurking, which means that it was known already. Fully agreed, it went through because of a lack of hours per day and the fact that the priority of developers was elsewhere. For me, for example, the priorities have mostly been with bugs that bothered me or that bothered Conectiva's customers. If you _really_ feel this strongly about the bug, you could either try to increase the number of hours a day for all of us or you could talk to my boss about hiring me as a consultant to fix the problem for you on an emergency basis :) The other two alternatives would be either waiting until somebody gets around to fixing the bug or sending in a patch yourself. Trying to piss off developers has adverse effect on all four of the methods above :) Rik -- Virtual memory is like a game you can't win; However, without VM there's truly nothing to lose... http://www.surriel.com/ http://distro.conectiva.com/ Send all your spam to [EMAIL PROTECTED] (spam digging piggy) - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM... (and 2.4.5-ac3)
On Wed, 30 May 2001, Vincent Stemen wrote: On Wednesday 30 May 2001 15:17, Mike Galbraith wrote: On Wed, 30 May 2001, Vincent Stemen wrote: On Wednesday 30 May 2001 01:02, Mike Galbraith wrote: On Tue, 29 May 2001, Vincent Stemen wrote: On Tuesday 29 May 2001 15:16, Alan Cox wrote: a reasonably stable release until 2.2.12. I do not understand why code with such serious reproducible problems is being introduced into the even numbered kernels. What happened to the plan to use only the Who said it was introduced ?? It was more 'lurking' than introduced. And unfortunately nobody really pinned it down in 2.4test. I fail to see the distinction. First of all, why was it ever released as 2.4-test? That question should probably be directed at Linus. If it is not fully tested, then it should be released it as an odd number. If it already existed in the odd numbered development kernel and was known, then it should have never been released as a production kernel until it was resolved. Otherwise, it completely defeats the purpose of having the even/odd numbering system. I do not expect bugs to never slip through to production kernels, but known bugs that are not trivial should not, and serious bugs like these VM problems especially should not. And you can help prevent them from slipping through by signing up as a shake and bake tester. Indeed, you can make your expectations reality absolutely free of charge, microfont and or compensation /microfont what a bargain! X ___ ;-) -Mike The problem is, that's not true. These problems are not slipping through because of lack of testers. As Alan said, the VM problem has Sorry, that's a copout. You (we) had many chances to notice. Don't push the problems back onto developers.. it's our problem. How is that a copout? The problem was noticed. I am only suggesting that we not be in such a hurry to put code in the production kernels until we are pretty sure it works well enough, and that we release major production versions more often so that they do not contain 2 or 3 years worth of new code making it so hard to debug. We probably should have had 2 or 3 code freezes and production releases since 2.2.x. As I mentioned in a previous posting, this way we do not have to run a 2 or 3 year old kernel in order to have reasonable stability. I don't think you or I can do a better job of release management than Linus and friends, so there's no point in us discussing it. If you want to tell Linus, Alan et al how to do it 'right', you go do that. -Mike - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM... (and 2.4.5-ac3)
On Wednesday 30 May 2001 01:02, Mike Galbraith wrote: On Tue, 29 May 2001, Vincent Stemen wrote: On Tuesday 29 May 2001 15:16, Alan Cox wrote: a reasonably stable release until 2.2.12. I do not understand why code with such serious reproducible problems is being introduced into the even numbered kernels. What happened to the plan to use only the Who said it was introduced ?? It was more 'lurking' than introduced. And unfortunately nobody really pinned it down in 2.4test. I fail to see the distinction. First of all, why was it ever released as 2.4-test? That question should probably be directed at Linus. If it is not fully tested, then it should be released it as an odd number. If it already existed in the odd numbered development kernel and was known, then it should have never been released as a production kernel until it was resolved. Otherwise, it completely defeats the purpose of having the even/odd numbering system. I do not expect bugs to never slip through to production kernels, but known bugs that are not trivial should not, and serious bugs like these VM problems especially should not. And you can help prevent them from slipping through by signing up as a shake and bake tester. Indeed, you can make your expectations reality absolutely free of charge, microfont and or compensation /microfont what a bargain! X ___ ;-) -Mike The problem is, that's not true. These problems are not slipping through because of lack of testers. As Alan said, the VM problem has been lurking, which means that it was known already. We currently have no development/production kernel distinction and I have not been able to find even one stable 2.4.x version to run on our main machines. Reverting back to 2.2.x is a real pain because of all the surrounding changes which will affect our initscripts and other system configuration issues, such as Unix98 pty's, proc filesystem differences, device numbering, etc. I have the greatest respect and appreciation for Linus, Alan, and all of the other kernel developers. My comments are not meant to criticize, but rather to point out some the problems I see that are making it so difficult to stabilize the kernel and encourage them to steer back on track. Here are some of the problems I see: There was far to long of a stretch with to much code dumped into both the 2.2 and 2.4 kernels before release. There needs to be a smaller number changes between major releases so that they can be more thoroughly tested and debugged. In the race to get it out there they are making the same mistakes as Microsoft, releasing production kernels with known serious bugs because it is taking to long and they want to move on forward. I enjoy criticizing Microsoft so much for the same thing that I do not want to have to stop in order to not sound hypocritical :-). The Linux community has built a lot of it's reputation on not making these mistakes. Please lets try not to destroy that. They are disregarding the even/odd versioning system. For example: There was a new 8139too driver added to the the 2.4.5 (I think) kernel which Alan Cox took back out and reverted to the old one in his 2.4.5-ac? versions because it is apparently causing lockups. Shouldn't this new driver have been released in a 2.5.x development kernel and proven there before replacing the one in the production kernel? I haven't even seen a 2.5.x kernel released yet. Based on Linus's original very good plan for even/odd numbers, there should not have been 2.4.0-test? kernels either. This was another example of the rush to increment to 2.4 long before it was ready. There was a long stretch of test kernels and and now we are all the way to 2.4.5 and it is still not stable. We are repeating the 2.2.x process all over again. It should have been 2.3.x until the production release was ready. If they needed to distinguish a code freeze for final testing, it could be done with a 4th version component (2.3.xx.xx), where the 4 component is incremented for final bug fixes. - Vincent Stemen - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM... (and 2.4.5-ac3)
There was a new 8139too driver added to the the 2.4.5 (I think) kernel which Alan Cox took back out and reverted to the old one in his 2.4.5-ac? versions because it is apparently causing lockups. Shouldn't this new driver have been released in a 2.5.x development kernel and proven there before replacing the one in the production kernel? I haven't even seen a 2.5.x kernel released yet. Nope. The 2.4.3 one is buggy too - but differently (and it turns out a little less) buggy. Welcome to software. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM... (and 2.4.5-ac3)
On Tue, 29 May 2001, Vincent Stemen wrote: > On Tuesday 29 May 2001 15:16, Alan Cox wrote: > > > a reasonably stable release until 2.2.12. I do not understand why > > > code with such serious reproducible problems is being introduced into > > > the even numbered kernels. What happened to the plan to use only the > > > > Who said it was introduced ?? It was more 'lurking' than introduced. And > > unfortunately nobody really pinned it down in 2.4test. > > > > I fail to see the distinction. First of all, why was it ever released > as 2.4-test? That question should probably be directed at Linus. If > it is not fully tested, then it should be released it as an odd > number. If it already existed in the odd numbered development kernel > and was known, then it should have never been released as a production > kernel until it was resolved. Otherwise, it completely defeats the > purpose of having the even/odd numbering system. > > I do not expect bugs to never slip through to production kernels, but > known bugs that are not trivial should not, and serious bugs like > these VM problems especially should not. And you can help prevent them from slipping through by signing up as a shake and bake tester. Indeed, you can make your expectations reality absolutely free of charge, and or compensation what a bargain! X ___ ;-) -Mike - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
Mike Galbraith ([EMAIL PROTECTED]) wrote: > > Emphatic yes. We went from cache collapse to cache bloat. Rik, I think Mike deserves his beer. ;) Agreed. Swap reclaim doesn't seem to be the root issue here, IMHO. But instead: his box was capable of maintaining a modest cache and the desired user processes without massive allocations (and use) of swap space. There was plenty of cache to reap, but VM decided to swapout instead. Seems we're out of balance here (IMHO). I see this too, and it's only a symptom of post-2.4.4 kernels. Example: on a 128M system w/2.4.5, loading X and a simulation code of RSS=70M causes the system to drop 40-50M into swap...with 100M of cache sitting there, and some of those cache pages are fairly old. It's not just allocation; there is noticable disk activity associated with paging that causes a lag in interactivity. In 2.4.4, there is no swap activity at all. And if the application causes heavy I/O *and* memory load (think StarOffice, or Quake 3), this situation gets even worse (because there's typically more competition/demand for cache pages). And on low-memory systems (ex. 32M), even a basic Web browsing test w/ Opera drops the 2.4.5 system 25M into swap where 2.4.4 barely cracks 5 MB on the same test (and the interactivity shows). This is all independent of swap reclaim. So is there an ideal VM balance for everyone? I have found that low-RAM systems seem to benefit from being on the "cache-collapse" side of the curve (so I prefer the pre-2.4.5 balance more than Mike probably does) and those low-RAM systems are the first hit when, as now, we're favoring "cache bloat". Should balance behaviors could be altered by the user (via sysctl's maybe? Yeah, I hear the cringes)? Or better, is it possible to dynamically choose where the watermarks in balancing should lie, and alter them automatically? 2.5 stuff there, no doubt. Balancing seems so *fragile* (to me). Cheers, Craig Kulesa [EMAIL PROTECTED] - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM... (and 2.4.5-ac3)
> a reasonably stable release until 2.2.12. I do not understand why > code with such serious reproducible problems is being introduced into > the even numbered kernels. What happened to the plan to use only the Who said it was introduced ?? It was more 'lurking' than introduced. And unfortunately nobody really pinned it down in 2.4test. > By the way, The 2.4.5-ac3 kernel still fills swap and runs out of > memory during my morning NFS incremental backup. I got this message > in the syslog. 2.4.5-ac doesn't do some of the write throttling. Thats one thing I'm still working out. Linus 2.4.5 does write throttling but Im not convinced its done the right way > completely full. By that time the memory was in a reasonable state > but the swap space is still never being released. It wont be, its copied of memory already in apps. Linus said 2.4.0 would need more swap than ram when he put out 2.4.0. Alan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM... (and 2.4.5-ac3)
On Tuesday 29 May 2001 10:37, elko wrote: > On Tuesday 29 May 2001 11:10, Alan Cox wrote: > > > It's not a bug. It's a feature. It only breaks systems that are run > > > w= ith "too > > > little" swap, and the only difference from 2.2 till now is, that the > > > de= finition > > > of "too little" changed. > > > > its a giant bug. Or do you want to add 128Gb of unused swap to a full > > kitted out Xeon box - or 512Gb to a big athlon ??? > > this bug is biting me too and I do NOT like it ! > > if it's a *giant* bug, then why is LK-2.4 called a *stable* kernel ?? This has been my complaint ever since the 2.2.0 kernel. I did not see a reasonably stable release until 2.2.12. I do not understand why code with such serious reproducible problems is being introduced into the even numbered kernels. What happened to the plan to use only the odd numbered kernels for debugging and refinement of the code? I never said anything because I thought the the kernel developers would eventually get back on track after the mistakes of the 2.2.x kernels but it has been years now and it still has not happened. I do not wish sound un-appreciative to those that have put so much wonderful work into the Linux kernel but this is the same thing we criticize Microsoft for. Putting out production code that obviously is not ready. Please lets not earn the same reputation of such commercial companies. By the way, The 2.4.5-ac3 kernel still fills swap and runs out of memory during my morning NFS incremental backup. I got this message in the syslog. May 29 06:39:15 (none) kernel: Out of Memory: Killed process 23502 (xteevee). For some reason xteevee is commonly the process that gets killed. My understanding is that it is part of Xscreensaver, but it was during my backup. This was the output of 'free' after I got up and found the swap completely full. By that time the memory was in a reasonable state but the swap space is still never being released. total used free sharedbuffers cached Mem:255960 220668 35292292 110960 80124 -/+ buffers/cache: 29584 226376 Swap:40124 40112 12 Configuration - AMD K6-2/450 256Mb RAM 2.4.5-ac3 Kernel compiled with egcs-1.1.2. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM...
On Tuesday 29 May 2001 11:10, Alan Cox wrote: > > It's not a bug. It's a feature. It only breaks systems that are run w= > > ith "too > > little" swap, and the only difference from 2.2 till now is, that the de= > > finition > > of "too little" changed. > > its a giant bug. Or do you want to add 128Gb of unused swap to a full > kitted out Xeon box - or 512Gb to a big athlon ??? this bug is biting me too and I do NOT like it ! if it's a *giant* bug, then why is LK-2.4 called a *stable* kernel ?? -- Elko Holl - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM...
> It's not a bug. It's a feature. It only breaks systems that are run w= > ith "too > little" swap, and the only difference from 2.2 till now is, that the de= > finition > of "too little" changed. its a giant bug. Or do you want to add 128Gb of unused swap to a full kitted out Xeon box - or 512Gb to a big athlon ??? - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM...
> Ouch! When compiling MySql, building sql_yacc.cc results in a ~300M > cc1plus process size. Unfortunately this leads the machine with 380M of > RAM deeply into swap: > > Mem: 381608K av, 248504K used, 133104K free, 0K shrd, 192K > buff > Swap: 255608K av, 255608K used, 0K free 215744K > cached That is supposed to hapen. The pages are existing both in swap and memory but not recovered. In that state the VM hasn't even broken yet. Where you hit a problem is that the 255Mb of stuff both in memory and swap won't be flushed from swap when you need more swap space. That is a giant size special edition stupid design flaw that is on the VM hackers list. But there are only a finite number of patches you can do in a day, and things like sucking completely came first I believe - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM...
On Tuesday 29 May 2001 11:10, Alan Cox wrote: It's not a bug. It's a feature. It only breaks systems that are run w= ith too little swap, and the only difference from 2.2 till now is, that the de= finition of too little changed. its a giant bug. Or do you want to add 128Gb of unused swap to a full kitted out Xeon box - or 512Gb to a big athlon ??? this bug is biting me too and I do NOT like it ! if it's a *giant* bug, then why is LK-2.4 called a *stable* kernel ?? -- Elko Holl - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM... (and 2.4.5-ac3)
On Tuesday 29 May 2001 10:37, elko wrote: On Tuesday 29 May 2001 11:10, Alan Cox wrote: It's not a bug. It's a feature. It only breaks systems that are run w= ith too little swap, and the only difference from 2.2 till now is, that the de= finition of too little changed. its a giant bug. Or do you want to add 128Gb of unused swap to a full kitted out Xeon box - or 512Gb to a big athlon ??? this bug is biting me too and I do NOT like it ! if it's a *giant* bug, then why is LK-2.4 called a *stable* kernel ?? This has been my complaint ever since the 2.2.0 kernel. I did not see a reasonably stable release until 2.2.12. I do not understand why code with such serious reproducible problems is being introduced into the even numbered kernels. What happened to the plan to use only the odd numbered kernels for debugging and refinement of the code? I never said anything because I thought the the kernel developers would eventually get back on track after the mistakes of the 2.2.x kernels but it has been years now and it still has not happened. I do not wish sound un-appreciative to those that have put so much wonderful work into the Linux kernel but this is the same thing we criticize Microsoft for. Putting out production code that obviously is not ready. Please lets not earn the same reputation of such commercial companies. By the way, The 2.4.5-ac3 kernel still fills swap and runs out of memory during my morning NFS incremental backup. I got this message in the syslog. May 29 06:39:15 (none) kernel: Out of Memory: Killed process 23502 (xteevee). For some reason xteevee is commonly the process that gets killed. My understanding is that it is part of Xscreensaver, but it was during my backup. This was the output of 'free' after I got up and found the swap completely full. By that time the memory was in a reasonable state but the swap space is still never being released. total used free sharedbuffers cached Mem:255960 220668 35292292 110960 80124 -/+ buffers/cache: 29584 226376 Swap:40124 40112 12 Configuration - AMD K6-2/450 256Mb RAM 2.4.5-ac3 Kernel compiled with egcs-1.1.2. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM
Mike Galbraith ([EMAIL PROTECTED]) wrote: Emphatic yes. We went from cache collapse to cache bloat. Rik, I think Mike deserves his beer. ;) Agreed. Swap reclaim doesn't seem to be the root issue here, IMHO. But instead: his box was capable of maintaining a modest cache and the desired user processes without massive allocations (and use) of swap space. There was plenty of cache to reap, but VM decided to swapout instead. Seems we're out of balance here (IMHO). I see this too, and it's only a symptom of post-2.4.4 kernels. Example: on a 128M system w/2.4.5, loading X and a simulation code of RSS=70M causes the system to drop 40-50M into swap...with 100M of cache sitting there, and some of those cache pages are fairly old. It's not just allocation; there is noticable disk activity associated with paging that causes a lag in interactivity. In 2.4.4, there is no swap activity at all. And if the application causes heavy I/O *and* memory load (think StarOffice, or Quake 3), this situation gets even worse (because there's typically more competition/demand for cache pages). And on low-memory systems (ex. 32M), even a basic Web browsing test w/ Opera drops the 2.4.5 system 25M into swap where 2.4.4 barely cracks 5 MB on the same test (and the interactivity shows). This is all independent of swap reclaim. So is there an ideal VM balance for everyone? I have found that low-RAM systems seem to benefit from being on the cache-collapse side of the curve (so I prefer the pre-2.4.5 balance more than Mike probably does) and those low-RAM systems are the first hit when, as now, we're favoring cache bloat. Should balance behaviors could be altered by the user (via sysctl's maybe? Yeah, I hear the cringes)? Or better, is it possible to dynamically choose where the watermarks in balancing should lie, and alter them automatically? 2.5 stuff there, no doubt. Balancing seems so *fragile* (to me). Cheers, Craig Kulesa [EMAIL PROTECTED] - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM...
Ouch! When compiling MySql, building sql_yacc.cc results in a ~300M cc1plus process size. Unfortunately this leads the machine with 380M of RAM deeply into swap: Mem: 381608K av, 248504K used, 133104K free, 0K shrd, 192K buff Swap: 255608K av, 255608K used, 0K free 215744K cached That is supposed to hapen. The pages are existing both in swap and memory but not recovered. In that state the VM hasn't even broken yet. Where you hit a problem is that the 255Mb of stuff both in memory and swap won't be flushed from swap when you need more swap space. That is a giant size special edition stupid design flaw that is on the VM hackers list. But there are only a finite number of patches you can do in a day, and things like sucking completely came first I believe - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM... (and 2.4.5-ac3)
a reasonably stable release until 2.2.12. I do not understand why code with such serious reproducible problems is being introduced into the even numbered kernels. What happened to the plan to use only the Who said it was introduced ?? It was more 'lurking' than introduced. And unfortunately nobody really pinned it down in 2.4test. By the way, The 2.4.5-ac3 kernel still fills swap and runs out of memory during my morning NFS incremental backup. I got this message in the syslog. 2.4.5-ac doesn't do some of the write throttling. Thats one thing I'm still working out. Linus 2.4.5 does write throttling but Im not convinced its done the right way completely full. By that time the memory was in a reasonable state but the swap space is still never being released. It wont be, its copied of memory already in apps. Linus said 2.4.0 would need more swap than ram when he put out 2.4.0. Alan - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM...
It's not a bug. It's a feature. It only breaks systems that are run w= ith too little swap, and the only difference from 2.2 till now is, that the de= finition of too little changed. its a giant bug. Or do you want to add 128Gb of unused swap to a full kitted out Xeon box - or 512Gb to a big athlon ??? - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM... (and 2.4.5-ac3)
On Tue, 29 May 2001, Vincent Stemen wrote: On Tuesday 29 May 2001 15:16, Alan Cox wrote: a reasonably stable release until 2.2.12. I do not understand why code with such serious reproducible problems is being introduced into the even numbered kernels. What happened to the plan to use only the Who said it was introduced ?? It was more 'lurking' than introduced. And unfortunately nobody really pinned it down in 2.4test. I fail to see the distinction. First of all, why was it ever released as 2.4-test? That question should probably be directed at Linus. If it is not fully tested, then it should be released it as an odd number. If it already existed in the odd numbered development kernel and was known, then it should have never been released as a production kernel until it was resolved. Otherwise, it completely defeats the purpose of having the even/odd numbering system. I do not expect bugs to never slip through to production kernels, but known bugs that are not trivial should not, and serious bugs like these VM problems especially should not. And you can help prevent them from slipping through by signing up as a shake and bake tester. Indeed, you can make your expectations reality absolutely free of charge, microfont and or compensation /microfont what a bargain! X ___ ;-) -Mike - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM...
On Tue, 29 May 2001, Jeff Garzik wrote: > > On Tuesday 29 May 2001 00:10, Jakob Østergaard wrote: > > > > > > > Mem: 381608K av, 248504K used, 133104K free, 0K shrd, 192K > > > > > buff > > > > > Swap: 255608K av, 255608K used, 0K free 215744K > > > > > cached > > > > > > > > > > Vanilla 2.4.5 VM. > > > > > It's not a bug. It's a feature. It only breaks systems that are run with > > > "too little" swap, and the only difference from 2.2 till now is, that the > > > definition of "too little" changed. > > I am surprised as many people as this are missing, > > * when you have an active process using ~300M of VM, in a ~380M machine, > 2/3 of the machine's RAM should -not- be soaked up by cache Emphatic yes. We went from cache collapse to cache bloat. IMHO, the bugfix for collapse exposed other problems. I posted a patch which I believe demonstrated that pretty well. (i also bet Rik a virtual beer that folks would knock on his mailbox when 2.4.5 was released. please cc him somebody.. i want my brewski;) -Mike - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM...
> On Tuesday 29 May 2001 00:10, Jakob Østergaard wrote: > > > > > Mem: 381608K av, 248504K used, 133104K free, 0K shrd, 192K > > > > buff > > > > Swap: 255608K av, 255608K used, 0K free 215744K > > > > cached > > > > > > > > Vanilla 2.4.5 VM. > > > It's not a bug. It's a feature. It only breaks systems that are run with > > "too little" swap, and the only difference from 2.2 till now is, that the > > definition of "too little" changed. I am surprised as many people as this are missing, * when you have an active process using ~300M of VM, in a ~380M machine, 2/3 of the machine's RAM should -not- be soaked up by cache * when you have an active process using ~300M of VM, in a ~380M machine, swap should not be full while there is 133M of RAM available. The above quoted is top output, taken during the several minutes where cc1plus process was ~300M in size. Similar numbers existed before and after my cut-n-paste, so this was not transient behavior. I can assure you, these are bugs not features :) -- Jeff Garzik | Disbelief, that's why you fail. Building 1024| MandrakeSoft | - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM...
On Tue, May 29, 2001 at 01:46:28PM +0900, G. Hugh Song wrote:
> Jakob,
>
> My Alpha has 2GB of physical memory. In this case how much swap space
> should
> I assign in these days of kernel 2.4.*? I had had trouble with 1GB of
> swap space
> before switching back to 2.2.20pre2aa1.
If you run a single mingetty and bash session, you need no swap.
If you run four 1GB processes concurrently, I would use ~5-6G of swap to be on
the safe side.
Swap is very cheap, even if measured in gigabytes. Go with the sum of the
largest process foot-prints you can imagine running on your system, and then
add some. Be generous. It's not like unused swap space is going to slow the
system down - it's a nice extra little safety to have. It's beyond me why
anyone would run a system with marginal swap.
On a compile box here with 392 MB physical, I have 900 MB swap. This
accomodates multiple concurrent 100-300 MB compile jobs. Never had a problem.
Oh, and I didn't have to change my swap setup between 2.2 and 2.4.
--
: [EMAIL PROTECTED] : And I see the elder races, :
:.: putrid forms of man:
: Jakob Østergaard : See him rise and claim the earth, :
:OZ9ABN : his downfall is at hand. :
:.:{Konkhra}...:
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM...
Jakob, My Alpha has 2GB of physical memory. In this case how much swap space should I assign in these days of kernel 2.4.*? I had had trouble with 1GB of swap space before switching back to 2.2.20pre2aa1. Thanks -- G. Hugh Song - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM...
On Tue, May 29, 2001 at 11:32:09AM +0900, G. Hugh Song wrote:
>
> Jeff Garzik wrote:
> >
> > Ouch! When compiling MySql, building sql_yacc.cc results in a ~300M
> > cc1plus process size. Unfortunately this leads the machine with 380M of
> > RAM deeply into swap:
> >
> > Mem: 381608K av, 248504K used, 133104K free, 0K shrd, 192K
> > buff
> > Swap: 255608K av, 255608K used, 0K free 215744K
> > cached
> >
> > Vanilla 2.4.5 VM.
> >
>
> This bug known as the swap-reclaim bug has been there for a while since
> around 2.4.4. Rick van Riel said that it is in the TO-DO list.
> Because of this, I went back to 2.2.20pre2aa1 on UP2000 SMP.
>
> IMHO, the current 2.4.* kernels should still be 2.3.*. When this bug
> is removed, I will come back to 2.4.*.
Just keep enough swap around. How hard can that be ?
Really, it's not like a memory leak or something. It's just "late reclaim".
If Linux didn't do over-commit, you wouldn't have been able to run that job
anyway.
It's not a bug. It's a feature. It only breaks systems that are run with "too
little" swap, and the only difference from 2.2 till now is, that the definition
of "too little" changed.
--
: [EMAIL PROTECTED] : And I see the elder races, :
:.: putrid forms of man:
: Jakob Østergaard : See him rise and claim the earth, :
:OZ9ABN : his downfall is at hand. :
:.:{Konkhra}...:
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM...
Jeff Garzik wrote: > > Ouch! When compiling MySql, building sql_yacc.cc results in a ~300M > cc1plus process size. Unfortunately this leads the machine with 380M of > RAM deeply into swap: > > Mem: 381608K av, 248504K used, 133104K free, 0K shrd, 192K > buff > Swap: 255608K av, 255608K used, 0K free 215744K > cached > > Vanilla 2.4.5 VM. > This bug known as the swap-reclaim bug has been there for a while since around 2.4.4. Rick van Riel said that it is in the TO-DO list. Because of this, I went back to 2.2.20pre2aa1 on UP2000 SMP. IMHO, the current 2.4.* kernels should still be 2.3.*. When this bug is removed, I will come back to 2.4.*. Regards, Hugh - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM...
Jeff Garzik wrote: > > Ouch! When compiling MySql, building sql_yacc.cc results in a ~300M > cc1plus process size. Unfortunately this leads the machine with 380M of > RAM deeply into swap: > > Mem: 381608K av, 248504K used, 133104K free, 0K shrd, 192K > buff > Swap: 255608K av, 255608K used, 0K free 215744K > cached > > Vanilla 2.4.5 VM. > Sorry. I just looked at your numbers again and saw you have 133 MB of real ram free. Is this during compile? -- = Mohammad A. Haque http://www.haque.net/ [EMAIL PROTECTED] "Alcohol and calculus don't mix. Project Lead Don't drink and derive." --Unknown http://wm.themes.org/ [EMAIL PROTECTED] = - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Plain 2.4.5 VM...
Ouch! When compiling MySql, building sql_yacc.cc results in a ~300M cc1plus process size. Unfortunately this leads the machine with 380M of RAM deeply into swap: Mem: 381608K av, 248504K used, 133104K free, 0K shrd, 192K buff Swap: 255608K av, 255608K used, 0K free 215744K cached Vanilla 2.4.5 VM. -- Jeff Garzik | Disbelief, that's why you fail. Building 1024| MandrakeSoft | - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Plain 2.4.5 VM...
Ouch! When compiling MySql, building sql_yacc.cc results in a ~300M cc1plus process size. Unfortunately this leads the machine with 380M of RAM deeply into swap: Mem: 381608K av, 248504K used, 133104K free, 0K shrd, 192K buff Swap: 255608K av, 255608K used, 0K free 215744K cached Vanilla 2.4.5 VM. -- Jeff Garzik | Disbelief, that's why you fail. Building 1024| MandrakeSoft | - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM...
Jeff Garzik wrote: Ouch! When compiling MySql, building sql_yacc.cc results in a ~300M cc1plus process size. Unfortunately this leads the machine with 380M of RAM deeply into swap: Mem: 381608K av, 248504K used, 133104K free, 0K shrd, 192K buff Swap: 255608K av, 255608K used, 0K free 215744K cached Vanilla 2.4.5 VM. Sorry. I just looked at your numbers again and saw you have 133 MB of real ram free. Is this during compile? -- = Mohammad A. Haque http://www.haque.net/ [EMAIL PROTECTED] Alcohol and calculus don't mix. Project Lead Don't drink and derive. --Unknown http://wm.themes.org/ [EMAIL PROTECTED] = - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM...
Jeff Garzik wrote: Ouch! When compiling MySql, building sql_yacc.cc results in a ~300M cc1plus process size. Unfortunately this leads the machine with 380M of RAM deeply into swap: Mem: 381608K av, 248504K used, 133104K free, 0K shrd, 192K buff Swap: 255608K av, 255608K used, 0K free 215744K cached Vanilla 2.4.5 VM. This bug known as the swap-reclaim bug has been there for a while since around 2.4.4. Rick van Riel said that it is in the TO-DO list. Because of this, I went back to 2.2.20pre2aa1 on UP2000 SMP. IMHO, the current 2.4.* kernels should still be 2.3.*. When this bug is removed, I will come back to 2.4.*. Regards, Hugh - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM...
On Tue, May 29, 2001 at 11:32:09AM +0900, G. Hugh Song wrote:
Jeff Garzik wrote:
Ouch! When compiling MySql, building sql_yacc.cc results in a ~300M
cc1plus process size. Unfortunately this leads the machine with 380M of
RAM deeply into swap:
Mem: 381608K av, 248504K used, 133104K free, 0K shrd, 192K
buff
Swap: 255608K av, 255608K used, 0K free 215744K
cached
Vanilla 2.4.5 VM.
This bug known as the swap-reclaim bug has been there for a while since
around 2.4.4. Rick van Riel said that it is in the TO-DO list.
Because of this, I went back to 2.2.20pre2aa1 on UP2000 SMP.
IMHO, the current 2.4.* kernels should still be 2.3.*. When this bug
is removed, I will come back to 2.4.*.
Just keep enough swap around. How hard can that be ?
Really, it's not like a memory leak or something. It's just late reclaim.
If Linux didn't do over-commit, you wouldn't have been able to run that job
anyway.
It's not a bug. It's a feature. It only breaks systems that are run with too
little swap, and the only difference from 2.2 till now is, that the definition
of too little changed.
--
: [EMAIL PROTECTED] : And I see the elder races, :
:.: putrid forms of man:
: Jakob Østergaard : See him rise and claim the earth, :
:OZ9ABN : his downfall is at hand. :
:.:{Konkhra}...:
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM...
Jakob, My Alpha has 2GB of physical memory. In this case how much swap space should I assign in these days of kernel 2.4.*? I had had trouble with 1GB of swap space before switching back to 2.2.20pre2aa1. Thanks -- G. Hugh Song - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Plain 2.4.5 VM...
On Tue, May 29, 2001 at 01:46:28PM +0900, G. Hugh Song wrote:
Jakob,
My Alpha has 2GB of physical memory. In this case how much swap space
should
I assign in these days of kernel 2.4.*? I had had trouble with 1GB of
swap space
before switching back to 2.2.20pre2aa1.
If you run a single mingetty and bash session, you need no swap.
If you run four 1GB processes concurrently, I would use ~5-6G of swap to be on
the safe side.
Swap is very cheap, even if measured in gigabytes. Go with the sum of the
largest process foot-prints you can imagine running on your system, and then
add some. Be generous. It's not like unused swap space is going to slow the
system down - it's a nice extra little safety to have. It's beyond me why
anyone would run a system with marginal swap.
On a compile box here with 392 MB physical, I have 900 MB swap. This
accomodates multiple concurrent 100-300 MB compile jobs. Never had a problem.
Oh, and I didn't have to change my swap setup between 2.2 and 2.4.
--
: [EMAIL PROTECTED] : And I see the elder races, :
:.: putrid forms of man:
: Jakob Østergaard : See him rise and claim the earth, :
:OZ9ABN : his downfall is at hand. :
:.:{Konkhra}...:
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
