Re: kernel: page allocation failure recommendations
A lot more functionality is ascribed to swappiness than it deserves. I'd be thinking this is more an issue in the buddy allocator. The slub allocator is much better at handling things in more recent kernels. Should be the default IMHO. Shane ... On Thu, 2010-08-05 at 11:38 -0600, Mark Post wrote: > >>> On 8/5/2010 at 11:36 AM, Marcy Cortes > >>> wrote: > > Decrease to 0 is what I've been advocating under VM. Keeps linux from doing > > preemptive moves to swap. Just let VM see the page is being not used and he > > will page it out. > > Probably not what you want when your Linux system is having memory > fragmentation problems. -- For LINUX-390 subscribe / signoff / archive access instructions, send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For more information on Linux on System z, visit http://wiki.linuxvm.org/
Re: kernel: page allocation failure recommendations
>>> On 8/5/2010 at 11:36 AM, Marcy Cortes >>> wrote: > Decrease to 0 is what I've been advocating under VM. Keeps linux from doing > preemptive moves to swap. Just let VM see the page is being not used and he > will page it out. Probably not what you want when your Linux system is having memory fragmentation problems. Mark Post -- For LINUX-390 subscribe / signoff / archive access instructions, send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For more information on Linux on System z, visit http://wiki.linuxvm.org/
Re: kernel: page allocation failure recommendations
Decrease to 0 is what I've been advocating under VM. Keeps linux from doing preemptive moves to swap. Just let VM see the page is being not used and he will page it out. Marcy. Sent from my BlackBerry. - Original Message - From: Linux on 390 Port To: LINUX-390@vm.marist.edu Sent: Thu Aug 05 10:32:10 2010 Subject: Re: [LINUX-390] kernel: page allocation failure recommendations Thanks to all that responded. I appreciate everyone's responsiveness. I increased vm.swappiness from 60 to 80. It is too early to tell if it made any significant difference. We have also accelerated our plans to move forward with our SLE-10-s390x-SP3 production implementation. Peter Shane Ginnane Sent by: Linux on 390 Port 08/04/2010 10:10 PM Please respond to Linux on 390 Port To LINUX-390@vm.marist.edu cc Subject Re: kernel: page allocation failure recommendations I quite liked this from one of the kernel devs: But please bear in mind, this "page allocation failure" message is purely a developer diagnostic thing. The reason it is there is so that if some random toaster driver oopses over a failure to handle an allocation failure, the person who reports the bug can say "I saw an allocation failure and then your driver crashed". Which tells the driver developer where to look. Although recoverable, I'd be looking at adding storage to the guest. I can't see swappiness helping in this context - but I don't see it harming either, and I generally agree with the sentiment that it should be lower in a z/ environment. Getting up to date on the kernel would also be a good idea as Mark suggested - lots of reports on late 2.5 and early 2.6 kernels. Shane ... (interestingly I don't see this post on the maillist archive) Linux on 390 Port wrote on 05/08/2010 02:56:54 AM: > We are running the following Linux: > > Linux linuxp01 2.6.5-7.315-s390x #1 SMP Wed Nov 26 13:03:18 UTC 2008 > s390x s390x s390x GNU/Linux with SLES-9-s390x-SP4 + "online updates" > under /VM Version 5 Release 4.0, service level 0902 (64-bit). Linux > is defined with 768MB of memory. > > We recently implemented CICS Transaction Gateway version 6.0.1 fix > pack and since then are seeing some page fault errors. However, I am > not really seeing memory shortage issues with the z/VM monitor or > the Linux free ?m errors. I am thinking about lowering vm.swappiness > from 60. What does everyone think, is that worth doing? Any other > recommendations? > > We are limited on memory resources but if push came to shove, I > could increase from 768MB to 1024MB and increase the swap space accordingly. > > I know, I know, all this stuff is old, but we are in the process of upgrading. This Email message and any attachment may contain information that is proprietary, legally privileged, confidential and/or subject to copyright belonging to Pepco Holdings, Inc. or its affiliates ("PHI"). This Email is intended solely for the use of the person(s) to which it is addressed. If you are not an intended recipient, or the employee or agent responsible for delivery of this Email to the intended recipient(s), you are hereby notified that any dissemination, distribution or copying of this Email is strictly prohibited. If you have received this message in error, please immediately notify the sender and permanently delete this Email and any copies. PHI policies expressly prohibit employees from making defamatory or offensive statements and infringing any copyright or any other legal right by Email communication. PHI will not accept any liability in respect of such communications. -- For LINUX-390 subscribe / signoff / archive access instructions, send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For more information on Linux on System z, visit http://wiki.linuxvm.org/
Re: kernel: page allocation failure recommendations
Thanks to all that responded. I appreciate everyone's responsiveness. I increased vm.swappiness from 60 to 80. It is too early to tell if it made any significant difference. We have also accelerated our plans to move forward with our SLE-10-s390x-SP3 production implementation. Peter Shane Ginnane Sent by: Linux on 390 Port 08/04/2010 10:10 PM Please respond to Linux on 390 Port To LINUX-390@vm.marist.edu cc Subject Re: kernel: page allocation failure recommendations I quite liked this from one of the kernel devs: But please bear in mind, this "page allocation failure" message is purely a developer diagnostic thing. The reason it is there is so that if some random toaster driver oopses over a failure to handle an allocation failure, the person who reports the bug can say "I saw an allocation failure and then your driver crashed". Which tells the driver developer where to look. Although recoverable, I'd be looking at adding storage to the guest. I can't see swappiness helping in this context - but I don't see it harming either, and I generally agree with the sentiment that it should be lower in a z/ environment. Getting up to date on the kernel would also be a good idea as Mark suggested - lots of reports on late 2.5 and early 2.6 kernels. Shane ... (interestingly I don't see this post on the maillist archive) Linux on 390 Port wrote on 05/08/2010 02:56:54 AM: > We are running the following Linux: > > Linux linuxp01 2.6.5-7.315-s390x #1 SMP Wed Nov 26 13:03:18 UTC 2008 > s390x s390x s390x GNU/Linux with SLES-9-s390x-SP4 + "online updates" > under /VM Version 5 Release 4.0, service level 0902 (64-bit). Linux > is defined with 768MB of memory. > > We recently implemented CICS Transaction Gateway version 6.0.1 fix > pack and since then are seeing some page fault errors. However, I am > not really seeing memory shortage issues with the z/VM monitor or > the Linux free ?m errors. I am thinking about lowering vm.swappiness > from 60. What does everyone think, is that worth doing? Any other > recommendations? > > We are limited on memory resources but if push came to shove, I > could increase from 768MB to 1024MB and increase the swap space accordingly. > > I know, I know, all this stuff is old, but we are in the process of upgrading. This Email message and any attachment may contain information that is proprietary, legally privileged, confidential and/or subject to copyright belonging to Pepco Holdings, Inc. or its affiliates ("PHI"). This Email is intended solely for the use of the person(s) to which it is addressed. If you are not an intended recipient, or the employee or agent responsible for delivery of this Email to the intended recipient(s), you are hereby notified that any dissemination, distribution or copying of this Email is strictly prohibited. If you have received this message in error, please immediately notify the sender and permanently delete this Email and any copies. PHI policies expressly prohibit employees from making defamatory or offensive statements and infringing any copyright or any other legal right by Email communication. PHI will not accept any liability in respect of such communications. -- For LINUX-390 subscribe / signoff / archive access instructions, send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For more information on Linux on System z, visit http://wiki.linuxvm.org/
Re: kernel: page allocation failure recommendations
I quite liked this from one of the kernel devs: But please bear in mind, this "page allocation failure" message is purely a developer diagnostic thing. The reason it is there is so that if some random toaster driver oopses over a failure to handle an allocation failure, the person who reports the bug can say "I saw an allocation failure and then your driver crashed". Which tells the driver developer where to look. Although recoverable, I'd be looking at adding storage to the guest. I can't see swappiness helping in this context - but I don't see it harming either, and I generally agree with the sentiment that it should be lower in a z/ environment. Getting up to date on the kernel would also be a good idea as Mark suggested - lots of reports on late 2.5 and early 2.6 kernels. Shane ... (interestingly I don't see this post on the maillist archive) Linux on 390 Port wrote on 05/08/2010 02:56:54 AM: > We are running the following Linux: > > Linux linuxp01 2.6.5-7.315-s390x #1 SMP Wed Nov 26 13:03:18 UTC 2008 > s390x s390x s390x GNU/Linux with SLES-9-s390x-SP4 + "online updates" > under /VM Version 5 Release 4.0, service level 0902 (64-bit). Linux > is defined with 768MB of memory. > > We recently implemented CICS Transaction Gateway version 6.0.1 fix > pack and since then are seeing some page fault errors. However, I am > not really seeing memory shortage issues with the z/VM monitor or > the Linux free –m errors. I am thinking about lowering vm.swappiness > from 60. What does everyone think, is that worth doing? Any other > recommendations? > > We are limited on memory resources but if push came to shove, I > could increase from 768MB to 1024MB and increase the swap space accordingly. > > I know, I know, all this stuff is old, but we are in the process of upgrading.
Re: kernel: page allocation failure recommendations
>>> On 8/4/2010 at 12:56 PM, "Peter E. Abresch Jr. - at Pepco" wrote: > We are running the following Linux: > > Linux linuxp01 2.6.5-7.315-s390x #1 SMP Wed Nov 26 13:03:18 UTC 2008 s390x > s390x s390x GNU/Linux with SLES-9-s390x-SP4 + "online updates" under /VM > Version 5 Release 4.0, service level 0902 (64-bit). Linux is defined with > 768MB of memory. > > We recently implemented CICS Transaction Gateway version 6.0.1 fix pack > and since then are seeing some page fault errors. However, I am not really > seeing memory shortage issues with the z/VM monitor or the Linux free ?m > errors. I am thinking about lowering vm.swappiness from 60. What does > everyone think, is that worth doing? Any other recommendations? > > We are limited on memory resources but if push came to shove, I could > increase from 768MB to 1024MB and increase the swap space accordingly. > > I know, I know, all this stuff is old, but we are in the process of > upgrading. It's old even for a SLES9 SP4 system. There have been 6 kernel updates since 2.6.5-7.315 was issued two years ago. Some of those included memory management fixes. (Hint.) > I have attached a copy of the zipped log for review. Thanks as always. After applying this command to the unzipped file: iconv -f EBCDICUS -t US-ASCII -c messages.txt | sed -e "s/Aug /\nAug /g" | less I was able to look at the messages file. The problem you're seeing is a result of memory fragmentation, not a total shortage. Aug 2 01:14:47 linuxp01 kernel: sh: page allocation failure. order:0, mode:0x50 Aug 2 01:14:47 linuxp01 kernel: sh: page allocation failure. order:0, mode:0x50 Aug 2 01:14:47 linuxp01 kernel: sh: page allocation failure. order:0, mode:0x50 Aug 4 09:00:07 linuxp01 kernel: cclclnt: page allocation failure. order:4, mode:0xd0 The first three messages indicate that the kernel wasn't able to find a single 4K page free to satisfy the allocation request. I don't know that playing with the swappiness setting will help at all. I suspect not, but I'm not a kernel hacker. I think adding more virtual storage might delay the onset of the problem, but not necessarily eliminate it. At least give the 2.6.5-7.322 kernel a try. Mark Post -- For LINUX-390 subscribe / signoff / archive access instructions, send email to lists...@vm.marist.edu with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 -- For more information on Linux on System z, visit http://wiki.linuxvm.org/