Re: [PATCHES] improve overcommit docs
That covers it extremely well. cheers andrew Tom Lane wrote: Andrew Dunstan <[EMAIL PROTECTED]> writes: At the time I wrote the original 2.6 was not out even in prerelease, which is why I was deliberately somewhat vague about it. It is still in prerelease, and it will in fact work slightly differently from what was in some 2.4 kernels - there are 2 settings that govern this instead of 1. Okay, I revised that section yet again based on this info: http://candle.pha.pa.us/main/writings/pgsql/sgml/kernel-resources.html#AEN17043 Thanks for the update. regards, tom lane ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] improve overcommit docs
Andrew Dunstan <[EMAIL PROTECTED]> writes: > At the time I wrote the original 2.6 was not out even in prerelease, > which is why I was deliberately somewhat vague about it. It is still in > prerelease, and it will in fact work slightly differently from what was > in some 2.4 kernels - there are 2 settings that govern this instead of > 1. Okay, I revised that section yet again based on this info: http://candle.pha.pa.us/main/writings/pgsql/sgml/kernel-resources.html#AEN17043 Thanks for the update. regards, tom lane ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PATCHES] improve overcommit docs
At the time I wrote the original 2.6 was not out even in prerelease, which is why I was deliberately somewhat vague about it. It is still in prerelease, and it will in fact work slightly differently from what was in some 2.4 kernels - there are 2 settings that govern this instead of 1. Here is the 2.6 description straight from linux-2.6.0-test9/Documentation/vm/overcommit-accounting: --- The Linux kernel supports three overcommit handling modes 0 - Heuristic overcommit handling. Obvious overcommits of address space are refused. Used for a typical system. It ensures a seriously wild allocation fails while allowing overcommit to reduce swap usage. root is allowed to allocate slighly more memory in this mode. This is the default. 1 - No overcommit handling. Appropriate for some scientific applications. 2 - (NEW) strict overcommit. The total address space commit for the system is not permitted to exceed swap + a configurable percentage (default is 50) of physical RAM. Depending on the percentage you use, in most situations this means a process will not be killed while accessing pages but will receive errors on memory allocation as appropriate. The overcommit policy is set via the sysctl `vm.overcommit_memory'. The overcommit percentage is set via `vm.overcommit_ratio'. - Also note that this is wrong for 2.4: run the machine out of memory. If your kernel supports the strict ! paranoid modes of overcommit handling, you can also relieve this There are 2 modes: strict (allow commit up to sizeof(swap plus 1/2 RAM) ) and paranoid (allow commit up to sizeof(swap) ). Wordsmith it however you like cheers andrew Neil Conway wrote: This patch makes some improvements to the section of the documentation that describes the Linux 2.4 memory overcommit behavior. I removed the almost content-free assertion that "You will need enough swap space to cover your memory needs." If this is intended to communicate anything meaningful, can someone rephrase it, please? This patch also includes a fix for a typo noticed by Robert Treat. Is this suitable for 7.4 (either the whole patch, or just the typo fix)? -Neil Index: doc/src/sgml/runtime.sgml === RCS file: /var/lib/cvs/pgsql-server/doc/src/sgml/runtime.sgml,v retrieving revision 1.218 diff -c -r1.218 runtime.sgml *** doc/src/sgml/runtime.sgml 14 Nov 2003 15:43:22 - 1.218 --- doc/src/sgml/runtime.sgml 16 Nov 2003 02:07:42 - *** *** 1294,1300 Unfortunately, there is no well-defined method for determining ideal values for the family of cost variables that ! below. You are encouraged to experiment and share your findings. --- 1294,1300 Unfortunately, there is no well-defined method for determining ideal values for the family of cost variables that ! appear below. You are encouraged to experiment and share your findings. *** *** 3267,3301 Linux Memory Overcommit ! Linux kernels of version 2.4.* have a poor default memory ! overcommit behavior, which can result in the PostgreSQL server ! (postmaster process) being killed by the ! kernel if the memory demands of another process cause the system ! to run out of memory. ! If this happens, you will see a kernel message looking like this ! (consult your system documentation and configuration on where to ! look for such a message): Out of Memory: Killed process 12345 (postmaster). ! And, of course, you will find that your database server has ! disappeared. To avoid this situation, run PostgreSQL on a machine where you can be sure that other processes will not run the machine out of memory. If your kernel supports the strict ! and/or paranoid modes of overcommit handling, you can also relieve ! this problem by altering the system's default behaviour. This can ! be determined by examining the function ! vm_enough_memory in the file mm/mmap.c ! in the kernel source. If this file reveals that the strict and/or ! paranoid modes are supported by your kernel, turn one of these ! modes on by using sysctl -w vm.overcommit_memory=2 --- 3267,3302 Linux Memory Overcommit ! In Linux 2.4, the default virtual memory configuration is not ! optimal for PostgreSQL. Because of the ! way that the kernel implements memory overcommit, the kernel may ! terminate the PostgreSQL server (the ! postmaster process) if the
Re: [PATCHES] improve overcommit docs
Neil Conway <[EMAIL PROTECTED]> writes: > This patch makes some improvements to the section of the documentation > that describes the Linux 2.4 memory overcommit behavior. Applied. I tweaked some of the wording a bit further. > I removed the almost content-free assertion that "You will need enough > swap space to cover your memory needs." If this is intended to > communicate anything meaningful, can someone rephrase it, please? Seemed pretty content-free to me too. regards, tom lane ---(end of broadcast)--- TIP 8: explain analyze is your friend
[PATCHES] improve overcommit docs
This patch makes some improvements to the section of the documentation that describes the Linux 2.4 memory overcommit behavior. I removed the almost content-free assertion that "You will need enough swap space to cover your memory needs." If this is intended to communicate anything meaningful, can someone rephrase it, please? This patch also includes a fix for a typo noticed by Robert Treat. Is this suitable for 7.4 (either the whole patch, or just the typo fix)? -Neil Index: doc/src/sgml/runtime.sgml === RCS file: /var/lib/cvs/pgsql-server/doc/src/sgml/runtime.sgml,v retrieving revision 1.218 diff -c -r1.218 runtime.sgml *** doc/src/sgml/runtime.sgml 14 Nov 2003 15:43:22 - 1.218 --- doc/src/sgml/runtime.sgml 16 Nov 2003 02:07:42 - *** *** 1294,1300 Unfortunately, there is no well-defined method for determining ideal values for the family of cost variables that ! below. You are encouraged to experiment and share your findings. --- 1294,1300 Unfortunately, there is no well-defined method for determining ideal values for the family of cost variables that ! appear below. You are encouraged to experiment and share your findings. *** *** 3267,3301 Linux Memory Overcommit ! Linux kernels of version 2.4.* have a poor default memory ! overcommit behavior, which can result in the PostgreSQL server ! (postmaster process) being killed by the ! kernel if the memory demands of another process cause the system ! to run out of memory. ! If this happens, you will see a kernel message looking like this ! (consult your system documentation and configuration on where to ! look for such a message): Out of Memory: Killed process 12345 (postmaster). ! And, of course, you will find that your database server has ! disappeared. To avoid this situation, run PostgreSQL on a machine where you can be sure that other processes will not run the machine out of memory. If your kernel supports the strict ! and/or paranoid modes of overcommit handling, you can also relieve ! this problem by altering the system's default behaviour. This can ! be determined by examining the function ! vm_enough_memory in the file mm/mmap.c ! in the kernel source. If this file reveals that the strict and/or ! paranoid modes are supported by your kernel, turn one of these ! modes on by using sysctl -w vm.overcommit_memory=2 --- 3267,3302 Linux Memory Overcommit ! In Linux 2.4, the default virtual memory configuration is not ! optimal for PostgreSQL. Because of the ! way that the kernel implements memory overcommit, the kernel may ! terminate the PostgreSQL server (the ! postmaster process) if the memory demands of ! another process cause the system to run out of virtual memory. ! If this happens, you will see a kernel message that looks like ! this (consult your system documentation and configuration on where ! to look for such a message): Out of Memory: Killed process 12345 (postmaster). ! This indicates that the postmaster process ! has been terminated due to memory pressure; ! PostgreSQL will need to be restarted. To avoid this situation, run PostgreSQL on a machine where you can be sure that other processes will not run the machine out of memory. If your kernel supports the strict ! paranoid modes of overcommit handling, you can also relieve this ! problem by altering the system's default behaviour. This can be ! determined by examining the function vm_enough_memory ! in the file mm/mmap.c in the kernel source. If this ! file reveals that the strict or paranoid modes are supported by ! your kernel, turn one of these modes on by using sysctl -w vm.overcommit_memory=2 *** *** 3304,3315 sysctl -w vm.overcommit_memory=3 for paranoid mode, or placing an equivalent entry in ! /etc/sysctl.conf. ! Using these settings in a kernel which does not support these modes will almost certainly increase the danger of the kernel killing the database server, rather than reducing it. If in any doubt, consult a kernel expert or your kernel vendor. --- 3305,3320 sysctl -w vm.overcommit_memory=3 for paranoid mode, or placing an equivalent entry in ! /etc/sysctl.conf. For more information on memory ! overcommit handling and the meaning of these kernel settings, ! consult the file ! Documentation/vm/overcommit-accounting in a ! recent version of the source code for the Linux kernel. ! Using these settings with a kernel which does not suppor
