Re: MPM sizing defaults and config
- Rainer Jung rainer.j...@kippdata.de wrote: OS2 ===conf default proposed StartServers 22 2 MinSpareThreads 55 5 MaxSpareThreads 10 10 20 MaxRequestsPerChild 01 0 Any comments? From docs/manual/upgrading.xml: + liPlatform support has been removed for BeOS, OS/2, TPF, and + even older platforms such as A/UX, Next, and Tandem. These were + believed to be broken anyway./li What piece am I missing? Regards, Rainer -- Igor Galić Tel: +43 (0) 699 122 96 338 Fax: +43(0) 1 91 333 41 Mail: i.ga...@brainsware.org URL: http://brainsware.org/
Re: MPM sizing defaults and config
On 21/06/10 09:20, Igor Galić wrote: - Rainer Jung rainer.j...@kippdata.de wrote: OS2 ===conf default proposed StartServers 22 2 MinSpareThreads 55 5 MaxSpareThreads 10 10 20 MaxRequestsPerChild 01 0 Any comments? From docs/manual/upgrading.xml: + liPlatform support has been removed for BeOS, OS/2, TPF, and + even older platforms such as A/UX, Next, and Tandem. These were + believed to be broken anyway./li What piece am I missing? The docs need updating. OS/2 support has been revived.
Re: MPM sizing defaults and config
On 21.06.2010 11:34, Brian Havard wrote: On 21/06/10 09:20, Igor Galić wrote: From docs/manual/upgrading.xml: +liPlatform support has been removed for BeOS, OS/2, TPF, and + even older platforms such as A/UX, Next, and Tandem. These were + believed to be broken anyway./li What piece am I missing? The docs need updating. OS/2 support has been revived. Fixed in r956524. Rainer
Re: MPM sizing defaults and config
On 16.06.2010 04:27, William A. Rowe Jr. wrote: On 6/15/2010 7:27 PM, Guenter Knauf wrote: Hi, Am 16.06.2010 00:37, schrieb William A. Rowe Jr.: Netware ===conf default proposed StartThreads250 50 50 MinSpareThreads 25 10 25 MaxSpareThreads 250 100100 MaxThreads 1000 2048 1000 MaxRequestsPerChild 00 0 ThreadStackSize 6553665536 65536 MaxMemFree 1000 - (remove) ThreadStackSize seems a bit dicey, would rather see 128k default. Because it is the only MPM with non system default used, I wondered whether there's a bit of history behind that value. I'm certain :) Guenter? Brad? well, I dont think a higher ThreadStackSize can hurt much beside higher memory usage (which shouldnt be an issue nowadays). I cant tell why we have 64k as default, but I can tell that this is 8 times as much as what our linker uses by default if you dont specify a stacksize, so brobably it was just a value which Brad thought would be enough when he ported 2.0 to NetWare ...; also IIRC 8k was way too less for modules like mod_rewrite ... - but everything else seems to be coded cleaner so that every greater memory block is aloocated rather than pushed on stack. Also I can add that for mysql I use 128k too since had probs with the testsuite where some tests crashed with 64k. So in general I consider a higher stackzise more safe for bad coding where hughe memory blocks are pushed on the stack. Ditto with win32, but ThreadStackSize should be expressing a MAX stack extent, not the base. Win32 will always be starting with 4kb minimum, but the ThreadStackSize constrains the number of threads which could be allocated based on max stacks. The Windows MPM uses ThreadStackSize for the reservation of stack memory (worker threads), it doesn't commit it. System default seems to be 1MB. Side comment: APR on Win instead seems to set the committed size. For Netware I can't find any documentation whether the argument to NXContextAlloc refers to stack reservation or actually allocation, whether to initial or to max. Could be some simple mode of a fixed stack size, already alloated when NXContextAlloc() is called. Not sure. For Unix we rely on APR, which uses pthreads and there it is the *minimum* stack size that's set (unclear whether only reserved or actually allocated). On Solaris (64Bit kernel) as an example, default values are 1MB for 32Bit processes and 2MB for 64Bit. Rainer
MPM sizing defaults and config
The default configuration for various MPMs differs from the example configuration file conf/extra/httpd-mpm.conf. Before bringing those two in sync, I want to propose the values we want to use as new defaults as well as for the extras configuration file. Prefork ===conf default proposed StartServers 55 5 MinSpareServers 55 3 MaxSpareServers 10 10 15 MaxClients 150 256256 MaxRequestsPerChild 01 0 Worker/Event conf default proposed StartServers 23 2 MinSpareThreads 25 75 25 MaxSpareThreads 75 250100 MaxClients 150 400400 ThreadsPerChild 25 25 25 MaxRequestsPerChild 01 0 Simple == conf default proposed SimpleProcCount 55 5? SimpleThreadCount 55 5? WinNT = conf default proposed ThreadsPerChild 150 64150 MaxRequestsPerChild 00 0 Netware ===conf default proposed StartThreads250 50 50 MinSpareThreads 25 10 25 MaxSpareThreads 250 100100 MaxThreads 1000 2048 1000 MaxRequestsPerChild 00 0 ThreadStackSize 6553665536 65536 MaxMemFree 1000 - (remove) OS2 ===conf default proposed StartServers 22 2 MinSpareThreads 55 5 MaxSpareThreads 10 10 20 MaxRequestsPerChild 01 0 Any comments? Regards, Rainer
Re: MPM sizing defaults and config
As a broad general question - why not equivalent number of MaxClients
across all MPMs?
On 6/15/2010 4:03 PM, Rainer Jung wrote:
The default configuration for various MPMs
differs from the example configuration file
conf/extra/httpd-mpm.conf.
Before bringing those two in sync, I want
to propose the values we want to use as new
defaults as well as for the extras configuration
file.
Prefork
===conf default proposed
StartServers 55 5
MinSpareServers 55 3
MaxSpareServers 10 10 15
MaxClients 150 256256
MaxRequestsPerChild 01 0
This seems bursty - can't we raise MaxSpare to at least 50?
Worker/Event
conf default proposed
StartServers 23 2
MinSpareThreads 25 75 25
MaxSpareThreads 75 250100
MaxClients 150 400400
ThreadsPerChild 25 25 25
MaxRequestsPerChild 01 0
If threaded, and stable, why not some 50 or 100 threads per child?
Simple
== conf default proposed
SimpleProcCount 55 5?
SimpleThreadCount 55 5?
WinNT
= conf default proposed
ThreadsPerChild 150 64150
MaxRequestsPerChild 00 0
+1
ThreadStackSize could probably be cut to 128k default (256k
processwide-default).
Netware
===conf default proposed
StartThreads250 50 50
MinSpareThreads 25 10 25
MaxSpareThreads 250 100100
MaxThreads 1000 2048 1000
MaxRequestsPerChild 00 0
ThreadStackSize 6553665536 65536
MaxMemFree 1000 - (remove)
ThreadStackSize seems a bit dicey, would rather see 128k default.
OS2
===conf default proposed
StartServers 22 2
MinSpareThreads 55 5
MaxSpareThreads 10 10 20
MaxRequestsPerChild 01 0
ThreadsPerChild?
Re: MPM sizing defaults and config
- William A. Rowe Jr. wr...@rowe-clan.net wrote:
As a broad general question - why not equivalent number of MaxClients
across all MPMs?
Because it might come expensive Memory-wise with some MPMs.
On 6/15/2010 4:03 PM, Rainer Jung wrote:
The default configuration for various MPMs
differs from the example configuration file
conf/extra/httpd-mpm.conf.
Before bringing those two in sync, I want
to propose the values we want to use as new
defaults as well as for the extras configuration
file.
Prefork
===conf default proposed
StartServers 55 5
MinSpareServers 55 3
MaxSpareServers 10 10 15
MaxClients 150 256256
MaxRequestsPerChild 01 0
This seems bursty - can't we raise MaxSpare to at least 50?
Not entirely sure how it makes sense to Start 5 but
only keep MinSpareServers 3, would you enlighten me please?
Worker/Event
conf default proposed
StartServers 23 2
MinSpareThreads 25 75 25
MaxSpareThreads 75 250100
MaxClients 150 400400
ThreadsPerChild 25 25 25
MaxRequestsPerChild 01 0
If threaded, and stable, why not some 50 or 100 threads per child?
For preforking/hybrinds I'd set StartServers/MinSpareServers
to $num CPU Cores (not threads!)
So long,
--
Igor Galić
Tel: +43 (0) 699 122 96 338
Fax: +43(0) 1 91 333 41
Mail: i.ga...@brainsware.org
URL: http://brainsware.org/
Re: MPM sizing defaults and config
On 15.06.2010 23:09, William A. Rowe Jr. wrote: As a broad general question - why not equivalent number of MaxClients across all MPMs? I was uncertain about that. Users often tend to try to fix performance problems by adding concurrency to the web server. If your web server is configured very tight, then this will help. But if you actually have a throughput problem (slow backends, network etc.), then allowing even more concurrency will make things worse. So I wanted to stick away from sizing to what's possible and stay closer to what's reasonable. Concerning Unix my gut feeling was that 256 processes (prefork) and 16x25=400 threads doesn't put much pressure on system resources of modern entry servers. I raised the MaxClients for the threaded MPMs, because I expect them to be a bit cheaper. But no strong opinion here about the best numbers. Keeping MaxClients in sync would make the MPMs better comparable out of the box, but on the other hand precisely this type of comparison might not make much sense. On 6/15/2010 4:03 PM, Rainer Jung wrote: The default configuration for various MPMs differs from the example configuration file conf/extra/httpd-mpm.conf. Before bringing those two in sync, I want to propose the values we want to use as new defaults as well as for the extras configuration file. Prefork ===conf default proposed StartServers 55 5 MinSpareServers 55 3 MaxSpareServers 10 10 15 MaxClients 150 256256 MaxRequestsPerChild 01 0 This seems bursty - can't we raise MaxSpare to at least 50? Right, we should in order to prevent continuous forking and dying. Worker/Event conf default proposed StartServers 23 2 MinSpareThreads 25 75 25 MaxSpareThreads 75 250100 MaxClients 150 400400 ThreadsPerChild 25 25 25 MaxRequestsPerChild 01 0 If threaded, and stable, why not some 50 or 100 threads per child? OK, especially since for Event I think that there is a problem of correctly balancing load between child processes. For worker it is clear, when a child is saturated, i.e. when each thread has a connection. For event it's quite possible, that the idleness of worker threads fluctuates. So bigger thread pools (per process) might reduce risk of starvation. Simple == conf default proposed SimpleProcCount 55 5? SimpleThreadCount 55 5? WinNT = conf default proposed ThreadsPerChild 150 64150 MaxRequestsPerChild 00 0 +1 ThreadStackSize could probably be cut to 128k default (256k processwide-default). Only the Netware MPM uses a thread stack size different from the OS default at the moment (OK, Windows uses hard-coded 65536 for the service thread and a stderr_thread but not for the worker threads). So do we want to now switch over to an httpd specific value or should we stick with system default? Netware ===conf default proposed StartThreads250 50 50 MinSpareThreads 25 10 25 MaxSpareThreads 250 100100 MaxThreads 1000 2048 1000 MaxRequestsPerChild 00 0 ThreadStackSize 6553665536 65536 MaxMemFree 1000 - (remove) ThreadStackSize seems a bit dicey, would rather see 128k default. Because it is the only MPM with non system default used, I wondered whether there's a bit of history behind that value. OS2 ===conf default proposed StartServers 22 2 MinSpareThreads 55 5 MaxSpareThreads 10 10 20 MaxRequestsPerChild 01 0 ThreadsPerChild? i read the code like this: number of threads per child is dynamic. Maximum is not configurable but instead defined by HARD_THREAD_LIMIT, which is set to 256 in the MPM source. Rainer
Re: MPM sizing defaults and config
On 6/15/2010 4:20 PM, Igor Galić wrote:
Worker/Event
conf default proposed
StartServers 23 2
MinSpareThreads 25 75 25
MaxSpareThreads 75 250100
MaxClients 150 400400
ThreadsPerChild 25 25 25
MaxRequestsPerChild 01 0
If threaded, and stable, why not some 50 or 100 threads per child?
For preforking/hybrinds I'd set StartServers/MinSpareServers
to $num CPU Cores (not threads!)
Servers? Threads can and do service different cpu's. Are you just
being optimistic that each process's listen queue will land on a different
CPU core, by happenstance?
Re: MPM sizing defaults and config
On 6/15/2010 4:42 PM, Rainer Jung wrote: On 15.06.2010 23:09, William A. Rowe Jr. wrote: As a broad general question - why not equivalent number of MaxClients across all MPMs? I was uncertain about that. Users often tend to try to fix performance problems by adding concurrency to the web server. If your web server is configured very tight, then this will help. But if you actually have a throughput problem (slow backends, network etc.), then allowing evengo more concurrency will make things worse. We don't disagree. So I wanted to stick away from sizing to what's possible and stay closer to what's reasonable. Concerning Unix my gut feeling was that 256 processes (prefork) and 16x25=400 threads doesn't put much pressure on system resources of modern entry servers. I raised the MaxClients for the threaded MPMs, because I expect them to be a bit cheaper. But no strong opinion here about the best numbers. Keeping MaxClients in sync would make the MPMs better comparable out of the box, but on the other hand precisely this type of comparison might not make much sense. Here again, keeping it tight is probably good. Threads do cost (stacks, fd's, lots of other resources) although they don't generally eat everything that a process would. None the less, if the 'out of the box' server should support 250 clients, that's what we aught to do. I'd hate people to jump from prefork to worker, for example, only to find their resource profile and concurrency is changed entirely under their nose. In fact my 'quick test' servers I hang onto all the time only have 32 workers each, for this very reason. I don't care to have them gobble resources and I rarely need more concurrency, myself. For event it's quite possible, that the idleness of worker threads fluctuates. So bigger thread pools (per process) might reduce risk of starvation. But that's true of prefork or worker as well, if they are blocked on something like CGI. Here again, a consistent MaxClients, with the user-predictable caviat that they might wish to raise this for those who have slow/blocking response content. Only the Netware MPM uses a thread stack size different from the OS default at the moment (OK, Windows uses hard-coded 65536 for the service thread and a stderr_thread but not for the worker threads). So do we want to now switch over to an httpd specific value or should we stick with system default? It does? I guess I never backported this from trunk, but on trunk it most certainly supports ThreadStackSize, and that number happens to default to 256k from the build schema (and yes, some trivial required threads get bumped down to a sensible 64k). Netware ===conf default proposed StartThreads250 50 50 MinSpareThreads 25 10 25 MaxSpareThreads 250 100100 MaxThreads 1000 2048 1000 MaxRequestsPerChild 00 0 ThreadStackSize 6553665536 65536 MaxMemFree 1000 - (remove) ThreadStackSize seems a bit dicey, would rather see 128k default. Because it is the only MPM with non system default used, I wondered whether there's a bit of history behind that value. I'm certain :) Guenter? Brad?
Re: MPM sizing defaults and config
Hi, Am 16.06.2010 00:37, schrieb William A. Rowe Jr.: Netware ===conf default proposed StartThreads250 50 50 MinSpareThreads 25 10 25 MaxSpareThreads 250 100100 MaxThreads 1000 2048 1000 MaxRequestsPerChild 00 0 ThreadStackSize 6553665536 65536 MaxMemFree 1000 - (remove) ThreadStackSize seems a bit dicey, would rather see 128k default. Because it is the only MPM with non system default used, I wondered whether there's a bit of history behind that value. I'm certain :) Guenter? Brad? well, I dont think a higher ThreadStackSize can hurt much beside higher memory usage (which shouldnt be an issue nowadays). I cant tell why we have 64k as default, but I can tell that this is 8 times as much as what our linker uses by default if you dont specify a stacksize, so brobably it was just a value which Brad thought would be enough when he ported 2.0 to NetWare ...; also IIRC 8k was way too less for modules like mod_rewrite ... - but everything else seems to be coded cleaner so that every greater memory block is aloocated rather than pushed on stack. Also I can add that for mysql I use 128k too since had probs with the testsuite where some tests crashed with 64k. So in general I consider a higher stackzise more safe for bad coding where hughe memory blocks are pushed on the stack. But lets also hear what Brad thinks - anyway its here only a config file change. Gün.
Re: MPM sizing defaults and config
On 6/15/2010 7:27 PM, Guenter Knauf wrote: Hi, Am 16.06.2010 00:37, schrieb William A. Rowe Jr.: Netware ===conf default proposed StartThreads250 50 50 MinSpareThreads 25 10 25 MaxSpareThreads 250 100100 MaxThreads 1000 2048 1000 MaxRequestsPerChild 00 0 ThreadStackSize 6553665536 65536 MaxMemFree 1000 - (remove) ThreadStackSize seems a bit dicey, would rather see 128k default. Because it is the only MPM with non system default used, I wondered whether there's a bit of history behind that value. I'm certain :) Guenter? Brad? well, I dont think a higher ThreadStackSize can hurt much beside higher memory usage (which shouldnt be an issue nowadays). I cant tell why we have 64k as default, but I can tell that this is 8 times as much as what our linker uses by default if you dont specify a stacksize, so brobably it was just a value which Brad thought would be enough when he ported 2.0 to NetWare ...; also IIRC 8k was way too less for modules like mod_rewrite ... - but everything else seems to be coded cleaner so that every greater memory block is aloocated rather than pushed on stack. Also I can add that for mysql I use 128k too since had probs with the testsuite where some tests crashed with 64k. So in general I consider a higher stackzise more safe for bad coding where hughe memory blocks are pushed on the stack. Ditto with win32, but ThreadStackSize should be expressing a MAX stack extent, not the base. Win32 will always be starting with 4kb minimum, but the ThreadStackSize constrains the number of threads which could be allocated based on max stacks.
