I'm a bit confused here... Perrin, isn't what Jani is mentioning here exactly what Stas wanted to accomplish (well, one specific detail of what he wanted to accomplish) with mp2, with the specific result in mind of eliminating the common Apache 1 issue of using the 2 backend (mp/static) with a single front-end, or am I dreaming?
Issac Perrin Harkins wrote: > On 6/4/07, Jani M. <[EMAIL PROTECTED]> wrote: >> With prefork, running ~ 600 processes results in roughly 700MB of memory >> consumption. This obviously gives us 600 client "download slots", and >> 600 perl interpreters, and leaves some memory for other processes and >> disk caching. >> >> With the threaded worker mpm, I can run ~150 processes, each with 20 >> threads, and 3 perl interpreters. This will use about the same about of >> memory, so roughly 700MB. This in turn gives us 3000 client "download >> slots", but "only" 450 perl interpreters. > > There's no reason to run more interpreters under prefork. Don't make > your mod_perl server handle static files. Serve them with a separate > apache or some other proxy server. > > But I think the bigger picture that you're missing here is > copy-on-write sharing. When you fork processes, most of the memory is > actually shared internally by the virtual memory system, so it isn't > using physical RAM. You might have very large processes which only > use a few MBs of actual RAM each. Threads don't get this benefit. > They have to copy everything when a new thread is started and there is > no copy-on-write sharing. It results in some really large differences > in terms of physical RAM needed per perl interpreter. If you run 100 > interpreters with a threaded MPM and 100 with prefork, the prefork > ones will use a lot less of your real memory. > >> When most of the time the client uses is not spent inside the mod_perl >> handlers or filters, but rather downloading the actual content, >> 'PerlInterpScope handler' can be a lifesaver - a few interpreters can >> easily handle the load for a much higher number of client threads. > > That's what the proxy setup is for. The mod_perl setup just dumps the > file very quickly to the proxy, and the proxy deals with spooning it > out the clients. > >> I know what you mean, but the problem here is that this mod_perl server >> *is* the reverse proxy :) There are several backend servers which this >> server will both proxy and cache the content for - mod_perl is, putting >> it simply, just needed for additional intelligence. > > If what you're saying is that you can't separate out the mod_perl bits > with a proxy because they do things like authentication, you might be > interested in seeing what LiveJournal does with their proxy called > perlbal. They use a system of internal redirects to let mod_perl > handle auth functions and pass the file serving off to perlbal. You > can read about it in Brad Fitzpatrick's presentation (around silde > 45): > http://www.danga.com/words/2007_04_linuxfest_nw/linuxfest.pdf > > - Perrin
