Re: pre-loaded modules on Solaris
> At 06:11 AM 11/10/00 -0500, barries wrote: > >> Address Kbytes Resident Shared Private > >> -- -- --- > >> total Kb 24720 227203288 19432 <<< pre-loaded modules > >> total Kb 14592 1297630969880 <<< not pre-loaed modules. > > >Stupid question, probably, but when running the non-pre-loaded version, > >are you sure all the same modules are being loaded? > > Yes. According to perl-status, anyway. Some modules are loaded into the > parent, of course, because of mod_perl. But when not pre-loading I start > the server, look at perl-status and then made some requests and looked > again to see what was loaded. The difference is what modules I'm use'ing > in my test. > > >I'm wondering if there's > >some set of modules that, for some reason, isn't being loaded by the > >sequence of requests you're firing against all of your httpds to > >get the servers "warmed up" to represent real-life state. > > When looking at pmap it looks like the main difference in "private" memory > usage is in the heap. I'm not clear why the heap would end up so much > bigger when pre-loading modules. > > Unfortunately, Linux doesn't seem to have the same reporting abilities as > Solaris, but using /proc//statm to show shared and private memory > under these same test showed that pre-loading was a big win. So it seems > like a Solaris issue. Uhm, http://perl.apache.org/guide/performance.html#Sharing_Memory which uses extensively libgtop library, which is available for Solaris... > > > > > Bill Moseley > mailto:[EMAIL PROTECTED] > _ Stas Bekman JAm_pH -- Just Another mod_perl Hacker http://stason.org/ mod_perl Guide http://perl.apache.org/guide mailto:[EMAIL PROTECTED] http://apachetoday.com http://jazzvalley.com http://singlesheaven.com http://perlmonth.com perl.org apache.org
Re: pre-loaded modules on Solaris
On 11/10/00 10:15 AM, John Hughes wrote: > A guess: When pmap says "shared" it means stuff that realy shared. > I.E. it's not counting stuff that is marked copy-on-write, which is > shared UNTIL YOU TRY TO MODIFY IT. Is there any way to measure the size of the shared copy-on-write pages on Solaris? It seems insane that everyone just "assumes" that all this copy-on-write goodness is happening, despite (seemingly) no way to measure it. I'll admit, it's always been my assumption too, but I'd really like some hard numbers to back me up. -John
RE: pre-loaded modules on Solaris
> At 06:11 AM 11/10/00 -0500, barries wrote: > >> Address Kbytes Resident Shared Private > >> -- -- --- > >> total Kb 24720 227203288 19432 <<< pre-loaded modules > >> total Kb 14592 1297630969880 <<< not pre-loaed modules. > > When looking at pmap it looks like the main difference in "private" memory > usage is in the heap. I'm not clear why the heap would end up so much > bigger when pre-loading modules. A guess: When pmap says "shared" it means stuff that realy shared. I.E. it's not counting stuff that is marked copy-on-write, which is shared UNTIL YOU TRY TO MODIFY IT. Using pre-loaded modules you will get some things (XS) mmaped in as shared, read-only and a whole bunch of stuff loaded into the heap. this is "shared" when you fork, until you change it. The trick is that most of the perl "bytecode" doesn't get modified after it's loaded, so it'll stay "shared". -- John Hughes <[EMAIL PROTECTED]>, CalvaEDI SA.Tel: +33-1-4313-3131 66 rue du Moulin de la Pointe, Fax: +33-1-4313-3139 75013 PARIS.
Re: pre-loaded modules on Solaris
At 06:11 AM 11/10/00 -0500, barries wrote: >> Address Kbytes Resident Shared Private >> -- -- --- >> total Kb 24720 227203288 19432 <<< pre-loaded modules >> total Kb 14592 1297630969880 <<< not pre-loaed modules. >Stupid question, probably, but when running the non-pre-loaded version, >are you sure all the same modules are being loaded? Yes. According to perl-status, anyway. Some modules are loaded into the parent, of course, because of mod_perl. But when not pre-loading I start the server, look at perl-status and then made some requests and looked again to see what was loaded. The difference is what modules I'm use'ing in my test. >I'm wondering if there's >some set of modules that, for some reason, isn't being loaded by the >sequence of requests you're firing against all of your httpds to >get the servers "warmed up" to represent real-life state. When looking at pmap it looks like the main difference in "private" memory usage is in the heap. I'm not clear why the heap would end up so much bigger when pre-loading modules. Unfortunately, Linux doesn't seem to have the same reporting abilities as Solaris, but using /proc//statm to show shared and private memory under these same test showed that pre-loading was a big win. So it seems like a Solaris issue. Bill Moseley mailto:[EMAIL PROTECTED]
Re: pre-loaded modules on Solaris
On Thu, Nov 09, 2000 at 04:24:31PM -0800, Bill Moseley wrote: > And here's comparing the totals returned by the pmap program that should > detail shared and private memory (according to the paper cited above). > > Address Kbytes Resident Shared Private > -- -- --- > total Kb 24720 227203288 19432 <<< pre-loaded modules > total Kb 14592 1297630969880 <<< not pre-loaed modules. > > Indeed there's a tiny bit more shared memory in the pre-loaded Apache, but > the amount of "private" memory is significantly higher, too. Ten megs a > child will add up. It doesn't really make sense to me, but that's what > pmap is showing. Stupid question, probably, but when running the non-pre-loaded version, are you sure all the same modules are being loaded? I'm wondering if there's some set of modules that, for some reason, isn't being loaded by the sequence of requests you're firing against all of your httpds to get the servers "warmed up" to represent real-life state. - Barrie
Re: pre-loaded modules on Solaris
Hi Mike, I've cc'd the mod_perl list for other Solaris users to consider. At 10:49 AM 11/01/00 -0800, Michael Blakeley wrote: >I saw a significant benefit from pre-loading modules. Let's take a >test case where we share via startup.pl: > >Without preloading: > ># tellshared.pl web >count vsz rss kB % >8 146304 67784 78520 54 > >With the "use" section above: > ># tellshared.pl web >count vsz rss kB % >8 132672 17032 115640 87 > >'rss' is the resident set size - that is, the amount of actual RAM in >use. The 'vsz' tells the size of the virtual address space - the swap >in use. The kB column shows the difference (ie, the "saved RAM" via >page sharing) and the % shows the %-shared. I'm not clear you can measure the shared memory space that way. I don't really understand the memory system much, but here's a paper that I found interesting: http://www.sun.com/solutions/third-party/global/SAS/pdf/vmsizing.pdf "The virtual address size of a process often bares no resemblance to the amount of memory a process is using because it contains all of the unallocated memory, libraries, shared memory and sometimes hardware devices (in the case of XSun). "The RSS figure is a measure of the amount of physical memory mapped into a process, but often there is more than one copy of the process running, and a large proportion of a process is shared with another. "MemTool provides a mechanism for getting a detailed look at a processes memory utilization. MemTool can show how much memory is in-core, how much of that is shared, and hence how much private memory a process has. The pmem command (or /usr/proc/bin/pmap -x in Solaris 2.6) can be used to show the memory utilization of a single process." Now, with a simple "hello world" mod_perl handler that loaded a bunch of modules I did see that pre-loading modules reduces memory usage -- both in looking at ps output, and with the pmap program, even after a number of requests. This is consistent with what you commented on above. I'm repeating, but I found with a real-world application that sharing the modules ended up using quite a bit more "private" memory. I don't know if that's only an issue with my specific OS, or with how my specific application is running. Here's ps output with pre-loaded modules. On these tests I'm running Apache 1.3.12 with mod_perl 1.24 static, but everything else is DSO. I've got maxclients set to one so there's only the parent and one child. I'm pre-loading modules in a section here: S USER PID PPID %CPU %MEM VSZ RSSSTIMETIME COMMAND S lii 318 1 0.0 0.3 8376 5464 15:50:380:00 httpd.mo S lii 319 318 0.8 1.1 24720 21288 15:50:380:05 httpd.mo And now without pre-loaded modules: S USER PID PPID %CPU %MEM VSZ RSSSTIMETIME COMMAND S lii 1260 1 0.0 0.2 4392 3552 15:56:250:00 httpd.mo S lii 1261 1260 0.9 0.6 14592 12304 15:56:250:05 httpd.mo And here's comparing the totals returned by the pmap program that should detail shared and private memory (according to the paper cited above). Address Kbytes Resident Shared Private -- -- --- total Kb 24720 227203288 19432 <<< pre-loaded modules total Kb 14592 1297630969880 <<< not pre-loaed modules. Indeed there's a tiny bit more shared memory in the pre-loaded Apache, but the amount of "private" memory is significantly higher, too. Ten megs a child will add up. It doesn't really make sense to me, but that's what pmap is showing. Maybe this isn't that interesting. Anyway, I'll try a non DSO Apache and see if it makes a difference, and also try with an Apache that forks off more clients than just one, but I can't imagine that making a difference. Later, Bill Moseley mailto:[EMAIL PROTECTED]