I've done extensive reading in both the guide and the maillist archives
and haven't found a very comprehensive explanation of this topic.
First I'll explain the setup of the scripts and webserver that we're
using:
I have 1 accessor script that the public would connect to "test.pl".
Inside that script, I "use" a single module "test.pm" and then call
"test::run()". When I just use this setup, and do no pre-loading, each
servlet compiles the test.pm + all included modules on the first call and
then it stays compiled for the duration of the servlet. This is ok, but
the entire compiled network of included pm's caused the servlet sizes to
be about 8 megs each. Fairly high if we wanted to run a lot of servlets
on a minimal system.
So I instead pre-loaded the test module in the startup.pl file during
apache's loading session. This then seemed to save a lot of memory. With
4 servlets running, we went from aproximatly 30megs of total memory used
to 20 megs. However, Once we started accessing the script and causing
calls to be made to man aspects of the script, the total memory used spike
to over 60megs! This seemed really strange since the memory used without
library sharing was less!! Is this a problem with the way system-monitor
(the way we were calculating the shared/real memory) reports stuff?
Because each servlet was now reported to be ~15megs in size..about 7-8
megs bigger than before in the "non-optimized" state :)
I understand that with shared libs, as global variables become dirty,
pages become unshared. Is that true? Is it only for global variables?
Or do local code pages become unshared when used? If so, then there would
be minimal advantage to sharing libs on a very diversly active system
since most code would become unshared quite quickly.
Is there a technique to minimize the ammount of non-dynamic variables that
get shared? Perhaps putting all dynamic variables into a seperate package
and module? This interests me the most as a number of our variables are
semi-large static data structures that may be contributing to the
wastefulness of space when they lie on pages that are forced to be shared
due to the global variable use.
I suppose another possibility is optimizing those static data structures
not to use too much memory. For example, a couple of those data
strucutres are hashes. They are dynamically created at compile time and
never change from that point onward so they should not causing a sharing
issue. However, I don't know the internal representation of hashes in
perl, though I can make some guesses. I assume it allocates a new chunk
of memory every so often as variables are added. But I also assume it
allocates more than needed to optimize for speed. Is there a way to
'prune' off that excess memory at some point? The advantage of doing that
would be that if those variables do become unshared, their wasted memory
doesn't get duplicated along side the data.
I have one other question...entirly unrelated to the above problem...that
is this. As i mentioned before, we have the 1 script, test.pl, that
requires test.pm into it and calls test::run() to access the loaded libs.
Is it possible to bypass this directly, and have all accesses to a
particular virtual host "test.blah.com" instead of routing to test.pl (as
it does now) be handled by the pre-loaded test.pm module (directly call
test::run()) ?
If anyone can help me on any of these issues, or point me to a page in
'the guide' or other web resource, I'd greatly appreciate it.
- Bryce
