unsubscribe
-- Stephen Griffin Realtime Solutions MSOP14 Mission Systems Solutions MSS Defence and Communication Systems DCS im Auftrag / authorised by EADS Deutschland GmbH Defence and Communications Systems 89077 Ulm Tel.: +49 (0) 731/392-5872 Mob.: +49 (0) 175 684 53 96 Fax.: +49 (0) 731/392-4225 E-Fax.: +49 (0) 731/392-20 5872 e-mail: stephen.griffin.sup...@eads.com EADS Deutschland GmbH Registered Office: Ottobrunn District Court of Munich HRB 107648 Chairman of the Supervisory Board: Dr. Thomas Enders Managing Directors: Dr. Stefan Zoller (Chairman), Michael Hecht This E-mail and any attachment(s) to it are for the addressee's use only. It is strictly confidential and may contain legally privileged information. No confidentiality or privilege is waived or lost by any mistransmission. If you are not the intended addressee, then please delete it from your system and notify the sender immediateley. You are hereby notified that any use, disclosure, copying or any action taken in reliance on it is strictly prohibited and may be unlawful. Thank you.
Re: unsubscribe
Try: modperl-unsubscr...@perl.apache.org Maybe it would help to have this bit of information, obvious as it should be, appended to each message, as is the case with many other mailing lists. (While even that may be ignored in certain cases, it certainly wouldn't be in all cases.) At the moment, the unsubscribe address appears only in the mail headers. Apart from being listed on the website, of course. Michael Ludwig
[OT] Re: unsubscribe
It is already included. Check your headers. list-help: mailto:modperl-h...@perl.apache.org list-unsubscribe: mailto:modperl-unsubscr...@perl.apache.org List-Post: mailto:modperl@perl.apache.org Michael Ludwig wrote: Try: modperl-unsubscr...@perl.apache.org Maybe it would help to have this bit of information, obvious as it should be, appended to each message, as is the case with many other mailing lists. (While even that may be ignored in certain cases, it certainly wouldn't be in all cases.) At the moment, the unsubscribe address appears only in the mail headers. Apart from being listed on the website, of course. Michael Ludwig
Re: Initializing Sleepycat::DbXml (Berkeley, Oracle) objects in startup.pl
I've referenced this thread on the Oracle Berkeley DB forum. Chances are it'll catch the attention of one of the BDB experts. If you're interested, please see: http://forums.oracle.com/forums/thread.jspa?threadID=848390 Michael Ludwig
Re: [OT] Re: unsubscribe
Issac Goldstand schrieb: It is already included. Check your headers. list-help: mailto:modperl-h...@perl.apache.org list-unsubscribe: mailto:modperl-unsubscr...@perl.apache.org List-Post: mailto:modperl@perl.apache.org I know. Michael Ludwig wrote: At the moment, the unsubscribe address appears only in the mail headers. Apart from being listed on the website, of course. Most people, however, won't bother to check the headers. While we're off-topic, I'd also think it beneficial to set the Reply-To header to the list. While having been a member of the list for a short time only, I've already received some answers privately that were obviously intended to appear on-list. Setting Reply-To to the list settles this issue, and avoids losing valuable content for the archives. As it settles the issue of receiving duplicates by those people who choose to reply to all recipients. I know, however, from having followed similar discussions on other lists, that some people prefer to *not* have the reply by default go to the list, on the grounds that one may not be aware that the possibly embarrassing and compromising contents of the mail go to the eternal archive of the public web place instead of just one poor individual's mailbox. So that's just my drei Groschen, and I'll refrain from further driving this issue or flogging this undead horse. Thanks for your patience. Michael Ludwig
Re: Initializing Sleepycat::DbXml (Berkeley, Oracle) objects in startup.pl
On Sun 18 Jan 2009, Michael Ludwig wrote:
From perldoc perlthrtut: In this model each thread runs in its own
Perl interpreter, and any data sharing between threads must be
explicit. This does not sound to me as if there is a significant
advantage over spawning child processes, at least not on UNIX.
Hmm. Not sure what to make of this threaded Perl.
In fact, it is worse than fork()ing off unix processes because the
interpreter data is completely copied while a new interpreter is
created. On the other hand a forked process copies only those pages
that are written to.
I haven't used Sleepycat::DbXml but I use BerkeleyDB in production with
prefork, see Apache2::Translation on CPAN. What I have learned is, you
can't open the BDB env in the parent apache. When I wrote that piece I
also had problems when I opened the env in the parent and then closed
it before forking children. Perhaps they are gone in a newer version.
I think with prefork opening the env in a ChildInit handler should work.
I ended up with the connect-at-first-usage approach
unless( $connected ) {
connect();
$connected=1;
}
because I test almost all my modules also with a threaded MPM and the
ChildInit hook is run once per process not once per Perl interpreter.
Torsten
--
Need professional mod_perl support?
Just hire me: torsten.foert...@gmx.net
Multiple mod_perl 2.0 installations on one Linux machine
What does a mod_perl 2.0 installation on Linux consist of? * Apache 2.0 or 2.2 * mod_perl 2.0 * libapreq (optional, true, but indispensable for convenience) The Apache, let's say the worker MPM, is going to be installed under /usr/local/apache22w. If I now want to install a prefork MPM next to it, this is no problem at all, it goes to, say, /usr/local/apache22pf. But what about the mod_perl 2.0 installation? The mod_perl DSO itself is installed to $ServerRoot/modules. Doing an ldd on the module doesn't reveal any dependencies on libraries under $ServerRoot/lib, but I guess that nevertheless, this mod_perl does very well depend on the particular Apache version it is configured and built to work with. True? There is also the Perl interface to Apache/mod_perl, a bunch of modules installed under, say, /usr/local/lib/perl/5.8.8 and /usr/local/lib/perl/5.8.8/auto. Doing a crude ldd on all those modules: find /usr/local/lib/perl/5.8.8/auto/Apache2 -type f -name \*.so \ | while read so; do ldd $so; done | sort | uniq -c I can only see dependencies on /lib/ld-linux.so.2, libc.so.6, and linux-gate.so.1. But this certainly doesn't mean that there is no dependency on the particular version of mod_perl that is installed? Or does it? Doing the same find/ldd loop for /usr/local/lib/perl/5.8.8/auto/APR, I can see more dependencies, including dependencies on the particular Apache installation: /usr/local/apache22w/lib/libapr-1.so.0 (0xb7eb8000) /usr/local/apache22w/lib/libaprutil-1.so.0 /usr/local/apache22w/lib/libexpat.so.0 And even - horrors - this one (which looks like an error in the configure/build logic): /usr/local/src/libapreq2-2.08/library/.libs/libapreq2.so.3 Now, what happens if I am going to install a new Apache 2.2/prefork with mod_perl in addition to the already existing Apache 2.2/worker with mod_perl? The new mod_perl modules are going to clobber the existing one, probably rendering the old mod_perl useless by supplanting their own versions linked to the prefork MPM libraries, and I imagine the worker MPM won't be happy when being asked to interact with these. True? Does anyone know what the dependencies are? And how would I deal with this situation if my above observation was true and I wanted a worker and a prefork MPM on the same machine, both with mod_perl? Michael Ludwig
Re: Initializing Sleepycat::DbXml (Berkeley, Oracle) objects in startup.pl
macke...@animalhead.com schrieb: Yes it is now Abend again. Timewise this is like relationships I have had with Philips (now NXP) European colleagues... And a new morning again, and it goes round and round ... On Jan 18, 2009, at 10:56 AM, Michael Ludwig wrote: macke...@animalhead.com schrieb: Thanks for replying again. What you've outlined above is what I had implemented following Mark's suggestion in his first reply to my initial post. Unfortunately, this quite reliably produces SEGVs, so I think this usage is not aligned with the Berkeley interface design. I don't know in which way this is wrong - I guess I'll have to ask the Berkeley folks in order to maybe clarify this. I was using berkeleyDB 4.?, but without the fancy threading stuff you mention below. The BerkeleyDB module is based on the C interface while Sleepycat::Db is based on the C++ extension to the C interface, as far as I can tell. Since you can't share a tied variable (sharing uses the tying mechanism), while I was futzing with threading, I tried to share the underlying filehandle and use the function interface. But that gave lots of SEGVs so I quit sharing anything about the DBs between threads, which got rid of the SEGVs. Good. So you got the BerkeleyDB module working on a threading worker MPM by avoiding any attempt at inter-thread sharing. I'm still getting SEGVs trying to do the same with Sleepycat::DbXml. What this means is that each thread must open the db's for itself. The amount of data stored for each open DB connection, times THREADS_PER_CHILD times the number of Apache children at any given point, makes for some memory. But 1) the separate connections help the DB package be thread-safe, So if coding the handler as above means that each thread, having its own global variables, opens its own handle to the Berkeley environment, I shouldn't need the DB_THREAD flag (which, according to the Berkeley documentation, [causes] the DbEnv handle returned by DbEnv::open to be free-threaded; that is, concurrently usable by multiple threads in the address space. This flag is needed if any handle is used by more than one thread (or process) concurrently. So it shouldn't be needed if the handler is coded as above and there isn't any concurrent access elsewhere. I have not worked with this flag but from your words it sounds right. 2) the first-used threads keep getting re-used in preference to threads not yet used. I noticed this is indeed what seems to happen - whether by chance or as a feature, I don't know. You can find it described proudly as a memory-minimization feature in some of the Apache docs about worker and/or event. Does anyone know where this is? Haven't found it here: http://httpd.apache.org/docs/2.2/mod/worker.html 3) if you consider each thread as more or less equivalent to a child process in prefork, your total memory requirement is less. From perldoc perlthrtut: In this model each thread runs in its own Perl interpreter, and any data sharing between threads must be explicit. This does not sound to me as if there is a significant advantage over spawning child processes, at least not on UNIX. Start a prefork Apache2 (if you want, let it run a while to give the children a chance to grow. Note the (largest or average) size of child processes. Start a worker or event Apache, let it run similarly (or not at all if you're impatient. Divide the (largest or average) size of child processes by THREADS PER CHILD. My point was that this number will be much smaller than the process size with prefork. Okay, I see. More concurrency bang for the memory buck, despite certain restrictions, if my understanding is correct, of the Perl threading implementation. Michael Ludwig
Re: Initializing Sleepycat::DbXml (Berkeley, Oracle) objects in startup.pl
Torsten Foertsch schrieb:
On Sun 18 Jan 2009, Michael Ludwig wrote:
Hmm. Not sure what to make of this threaded Perl.
In fact, it is worse than fork()ing off unix processes because the
interpreter data is completely copied while a new interpreter is
created. On the other hand a forked process copies only those pages
that are written to.
Thanks. If I'm understanding correctly and this is true (and not
considering SEGVs), the total memory cost for a mod_perl on a worker
MPM should outweigh that for mod_perl on a prefork MPM, which seems
to not be aligned with what Craig has observed.
I haven't used Sleepycat::DbXml but I use BerkeleyDB in production
with prefork, see Apache2::Translation on CPAN.
Looks interesting. I'm going to take a look.
What I have learned is, you can't open the BDB env in the parent
apache. When I wrote that piece I also had problems when I opened the
env in the parent and then closed it before forking children. Perhaps
they are gone in a newer version.
I think with prefork opening the env in a ChildInit handler should
work. I ended up with the connect-at-first-usage approach
unless( $connected ) {
connect();
$connected=1;
}
because I test almost all my modules also with a threaded MPM and the
ChildInit hook is run once per process not once per Perl interpreter.
In order to be on the safe side as far as Berkeley DB is concerned,
you'd have to make sure up front that the environment is safe to use,
which is done by running recovery (DB_RECOVER flag, DB_RUNRECOVERY error
code) in a single thread or process that has exclusive access to the
environment.
Given that we're experiencing problems (SEGVs) when trying to open
the environment up front in the parent process, it looks like this
requirement is not easy to meet on the Apache/mod_perl platform.
Michael Ludwig
Re: Initializing Sleepycat::DbXml (Berkeley, Oracle) objects in startup.pl
On Mon 19 Jan 2009, Michael Ludwig wrote: In fact, it is worse than fork()ing off unix processes because the interpreter data is completely copied while a new interpreter is created. On the other hand a forked process copies only those pages that are written to. Thanks. If I'm understanding correctly and this is true (and not considering SEGVs), the total memory cost for a mod_perl on a worker MPM should outweigh that for mod_perl on a prefork MPM, which seems to not be aligned with what Craig has observed. Of course it depends on your setup. If you configure only a small number of interpreters then the overall memory footprint can be lower then with prefork where each process runs its own interpreter. But the effect of the copied interpreters outweighs this very soon. Measuring memory consumption on Linux is not simple. You can't simply add up all RSS or VSZ figures or something like this. I know of 2 approaches that work more or less well. 1) call mlockall in the parent and then use /proc/PID/smaps (or Linux::Smaps) to measure shared memory 2) take a system with enough RAM and nothing else to do, start apache with different numbers of children and watch the free memory with vmstat while doing so. Then you can estimate the amount of memory that is consumed by one apache process. See the attached picture for example. In order to be on the safe side as far as Berkeley DB is concerned, you'd have to make sure up front that the environment is safe to use, which is done by running recovery (DB_RECOVER flag, DB_RUNRECOVERY error code) in a single thread or process that has exclusive access to the environment. You can run recovery in a separate process spawned at startup by the parent. Torsten -- Need professional mod_perl support? Just hire me: torsten.foert...@gmx.net attachment: experiment-scrambled.png
Re: Initializing Sleepycat::DbXml (Berkeley, Oracle) objects in startup.pl
Torsten Foertsch schrieb: Of course it depends on your setup. If you configure only a small number of interpreters then the overall memory footprint can be lower then with prefork where each process runs its own interpreter. But the effect of the copied interpreters outweighs this very soon. I see. Guess I have some reading to do to better understand what interpreters as in Perl ithreads really means. Measuring memory consumption on Linux is not simple. You can't simply add up all RSS or VSZ figures or something like this. I know of 2 approaches that work more or less well. 1) call mlockall in the parent and then use /proc/PID/smaps (or Linux::Smaps) to measure shared memory 2) take a system with enough RAM and nothing else to do, start apache with different numbers of children and watch the free memory with vmstat while doing so. Then you can estimate the amount of memory that is consumed by one apache process. See the attached picture for example. Thanks for sharing this. You can run recovery in a separate process spawned at startup by the parent. That's a good idea. And a simple one at that. Thank you! Michael
Re: Multiple mod_perl 2.0 installations on one Linux machine
Michael Ludwig wrote: The mod_perl DSO itself is installed to $ServerRoot/modules. Doing an ldd on the module doesn't reveal any dependencies on libraries under $ServerRoot/lib, but I guess that nevertheless, this mod_perl does very well depend on the particular Apache version it is configured and built to work with. True? Very true. I can only see dependencies on /lib/ld-linux.so.2, libc.so.6, and linux-gate.so.1. But this certainly doesn't mean that there is no dependency on the particular version of mod_perl that is installed? Or does it? it most definitely dpends on libperl.so, httpd, apr, apr-util, and mod_perl.so Doing the same find/ldd loop for /usr/local/lib/perl/5.8.8/auto/APR, I can see more dependencies, including dependencies on the particular Apache installation: libapreq2.so and mod_apreq2.so + all the above ones. /usr/local/apache22w/lib/libapr-1.so.0 (0xb7eb8000) /usr/local/apache22w/lib/libaprutil-1.so.0 /usr/local/apache22w/lib/libexpat.so.0 And even - horrors - this one (which looks like an error in the configure/build logic): /usr/local/src/libapreq2-2.08/library/.libs/libapreq2.so.3 we [joes] just fixed this though it doesn't affect everyone. Its will be in the next release which will be soonish Now, what happens if I am going to install a new Apache 2.2/prefork with mod_perl in addition to the already existing Apache 2.2/worker with mod_perl? The new mod_perl modules are going to clobber the existing one, probably rendering the old mod_perl useless by supplanting their own versions linked to the prefork MPM libraries, and I imagine the worker MPM won't be happy when being asked to interact with these. True? Correct, you get to play magical paths. Consider yourself a developer now and the following tree: /sfw /perl 5.8.1 5.8.8 none-stdio-mymalloc ithread-perlio-perlmalloc /cpans 5.8.8-none-perlio-perlmalloc /apr 1.3.3 threaded none /apu 1.3.4 apr-1.3.3-threaded threaded none apr-1.3.3-none /httpd 2.2.11 prefork apr-1.3.3-none=apu-1.3.4-none worker event /mod_perl 2.0.3 2.0.4 r732904 5.8.8-none-perlio-mymalloc=httpd-2.2.11-apr-1.3.3-none-apu-1.3.4-none /libapreq2 2.0.7 2.0.8 2.0.4=5.8.8-none-perlio-mymalloc=httpd-2.2.11-apr-1.3.3-none-apu-1.3.4 After doing this with a bug of ./configure, Makefile.PL and Configure flags you have 2 options 1) use the path to the .so in the install directories 2) install the .so into modules/ BUT rename it to something like mod_perl-5.8.8-none-perlio-mymalloc=httpd-2.2.11-apr-1.3.3-none-apu-1.3.4-none.so Then load the correct one. The good part is that Apache-Test lets you pass flags to name the mod_perl.so you want. When testing stuff or running the perl test suite. This also applies to libapreq2.so, but not quite yet with the Apache-Test cli naming. Doing the above will allow you to install ALMOST the full permutations of what we support. To get the full amount, you have to through GCC into the mix for older httpd's. Be warned, you'll need a good sized disk for this. Of course, if there are combinations you know you'll never care about, then you leave them out and your directory names get shorter. my @perl_io_types = qw(perlio stdio); my @perl_thread_types = qw(none ithreads); my @perl_malloc_types = qw(mymalloc perlmalloc); ## only some were actually bundled ## 0.x for 2.0.xx 1.2.x and 1.3.x for 2.2.xx) ## threadedness of apr/apu should match my @apr_vers = qw(lots); my @apu_vers = qw(lots); my @apr_thread_types = qw(threaded none); my @apu_thread_types = qw(threaded none) ## XXX: dbd, gdbm, mysql, pgsql, oracle, freetds, ldap, etc... ## prefork is apr/apu without threads ## prefork_threaded is them with threads my @httpd2_vers = qw(2.0.48...2.0.63, 2.2.0..2.2.11); my @httpd2_mpms = qw(prefork prefork_threaded worker event); my @mod_perl_vers = qw(2.0.4); MP_MAINTAINER MP_DEBUG MP_APXS my @libapreq2_vers = qw(2.0.8) -- Philip M. Gollucci (pgollu...@p6m7g8.com) c: 703.336.9354 Consultant - P6M7G8 Inc. http://p6m7g8.net Senior System Admin - RideCharge, Inc. http://ridecharge.com 1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70 3F8C 75B8 8FFB DB9B 8C1C Work like you don't need the money, love like you'll never get hurt, and dance like nobody's watching.
Re: Multiple mod_perl 2.0 installations on one Linux machine
On Mon, Jan 19, 2009 at 5:56 AM, Michael Ludwig m...@as-guides.com wrote: Now, what happens if I am going to install a new Apache 2.2/prefork with mod_perl in addition to the already existing Apache 2.2/worker with mod_perl? Hmm, isn't the MPM just a runtime configuration? Why would you need another httpd install? It sounds like a separate config file would do it, if that's all you want to change. If you want to change other things, like using different version of httpd, the safest thing is to run two separate perls, one for each mod_perl setup. - Perrin
Re: Multiple mod_perl 2.0 installations on one Linux machine
Perrin Harkins wrote: Hmm, isn't the MPM just a runtime configuration? Why would you need another httpd install? It sounds like a separate config file would do it, if that's all you want to change. MPM is a build time configuration via the --with-mpm= to ./configure Adam
Re: Multiple mod_perl 2.0 installations on one Linux machine
On Mon, Jan 19, 2009 at 1:37 PM, Adam Prime adam.pr...@utoronto.ca wrote: Perrin Harkins wrote: Hmm, isn't the MPM just a runtime configuration? Why would you need another httpd install? It sounds like a separate config file would do it, if that's all you want to change. MPM is a build time configuration via the --with-mpm= to ./configure I thought you could build more than one and load the one you want at runtime. Isn't that what Red Hat does? - Perrin
Re: Multiple mod_perl 2.0 installations on one Linux machine
Perrin Harkins wrote: On Mon, Jan 19, 2009 at 1:37 PM, Adam Prime adam.pr...@utoronto.ca wrote: Perrin Harkins wrote: Hmm, isn't the MPM just a runtime configuration? Why would you need another httpd install? It sounds like a separate config file would do it, if that's all you want to change. MPM is a build time configuration via the --with-mpm= to ./configure I thought you could build more than one and load the one you want at runtime. Isn't that what Red Hat does? If redhat's shipping more than one mpm, they've patched httpd. According to the 2.2 docs, you have to choose exactly one MPM: --with-mpm=MPM Choose the process model for your server. You have to select exactly one Multi-Processing Module. Otherwise the default MPM for your operating system will be taken. Possible MPMs are beos, mpmt_os2, prefork, and worker. http://httpd.apache.org/docs/2.2/programs/configure.html Adam
Re: Multiple mod_perl 2.0 installations on one Linux machine
Why not just run multiple httpds? I do it, but I have multiple IPs and that may be a requirement for running different httpds. On Jan 19, 2009, at 12:45 PM, Adam Prime wrote: Perrin Harkins wrote: On Mon, Jan 19, 2009 at 1:37 PM, Adam Prime adam.pr...@utoronto.ca wrote: Perrin Harkins wrote: Hmm, isn't the MPM just a runtime configuration? Why would you need another httpd install? It sounds like a separate config file would do it, if that's all you want to change. MPM is a build time configuration via the --with-mpm= to ./configure I thought you could build more than one and load the one you want at runtime. Isn't that what Red Hat does? If redhat's shipping more than one mpm, they've patched httpd. According to the 2.2 docs, you have to choose exactly one MPM: --with-mpm=MPM Choose the process model for your server. You have to select exactly one Multi-Processing Module. Otherwise the default MPM for your operating system will be taken. Possible MPMs are beos, mpmt_os2, prefork, and worker. http://httpd.apache.org/docs/2.2/programs/configure.html Adam
Re: Multiple mod_perl 2.0 installations on one Linux machine
On Mon 19 Jan 2009, Adam Prime wrote: I thought you could build more than one and load the one you want at runtime. Isn't that what Red Hat does? If redhat's shipping more than one mpm, they've patched httpd. Probably not httpd. At least Suse has not. Instead they configure and build a prefork httpd and a worker httpd one by one and then merge the result. The apache API does not depend on the chosen MPM. So, almost certainly you can build modules (including mod_perl) with one MPM and use it with another. Usually I build mod_perl against a httpd with worker MPM and TEST is with both worker and prefork. So far I have never found something that went bad only because of the other MPM. Torsten -- Need professional mod_perl support? Just hire me: torsten.foert...@gmx.net
Re: Multiple mod_perl 2.0 installations on one Linux machine
Torsten Foertsch wrote: On Mon 19 Jan 2009, Adam Prime wrote: I thought you could build more than one and load the one you want at runtime. Isn't that what Red Hat does? If redhat's shipping more than one mpm, they've patched httpd. Probably not httpd. At least Suse has not. Instead they configure and build a prefork httpd and a worker httpd one by one and then merge the result. The apache API does not depend on the chosen MPM. So, almost certainly you can build modules (including mod_perl) with one MPM and use it with another. Built with a threaded MPM and any module should also work on prefork etc. But not visa-versa!!!
Re: Multiple mod_perl 2.0 installations on one Linux machine
William A. Rowe, Jr. wrote: Built with a threaded MPM and any module should also work on prefork etc. But not visa-versa!!! 100% correct, the linkage is wrong the other way. -- Philip M. Gollucci (pgollu...@p6m7g8.com) c: 703.336.9354 Consultant - P6M7G8 Inc. http://p6m7g8.net Senior System Admin - RideCharge, Inc. http://ridecharge.com 1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70 3F8C 75B8 8FFB DB9B 8C1C Work like you don't need the money, love like you'll never get hurt, and dance like nobody's watching.
