Re: Where is Perl compilation output when using modperl ?
Hello, Just to confrm that I have the same symptom as yours when I was developing on Win32: when a module couldn't load, the only thing I could get was a bare "Can't load module", without any explanation. Then, th eonly thing I could do was trying to isolate the problem, progressively stripping stuff from my code. I never investigated so as to get the true perl diagnose. Lionel. - Original Message - From: "Jens Helweg" <[EMAIL PROTECTED]> To: Sent: Thursday, June 28, 2007 12:09 PM Subject: Where is Perl compilation output when using modperl ? Hi erveryone, I am using modperl2 with apache2 on win32 (activestate Perl 5.8). I have my own perl module included in the apache conf. Whenever I have an error in my module apache does not start and the only error message I can find is in apache's error.log: Can't load Perl file: D:/path_to_my_perl_module for server myserver, exiting... It doesn't say anything about what is wrong in the code. Is there a way to get the compilers output from perl, so I can get details on what is wrong in the code ? (when I run perl -c mymodule.pm on command line it complains about missing modules - so that doesn't seem to be an option when using modperl.) Thanks in advance, Jens
Re: Howto develop with modperl 2 ? (Restart Apache all the time ?)
Yes, easily.
In any of your handler, you make a reference to one test module.
For example, in your response handler:
use myModule();
use Apache2::Const -compile => qw(OK);
sub handler {
myModule::printHello();
return Apache2::Const::OK;
}
1;
And then: myModule.pm:
sub printHello {
print 'Hello';
}
1;
Then, you start your server, and test your handler.
It should print 'Hello';
Then, without restarting your server, you change myModule.pm into:
sub printHello {
print 'Hi';
}
1;
And you run your page again.
This time, it should print 'Hi', showing that Apache took the update into
account on the fly, without having to restart anything.
HTH.
Lionel.
- Original Message -
From: "Jens Helweg" <[EMAIL PROTECTED]>
To:
Sent: Thursday, June 21, 2007 11:12 AM
Subject: Re: Howto develop with modperl 2 ? (Restart Apache all the time ?)
Thank you. I really missed the second ':'.
Now it's not complaining anymore but the reload still doesn't work...
Is there a way to verify that the Reload module has been loaded and that
is works at all ?
Sean Davis wrote:
Jens Helweg wrote:
Thanks Linonel, Jonathan.
I have added this to my configuration and now get the error message:
failed to resolve handler `Apache2:Reload':
I haven't had the chance to look at this any closer. I will do this
tomorrow, though. I don't want to overstrain your help - but do you have
an idea ?
Is this error cut-and-paste? If so, you have only one ':'.
Sean
Re: Howto develop with modperl 2 ? (Restart Apache all the time ?)
Hi, You can put that in your Apache conf file: PerlModule Apache2::Reload PerlInitHandler Apache2::Reload This way, every time you alter one of your Apache modules, it gets reloaded. This incurs a small performance hit (the server has to fstat the related files every time a request is run, to know if any module has been updated on disk), but that's definitely a good choice when developing (no need to restart your server) Lionel. - Original Message - From: "Jens Helweg" <[EMAIL PROTECTED]> To: Sent: Tuesday, June 19, 2007 1:34 PM Subject: Howto develop with modperl 2 ? (Restart Apache all the time ?) Hi everyone, I have just started using modperl 2 on Windows. Apache is 2.2.4. Perl ist ActiveState Perl 5.8.8. I have setup my first perl module an included everything into the apache configuration and up to now everything works fine. But one thing that I really wonder is: I need to restart apache every time I make a change to my perl module. Is that the way one would develop using modperl ? (I don't think so). Running the perl module on command line neither works because then it complains about the modperl/apache stuff that isn't available. How would one professionally develop code using modperl ? Regards, Jens
Re: which module for this purpose?
All you need to do (assuming mod_perl2) is:
- set the content-type (to whatever file type you are sending)
- use $r->sendfile('filename')
See
http://perl.apache.org/docs/2.0/api/Apache2/RequestIO.html#C_sendfile_
Clint
Unless Im mistaken, why not use instead the Apache default handler for these
static files to be served?
Using the right directive for the directory containing the static files to
be served:
SetHandler default-handler
Re: Loading Win32::OLE in a modperl package
Hi again, Did you try finally try out with Apache 2.0? If this works with Apache 2.0, but not with Apache 2.2, then, it would perhaps be useful to point this out. Lionel. - Original Message - From: "Foo JH" <[EMAIL PROTECTED]> To: "Lionel MARTIN" <[EMAIL PROTECTED]> Sent: Friday, June 08, 2007 11:21 AM Subject: Re: Loading Win32::OLE in a modperl package Thanks Lionel. Damn. I was hoping I won't need to 'regress' into Apache 2.0. But thanks a lot for suggesting a way out for me. Lionel MARTIN wrote: Hi again, I tried our code, using as well this directive in my httpd.conf file: PerlResponseHandler MyPackage To make your handler work as the main Response Handler. Moreover I had to alter your code to make it work, adding a: use Apache2::RequestRec; because otherwise, the content_type method is not defined. And it worked perfectly (returning 'hello';)
Re: Loading Win32::OLE in a modperl package
Hi again,
I tried our code, using as well this directive in my httpd.conf file:
PerlResponseHandler MyPackage
To make your handler work as the main Response Handler.
Moreover I had to alter your code to make it work, adding a:
use Apache2::RequestRec;
because otherwise, the content_type method is not defined.
And it worked perfectly (returning 'hello';)
#**
package MyPackage;
use strict;
use warnings;
use Win32::OLE;
use Apache2::RequestRec;
sub handler
{
my $r = shift;
$r->content_type('text/plain');
print 'hello';
return Apache2::Const::OK;
}
1;
#**
For your information, I am using:
Apache/2.0.59 (Win32) mod_perl/2.0.3 Perl/v5.8.8
perl and MP are ActiveState builds.
And I downloaded the Win32 binary from the Apache httpd website.
Lionel.
- Original Message -
From: "Foo JH" <[EMAIL PROTECTED]>
To: "Lionel MARTIN" <[EMAIL PROTECTED]>
Cc: "mod_perl"
Sent: Friday, June 08, 2007 10:43 AM
Subject: Re: Loading Win32::OLE in a modperl package
Thanks Lionel,
I'm running Apache2.2 + Perl5.88.
Try this piece of code:
package MyPackage;
use strict;
use warnings;
use Win32::OLE;
sub handler
{
my $r = shift;
$r->content_type('text/plain');
print 'hello';
return Apache2::Const::OK;
}
1;
Lionel MARTIN wrote:
Basically,
Doing, in test.pl, run by ModPel::Registry, something like:
use Win32::OLE;
print 'Hello';
Works prefectly...
Which versions (Apache, MP and Perl) are you running?
- Original Message - From: "Foo JH" <[EMAIL PROTECTED]>
To: "mod_perl"
Sent: Friday, June 08, 2007 10:29 AM
Subject: Loading Win32::OLE in a modperl package
Hi all,
I wonder if I am alone in experiencing this. Simply put: putting 'use
Win32::OLE' in my modperl package will cause the Apache to fault. It
basically can't start at all.
A window will pop up complaining about 'Apache HTTP Server has
encountered a problem and needs to close'. No entry into the error log.
Does anyone know if this can be worked around?
Re: Loading Win32::OLE in a modperl package
Basically, Doing, in test.pl, run by ModPel::Registry, something like: use Win32::OLE; print 'Hello'; Works prefectly... Which versions (Apache, MP and Perl) are you running? - Original Message - From: "Foo JH" <[EMAIL PROTECTED]> To: "mod_perl" Sent: Friday, June 08, 2007 10:29 AM Subject: Loading Win32::OLE in a modperl package Hi all, I wonder if I am alone in experiencing this. Simply put: putting 'use Win32::OLE' in my modperl package will cause the Apache to fault. It basically can't start at all. A window will pop up complaining about 'Apache HTTP Server has encountered a problem and needs to close'. No entry into the error log. Does anyone know if this can be worked around?
Re: Is it advisable NOT to limit MaxRequestsPerChild in Win32 modperl?
When developing for PHP, I'm using a colpletely different httpd.conf to start my server, and in this paeticular httpd.conf file, there's no mention about modperl, perl or whatever required libapreq. The only modules I'm loading are: mime_module log_config_module status_module dir_module php5_module And the same error still happens again and again. Everytime I'm reinstalling my development OS, I'm using a brand new version of 2.0.xx version of Apache (currently using 2.0.59) and I have to confess that this nasty problem you are talking about was not appearing in my previous Apache installations (2.0.55 if I remember well), or at least not that often. As explained below, that's a real pity, because under Win32, a thread/child dying meamns the whole dying, and everything is forked again from the initial parent process. If I had to use Win32 on a production server, I would either investigate more about this problem (writing to Apache Win32 porters or trying to study the souce code, (not easy when you are not used to) either...would avoid using it at all prices. For now, you might perhaps give a look at previous Apache 2.0 versions and as well signal this bug to Apache Win32 porters. (are you subscribed to any Apache mailing list?) Lionel. - Original Message - From: "Foo JH" <[EMAIL PROTECTED]> To: "Lionel MARTIN" <[EMAIL PROTECTED]> Cc: "Perrin Harkins" <[EMAIL PROTECTED]>; Sent: Saturday, May 26, 2007 3:44 AM Subject: Re: Is it advisable NOT to limit MaxRequestsPerChild in Win32 modperl? Thanks Lionel, You meant when you dev. php, your apache did not load modperl at all, or it's just not used? Will there be a performance enhancement (as in no unexpected server restart) if you unload modperl totally? Yeah, that silly error page makes people unsure about Apache on Windows. A real bummer. Lionel MARTIN wrote: Hi, Actually I didn't see your original question, about the following error: [Mon Mar 05 21:19:47 2007] [notice] Parent: child process exited with status 3221225477 -- Restarting. Actually, I am not even sure that this is due to MP2. Indeed, on my development computer (running under Win XP, Apache/2.0.59), I am developing both PHP and Perl applications. When developing for PHP, my Apache Server doesn't know about Mod Perl. And yet, I'm keeping having the error you have, time to time, when loading my PHP pages. Actually, the messages below are taken from my logs (Apache/2.0.59 and PHP 5.2.1) [Fri May 25 10:01:24 2007] [notice] Parent: child process exited with status 3221225477 -- Restarting. [Fri May 25 10:01:24 2007] [notice] Apache/2.0.59 (Win32) PHP/5.2.1 configured -- resuming normal operations [Fri May 25 10:01:24 2007] [notice] Server built: Jul 27 2006 15:55:03 [Fri May 25 10:01:24 2007] [notice] Parent: Created child process 3828 [Fri May 25 10:01:24 2007] [notice] Child 3828: Child process is running [Fri May 25 10:01:24 2007] [notice] Child 3828: Acquired the start mutex. [Fri May 25 10:01:24 2007] [notice] Child 3828: Starting 64 worker threads. But that's quite transparent to the client, as it seems that after the restart, the page is served anyway. (but I get an ugly Windows error message box) So, I think the problems we are encountering are more due to Apache than to ModPerl. Lionel. - Original Message - From: "Foo JH" <[EMAIL PROTECTED]> To: "Lionel MARTIN" <[EMAIL PROTECTED]> Cc: "Perrin Harkins" <[EMAIL PROTECTED]>; Sent: Friday, May 25, 2007 10:29 AM Subject: Re: Is it advisable NOT to limit MaxRequestsPerChild in Win32 modperl? Hello Perrin and Lionel, Lionel MARTIN wrote: Unless I'm mistaken, and even if there is only one (child) Apache process and several thread under Windows (winnt MPM), the only directive that can be used is MaxRequestsPerChild and not MaxRequestsPerThread. It is not possible to restart individual threads (and associated Perl Interpreteers), but only restart the child (which means restarting all the threads at once). So, basically, the value indicated by MaxRequestsPerChild is shared among all threads. (and its defaults value is 0, which means that your child is never restarted because of having handled too many requests). Makes sense. I've tried inserting: MaxRequestsPerThread 2 to httpd.conf, but it complains that 'perhaps misspelled or defined by a module not included in the configuration'. Which is quite true: I don't see any tags for WinNT. Any suggestions? That's a bit of pity, because that's like an "all or nothing" behaviour, you can't kill individual threads because they are growing too much. I know. It's the curse of being a Windows developer. We're all screwed.
Re: Is it advisable NOT to limit MaxRequestsPerChild in Win32 modperl?
Hi, Actually I didn't see your original question, about the following error: [Mon Mar 05 21:19:47 2007] [notice] Parent: child process exited with status 3221225477 -- Restarting. Actually, I am not even sure that this is due to MP2. Indeed, on my development computer (running under Win XP, Apache/2.0.59), I am developing both PHP and Perl applications. When developing for PHP, my Apache Server doesn't know about Mod Perl. And yet, I'm keeping having the error you have, time to time, when loading my PHP pages. Actually, the messages below are taken from my logs (Apache/2.0.59 and PHP 5.2.1) [Fri May 25 10:01:24 2007] [notice] Parent: child process exited with status 3221225477 -- Restarting. [Fri May 25 10:01:24 2007] [notice] Apache/2.0.59 (Win32) PHP/5.2.1 configured -- resuming normal operations [Fri May 25 10:01:24 2007] [notice] Server built: Jul 27 2006 15:55:03 [Fri May 25 10:01:24 2007] [notice] Parent: Created child process 3828 [Fri May 25 10:01:24 2007] [notice] Child 3828: Child process is running [Fri May 25 10:01:24 2007] [notice] Child 3828: Acquired the start mutex. [Fri May 25 10:01:24 2007] [notice] Child 3828: Starting 64 worker threads. But that's quite transparent to the client, as it seems that after the restart, the page is served anyway. (but I get an ugly Windows error message box) So, I think the problems we are encountering are more due to Apache than to ModPerl. Lionel. - Original Message - From: "Foo JH" <[EMAIL PROTECTED]> To: "Lionel MARTIN" <[EMAIL PROTECTED]> Cc: "Perrin Harkins" <[EMAIL PROTECTED]>; Sent: Friday, May 25, 2007 10:29 AM Subject: Re: Is it advisable NOT to limit MaxRequestsPerChild in Win32 modperl? Hello Perrin and Lionel, Lionel MARTIN wrote: Unless I'm mistaken, and even if there is only one (child) Apache process and several thread under Windows (winnt MPM), the only directive that can be used is MaxRequestsPerChild and not MaxRequestsPerThread. It is not possible to restart individual threads (and associated Perl Interpreteers), but only restart the child (which means restarting all the threads at once). So, basically, the value indicated by MaxRequestsPerChild is shared among all threads. (and its defaults value is 0, which means that your child is never restarted because of having handled too many requests). Makes sense. I've tried inserting: MaxRequestsPerThread 2 to httpd.conf, but it complains that 'perhaps misspelled or defined by a module not included in the configuration'. Which is quite true: I don't see any tags for WinNT. Any suggestions? That's a bit of pity, because that's like an "all or nothing" behaviour, you can't kill individual threads because they are growing too much. I know. It's the curse of being a Windows developer. We're all screwed.
Re: Is it advisable NOT to limit MaxRequestsPerChild in Win32 modperl?
Unless I'm mistaken, and even if there is only one (child) Apache process and several thread under Windows (winnt MPM), the only directive that can be used is MaxRequestsPerChild and not MaxRequestsPerThread. It is not possible to restart individual threads (and associated Perl Interpreteers), but only restart the child (which means restarting all the threads at once). So, basically, the value indicated by MaxRequestsPerChild is shared among all threads. (and its defaults value is 0, which means that your child is never restarted because of having handled too many requests). That's a bit of pity, because that's like an "all or nothing" behaviour, you can't kill individual threads because they are growing too much. Lionel. - Original Message - From: "Perrin Harkins" <[EMAIL PROTECTED]> To: "Foo JH" <[EMAIL PROTECTED]> Cc: Sent: Friday, May 25, 2007 7:05 AM Subject: Re: Is it advisable NOT to limit MaxRequestsPerChild in Win32 modperl? On 5/24/07, Foo JH <[EMAIL PROTECTED]> wrote: This I accomplish by setting a small number (for testing) to MaxRequestsPerChild to see how modperl recovers from a reload. I think you're looking for MaxRequestsPerThread. There is only one process on the Windows MPM. - Perrin
Re: After retrieving data from DB, the memory doesn't seem to be freed up
On 5/12/07, Lionel MARTIN <[EMAIL PROTECTED]> wrote: Of course, I understand why $tmp value is reset during each call (the lexical variable goes out of scope) but, as we can see, the address used by the variable is changing as well during each request, and this fact doesn't seem consistent with what has been said before (i.e. that memory stayed allocated to lexical variables across requests)... I don't know but my guess would be that it's either a problem with your test or a quirk of Windows/threads. When you want to look at internals, you'll get better information using something like Devel::Peek. Actually, I just tried the same tests on a Linux distribution (Ubuntu 6.10 running Perl 5.8.8, Apache 2.0.5x and mod_perl 2.03) and the results are exactly the same: Test1 => changing memory location Test2 => fix memory location Test3 => changing memory location Test4 => fix memory location (I swapped Test3 and Test4 results in my previous post) To be sure there's only one Perl Interpreter, I used these directives: ThreadsPerChild 1 MaxClients 1 So, I can conclude that this not coming from my Test Environment, nor from Windows MPM, and that anyone should obtain the same results. Lionel.
Re: After retrieving data from DB, the memory doesn't seem to befreed up
Nope, I'm sure that all the requests are served by the same thread. Anyway, just to make you sure, I'm adding a directive to my httpd.conf file: ThreadsPerChild 1 This way, only one thread is started and can be serving requests. Actually, making several tests brought interesting results: Test 1: #** use strict; package mypack; my $tmp; $tmp .= 'a'; print '$tmp value: ' . $tmp; print ' - address: ' . \$tmp . "\n"; #** It brings: $tmp value: a - address: SCALAR(0xd7059c) $tmp value: a - address: SCALAR(0xcf4178) $tmp value: a - address: SCALAR(0xcf42a4) $tmp value: a - address: SCALAR(0xcf42bc) So, different memory blocks used for the same lexical variable. (not what we could expect) Test 2: #** use strict; package mypack; our $global; my $tmp; $tmp .= 'a'; print '$tmp value: ' . $tmp; print ' - address: ' . \$tmp; print ' - global address: ' . \$global . "\n"; #** Here, I'm testing the memory block of a global variable as well. This brings: $tmp value: a - address: SCALAR(0xcf4634) - global address: SCALAR(0xcf464c) $tmp value: a - address: SCALAR(0xcf4634) - global address: SCALAR(0xcf464c) $tmp value: a - address: SCALAR(0xcf4634) - global address: SCALAR(0xcf464c) $tmp value: a - address: SCALAR(0xcf4634) - global address: SCALAR(0xcf464c) As you can see, now, both the lexical and global variables are keeping their memory block (but the lexical is reset after each call, because going out of scope) So, when any global suymbol comes into play, it seems that lexicals are keeping their memory location. And to prove that, let's try: Test 3: #** use strict; package mypack; my $tmp; $tmp .= 'a'; print '$tmp value: ' . $tmp; print ' - address: ' . \$tmp . "\n"; $mypack::dummy; #** Here, this is the same as test 1, except that I'm referring to a global variable at the end of the script. Here, and contrary to test 1, the lexical is keeping its memory block: $tmp value: a - address: SCALAR(0xa39f3c) $tmp value: a - address: SCALAR(0xa39f3c) $tmp value: a - address: SCALAR(0xa39f3c) $tmp value: a - address: SCALAR(0xa39f3c) (=> consistant with Test2: referring to a global variable seems to force lexicals to keep their memory block) But surprisingly enough, if I run another test, similar to test3, but this time, my reference to the global variable is the last but one statement, instead of being the last one: Test 4: #** use strict; package mypack; my $tmp; $tmp .= 'a'; print '$tmp value: ' . $tmp; $mypack::dummy; print ' - address: ' . \$tmp . "\n"; #** This time, this brings: $tmp value: a - address: SCALAR(0xaf73c4) $tmp value: a - address: SCALAR(0xcf48d4) $tmp value: a - address: SCALAR(0xcf45c8) $tmp value: a - address: SCALAR(0xcf4598) So, just moving the global symbol statement one statement up messes up the results... All that is surprising, isn't it? Does anyone has any explanation? Anyway, if noone can explain that, I think I'll stop trying to understand everything and just assume that lexical are keeping their memory block, even if this doesn't seem to always be the case. Lionel. - Original Message - From: "Clinton Gormley" <[EMAIL PROTECTED] > To: "Lionel MARTIN" <[EMAIL PROTECTED]> Cc: "Perrin Harkins" <[EMAIL PROTECTED]>; Sent: Saturday, May 12, 2007 8:59 AM Subject: Re: After retrieving data from DB, the memory doesn't seem to befreed up #** use strict; package mypack; my $tmp; $tmp .= 'a'; print '$tmp value: ' . $tmp . ' - address: ' . \$tmp . "\n"; #** I am running this script in a ModPerl::Registry environment. In this situation, the first time I'm running the script, I can see this: main $tmp: a - address: SCALAR(0xd725e0) And then, in subsequent calls: $tmp value: a - address: SCALAR(0xd77bf4) $tmp value: a - address: SCALAR(0xd77d20) $tmp value: a - address: SCALAR(0xd77d38) Could it be that the request is being answered by different threads, each one with their own copy of the $tmp variable? Try restricting apache to using only a single thread, and repeat the test. Clint
Re: After retrieving data from DB, the memory doesn't seem to be freed up
Hi all (and Perrin), I understand the concepts being discussed here. However, no on replied one of my questions: Let's conside this script: #** use strict; package mypack; my $tmp; $tmp .= 'a'; print '$tmp value: ' . $tmp . ' - address: ' . \$tmp . "\n"; #** I am running this script in a ModPerl::Registry environment. In this situation, the first time I'm running the script, I can see this: main $tmp: a - address: SCALAR(0xd725e0) And then, in subsequent calls: $tmp value: a - address: SCALAR(0xd77bf4) $tmp value: a - address: SCALAR(0xd77d20) $tmp value: a - address: SCALAR(0xd77d38) Of course, I understand why $tmp value is reset during each call (the lexical variable goes out of scope) but, as we can see, the address used by the variable is changing as well during each request, and this fact doesn't seem consistent with what has been said before (i.e. that memory stayed allocated to lexical variables across requests)... I would have believed that, for the sake of efficiency (and that's why memory allocated to lexical variables is not released to Perl), the exact same memory location would have been used by the lexical variable in subsequent calls. So, obviously, as the memory is not released, and as new memory blocks are used for every call to host the lexical, this seems to me to be memory leak. Am I wrong or am I missing something? Lionel. I suppose I am still missing something, but what? Lionel.
Re: After retrieving data from DB, the memory doesn't seem to be freed up
-another example hat comes to my mind is a project (implemented in PHP) where I had to work. Part of the process was retrieving cached HTML template pages from the server and do BB tags parsing before serving the client. Of course, you would tell me that I could retrieve the data chunk by chunk, but this is not as obvious, as some BB tags could spread over several lines. So, this would need to devise an algorithm to be sure we are not cutting in the middle of a tag. In this kind of situation and if the file to be retrieved don't grow too large, I would prefer to retrieve the file all at once, do the processing, serve it to the client, and then, undef the buffer. Far easier than doing chunk by chunk. Same thing: you'd presumably wrap that data in an object. OK. Basically, I would use an object here, containing why not a scalar member. This way, when the reference to the object goes out of scope, the memory associated to the object would be freed as well. Finally, that's a devious way of doing what I was suggesting at the beginning: use a lexical variable (a scalar) and undef it. Wrapping that in an object would make the whole lot automatic (if I'm not mistaking). Lionel.
Re: After retrieving data from DB, the memory doesn't seem to be freed up
(sorry, I posted the message below from another email address hours ago, and
it didn't get publish - see below if anyone could reply)
On May 10, 2007, at 6:52 PM, Andy Armstrong wrote:
On 10 May 2007, at 23:48, Jonathan Vanasco wrote:
that also means that variables are sticky -- if you change something
from $a= "b" to $a.= "b" , you'll see the var appending b on every
iteration
No you don't:
sub frog {
my $x;
$x .= 'x';
return $x;
}
sorry, i should have clarified, I meant that to be:
| - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- - - - - - - - - - - - - - - - -
package frog;
my $x;
sub leap {
$x .= 'x';
return $x;
}
Yes, here, that's consistant with what we were saying yesterday:
$x is a lexical variable, so the memory allocated to it remains allocated
across requests.
But here, as $x is referenced by your leap sub, which is a global symbol, $x
content is never erased.
Actually, I tried something else that surprised me a bit:
use strict;
package mypack;
my $tmp;
$tmp .= 'a';
print 'main $tmp: ' . $tmp . ' - address: ' . \$tmp . "\n";
I am running this script in a ModPerl::Registry environment.
In this situation, the first time I'm running the script, I can see this:
main $tmp: a - address: SCALAR(0xd725e0)
And then, in subsequent calls:
main $tmp: a - address: SCALAR(0xd77bf4)
main $tmp: a - address: SCALAR(0xd77d20)
main $tmp: a - address: SCALAR(0xd77d38)
Of course, I understand why $tmp value is reset during each call (the
lexical variable goes out of scope) but, as we can see, the address used by
the variable is changing as well during each request, and this fact doesn't
seem consistent with what has been said before (i.e. that memory stayed
allocated to lexical variables across requests)...
I suppose I am still missing something, but what?
Lionel.
Re: After retrieving data from DB, the memory doesn't seem to befreed up
Hi, I think you got me wrong. My initial question was basically something like "how could I preserve/give back memory if needed" (i.e. in rare situations) and the reply turned up into a "don't use large scalars". (which is relevant, I agree, but was not directly replying my initial question) - use Apache2::SizeLimit to kill off a process if it gets too big (but doesn't work under windows) Unfortunately, I am developing under Windows, so... - force your child process to exit after serving the request if you have to do something big (eg process a large image, generate a PDF) Obviously, for the second case, I'm assuming that you would do these things on a small percentage of your total requests, otherwise killing off your child would be a major bottleneck. Here, as well, killing a child process under Windows, means killing my whole Apache server (and so all the serving threads). So, that's out of the equation. Lionel.
Re: After retrieving data from DB, the memory doesn't seem to be freed up
Lionel MARTIN wrote: - Don't load large amounts of data into scalars. Fine. Now I know why. But sometimes, you don't have the choice. I'd like to know what situations you encounter where you are forced to load large amounts of data into scalars. I can't think of any. I don't have any clear situations right here in mind, but we could imagine many: -for example, a bulletin board system, where you are retrieving posted message from a DB. Each message could weigh several dozens of kilo. (especially if you store HTML formatting in the DB) -another example hat comes to my mind is a project (implemented in PHP) where I had to work. Part of the process was retrieving cached HTML template pages from the server and do BB tags parsing before serving the client. Of course, you would tell me that I could retrieve the data chunk by chunk, but this is not as obvious, as some BB tags could spread over several lines. So, this would need to devise an algorithm to be sure we are not cutting in the middle of a tag. In this kind of situation and if the file to be retrieved don't grow too large, I would prefer to retrieve the file all at once, do the processing, serve it to the client, and then, undef the buffer. Far easier than doing chunk by chunk. Lionel.
Re: After retrieving data from DB, the memory doesn't seem to be freed up
Hi again,
If you have a lot of scripts that, for
example, load large data files, make a module with a sub that you call
to load the files and use it from all your scripts. Pass the data by
reference to the calling scripts. Then you will isolate the large
lexical in that one module.
Better yet, don't load a large file into memory.
I'll see what I can do when the problem arises, but of course, I now know I
should better avoid loading too much data in RAM at once, because of data
persistance.
All this discussion has been made from a theoritical point of view, I don't
have any practical situation in mind right now, but knowing the pros and
cons, I'll choose the best way when problems arise.
I tried to use "mysql_use_result" this morning, but without success.
There's probably just a syntax mistake in your script somewhere. This
is what I use, to turn it on for specific statement handles:
my $sth = $dbh->prepare($sql, {mysql_use_result => 1});
I am not sure this is a syntax problem (and I tried your syntax as well).
for example, when I then create a statement like:
$sth = $dbh->prepare($sql);
I can test $sth->{mysql_use_result}, and it brings 1, showing that my wish
has been taken into account by the compiler.
Actually, I tried to do the same with PHP, using mysql_unbuffered_query
(because here, when we talk about mysql_use_result, we are talking about
unbuffered queries) and the result is the same: my query data is fetched
from the server all at once.
So, actually, I suspect there is a problem somwhere with my MySQL libraries
or so (I don't really know where to investigate).
So, for now, there's no way for me to use unbuffered queries...
Lionel.
Re: After retrieving data from DB, the memory doesn't seem to be freed up
Hi Perrin,
No, if the message you're getting is to use globals instead of
lexicals then you've got it all wrong.
- Use lexicals for everything, unless you actually want the value to
persist across requests.
I was thinking, for large amounts of data, about using globals.
Indeed, imagine you have several different scripts that could potentially
use large amounts of data.
Then, using a global variable would save me the need to allocate for each
script different memory locations that would anyway never be used at once
(memory is allocated per Perl Intepreter)
For example, instead of using a "my $bigdata" in 50 different script, that
could this result in big memory allocation, I would use a unique
$mypack::bigdata in each script, so that each script would share the same
memory address for this potentially large data.buffer.
I know it is not elegant, but this could help me save lots of memory.
Another solution would be to undef potentially big lexical variables after
they are used, so that the memory could be used by other scripts.
- Don't load large amounts of data into scalars.
Fine. Now I know why. But sometimes, you don't have the choice.
- Use mysql_use_result if you have to load a very large result set.
Ideally you should avoid this by using LIMIT and OFFSET.
I tried to use "mysql_use_result" this morning, but without success.
Actually, I tried this script:
use strict;
package mypack;
use DBI;
my $dbh;
my $sth;
$dbh = DBI->connect('DBI:mysql:boardwalk;host=82.228.218.209;', 'root', '');
$dbh->{mysql_use_result} = 1;
$sth = $dbh->prepare('SELECT * FROM account');
$sth->execute();
Normally, as mysql_use_result is set, if I understand well how things should
be working, , no data should be retrieved from the DB at this stage (but
only when fetching rows).
However, all the query data is fetched anyway. (I could see the packets
transitting from MySQL). I don't exactly know why, and am wondering if there
is a setting to enable its use from MySQL point of view. (I am using MySQL
Server 5.0.27 in a Windows installation)
Does that make sense?
- Perrin
Re: After retrieving data from DB, the memory doesn't seem to be freed up
I would even say PER THREAD or PER PERL INTERPRETER. Indeed, I'm running Per/mod perl under Windows, and there's one unique Apache process (except the parent one) hosting all perl interpreters. Something to ask about modperl and memory in Windows. I know modperl uses threads in Windows. But does modperl respond to the MaxRequestsPerChild setting? In other words, if I specify that the 'child' should serve only 100 requests before respawning, does Perl free up all memory allocated by the thread before respawning the thread? Actually, under Windows, all Apache Threads and Perl Interpreters are living under the same child process. (which is spawned from the Apache parent process at server startup) The MaxRequestsPerChild directives is working under windows, and it specifies the total number of requests all your threads can process. After this number of threads is processed by all the threads, the child process is killed (and all its threads altogether), and spawned again from the parent process. So, for example, if you specify MaxRequestsPerChild 100 And then, if thread 1 processes 40 requests thread 2 processes 35 requests thread 3 processes 25 requests The whole process is started again. Of course, I'm talking here about independant requests being processed (i.e. requests under the same "Keep Alive timeout" count for one) Lionel.
Re: After retrieving data from DB, the memory doesn't seem to be freed up
Thanks for this. well, to complicate things, this behavior is PER CHILD... meaning: if you run test1.pl 2x and they are both served by the same apache child, then the memory will be reused. if you run test1.pl 2x and they are both served by the different apache children, then the memory will be not be reused. I would even say PER THREAD or PER PERL INTERPRETER. Indeed, I'm running Per/mod perl under Windows, and there's one unique Apache process (except the parent one) hosting all perl interpreters. Many docs I'm reading are assuming there isonly one Perl interpreter per Apache Process, which is not true for me. (and even for *nix users that are using the worker MPM)
Re: After retrieving data from DB, the memory doesn't seem to be freed up
- Original Message -
From: "Andy Armstrong" <[EMAIL PROTECTED]>
To: "Lionel MARTIN" <[EMAIL PROTECTED]>
Cc:
Sent: Friday, May 11, 2007 12:34 AM
Subject: Re: After retrieving data from DB, the memory doesn't seem to be
freed up
On 10 May 2007, at 23:25, Lionel MARTIN wrote:
OK, fine.
So, to sum up, if I have got 10 different scripts in a mod perl
environment (let's call them test1.pltest10.pl), and using lexical
variables there.
If I first run test1.pl and then, run test2.pl, the only way for test2.p
to get access to the memory used by test2.pl is freeing up test1.pl
lexical variables, by undefining them?
And what if I run test1.pl twice without undefining its lexical
variables? Will the same memory space be used twice, or will each call
use different memory space (I'm talking here about situation where the
same Perl interpreter is running the script twice in a mod perl
environment)?
The memory allocated /directly/ to a lexical remains allocated at the end
of the scope on the assumption that it might be needed again in the
future. That's so that when you call something like
sub string_builder {
my $s = '';
$s .= 'x' for 1 .. 1_000_000;
return $s;
}
repeatedly it's able to reuse memory from the previous invocation rather
than allocating it all from scratch each time. That's mainly a
performance optimisation.
However memory referred to indirectly as in your example
$x = [ 1 .. 1_000_000 ];
is freed as soon as nothing refers to it any more. So when $x goes out of
scope that storage /is/ freed - to Perl at least.
Please don't get the idea that Perl never frees anything and please don't
start littering your code with unnecessary explicit memory management.
--
Andy Armstrong, hexten.net
That's why I have been asking all these questions. That's only when
understanding things that you don't do unnecessary or take counter
productive measures.
Thanks,
Lionel.
Re: After retrieving data from DB, the memory doesn't seem to be freed up
OK, fine. That's something important to know then. If my website is made of hundreds of different pages, then, I should better ensure that lexical variables are properly freed up. Even if that's not good programming practive, at least, with global variables, we are sure that the same memory locations are used across different scripts. (which is not the case with lexical variables) Thanks for all your help, I now have a clearer picture. Lionel. - Original Message - From: "Perrin Harkins" <[EMAIL PROTECTED]> To: "Lionel MARTIN" <[EMAIL PROTECTED]> Cc: Sent: Friday, May 11, 2007 12:29 AM Subject: Re: After retrieving data from DB, the memory doesn't seem to be freed up On 5/10/07, Lionel MARTIN <[EMAIL PROTECTED]> wrote: So, to sum up, if I have got 10 different scripts in a mod perl environment (let's call them test1.pltest10.pl), and using lexical variables there. If I first run test1.pl and then, run test2.pl, the only way for test2.p to get access to the memory used by test2.pl is freeing up test1.pl lexical variables, by undefining them? Yes. And what if I run test1.pl twice without undefining its lexical variables? Will the same memory space be used twice, or will each call use different memory space (I'm talking here about situation where the same Perl interpreter is running the script twice in a mod perl environment)? It will reuse the already allocated memory. And the second pass is faster because the memory is already allocated. - Perrin
Re: After retrieving data from DB, the memory doesn't seem to be freed up
OK, fine. So, to sum up, if I have got 10 different scripts in a mod perl environment (let's call them test1.pltest10.pl), and using lexical variables there. If I first run test1.pl and then, run test2.pl, the only way for test2.p to get access to the memory used by test2.pl is freeing up test1.pl lexical variables, by undefining them? And what if I run test1.pl twice without undefining its lexical variables? Will the same memory space be used twice, or will each call use different memory space (I'm talking here about situation where the same Perl interpreter is running the script twice in a mod perl environment)? Lionel. - Original Message - From: "Perrin Harkins" <[EMAIL PROTECTED]> To: "Lionel MARTIN" <[EMAIL PROTECTED]> Cc: Sent: Friday, May 11, 2007 12:17 AM Subject: Re: After retrieving data from DB, the memory doesn't seem to be freed up On 5/10/07, Lionel MARTIN <[EMAIL PROTECTED]> wrote: > There's really no difference in the way globals behave under mod_perl. > You just don't notice it under CGI because the process quits right > after the request has been served. [...] So, this clearly shows that the global variables sticks in memory, while the lexical one doesn't. So, I would imagine that after the script is run, space used by the lexical variable would be freed up. That's what I'm telling you -- it won't be. The value will be gone, but the memory allocated to it will still be allocated to it. It won't be reused for other variables unless you undef $lexical. The other thing I was trying to explain is that the behavior of globals and lexicals is not affected by mod_perl. They behave the same way in any perl program. - Perrin
Re: After retrieving data from DB, the memory doesn't seem to be freed up
-I'll try to use, in my Perl scripts, lexical variables instead of global variables, so that I'm sure the used memory can be resued for later requests (global variables, on the other hand, stick in memory, due to mod perl way of operating) Not really. In terms of memory, there's no difference. The reason you should use lexicals is that it's a better programming style. There's really no difference in the way globals behave under mod_perl. You just don't notice it under CGI because the process quits right after the request has been served. Here, I'm getting confused. Let's imagine I'm running the following script though ModPerl::PerlRun or ModPerl::Registry: use strict; package mypack; my $lexical; our $global; print lexical: ' . $lexical. ' # global: ' . $global; $lexical= 2007; $global = 2007; This script would print: lexical: # global: the first time it is run and then: lexical: # global: 2007 So, this clearly shows that the global variables sticks in memory, while the lexical one doesn't. So, I would imagine that after the script is run, space used by the lexical variable would be freed up. Am I wrong?
Re: After retrieving data from DB, the memory doesn't seem to be freed up
When the array goes out of scope its reference count is decremented. If the reference count goes to zero (implying there are no more references) the memory is released. I would have believed the same, and that's why I believed that $tmp = [0..10]; followed by $tmp = 1; would free memory (no more reference to the anonymous array), but Perrin is telling me this is not the case.
Re: After retrieving data from DB, the memory doesn't seem to be freed up
Hi, I can see this is a very complex topic, that would require me to delve into Perl C surce code in order to really understand what's going on behind the scenes. So, for now, I'll limit myself to these strategies (and please tell me if I'm wrong): -I'll try avoid using large chunks of data so that interpreters memory footprint don't grow too big. -I'll try to use, in my Perl scripts, lexical variables instead of global variables, so that I'm sure the used memory can be resued for later requests (global variables, on the other hand, stick in memory, due to mod perl way of operating) I found an interesting topic here: http://www.nntp.perl.org/group/perl.perl5.porters/2006/03/msg111095.html Lionel. - Original Message - From: "Perrin Harkins" <[EMAIL PROTECTED]> To: "Lionel MARTIN" <[EMAIL PROTECTED]> Cc: Sent: Thursday, May 10, 2007 7:57 PM Subject: Re: After retrieving data from DB, the memory doesn't seem to be freed up On 5/10/07, Lionel MARTIN <[EMAIL PROTECTED]> wrote: On Windows XP, it seems to me that Perl never gves back memory to the OS, even after I undef variables or after they go out of scope. That's pretty common. Perl will be able to use the memory for other variables though. But then, I'm wondering why and how comes, in the example I was giving below, after $sth->finish, I could see that the Perl.exe process was shrinked and used less memory? (meaning that $sth->finish made Perl renders the memory to the OS) Some of this is dependent on your OS and compiler. I can't tell you why it sometimes gives memory back to the OS. I can tell you not to count on it. Are you saying this because, for example, if a Perl interpreter is using a 100Megs buffer to read a file, after the file is read, even if the memory can be used again by the Perl interpreter (which means we are not talking about memory leak here), it will never be given back to other processes/interpreters? Yes. 1) When a variable is undefined or goes out of scope, can I be sure that the memory that was used by it is straight away rendered to Perl so that it can use it for other variables? No. Those are two different things. If you explicitly undef it, the memory gets handed back to Perl: undef $foo; If it just goes out of scope, the memory stays allocated to that variable. 2) If I have a reference to a big array, like: $tmp = [1..100]; Does a : $tmp = 1; or a $tmp = undef; or a $#$tmp = -1; gives the memory back to Perl so that it can use it for other purposes? $tmp is just a reference, and doesn't take much memory at all. I'm not sure how you can clear memory that might be allocated to the anonymous array, or exactly what perl will do with it when the array goes out of scope. You can ask on p5p or perlmonks.org if you're really interested. Clearing an array is something like this: @array = (); If I am asking this, it is specifically to know if, when using large amounts of data, I should undef if possible the variables before allocating others, so that processes don't grow too big? No, you shouldn't. That would be a painful way to code. Instead, you should structure your program so that it never loads large amounts of data into memory. - Perrin
Re: After retrieving data from DB, the memory doesn't seem to be freed up
Hi Perrin, Thanks for your reply. On Windows XP, it seems to me that Perl never gves back memory to the OS, even after I undef variables or after they go out of scope. For example, I can see it with this code: use strict; print "Step 1\n"; ; my $x = 'a'x10**7; print "Step 2\n"; ; $x = undef; print "Step 3\n"; ; which gives: Step 1 > 1800k Step 2 > 21412k Step 3 > 21416k #no memory given back to the OS here But then, I'm wondering why and how comes, in the example I was giving below, after $sth->finish, I could see that the Perl.exe process was shrinked and used less memory? (meaning that $sth->finish made Perl renders the memory to the OS) What this means for you is that you should never do things like read an entire large file into a scalar, since that will permanently increase the size of that apache process. There's a lot of advice about this here: http://modperlbook.org/html/ch14_02.html Are you saying this because, for example, if a Perl interpreter is using a 100Megs buffer to read a file, after the file is read, even if the memory can be used again by the Perl interpreter (which means we are not talking about memory leak here), it will never be given back to other processes/interpreters? (I'm saying interpreters, because on Windows, all Perl interpreters are running withinin the same Apache child process) I have got two more questions: 1) When a variable is undefined or goes out of scope, can I be sure that the memory that was used by it is straight away rendered to Perl so that it can use it for other variables? Or does the garbage collection process runs "sometimes later on, but we don't know when"? (or better said: is the garbage colllection process synchronous or asynchronous?) 2) If I have a reference to a big array, like: $tmp = [1..100]; Does a : $tmp = 1; or a $tmp = undef; or a $#$tmp = -1; gives the memory back to Perl so that it can use it for other purposes? If I am asking this, it is specifically to know if, when using large amounts of data, I should undef if possible the variables before allocating others, so that processes don't grow too big? Thanks, Lionel. ----- Original Message - From: "Perrin Harkins" <[EMAIL PROTECTED]> To: "Lionel MARTIN" <[EMAIL PROTECTED]> Cc: Sent: Thursday, May 10, 2007 4:17 PM Subject: Re: After retrieving data from DB, the memory doesn't seem to be freed up On 5/10/07, Lionel MARTIN <[EMAIL PROTECTED]> wrote: I suspect that this is coming from data retrieved from my DB not being well freed. That's right. There are two things you need to understand here. The first is that Perl doesn't free memory from variables unless you tell it to, even when they go out of scope. This is an optimization, and it works in your favor if you use the memory again. You can free the memory from a variable by undef'ing it, but that just frees it up for Perl to use elsewhere in the current process. (This is somewhat operating system specific. You may see processes shrink on some systems after freeing memory, but not always.) What this means for you is that you should never do things like read an entire large file into a scalar, since that will permanently increase the size of that apache process. There's a lot of advice about this here: http://modperlbook.org/html/ch14_02.html The second is that the MySQL client library will load the entire result set into your process unless you tell it not to. This is done with the mysql_use_result option, discussed in the MySQL docs and also here: http://modperlbook.org/html/ch20_02.html - Perrin
After retrieving data from DB, the memory doesn't seem to be freed up
Hi,
I am running an Apache 2.059 server, on a Win32 platform (Perl 5.8 & mod_perl
2.03).
During the lifetime of the server, I cn see that the memory footprint of the
Apache process is getting bigger and bigger, so I would like to understand why.
I can't say this is really a problem, but I would like to foresee any potential
problem during production.
I suspect that this is coming from data retrieved from my DB not being well
freed.
In order to isolate the problem, I tried to run some Perl script (using the
Perl command line) and see how the memory footprint is evolving.
Here is a simple script:
use strict;
use DBI;
print "Before anything\n";
; #step1
my $dbh = DBI->connect('DBI:mysql:mydb;host=localhost;', 'root', '');
my $sth = $dbh->prepare('SELECT * FROM mytable);
print "After prepare\n";
; #step2
$sth->execute;
print "After execute\n";
; #step3
$sth->finish();
print "After finish\n";
; #step4
As you can see, I am using some in order to pause the program.
In order to make things stand out more, I am running a query that retrieves
over 100 000 rows, and 8 fields.
The memory taken by the Perl.exe process is, depending on the step:
-step1: 4196k (here, nothing is run)
-step2: 6008k (here, the statement has been prepared)
-step3: 23888k (here, the statement has been executed, and so, the data
retrieved from the MySQL server. I suppose DBI is getting it in a raw format)
-step4: 6092k (I am calling finish to tell DBI I don't need the data anymore,
and we are back to a fair memory level, so, data is freed well)
So, as you can see, in the situation described above, a lot of data is
retrieved from the DB, but is then freed up well, so I can be confident that
this wouldn't cause any problem in a mod_perl environment.
Now, let's tweak up a bit my script, and try to get a grip over the retrieved
data this way:
use strict;
use DBI;
print "Before anything\n";
; #step1
my $dbh = DBI->connect('DBI:mysql:mydb;host=localhost;', 'root', '');
my $sth = $dbh->prepare('SELECT * FROM mytable);
print "After prepare\n";
; #step2
$sth->execute;
print "After execute\n";
; #step3
my $ary_ref = $sth->fetchall_arrayref();
$#$ary_ref = -1;
print "After fetch\n";
; #step4
Here, the beginning is the same, and the memory footprint as well. Thing start
t differ in step4:
-step1: 4196k (here, nothing is run)
-step2: 6008k (here, the statement has been prepared)
-step3: 23888k (here, the statement has been executed, and so, the data
retrieved from the MySQL server. I suppose DBI is getting it in a raw format)
-step4: 71940k (!!)
As you can see, in step4, I am retrieving in a reference to a big array all the
data from the DB handle, but then ask the array to free itself. Nevertheless,
the memory footprint doesn't go back to normal.
And when run into mod_perl, the same happens: in the first situation, the
Apache.exe memory footprint goes back to normal, while it remains high in the
second situation (which could lead to problem for further requests)
So, I would like to know if this really matters and if I should worry or
perhaps if I am missing something.
I am really afraid of memory leaks.
Of course, in a real scenario, I should never retrieve that much data but I did
it this way to try unveiling potential problems.
Lionel.
Re: Apache::DBI - Can two different connection handles have the same reference/address at different times?
Yes absolutely. What you are trying to do is re-invent some of the existing functionality of both Apache::DBI and DBI. DBI can cache prepared statements so you don't have to prepare them everytime and Apache::DBI can pool those statements' connections so you don't have to re-create them everytime. In a mod_perl environment you should almost always be using prepare_cached(). In fact, in this precise situation, what I have to do is examine some old code, written by another team, and try to point out potential bugs, and that's exactly why I was asking this question. (I am not trying to reinvent the wheel, that's forbidden in Perl ;) ) Basically, this old code, for each request, was using Apache::DBI to retrieve DB connection handles, but was storing prepared statement handles in package variables. Then, during later calls, they assumed that if the current DB handle was the same as the previous one (i.e. same address), then, the stored statement handle could be reused wihout being prepared again. So,according to what you are telling me, in rare situations, I think this could lead to bugs. I had a quick look at Apache::DBI::connect code, and basically, what it does, when you ask for a DB connection handle (with the same attributes as previous calls), is first of all testing the current cached DB connection handle (via a ping call) and in case pinging fails, it calls DBI::connect to get a new connection. So, in case DBI::connect returns the same address for the DB handle (but with a new connection), using an already prepared statement would fail... Thanks for your reply. Lionel.
Re: Apache::DBI - Can two different connection handles have the same reference/address at different times?
OK Fine. But will this be compatible with Apache::DBI? Moreover, I would be happy to have a reply to my original question, at least for the sake of knowledge. Thanks. - Original Message - From: "Michael Peters" <[EMAIL PROTECTED]> To: "Lionel MARTIN" <[EMAIL PROTECTED]> Cc: Sent: Monday, May 07, 2007 8:35 PM Subject: Re: Apache::DBI - Can two different connection handles have the same reference/address at different times? Lionel MARTIN wrote: This would save me the need to prepare at every request, while I could benefit from old prepared statements. Just use prepare_cached() -- Michael Peters Developer Plus Three, LP
Apache::DBI - Can two different connection handles have the same reference/address at different times?
Hello,
As you know,
use Apache::DBI;
DBI->connect()
returns a db/connection handle.
I am wondering if it is possible, at different times during the server life,
that Apache::DBI returns twice the same memory reference, while the connection
is in fact physically not the same.
The reason for this question is simple: I would like to know if I can reuse
statement handles (for example doing a $sth->execute) without having to
recreate them if the conection handle is the same with an algorithm like this
simplified one:
our $sth;
our $currentdbh ;
our $olddbh;
$currentdbh = DBI->connect();
if ($currentdbh != $olddbh){
$sth = $currentdbh->prepare(...)
}
$sth->execute();
$olddbh = $currentdbh;
This would save me the need to prepare at every request, while I could benefit
from old prepared statements.
What I'm afraid of, by doing that, are situations where db handles appear to be
the same (same address) while they are not in fact the same.
Thanks,
Lionel.
Re: Weird behaviour with strings and accents
Hi,
Having $VAR1 = ["\x{e0} pr\x{e9}sent prot\x{e9}g\x{e9}"] or $VAR1 =
["\x{e0} pr\x{e9}sent prot\x{e9}g\x{e9}login774"];
is really what you were expecting, i.e. true latin1 one byte characters. (it
appears that your dump shows the hexa code of non ASCII chars, i.e. with hex
code above \x{a0}, but that's the same).
On the other hand, it appears that when you are not doing "use UTF8", then,
everytime you are handling a string, it is turned into UTF8, like your "[no
utf8]" section shows. (perhaps because your source code is written in UTF8?)
I don't know why, but anyway, it appears that [use uft8] solves your
problem.
Lionel.
- Original Message -
From: "Cyril SCETBON" <[EMAIL PROTECTED]>
To: <[EMAIL PROTECTED]>
Cc:
Sent: Thursday, December 28, 2006 7:18 PM
Subject: Re: Weird behaviour with strings and accents
[EMAIL PROTECTED] wrote:
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On Wed, Dec 27, 2006 at 04:56:50PM +0100, Cyril SCETBON wrote:
I'm using Modperl 2.0.2 with apache 2.0.55 and modperl is doing
something wrong with my string :
my $var="à présent protégé";
warn Dumper [$var];
my $var="à présent protégé".$login;
warn Dumper [$var];
$VAR1 = [
'à présent protégé'
];
$VAR1 = [
"\x{c3}\x{a0} pr\x{c3}\x{a9}sent
prot\x{c3}\x{a9}g\x{c3}\x{a9}login774"
];
__END__;
The accents are no more printed correctly !!
Hm. So it seems that Perl changes its notion of string encoding when
modifying it. Curious.
Your source seems to be UTF-8. Have you tried to invoke the pragma "use
utf-8"?
Here you can see the results when I use pragma utf8 :
[use utf8]
$VAR1 = [
"\x{e0} pr\x{e9}sent prot\x{e9}g\x{e9}"
];
$VAR1 = [
"\x{e0} pr\x{e9}sent prot\x{e9}g\x{e9}login774"
];
[no utf8]
$VAR1 = [
'à présent protégé'
];
$VAR1 = [
"\x{c3}\x{a0} pr\x{c3}\x{a9}sent
prot\x{c3}\x{a9}g\x{c3}\x{a9}login774"
];
It's really weird, isn't it ???
Regards
- -- tomás
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.1 (GNU/Linux)
iD8DBQFFk1WbBcgs9XrR2kYRAuWUAJ4jOouhO0F5SjMIlRQFRZl5aGIGUwCfSi4Y
9wzCCGflvOVt4nd8M7bVNjI=
=AAPm
-END PGP SIGNATURE-
Re: Getting post data
Hi,
You should use libapreq2, which is a library that globally allows to deal
with data submitted by the client.
You can find documentation about it here:
http://search.cpan.org/~joesuf/libapreq2-2.08/glue/perl/lib/Apache2/Request.pm
Concerning CGI.pm, except if you are already using it for other purposes, I
would advise you not to load and use it, and use libapreq2 instead which is
(far) lighter and more efficient.
Re large data uploads efficiency, I'm not able to reply right now, but I'm
sure others in this list could provide a reply.
Lionel.
- Original Message -
From: "Erik Norgaard" <[EMAIL PROTECTED]>
To:
Sent: Thursday, September 28, 2006 10:03 AM
Subject: Getting post data
Hi:
I am using Apache 2.2 with mod_perl2. Reading the documentation I have
found that I should get POST data using read():
$r->read($buffer, $r->headers_in('content-length'))
A few questions - ok, some may really be related to the http protocol:
1) Is the headers_in case sensitive? (content-length ne Content-Length)?
2) Once in, any tricks or modules for parsing the data?
3) How about file-uploads, are these binary or base64?
4) How do I efficiently handle large posts (such as file uploads)?
I know I could bang my head against the keyboard and eventually get some
working code out, but it seems that these are quite general issues, so
some solution must exist.
Other question: modperlbook.org is down, is there a mirror or place I can
download it?
Thanks, Erik
--
Ph: +34.666334818 web: http://www.locolomo.org
X.509 Certificate: http://www.locolomo.org/crt/8D03551FFCE04F0C.crt
Key ID: 69:79:B8:2C:E3:8F:E7:BE:5D:C3:C3:B1:74:62:B8:3F:9F:1F:69:B9
Re: mod_perl make test failed tests
To be honest, I'm running Apache, mod_perl and DBI under Win32 (+Mason) and I can't complain about it. I'd like to test and compare with LAMP to see what's the best (in term of speed and memory managament), but what you told me made me jump, thinking "Am I doing a mistake when targetting to use Win32 on a production server?" Let me have your conclusuions if you can, Thanks, Lionel. - Original Message - From: "Philip M. Gollucci" <[EMAIL PROTECTED]> To: "Lionel MARTIN" <[EMAIL PROTECTED]> Cc: "Maurice Yarrow" <[EMAIL PROTECTED]>; Sent: Monday, June 05, 2006 8:12 PM Subject: Re: mod_perl make test failed tests Lionel MARTIN wrote: I saw that in DBI docs, but I just thought this piece of info was a bit dated. But, now, you're saying the same. Does this mean that you would not use any Win32 platform for production, as Win32 Apache MPMs are threaded, and Win32 Perl distributions compiled with ithreads enabled to fit the threaded MPMs? I can't speak about Win32 at all. Personally, I wouldn't use it anywhere if you paid me, but hey thats just me. I haven't actually tried worker + DBI + mod_perl2 recently. Last time I got ugly warnings in the logs telling me not to do this. I'll try again tonightish and see what happens. -- Philip M. Gollucci ([EMAIL PROTECTED]) 323.219.4708 Consultant / http://p6m7g8.net/Resume/resume.shtml Senior Software Engineer - TicketMaster - http://ticketmaster.com 1024D/A79997FA F357 0FDD 2301 6296 690F 6A47 D55A 7172 A799 97F "It takes a minute to have a crush on someone, an hour to like someone, and a day to love someone, but it takes a lifetime to forget someone..."
Re: mod_perl make test failed tests
Hi all, This post is not related to the main topic, but Philip, you said: I think DBI seems to be making progress on CLONE() ing $dbh handles, at least from the test results I got/saw over on dbi-dev (at) perl (dot) org for the 1.51-RC1 just posted. Its docs still say not to use DBI with ithread enabled perl in production, so I'd go with that recommendation. I saw that in DBI docs, but I just thought this piece of info was a bit dated. But, now, you're saying the same. Does this mean that you would not use any Win32 platform for production, as Win32 Apache MPMs are threaded, and Win32 Perl distributions compiled with ithreads enabled to fit the threaded MPMs? This is a bit frightening.
Re: Where do the "warn" message go? (Ap2 & MP2 on Win32)
Hi, Thanks for trying to help me. Replying each of your questions... - Original Message - From: "Malcolm J Harwood" <[EMAIL PROTECTED]> To: Cc: "Lionel MARTIN" <[EMAIL PROTECTED]> Sent: Thursday, May 18, 2006 3:41 PM Subject: Re: Where do the "warn" message go? (Ap2 & MP2 on Win32) On Thursday 11 May 2006 02:00 am, Lionel MARTIN wrote: Hi, I'v really tried to reduce everything to a minimum to insulate the problem. So, here's the config file: # ServerName MYSERVER ServerRoot "C:\Program Files\Apache Group\Apache2\TestingFromScratch\ApacheModPerl" On the offchance, try taking the space out of the filename. DocumentRoot "htdocs" Listen 80 LoadModule mime_module ../../modules/mod_mime.so LoadModule log_config_module ../../modules/mod_log_config.so # for mod_perl LoadFile "c:/TempApache/Perl/bin/perl58.dll" LoadModule perl_module ../../modules/mod_perl.so SetHandler perl-script PerlResponseHandler ModPerl::Registry PerlOptions +ParseHeaders What happens if you change that to ModPerl::PerlRun? This could have been the issue, as PerlRun is not setting the environments variables, but I tried, and this is still not working. TransferLog logs/access.log ErrorLog logs/error.log TypesConfig conf/mime.types LogLevel debug # you get an appropriate entry in the access log? Yes. and here's the script (/perl/warning.pl): # warn "Hello You Script\n"; use strict; print "Content-Type: text/html\n\n"; print "I think I issued a warning"; # Does it make any difference if you move the warn to the end (it shouldn't)? No. Does anything different happen if you access the page more than once? If you shut down the server afterwards? Things I tried many times. No. What happens if you run in single process mode? Here, I'm not sure to understand what you mean. Under Windows, Apache is running in a threaded MPM, which means there are always 2 distinct processes: the parent process, and the unique child process, containing the many different threads replying the requests. I don't know the windows side of things, so I don't know if that makes any difference. I'm not seeing anything wrong with your config. On unix I'd look for permissions problems. Does apache run as a seperate user or with changed permissions after startup? In fact, I'm really surprised that my warn messages are not working, while I stripped down my configuration to a minimum level. Interestingly, if I'm doing: print STDERR "test1\n"; warn "test2\n"; in httpd.conf, both test1 and test2 messages go to shell and to error.log as well. Is there any Perl variables I could be checking to understand a bit more the reasons of my warn messages not working? Thanks, Lionel. -- "But life has to be more than just a pulse-beat. What we hold sacred gives our lives meaning." Sinclair in Babylon 5: Believers
Re: Cannot load mod_perl.so
Hi Tracy, You're telling me that mod_perl is the binary you downloaded from the mod_perl site... Could you be more precise? How can you confirm that it has been compiled with the same compiler as the ActivePerl binary? That's likely causing the problem you are describing. I think that the best is really to install the modperl binary from the ActiveState repository. You can easily do that following these steps: -go to a command prompt and type ppm => this launches the ppm tool -type "rep add winnipeg2 http://theoryx5.uwinnipeg.ca/cgi-bin/ppmserver?urn:/PPMServer58"; (without the quotes) -type "install mod_perl-2.2" and follow instructions from there. This will give you a fresh mod_perl.so file, and you'll then load it in your httpd.conf. I hope this will be working. Lionel. "Tracy E Schreiber" <[EMAIL PROTECTED]> a écrit dans le message de news: [EMAIL PROTECTED] "Tracy E Schreiber" <[EMAIL PROTECTED]> wrote in message news:... Hi Lionel, Thank you for your response I have these lines in httpd.conf: LoadFile "D:/Program Files/Perl/bin/perl58.dll" LoadModule perl_module modules/mod_perl.so I have ActivePerl 5.8.8. The mod_perl.so I have is the the most recent binary available from the mod_perl site. Tracy "Lionel MARTIN" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] > Hi, > > I don't think that's a question of path, as you are telling us that you > checked that, and that the file was there. > Actually, I think this is a compilation problem, between mod_perl and perl > itself (Apache has nothing to do with that right now). > > I'm running under Windows as well, and I made extensive tests, to find out > which version of mod_perl.so is working with which version of perl58.dll. > (hoping that you didn't forget to load this latter one in your httpd.conf, > before mod_perl.so !) > > I suppose that, as a Win32 user, you're using the ActivePerl distribution? > If this is the case, then, I suggest that you use and install the latest > version of Perl (v5.8.8) with the latest version of mod_perl (including > among others mod_perl.so). You can use the ppm utility to check for your > versions and/or update accordingly. > > Lionel. > > - Original Message - > From: "Philip M. Gollucci" <[EMAIL PROTECTED]> > To: "Tracy E Schreiber" <[EMAIL PROTECTED]> > Cc: > Sent: Tuesday, May 16, 2006 12:01 AM > Subject: Re: Cannot load mod_perl.so > > >> Tracy E Schreiber wrote: >>> Hi, >>> >>> I hope this isn't too much of a newbie question... >>> >>> I am trying to upgrade from Apache 2.0.55 using mod_perl V1.0 to Apache >>> 2.2.2 using mod_perl V2.0. This installation is on a Windows XP Pro SP2 >>> machine. I get this error: >> You mean V2.0 -- no 1.x version works with 2.x and vice verusa. >> >> >>> httpd.exe: Syntax error on line 31 of D:/Apache2.2/conf/httpd.conf: >>> Cannot load >>> D:/Apache2.2/modules/mod_perl.so into server: The specified module could >>> not be >>> found. >> Permisions ? >> >> >>> The module is in the directory. Am I doing something wrong or is the >>> module not compatible with Apache 2.2.2? >> LoadModule perl_module libexec/apache22/mod_perl.so >> >> changing the paths appropriately of course. >> >> >> Philip M. Gollucci ([EMAIL PROTECTED]) 323.219.4708 >> Consultant / http://p6m7g8.net/Resume/resume.shtml >> Senior Software Engineer - TicketMaster - http://ticketmaster.com >> 1024D/A79997FA F357 0FDD 2301 6296 690F 6A47 D55A 7172 A799 97F >> >> "It takes a minute to have a crush on someone, an hour to like someone, >> and a day to love someone, but it takes a lifetime to forget someone..." >> >
Re: Cannot load mod_perl.so
Hi, I don't think that's a question of path, as you are telling us that you checked that, and that the file was there. Actually, I think this is a compilation problem, between mod_perl and perl itself (Apache has nothing to do with that right now). I'm running under Windows as well, and I made extensive tests, to find out which version of mod_perl.so is working with which version of perl58.dll. (hoping that you didn't forget to load this latter one in your httpd.conf, before mod_perl.so !) I suppose that, as a Win32 user, you're using the ActivePerl distribution? If this is the case, then, I suggest that you use and install the latest version of Perl (v5.8.8) with the latest version of mod_perl (including among others mod_perl.so). You can use the ppm utility to check for your versions and/or update accordingly. Lionel. - Original Message - From: "Philip M. Gollucci" <[EMAIL PROTECTED]> To: "Tracy E Schreiber" <[EMAIL PROTECTED]> Cc: Sent: Tuesday, May 16, 2006 12:01 AM Subject: Re: Cannot load mod_perl.so Tracy E Schreiber wrote: Hi, I hope this isn't too much of a newbie question... I am trying to upgrade from Apache 2.0.55 using mod_perl V1.0 to Apache 2.2.2 using mod_perl V2.0. This installation is on a Windows XP Pro SP2 machine. I get this error: You mean V2.0 -- no 1.x version works with 2.x and vice verusa. httpd.exe: Syntax error on line 31 of D:/Apache2.2/conf/httpd.conf: Cannot load D:/Apache2.2/modules/mod_perl.so into server: The specified module could not be found. Permisions ? The module is in the directory. Am I doing something wrong or is the module not compatible with Apache 2.2.2? LoadModule perl_module libexec/apache22/mod_perl.so changing the paths appropriately of course. Philip M. Gollucci ([EMAIL PROTECTED]) 323.219.4708 Consultant / http://p6m7g8.net/Resume/resume.shtml Senior Software Engineer - TicketMaster - http://ticketmaster.com 1024D/A79997FA F357 0FDD 2301 6296 690F 6A47 D55A 7172 A799 97F "It takes a minute to have a crush on someone, an hour to like someone, and a day to love someone, but it takes a lifetime to forget someone..."
Re: Where do the "warn" message go? (Ap2 & MP2 on Win32)
Hi, I'v really tried to reduce everything to a minimum to insulate the problem. So, here's the config file: # ServerName MYSERVER ServerRoot "C:\Program Files\Apache Group\Apache2\TestingFromScratch\ApacheModPerl" DocumentRoot "htdocs" Listen 80 LoadModule mime_module ../../modules/mod_mime.so LoadModule log_config_module ../../modules/mod_log_config.so # for mod_perl LoadFile "c:/TempApache/Perl/bin/perl58.dll" LoadModule perl_module ../../modules/mod_perl.so SetHandler perl-script PerlResponseHandler ModPerl::Registry PerlOptions +ParseHeaders TransferLog logs/access.log ErrorLog logs/error.log TypesConfig conf/mime.types LogLevel debug # and here's the script (/perl/warning.pl): # warn "Hello You Script\n"; use strict; print "Content-Type: text/html\n\n"; print "I think I issued a warning"; # There's really nothing more. Lionel. - Original Message - From: "Malcolm J Harwood" <[EMAIL PROTECTED]> To: Cc: "Lionel MARTIN" <[EMAIL PROTECTED]> Sent: Thursday, May 11, 2006 7:52 AM Subject: Re: Where do the "warn" message go? (Ap2 & MP2 on Win32) On Thursday 11 May 2006 01:40 am, Lionel MARTIN wrote: I tried to put a "LogLevel debug" in my httpd.conf, so that everything gets logged, but this didn't make a difference: my warn messages don't get logged. What else could I try, in order to find out why? Are your logs going through syslog, or directly to file? If it's through syslog that could be filtering for you. Do you have a signal handler for warn installed at all? It's possible that's eating your warns. What modules do you have loaded? (I've seen odd cases with some modules causing warns to get eaten). Have you tried reducing your script to just the warn statement? What happens under ModPerl::PerlRun as opposed to ModPerl::Registry? -- To know and yet think we do not know is the highest attainment. Not to know and yet think we do know is a disease. - Lao-tzu
Re: Where do the "warn" message go? (Ap2 & MP2 on Win32)
Hi, I tried to put a "LogLevel debug" in my httpd.conf, so that everything gets logged, but this didn't make a difference: my warn messages don't get logged. What else could I try, in order to find out why? If I'm running a script with that: warn "warning\n"; die "dying\n"; Then, "waning" doesn't get logged while "dying" does... - Original Message - From: "Malcolm J Harwood" <[EMAIL PROTECTED]> To: Cc: "Lionel MARTIN" <[EMAIL PROTECTED]> Sent: Tuesday, May 09, 2006 3:53 PM Subject: Re: Where do the "warn" message go? (Ap2 & MP2 on Win32) On Thursday 04 May 2006 06:37 am, Lionel MARTIN wrote: I'm sorry, but I may have misled you: I "said" script, but I'm not talking about CGI scripts here. In fact, even when puttting my "warn $msg;" in a custom MP handler, the message doesn't go to the error log. So, I'd like to understand when the content of the warn message is going. (please have a look at the details below for the original question) Did you check your apache logging settings? If LogLevel isn't set appropriately, apache might be discarding the warns. - Original Message - >> I'm currently running Apache 2.0.54 with MP 2.0.2, on Win32. >> When I'm doing a >> print STDERR "Hello\n"; >> warn "Hello\n"; >> at server startup, i.e.in a >Perl> block in httpd.cong, both messages >> go to the console, and to error.log as well. >> But when I'm doing the same thing within a script (handleed by >> ModPerl::Registry), these message don't appear anywhere. So, I'm >> wondering where they get redirected? The fact I'm running that under >> Win32 (threaded MPMs) may be important? -- Anywhere is walking distance, if you've got the time. - Steven Wright
Re: Using Apache::DBI
You're saying: * The connect_on_init() method doesn't connect until the PerlInitHandler phase, which is when a new child starts up. * I'm not sure this is right. Acording to what I read, and to my test, the PerlInitHandler phase happens for every request, and not just when threads are created. That's why I'm asking for threaed MPMs. The ideal would be to create the connection when threads are started, create statement handles against this connection, and keep all that until threads are killed. - Original Message - From: "Perrin Harkins" <[EMAIL PROTECTED]> To: "Lionel MARTIN" <[EMAIL PROTECTED]> Cc: "Octavian Rasnita" <[EMAIL PROTECTED]>; Sent: Monday, May 08, 2006 5:00 PM Subject: Re: Using Apache::DBI On Mon, 2006-05-08 at 16:56 +0200, Lionel MARTIN wrote: And what about the best usage when using threaded MPMs, like Win32 MPMs, concerning connections? It's no different. The threads are created after startup, and the init phase happens in each thread. - Perrin
Re: Using Apache::DBI
And what about the best usage when using threaded MPMs, like Win32 MPMs,
concerning connections?
- Original Message -
From: "Perrin Harkins" <[EMAIL PROTECTED]>
To: "Octavian Rasnita" <[EMAIL PROTECTED]>
Cc:
Sent: Monday, May 08, 2006 3:26 PM
Subject: Re: Using Apache::DBI
Octavian Rasnita wrote:
I thought the best way of using Apache::DBI is to put the following code
in
the startup.pl script:
use Apache::DBI ();
Apache::DBI->connect_on_init('DBI:mysql:database=database_name;host=10.50.28
.37', 'user', undef, {PrintError => 1, RaiseError => 0, AutoCommit =>
1});
use DBI ();
But I understand that it is better to disconnect from the database after
the
server starts or to connect to the database only after the server starts.
What you're doing is correct. The connect_on_init() method doesn't
connect until the PerlInitHandler phase, which is when a new child starts
up. Even if you just call DBI->connect during startup, Apache::DBI will
not cache connections until after the server starts.
- Perrin
Re: Where do the "warn" message go? (Ap2 & MP2 on Win32)
Thanks, I'm sorry, but I may have misled you: I "said" script, but I'm not talking about CGI scripts here. In fact, even when puttting my "warn $msg;" in a custom MP handler, the message doesn't go to the error log. So, I'd like to understand when the content of the warn message is going. (please have a look at the details below for the original question) - Original Message - From: "William A. Rowe, Jr." <[EMAIL PROTECTED]> To: "Lionel MARTIN" <[EMAIL PROTECTED]> Cc: Sent: Thursday, May 04, 2006 12:28 PM Subject: Re: Where do the "warn" message go? (Ap2 & MP2 on Win32) Lionel MARTIN wrote: Hi, I'm currently running Apache 2.0.54 with MP 2.0.2, on Win32. When I'm doing a print STDERR "Hello\n"; warn "Hello\n"; at server startup, i.e.in a >Perl> block in httpd.cong, both messages go to the console, and to error.log as well. But when I'm doing the same thing within a script (handleed by ModPerl::Registry), these message don't appear anywhere. So, I'm wondering where they get redirected? The fact I'm running that under Win32 (threaded MPMs) may be important? http://httpd.apache.org/docs/2.0/mod/mod_cgi.html#scriptlog Bill
Where do the "warn" message go? (Ap2 & MP2 on Win32)
Hi, I'm currently running Apache 2.0.54 with MP 2.0.2, on Win32. When I'm doing a print STDERR "Hello\n";warn "Hello\n"; at server startup, i.e.in a >Perl> block in httpd.cong, both messages go to the console, and to error.log as well. But when I'm doing the same thing within a script (handleed by ModPerl::Registry), these message don't appear anywhere. So, I'm wondering where they get redirected? The fact I'm running that under Win32 (threaded MPMs) may be important? Thanks, Lionel.
Re: How many people use the Windows combo of Apache2 + mod_perl2 ?
Hi, I'm replying below. On Wed, 2006-04-26 at 17:20 +0200, Lionel MARTIN wrote: I'm preloading this module, using "PerlModule MyLoadingModule" in httpd.conf. Does that change if you load them via "use MyLoadingModule" in a startup.pl? I can't understand how it could change using a startup.pl file? After all, doing a PerlRequire startup.pl is just a way to incoporate code in the main flow, isn't it? Anyway, I gave it a try, and it gave exactly the same results: Module Loaded: restart_count: 1 # Perl Interpreter:0x26ebbc Module Loaded: restart_count: 2 # Perl Interpreter:0x150f29c Module Loaded: restart_count: 1 # Perl Interpreter:0x26ec54 Module Loaded: restart_count: 2 # Perl Interpreter:0x1ac6014 I really would like to understand how the WinNT MPM is working, how memory is shared, in order to make our coding more efficient. The problem is that most docs about memory management under MP is abou LAMP, and I'm not sure the same rules apply... With Perl threads, no memory is shared. There is no advantage to preloading when using threads. It just makes the threads take longer to spawn. I'm really sceptical (and probably lost) with what you're saying here. First of all, from the MP docs itself (http://perl.apache.org/docs/2.0/user/intro/overview.html#Threads_Support), I can read that: Rather than create a PerlInterperter per-thread by default, mod_perl creates a pool of interpreters. The pool mechanism helps cut down memory usage a great deal. As already mentioned, the syntax tree is shared between all cloned interpreters. And concerning me, I couldn't believe there's no difference between preloading components, and loading them at runtime, in term of memory consumption. So, I made extensive tests, creating a simple component (a PerlResponseHandler). I ma deocmparisons between preloading the handler, and loading it at runtime, and then, using 8 concurrent accesses (with 8 Perl Interpreters enabled by default), and I could see some difference in terms of memory consumption. I have got a lot of different results, but for example: When preloading my compoennts, and asking Apache to start 8 Perl Interpreters and then, making 8 concurtrent requests, this was taking 27968 kb of RAM for the main process and 29596 kb of RAM for the child process. On the other hand, without preloading the handler, and then, making 8 concurrent requests (=> handler loaded on the fly), this was taking 11740kb for the main process and 35252kb for the child process... I don't know how to exactly interpret that, but that shows there are some differences. I really would love to understand how sharing is working, and what's exactly shared (or perhaps have the confirmation that nothing is shared under winnt MPMs...?) I think that I should better try with LAMP as well, but to be honest, I have never tried to install Linux. Have you got any resources where I could read about how things are managed under Windows? - Perrin
Re: How many people use the Windows combo of Apache2 + mod_perl2 ?
Hi, modules, that are loaded 4 times in all. You might we watching each Interpreter thread start, IIRC they start in sequence and not simultaneously. At any rate, thats a feature of the winnt mpm. Unless I'm missing a point, I'm not convinced by that. Indeed, I made further tests, and no matter how many Perl Interpreters I'm setting at start time, preloaded modules are loaded 4 times. To make that stand out, I created a very simple module: package MyLoadingModule; use Apache2::ServerUtil(); use ModPerl::Util(); open FILE, ">>", "MyModule.txt"; print FILE "Module Loaded: restart_count: ", Apache2::ServerUtil::restart_count(), " # Perl Interpreter:", ModPerl::Util::current_perl_id(), "\n"; close FILE; 1; which sole purpose is to show that it's being loaded, and to show the restart_count value. I'm preloading this module, using "PerlModule MyLoadingModule" in httpd.conf. And when starting the server, this shows in "MyModule.txt": Module Loaded: restart_count: 1 # Perl Interpreter:0x26ebbc Module Loaded: restart_count: 2 # Perl Interpreter:0x8901dc Module Loaded: restart_count: 1 # Perl Interpreter:0x26ec54 Module Loaded: restart_count: 2 # Perl Interpreter:0xe6e29c This clearly shows that the server is started 4 times. I suppose that the main parent process is start then restarted. And then, the child process is started and restarted? I'm not sure about the details, but this is what I'm guessing. One could notice that the Perl Interpreter aaddress is changing everytime. I really would like to understand how the WinNT MPM is working, how memory is shared, in order to make our coding more efficient. The problem is that most docs about memory management under MP is abou LAMP, and I'm not sure the same rules apply... Moreover, I'm not sure how efficient is the Winnt MPM, related to Perl. As you know, under Windows, the server is working in a threaded environment Perl + Threads ... hmmm I choose to run away in most cases :) "PerlInterpInitHandler" to be able to initialize Perl Interpreter specific stuff) You might actually be able to add something to the registry depending on how complex you want to get... But like I said before, 6 years ago now :) Philip M. Gollucci ([EMAIL PROTECTED]) 323.219.4708 Consultant / http://p6m7g8.net/Resume/resume.shtml Senior Software Engineer - TicketMaster - http://ticketmaster.com 1024D/A79997FA F357 0FDD 2301 6296 690F 6A47 D55A 7172 A799 97F "It takes a minute to have a crush on someone, an hour to like someone, and a day to love someone, but it takes a lifetime to forget someone..."
Re: How many people use the Windows combo of Apache2 + mod_perl2 ?
Hi, I'm having the same kind of interrogation. Concerning me, I'm working on a project that's still in development, and I'm developing under Windows (MP2, ActivePerl 5.8.8 Build 816 , Apache 2.055) I'll have to make a choice for the production environment, and I'm not sure what to choose. I'll have to make tests. Of course, most people seem to use a LAMP environment, and most docs are Linux flavoured. Under Windows, when launching my Apache Server, there seems to be 4 starts in all (Start -> Restart -> Start -> Restart). I can see that with preloaded modules, that are loaded 4 times in all. Moreover, I'm not sure how efficient is the Winnt MPM, related to Perl. As you know, under Windows, the server is working in a threaded environment (one child process containing several threads). As there's just one child, the PerlChildInitHandler is quite useless, and it would be good to have a grasp at Thread starts (having something like a "PerlThreadInitHandler" or a "PerlInterpInitHandler" to be able to initialize Perl Interpreter specific stuff) But I'm quite confident I'll be able to use a Windows environment for my production server, things tseem to have improved a lot (even if I'll make comparisons anyway with a LAMP environment, in term of CPU usage, and memory sharing efficiency) - Original Message - From: "Foo Ji-Haw" <[EMAIL PROTECTED]> To: Sent: Wednesday, April 26, 2006 8:21 AM Subject: How many people use the Windows combo of Apache2 + mod_perl2 ? Just want to do a quick poll here. Most of the comments I read here, come from people who use the Linux/ BSD platform. Is there anybody who actually deploy the Windows combo on a production server? For those who do, do you have any issues that are Windows-specific? For me for example, my gripe is that I cannot use ServerUtil::restart_count to determine the instance of the final startup of the service, due to the implementation nature of mp2
[MP2] Is there anything shared between threads in threaded MPMs like the Winnt one?
Hi,
I'm using MP v2.02, along with ActivePerl
v5.8.8 and Apache v2.054, all that under Windows XP.
Under Win32, the Apache memory model is a threaded
model, i.e. we have one master Apache process,and, then, one unique child
process, containing several threads, that can, if needed, use Perl Interpreters
taken from an interpreter pool.
This is what I have understood so far.
So, I'm trying to understand what's really shared
when preloading modules from httpd.conf, and which benefits I could get from
doing that.
The problem is that I can't really understand
what's kept shared when new Perl interpreters are started.
From the MP docs, I have read that, among
others:
Care is taken by Perl to copy only mutable data,
which means that no runtime locking is required and read-only data such as the
syntax tree is shared from the parent, which should reduce the overall mod_perl
memory footprint.
I made tests to understand that. For example, a
simple basic test has been to create a module with a simple basic
sub:
package Modules::MyPack;sub
test{ 1;}1;
Then, I preloaded it in the parent process,
including a "PerlRequire Modules/MyPack.pm" in httpd.conf.
At last, I wanted to make a test in a siple script,
like that:
my $sub = \&Modules::MyPack::test;open
FILE, ">>", "test.txt";print FILE $sub, "\n";close
FILE;
The purpose of this test is to see what is the
actual memory address of &Modules::MyPack::test.
I ran this test with 3 concurrent Perl Intepreters
and noticed that the returned address is different for each of them, which makes
me think that there isn't any memory sharing, even for a simple preloaded
subroutine. Indeed, I would have believed, if this very basic subroutine had
been shared among intepreters, them to have the same actual memory
address.
Am I getting wrong?
What's actually being shared among intepreters? How
could I simply test the fact stuff is actually shared?
Thanks in advance,
Lionel.
