Re: Confusion about SQL results - row counts and contents differ

[Perrin Harkins]

Maybe you haven't committed some manual change on the server, so it isn't
visible to other connections.

On Thu, Jan 26, 2023 at 8:56 PM Steven Haigh via modperl <
modperl@perl.apache.org> wrote:

>
>
> On Fri, Jan 27 2023 at 09:50:42 +0800, demerphq 
> wrote:
>
> On Fri, 27 Jan 2023, 09:43 Steven Haigh via modperl, <
> modperl@perl.apache.org> wrote:
>
>> Hi all,
>>
>> I'm a little confused about my little mod_perl web site at the moment.
>>
>> I'm doing an SQL query, then punting the results into a dataset to
>> display via the web page.
>>
>> When I do the query directly on the SQL server, I see:
>>
>> SELECT COUNT(*) FROM printers;
>> 30
>>
>> When I do the same in code via the web page, I get:
>> $sth = $dbh->prepare("SELECT COUNT(*) FROM printers");
>> $vars->{"count"} = $dbh->selectcol_arrayref($sth, undef);
>>
>> 'count' => [ 29 ]
>>
>> Even trying a slightly different approach:
>> $sth = $dbh->prepare("SELECT COUNT(*) FROM printers");
>> $sth->execute();
>> $vars->{"count"} = $sth->fetchrow_hashref();
>>
>> count' => { 'COUNT(*)' => 29 }
>>
>> This table is only ever updated manually - so there is no possible way
>> that the contents are being updated during these runs - however I also note
>> that the data returned in something like `SELECT * FROM printers` is
>> different than what I get by running the query directly.
>>
>
> What do you mean by different?
>
>>
>> Annoyingly, even after restarting apache, restarting mariadb, and even
>> restarting the entire machine, I still see this issue.
>>
>> That leaves me completely stuck - how can old data be returned - even
>> after a full machine reboot?
>>
>> Am I missing something kinda obvious?
>>
>> The system is Fedora 37 with the following versions:
>> mariadb-10.9.4-1
>> mod_perl-2.0.12-5
>> httpd-2.4.54-5
>> perl-DBD-MariaDB-1.22-3
>> perl-DBD-MySQL-4.050-15
>>
>> I've tried both the dbi:mysql and dbi:MariaDB - and the same occurs.
>>
>
> Are you sure you are connecting to the same dB?
>
> I'd debug this with a simple perl script and no mod_perl, and then work
> forward from there.
>
>
> I stripped this back to:
>
> #!/usr/bin/perl
> use strict;
> use warnings;
> use POSIX qw(strftime);
> use Data::Dumper;
> use DBI;
>
> ## Connect to the database...
> my $sqldb = "dbname";
> my $sqlusername = "dbusername";
> my $sqlpassword = "dbpassword";
> my $sqlhost = "localhost";
> my $dbh = DBI->connect("dbi:MariaDB:$sqldb:$sqlhost", $sqlusername,
> $sqlpassword) || die "Unable to connect to database! $!";
>
> my $sth = $dbh->prepare("SELECT COUNT(*) FROM printers");
> $sth->execute();
>
> print Dumper($sth->fetchrow_hashref());
>
> Yet that also returns 29.
>
> So now I'm *really* confused - but at least that rules out mod_perl.
>
> Yves
>
>
> --
>> Steven Haigh  net...@crc.id.au  https://crc.id.au
>> 
>>
>

Re: mod_perl alternatives

[Perrin Harkins]

I don't think there's any sudden urgency to switch to something else, since
nothing has actually changed about the state of mod_perl. However, if you
do want to migrate to something else, you should look at Plack:
https://metacpan.org/pod/Plack

Plack is not a drop-in replacement for mod_perl, but it's a high-quality
project with a well-developed ecosystem that will have some sort of
solution for most (not all) things that people are doing in mod_perl. But
again, I don't think you need to rush to do anything here.

- Perrin

On Thu, Mar 18, 2021 at 12:01 AM Jim Albert  wrote:

> Given the recent discussion on the need for mod_perl PMC members and the
> disclosure that there is no active development on mod_perl this seems
> like an appropriate time to start a thread on a discussion of mod_perl
> alternatives inline with the various means of using mod_perl from the
> low level use of interfacing with the Apache server to the quick and
> dirty stuff (ModPerl::PerlRun, I believe to keep Perl and modules in
> memory).
>
> I've seen mod_fcgid proposed in posts on other forums. Has anyone played
> with alternatives? I expect the low level Apache interaction might be
> difficult to duplicate at least to continue to do so in Perl. Perhaps
> the ModPerl::PerlRun approach of keeping Perl and modules in memory is a
> potential starting point for discussion for those using mod_perl at the
> most basic level.
>
> Jim
>
>
>
>

Re: Does anyone know what MP_NEED_HASH_SEED_FIXUP is for?

[Perrin Harkins]

Hi Steve,

I remember the original problem with rand(), but not what this change was
about. There used to be a problem with all mod_perl child processes
producing the same series of random numbers, because the seed was
initialized before forking. This sounds like a different problem.

- Perrin

On Mon, Mar 15, 2021 at 9:58 AM Steve Hay
 wrote:

> The reason I ask is that it is broken as of perl 5.33.7 and it looks
> like it will remain that way. See:
> https://github.com/Perl/perl5/issues/18617
>
> The problem is that the assignment to PL_hash_seed done in
> modperl_hash_seed_set() is no longer possible, and there is a
> suggestion that it was terribly wrong of mod_perl to be doing this
> anyway so the bug is really in mod_perl, not the perl change that
> causes the build breakage.
>
> If nobody knows any better than I'm inclined to apply the following
> diff to simply switch off MP_NEED_HASH_SEED_FIXUP for perl 5.33.7 and
> higher:
>
> Index: src/modules/perl/modperl_perl.c
> ===
> --- src/modules/perl/modperl_perl.c (revision 1887671)
> +++ src/modules/perl/modperl_perl.c (working copy)
> @@ -210,7 +210,10 @@
>   }
>  }
>
> -#if !(MP_PERL_VERSION_AT_MOST(5, 8, 0)) && \
> +/* The need for MP_NEED_HASH_SEED_FIXUP is unclear, and it no longer
> works as
> + * of Perl 5.33.7 anyway: See https://github.com/Perl/perl5/issues/18617
> */
> +#if (!(MP_PERL_VERSION_AT_MOST(5, 8, 0)) && \
> + MP_PERL_VERSION_AT_MOST(5, 33, 6)) && \
>  (defined(USE_HASH_SEED) || defined(USE_HASH_SEED_EXPLICIT))
>  #define MP_NEED_HASH_SEED_FIXUP
>  #endif
>
> Using Apache 2.4.46 and mod_perl 2.0.11 on Windows 10 with VS2019 x64
> I get the following test failures from a build with perl 5.33.6:
>
> t\filter\in_bbs_inject_header.t   (Wstat: 0 Tests: 36 Failed: 3)
>   Failed tests:  22, 26, 30
> t\modperl\env.t   (Wstat: 0 Tests: 164 Failed: 17)
>   Failed tests:  25, 28, 31, 43, 46, 49, 55, 58, 73, 82
> 85, 88, 91, 97, 142, 145, 163
> t\modperl\setupenv.t  (Wstat: 0 Tests: 63 Failed: 6)
>   Failed tests:  8, 22, 29, 36, 50, 57
> t\preconnection\note.t(Wstat: 0 Tests: 1 Failed: 1)
>   Failed test:  1
>
> As of 5.33.7, the build now fails:
>
> modperl_perl.c(271): error C2102: '&' requires l-value
>
> But with the above patch applied the build works again, and I actually
> have fewer test failures!:
>
> t\modperl\env.t   (Wstat: 0 Tests: 158 Failed: 17)
>   Failed tests:  25, 28, 31, 43, 46, 49, 55, 58, 67, 76
> 79, 82, 85, 91, 136, 139, 157
> t\preconnection\note.t(Wstat: 0 Tests: 1 Failed: 1)
>   Failed test:  1
>
> Please can people test this patch on other setups and report whether
> it causes any breakage.
>
> And if anyone can remember what this hash seed fix-up was intended to
> do then it would be interesting to hear. I worry that something that I
> haven't tested may be requiring it.
>
> The Changes file has this to say about it:
>
> "perl 5.8.1 randomizes the hash seed, because we precalculate the hash
> values of mgv elements the hash seed has to be the same across all
> perl interpreters. So mod_perl now intercepts cases where perl would
> have randomize it, do the seed randomization by itself and tell perl
> to use that value."
>
> That change is rev. 70047 in SVN (30 July 2003), but the commit
> comment is just the same as the Changes comment above. No tests seem
> to have been added, and I don't have an example of anything that
> requires this fix-up in order to work.
>
> Steve
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@perl.apache.org
> For additional commands, e-mail: dev-h...@perl.apache.org
>
>

Re: MP framework

[Perrin Harkins]

You can run Dancer on mod_perl using Plack. The Dancer documentation covers
it. Or there's Catalyst.

These frameworks are trying to be independent of the web server they run
on, so they don't tie in to mod_perl beyond taking advantage of the speed.
That doesn't stop you from mixing them with mod_perl handlers that work on
different phases of the request though, like authentication. But if you're
asking if there's a high-level framework that takes full advantage of the
mod_perl API, the answer is no.

- Perrin

On Wed, Aug 2, 2017 at 5:49 AM, 风河  wrote:

> We actually use both dancer and modperl. but dancer works separately, it
> doesn't require modperl installed.
> So I was asking if there is a framework for modperl which behaves as
> something like dancer.
>
>
> On Wed, Aug 2, 2017, at 05:42 PM, André Warnier (tomcat) wrote:
> > On 02.08.2017 11:19, 风河 wrote:
> > > Hi
> > >
> > > currently we were developing the methods mostly as handlers.
> > > is there a new popular framework for full stack development of modperl?
> > >
> > Yes, many. See here :
> > http://lmgtfy.com/?q=perl+web+development+frameworks
> >
> > We (still) use Template::Toolkit, which many people would consider a bit
> > old and not
> > really "in fashion". But it works, and is simple to learn and use.
> > Other frameworks can be said to be more modern, more elegant, more
> > fashionable, etc..
> > But in my opinion, the more elegant and abstract a framework is, the
> > longer it takes to
> > really dominate it, the more difficult it is to figure out what really
> > happens when you
> > have a problem, and the less you are really "in control".
> > So the choice is yours, depending on
> > - where you are coming from
> > - where you want to go
> > - what kind of problems you have to solve
> > - how deep you still need to go into controlling what Apache does
> > - etc..
> > There is no "one size fits all", and it is very easy to get into endless
> > on-line
> > discussions about which is "the best".
> > (Including with people who say that they are looking for "a framework",
> > but really mean "a
> > CMS system").
> >
> >
>

Re: Multiple documents/files with same subroutine name

[Perrin Harkins]

Hmm, that's not expected behavior with the code you're showing here. Are
you running this under ModPerl::Registry? It's supposed to generate a
unique package name for every file that prevents subs from colliding.

There are some things to watch out for when using subroutines in code you
run under Registry, but not what you're seeing here. The doc on those is
here:
http://perl.apache.org/docs/general/perl_reference/perl_reference.html

- Perrin

On Tue, Aug 1, 2017 at 12:45 PM, John Gigawatt 
wrote:

> Hi,
>
> I have a question on the best way to handle having multiple "web pages"
> having a subroutine with the same name in it.  I apologize in advance if I
> use the wrong terminology.  I know there are issues with library modules
> (use, etc) and also global variables.  I am assuming this is similar but
> not sure the best way to handle.
>
> Example:
>
> test1.html
> =
> #!/usr/bin/perl
>
> testSub();
>
> sub testSub
> {
> print "A";
> }
>
> test2.html
> =
> #!/usr/bin/perl
>
> testSub();
>
> sub testSub
> {
> print "B";
> }
>
> If I run both of these over and over, ultimately I will start to see A's
> on test2 and B's on test1, etc.  I do know about a "counter" issue within
> the same subroutine -- but this is slightly different (at least from the
> outside).
>
> One solution obviously has been to make sure I have a custom name for
> every subroutine... but that can get messy to maintain and is not perfect.
>
> My thoughts are:
>
> * Is there a way, when calling "testSub" to say "only call the testSub
> local to this file"?
>
> * Is there a way, when defining "sub testSub" to force it to be locally
> scoped?  I believe the equivalent with variable is to say "our" rather than
> "my"... but not sure how that would happen here.
>
> I have not been able to find answers related to this specific issue --
> only to global variable issues and modules with conflicting names.
>
> NOTE: I added "use warnings" but it did not provide any warning of this
> issue.
>
> Any help would be appreciated.
>
> Thanks,
> -John
>
>

Re: capture exception

[Perrin Harkins]

https://www.effectiveperlprogramming.com/2011/03/know-the-different-evals/

On Tue, May 30, 2017 at 10:49 AM, Dirk-Willem van Gulik <
di...@webweaving.org> wrote:

>
> On 30 May 2017, at 16:43, John Dunlap  wrote:
>
> How is it a security hole?
>
> ….
>
> > my $ret = eval { $m->...() };
>
>
> Just imagine $m->…() returning something containing a valid perl
> expression such as " `rm -rf /‘; “, system(“rm -rf /“);  or something that
> wires up a shell to a TCP socket.
>
> Dw.
>
>

Re: Apparent memory leak

[Perrin Harkins]

Sorry, I should have explained what I meant better. You would add a handler
BEFORE the request gets to your regular application, so you catch the
details of the request that dies. I mis-remembered about the access_log. I
was actually thinking of a custom C module I used once that did this type
of thing -- logged the request in an early stage before handling it. But
you can do that with mod_perl.

This does not sound like a memory leak to me. Those are slow and can be
handled with something like Apache::SizeLimit. This sounds like some of
your Perl code is loading too much into memory for some reason, like
reading a giant file or database result. And that points to a specific type
of request or parameters being the problem.

On Tue, Sep 6, 2016 at 12:53 PM, John Dunlap <j...@lariat.co> wrote:

> If this is a memory leak, won't the last request to be sent to the
> mod_perl worker process be the last straw and not necessarily the culprit?
> What if the leak is in some library code that's used in every request?
>
> On Tue, Sep 6, 2016 at 12:43 PM, John Dunlap <j...@lariat.co> wrote:
>
>> My fear with logging the complete data input is that it will make the
>> problem worse for my customers because this problem is only happening on
>> heavily loaded servers. I can't reproduce it locally.
>>
>> On Tue, Sep 6, 2016 at 11:26 AM, Perrin Harkins <phark...@gmail.com>
>> wrote:
>>
>>> Hi John,
>>>
>>> The key is usually finding out what the request was that caused it. You
>>> can add the pid to your access logging, or write a more complete mod_perl
>>> handler to log the complete data input along with the pid. Then you just go
>>> back and look at what it was after you see which process was killed.
>>>
>>> - Perrin
>>>
>>> On Tue, Sep 6, 2016 at 10:00 AM, John Dunlap <j...@lariat.co> wrote:
>>>
>>>> The system load reported by the uptime command, on one of my servers,
>>>> periodically spikes to 20-30 and then, shortly thereafter, I see this in
>>>> dmesg:
>>>>
>>>> [2887460.393402] Out of memory: Kill process 12533 (/usr/sbin/apach)
>>>> score 25 or sacrifice child
>>>> [2887460.394880] Killed process 12533 (/usr/sbin/apach)
>>>> total-vm:476432kB, anon-rss:204480kB, file-rss:0kB
>>>>
>>>> Several gigs of memory then becomes available and the system load
>>>> quickly returns to normal. I'm pretty sure it's a mod perl process that's
>>>> doing this but I'm not entirely sure how to track down the problem.
>>>>
>>>> How would you guys approach this problem?
>>>>
>>>> --
>>>> John Dunlap
>>>> *CTO | Lariat *
>>>>
>>>> *Direct:*
>>>> *j...@lariat.co <j...@lariat.co>*
>>>>
>>>> *Customer Service:*
>>>> 877.268.6667
>>>> supp...@lariat.co
>>>>
>>>
>>>
>>
>>
>> --
>> John Dunlap
>> *CTO | Lariat *
>>
>> *Direct:*
>> *j...@lariat.co <j...@lariat.co>*
>>
>> *Customer Service:*
>> 877.268.6667
>> supp...@lariat.co
>>
>
>
>
> --
> John Dunlap
> *CTO | Lariat *
>
> *Direct:*
> *j...@lariat.co <j...@lariat.co>*
>
> *Customer Service:*
> 877.268.6667
> supp...@lariat.co
>

Re: Apparent memory leak

[Perrin Harkins]

Hi John,

The key is usually finding out what the request was that caused it. You can
add the pid to your access logging, or write a more complete mod_perl
handler to log the complete data input along with the pid. Then you just go
back and look at what it was after you see which process was killed.

- Perrin

On Tue, Sep 6, 2016 at 10:00 AM, John Dunlap  wrote:

> The system load reported by the uptime command, on one of my servers,
> periodically spikes to 20-30 and then, shortly thereafter, I see this in
> dmesg:
>
> [2887460.393402] Out of memory: Kill process 12533 (/usr/sbin/apach) score
> 25 or sacrifice child
> [2887460.394880] Killed process 12533 (/usr/sbin/apach) total-vm:476432kB,
> anon-rss:204480kB, file-rss:0kB
>
> Several gigs of memory then becomes available and the system load quickly
> returns to normal. I'm pretty sure it's a mod perl process that's doing
> this but I'm not entirely sure how to track down the problem.
>
> How would you guys approach this problem?
>
> --
> John Dunlap
> *CTO | Lariat *
>
> *Direct:*
> *j...@lariat.co *
>
> *Customer Service:*
> 877.268.6667
> supp...@lariat.co
>

Re: Bad rap

[Perrin Harkins]

As mentioned in this article, nginx is a great reverse proxy for mod_perl
sites.

On Tue, Jun 14, 2016 at 10:28 AM, John Dunlap  wrote:

> https://www.nginx.com/blog/nginx-vs-apache-our-view/
>
> On Tue, Jun 14, 2016 at 2:35 AM, André Warnier  wrote:
>
>> On 13.06.2016 14:09, John Dunlap wrote:
>>
>>> We use Amazon Cloudfront for serving all of our static content. The only
>>> thing we load from Apache is an index.html file to bootstrap into
>>> Ember.js.
>>> In our experience, Cloudfront delivers static content to the browser 5-6
>>> times faster than our servers can. So, practically all of our requests
>>> serve dynamic content.
>>>
>>> Also, I didn't mean that Apache is slow or that it isn't a great web
>>> server
>>> per say but rather that, due to its single thread per request model,
>>>
>>
>> does anyone do better ? multiple threads per request ? some new kind of
>> parallel quantum computing ?
>> Sorry, I guess you meant something else, but in this case maybe it helps
>> to be precise ?
>> (or, I am willng to learn if there is a model which I don't know yet)
>>
>>
>>  it
>>
>>> cannot accept as many concurrent connections as Nginx can. Now, as I have
>>> not had time to experiment with Perl+Nginx, I cannot speak to whether or
>>> not there are offsetting performance penalties incurred by FCGI. I can
>>> tell
>>> you that, at some point, I'm going to experiment with it.
>>>
>>> On Mon, Jun 13, 2016 at 7:16 AM, James Smith  wrote:
>>>
>>> All our experiences at work with nginx/psgi & nginx/fastcgi are poor - it
 is very good if any of your queries takes any length of time and/or the
 fastcgi/psgi requests are requested a lot relative to the static content
 served by nginx then there are quite significant error/performance
 issues In our case the only static files are mainly images.. The
 rest
 of the content is dynamic - whether it is server cached pages or real
 dynamic content...

 We have a load balancing proxy in-front of our apaches so we can fork
 content elsewhere that is to be served fast! We don't because Apache
 itself
 is fast enough! Admittedly we have taken a lot of care to reduce the
 overall number of requests to a minimum (page, 1 CSS, 1 JS + a handful
 of
 images per page)

 The hacks we would have to do in PSGI/FastCGI to get these features
 would
 probably be negated by the move away from Apache

 Apache is fast enough if you use it properly!!

 On 6/13/2016 11:58 AM, John Dunlap wrote:

 Speaking as someone would like to migrate to Nginx, at some point, the
 big
 advantage of Nginx really has nothing to do with mod_perl. It has to do
 with Apache. The way Apache processes requests is fundamentally slower
 than
 Nginx and, consequently, Nginx scales better.
 On Jun 13, 2016 6:54 AM, "James Smith"  wrote:

 Just posted:
>
> mod_perl is a much better framework that PSGI, FastCGI IF you make use
> of
> the integration of perl into all the stages of apache (you can hook
> into
> about 15 different stages in the Apache life cycle.
>
> We make of extensive use of the input, output filters, AAA-layers,
> clean
> up, logging, server startup, etc processes then it is one of the best
> web
> frameworks you can use.
>
> We have sites where content is produced by either being static,
> mod_perl,
> php, and java (or proxied back from some ancient CGI software) all
> processed by the same mod_perl code in the output filter to look the
> same!
> or different if was using a different site!
>
> If all you are interested in is wrapping CGI scripts in a cached
> interpreter for performance then yes you can move to one of these other
> frameworks - but then you have already spent lots of time and effort
> implementing the features that are virtually free with apache/mod_perl!
>
> On 6/11/2016 7:11 PM, Vincent Veyron wrote:
>
> Hi all,
>
> See this post on reddit :
>
> https://www.reddit.com/r/linuxadmin/comments/4n5seo/apache_22_mod_perl_to_nginx/
>
> Please help set the record straight. Ancient technology WTF?
>
>
>
>
>
> -- The Wellcome Trust Sanger Institute is operated by Genome Research
> Limited, a charity registered in England with number 1021457 and a
> company
> registered in England with number 2742969, whose registered office is
> 215
> Euston Road, London, NW1 2BE.
>
>

 -- The Wellcome Trust Sanger Institute is operated by Genome Research
 Limited, a charity registered in England with number 1021457 and a
 company
 registered in England with number 2742969, whose registered office is
 215
 Euston Road, London, NW1 2BE.


>>>
>>>
>>>
>>
>
>
> --
> John Dunlap
> 

Re: Bad rap

[Perrin Harkins]

I invited him to stop by for help understanding what he's got running and
what his options are. He doesn't know perl, so this is probably all a black
box to him.

On Sat, Jun 11, 2016 at 2:11 PM, Vincent Veyron  wrote:

> Hi all,
>
> See this post on reddit :
>
>
> https://www.reddit.com/r/linuxadmin/comments/4n5seo/apache_22_mod_perl_to_nginx/
>
> Please help set the record straight. Ancient technology WTF?
>
>
> --
> Bien à vous, Vincent Veyron
>
> https://libremen.com
> Logiciels de gestion, libres
>

Re: close connection for request, but continue

[Perrin Harkins]

On Thu, Apr 21, 2016 at 9:48 AM, Iosif Fettich  wrote:

> I'm afraid that won't fit, actually. It's not a typical Cleanup I'm after
> - I actually want to not abandon the request I've started, just for closing
> the incoming original request. The cleanup handler could relaunch the slow
> back request - but doing so I'd pay twice for it.


You don't have to. You can just return immediately, and do all the work in
the cleanup (or a job queue) while you let the client poll for status. It's
a little extra work for simple requests, but it means all requests are
handled the same and you never make extra requests to your expensive
backend.

If you're determined not to do polling from the client, your best bet is
probably to fork immediately and do the work in the fork, while you poll to
check if it's done in your original process. You'd have to write the
response to a database or something that the original process can pick it
up from. But forking from mod_perl is a pain and easy to mess up, so I
recommend doing one of the other approaches.

- Perrin

Re: close connection for request, but continue

[Perrin Harkins]

On Thu, Apr 21, 2016 at 5:20 AM, Iosif Fettich  wrote:

>
> I'm trying to achieve the following: when there is an incoming request, I
> want to set a time limit in which an answer should be delivered to the
> client, no matter what.
>
> However, since the work triggered by the initial request (there is another
> request to other site involved)  might take much longer than that time
> limit, I want that work to properly finish, despite the fact that the
> initial request was 'served' already.


TMTOWTDI, but the common way to do this is to add the long-running job to a
job queue, and then redirect the user to a page that periodically checks if
the job is done by using JavaScript requests.

If you don't have a job queue and don't want to add one just for this, you
could use a cleanup handler to run the slow stuff after disconnecting:
http://perl.apache.org/docs/2.0/user/handlers/http.html#PerlCleanupHandler

That will tie up a mod_perl process though, so it's not a good way to go
for large sites.

- Perrin

Re: Thread-safe & vars scope clarification

[Perrin Harkins]

If you want to try it yourself and report back, I'm sure we'd all be
interested in what you find out. The main thing I'm aware of is that
copy-on-write works very well for preforking and threads usually can't
match it, but maybe your application is different.

Regarding your thread-safety questions, chdir() is global to the process,
so not thread-safe. I think the issue with readdir() is that open directory
handles don't get copied when threads are spawned, but maybe there's more
to it. The special Perl vars are not a problem with threads, since each
thread is a separate interpreter.

You can read about the limitations of threads here:
http://perldoc.perl.org/threads.html#BUGS-AND-LIMITATIONS

- Perrin

On Mon, Apr 4, 2016 at 5:03 PM, Ben RUBSON <ben.rub...@gmail.com> wrote:

> Hi Perrin,
>
> Thank you for your answer.
>
> Well, for the moment I only use the prefork (non-threaded) MPM.
> But I would have liked to test performance of the threaded MPM, so I would
> have liked my code to be thread-ready.
>
> According to this :
>
> https://perl.apache.org/docs/2.0/user/performance/mpm.html#toc_Memory_Requirements_in_Threaded_MPM
> Sounds like threads could have an interest, but according to your message,
> should not be a general case.
>
> Thank you,
>
> Ben
>
> Le 4 avr. 2016 à 22:49, Perrin Harkins <phark...@gmail.com> a écrit :
>
> Hi Ben.
>
> Before you get too far into the details of using threads, can I ask why
> you're considering it? The memory footprint and performance of using forked
> processes with Perl is generally going to be better than that of threads,
> due to copy-on-write.
>
> - Perrin
>
> On Mon, Apr 4, 2016 at 4:44 PM, Ben RUBSON <ben.rub...@gmail.com> wrote:
>
>> Hello,
>>
>> I have some questions regarding mod_perl, threads, thread-safe functions,
>> special vars scope...
>>
>> From
>> https://perl.apache.org/docs/2.0/user/coding/coding.html#toc_Thread_environment_Issues
>> :
>> "if you chdir() in one thread, all other thread now see the current
>> working directory of that thread that chdir()'ed to that directory"
>> Do we speak about threads from threaded MPM ?
>> (so the specific Apache process which owns the thread running this Perl
>> script, and all its (the process) threads, will chdir()).
>>
>> From the same page :
>> "Other usually problematic (non thread-safe) functions include readdir(),
>> srand(), etc."
>> Why readdir() would not be thread-safe ? I can't see why. Perhaps
>> readdir() chdir() to read a directory content ?
>> How could we then read dir content being thread-safe ?
>>
>> From
>> https://perl.apache.org/docs/general/perl_reference/perl_reference.html#toc_The_Scope_of_the_Special_Perl_Variables
>> :
>> "In the mod_perl situation where the child server doesn't usually exit,
>> if in one of your scripts you modify a global variable it will be changed
>> for the rest of the process' life and will affect all the scripts executed
>> by the same process."
>> So in a threaded MPM, if one of the threads modifies one of the special
>> Perl variables, it will instantly affect all other threads of this process ?
>>
>> What about variables like $? ? (return code of system() calls)
>> And ${^CHILD_ERROR_NATIVE} ? (return code too)
>> Does their value stay local to the thread ? / Does each thread have these
>> variables with their own proper value ?
>> Or is it unsafe to read them because they could be modified by another
>> thread ?
>>
>> Thank you very much for your clarification,
>>
>> Best regards,
>>
>> Ben
>>
>>
>
>

Re: Thread-safe & vars scope clarification

[Perrin Harkins]

Hi Ben.

Before you get too far into the details of using threads, can I ask why
you're considering it? The memory footprint and performance of using forked
processes with Perl is generally going to be better than that of threads,
due to copy-on-write.

- Perrin

On Mon, Apr 4, 2016 at 4:44 PM, Ben RUBSON  wrote:

> Hello,
>
> I have some questions regarding mod_perl, threads, thread-safe functions,
> special vars scope...
>
> From
> https://perl.apache.org/docs/2.0/user/coding/coding.html#toc_Thread_environment_Issues
> :
> "if you chdir() in one thread, all other thread now see the current
> working directory of that thread that chdir()'ed to that directory"
> Do we speak about threads from threaded MPM ?
> (so the specific Apache process which owns the thread running this Perl
> script, and all its (the process) threads, will chdir()).
>
> From the same page :
> "Other usually problematic (non thread-safe) functions include readdir(),
> srand(), etc."
> Why readdir() would not be thread-safe ? I can't see why. Perhaps
> readdir() chdir() to read a directory content ?
> How could we then read dir content being thread-safe ?
>
> From
> https://perl.apache.org/docs/general/perl_reference/perl_reference.html#toc_The_Scope_of_the_Special_Perl_Variables
> :
> "In the mod_perl situation where the child server doesn't usually exit, if
> in one of your scripts you modify a global variable it will be changed for
> the rest of the process' life and will affect all the scripts executed by
> the same process."
> So in a threaded MPM, if one of the threads modifies one of the special
> Perl variables, it will instantly affect all other threads of this process ?
>
> What about variables like $? ? (return code of system() calls)
> And ${^CHILD_ERROR_NATIVE} ? (return code too)
> Does their value stay local to the thread ? / Does each thread have these
> variables with their own proper value ?
> Or is it unsafe to read them because they could be modified by another
> thread ?
>
> Thank you very much for your clarification,
>
> Best regards,
>
> Ben
>
>

Re: Weird variable scope problem

[Perrin Harkins]

Hi John,

Pretty hard to know what's going on without looking at actual code, but it
could be some variation of this type of problem:
http://perl.apache.org/docs/1.0/guide/porting.html#Name_collisions_with_Modules_and_libs

You can easily run into trouble with code that either doesn't use package
names or exports things from use() calls.

> Is it possible for that kind of dynamic loading to result in multiple
lexical scopes for the same package?

No, Perl doesn't support that.

- Perrin


On Fri, Mar 25, 2016 at 4:23 PM, John Dunlap <j...@lariat.co> wrote:

> They are initialized in my handler method, at the start of the request so
> they should be initialized at runtime. Basically, I have a singleton object
> which extends Class::Singleton which functions as an IOC container.
> However, because I have multiple virtual hosts and because the database
> connection is inside the IOC container, I have to have a separate object
> graph for each virtual host to prevent them from inadvertently talking to
> each other's databases. When I get an instance of the IOC container, I pass
> it the name of the site. Internally, it then maintains 2 lexically scoped
> variables; A hash, keyed by the name of the virtual host, and a scalar for
> storing the name of the current virtual host so that subsequent calls to
> the IOC container to get references to objects don't require a site name.
> This initialization happens at the start of every request to ensure that
> the right object graph is referenced when executing the request. However,
> in some older sections of the code base, when I attempt to references the
> IOC container these 2 lexically scoped variables are somehow empty, even
> though they were initialized at the start of the request, and then I get an
> error for attempting to call a function on an undefined value.
>
> I know that the code for some older areas of the application are loaded
> dynamically through evals and requires. Is it possible for that kind of
> dynamic loading to result in multiple lexical scopes for the same package?
>
> On Fri, Mar 25, 2016 at 3:58 PM, Perrin Harkins <phark...@gmail.com>
> wrote:
>
>> On Fri, Mar 25, 2016 at 10:11 AM, John Dunlap <j...@lariat.co> wrote:
>>>
>>> Basically, I have a module which has a some lexically scoped variables
>>> in it. These variables are used by accessor methods in the package. The
>>> problem is that, if I use this module in certain parts of my application,
>>> these variables are mysteriously empty even though I am certain that I've
>>> already initialized them.
>>>
>>
>> How do you initialize them? In code that runs at compile-time in the same
>> module? In calls from httpd.conf or startup.pl? Is this an OO module
>> with a declared package name?
>>
>>
>>> Is there a way for a single perl interpreter to have multiple copies of
>>> a lexical variable?
>>>
>>
>> No, but keep in mind that mod_perl is running inside multiple httpd
>> processes and each one has a separate interpreter with unique copies of all
>> lexicals.
>>
>> - Perrin
>>
>>
>
>
> --
> John Dunlap
> *CTO | Lariat *
>
> *Direct:*
> *j...@lariat.co <j...@lariat.co>*
>
> *Customer Service:*
> 877.268.6667
> supp...@lariat.co
>

Re: Weird variable scope problem

[Perrin Harkins]

On Fri, Mar 25, 2016 at 10:11 AM, John Dunlap  wrote:
>
> Basically, I have a module which has a some lexically scoped variables in
> it. These variables are used by accessor methods in the package. The
> problem is that, if I use this module in certain parts of my application,
> these variables are mysteriously empty even though I am certain that I've
> already initialized them.
>

How do you initialize them? In code that runs at compile-time in the same
module? In calls from httpd.conf or startup.pl? Is this an OO module with a
declared package name?


> Is there a way for a single perl interpreter to have multiple copies of a
> lexical variable?
>

No, but keep in mind that mod_perl is running inside multiple httpd
processes and each one has a separate interpreter with unique copies of all
lexicals.

- Perrin

Re: memcache and perl question

[Perrin Harkins]

On Tue, Nov 24, 2015 at 12:21 PM, Joseph Norris  wrote:

> process 1 -> set($tag1,$hash);data unique to process 1
> process 1 -> get($tag1,$hash);
> process 2 -> set ($tag2,$hash);   data unique to process 2
> process 2 -> get($tag2,$hash);
>

I'm not sure what you're trying to do. Assuming that $tag ne $tag2, there
is no data being shared here and no point to using memcached. The normal
thing would be for the key to be the same in both processes so they can
both get and set the same data. The actual hashes in local memory are
different of course.

each process must have a unique key to manipulate the hash via set and get
>

No, if they each have a unique key, they will be looking at totally
different data in memcached and not sharing.

- Perrin

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"memcached" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to memcached+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: memcache and perl question

[Perrin Harkins]

On Tue, Nov 24, 2015 at 11:05 AM, Joseph Norris  wrote:

> if I set a tag to a value of a hash:
>
> $self->{cache}->set('tag',$hash);
>
> is the hash ref set in memcache that will point back to my hash or do I
> have to actually have a %hash to be used in
> $self->{cache}->set('tag',%hash);
>

Like all other caching modules that do anything outside of your local
process, Cache::Memcached passes your data to Storable to flatten it into a
blob of data, so what gets sent over the wire is just that blob with the
complete contents of your hash. Storable will follow references.

and do I have to have some unique identifier for the the tag so that the in
> memory hash will be related to the actual script that is calling it.
> in other words will each run of the script get a different instance of
> 'tag' with its own unique %hash?
>

I'm not sure I understand what you're asking. When you call get() on the
cache to load the hash, binary data gets passed to Storable and turned into
a hash in your local process. If you make changes to the local hash after
that without storing them back to memcached, no other process will see
them. If you call get() on the same key in another process, that process
will also get a hash in local memory, and no connection will exist between
that and the one in your first process.

- Perrin

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"memcached" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to memcached+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: memcached on systems with small RAM

[Perrin Harkins]

You might want to look at alternatives in that kind of setup. For example,
BerkeleyDB is quite a bit faster than memcached when your cache is just on
one local system. The advantages of memcached come into play when you have
a large cache across multiple machines.

- Perrin

On Mon, Apr 27, 2015 at 6:00 AM, Hans Kramer jlam.kra...@gmail.com wrote:

 Hi,

 I am using memcached on a system with very modest RAM: 64Mb to be precise
 (an Artila Matrix 504).
 I have specified -M 1 -t 1 to reduce the memory allocated by memcached.
 Now RSS is always modest and around an explainable value, however, VSZ is
 at least 20Mb and with every
 thread I add VSZ increases with at least 6 Mbytes. If I understand this
 correctly, this relative high VSZ, increases the oom_score
 a lot. Hence I would like to reduce unnecessary alloc as much as possible.
 I have briefly looked at the memcached source and I cannot any significant
 mallocs. Is this perhaps coming from the libevent
 library?

 Any insights will be helpful.

 Kind regards

 Hans

 --

 ---
 You received this message because you are subscribed to the Google Groups
 memcached group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to memcached+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.


-- 

--- 
You received this message because you are subscribed to the Google Groups 
memcached group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to memcached+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: memcached on systems with small RAM

[Perrin Harkins]

It does write to disk, so maybe it's not appropriate for your use. It is
widely used in embedded systems though.

On Tue, Apr 28, 2015 at 9:08 AM, Hans Kramer jlam.kra...@gmail.com wrote:

 Doesn't BerkelyDB write to disk? (of course I could use a tmpfs.. hmmm
 )
 And I us  the auto expire of items in memcache

 Thanks for the tip, I definitely will consider it!

 Hans

 On Tue, Apr 28, 2015 at 2:43 PM, Perrin Harkins phark...@gmail.com
 wrote:

 You might want to look at alternatives in that kind of setup. For
 example, BerkeleyDB is quite a bit faster than memcached when your cache is
 just on one local system. The advantages of memcached come into play when
 you have a large cache across multiple machines.

 - Perrin

 On Mon, Apr 27, 2015 at 6:00 AM, Hans Kramer jlam.kra...@gmail.com
 wrote:

 Hi,

 I am using memcached on a system with very modest RAM: 64Mb to be
 precise (an Artila Matrix 504).
 I have specified -M 1 -t 1 to reduce the memory allocated by memcached.
 Now RSS is always modest and around an explainable value, however, VSZ
 is at least 20Mb and with every
 thread I add VSZ increases with at least 6 Mbytes. If I understand this
 correctly, this relative high VSZ, increases the oom_score
 a lot. Hence I would like to reduce unnecessary alloc as much as
 possible.
 I have briefly looked at the memcached source and I cannot any
 significant mallocs. Is this perhaps coming from the libevent
 library?

 Any insights will be helpful.

 Kind regards

 Hans

 --

 ---
 You received this message because you are subscribed to the Google
 Groups memcached group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to memcached+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.


  --

 ---
 You received this message because you are subscribed to a topic in the
 Google Groups memcached group.
 To unsubscribe from this topic, visit
 https://groups.google.com/d/topic/memcached/c-jJSFKZ1d0/unsubscribe.
 To unsubscribe from this group and all its topics, send an email to
 memcached+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.


  --

 ---
 You received this message because you are subscribed to the Google Groups
 memcached group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to memcached+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.


-- 

--- 
You received this message because you are subscribed to the Google Groups 
memcached group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to memcached+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Apache::Reload versioning

[Perrin Harkins]

Sorry Steve, I don't know what that's about. I don't see anything relevant
in my dev mail archives.

- Perrin

On Wed, Apr 22, 2015 at 1:33 PM, Steve Hay steve.m@googlemail.com
wrote:

 The current RELEASE notes for Apache::Reload say that after bumping
 the version number from X.YY-dev to X.YY-rc1 in the Changes file, this
 change should not be committed, and says (see dev@ archives) by way
 of an explanation.

 However, I have searched the archives and cannot find any explanation
 for this bizarre directive. Does anyone know the reason?

 (I'm about to make an RC1 for Apache-Reload so that we can include the
 latest code via an official release into the forthcoming
 mod_perl-2.0.9.)

 For the record, when I made Apache-Test-1.39, I *did* commit the -rc1
 edit to the Changes file since there was no instruction not to in that
 case. I cannot see that any harm has come from it, nor how it could do
 in the future.

 We don't bother tagging release candidates, but it might help someone
 in the future looking back through the svn history to see which
 commits represented different RCs, if anyone is interested.

 Unless I'm overlooking something that anyone can shed some light on
 then I will remove the instruction from Apache-Reload before making
 RC1.

 -
 To unsubscribe, e-mail: dev-unsubscr...@perl.apache.org
 For additional commands, e-mail: dev-h...@perl.apache.org

Re: Large File Download

[Perrin Harkins]

Yeah, sendfile() is how I've done this in the past, although I was using
mod_perl 1.x for it.

On Sat, Mar 28, 2015 at 5:55 AM, André Warnier a...@ice-sa.com wrote:

 Randolf Richardson wrote:

 I know that it's possible(and arguably best practice) to use Apache to
 download large files efficiently and quickly, without passing them
 through
 mod_perl. However, the data I need to download from my application is
 both
 dynamically generated and sensitive so I cannot expose it to the internet
 for anonymous download via Apache. So, I'm wondering if mod_perl has a
 capability similar to the output stream of a java servlet. Specifically,
 I
 want to return bits and pieces of the file at a time over the wire so
 that
 I can avoid loading the entire file into memory prior to sending it to
 the
 browser. Currently, I'm loading the entire file into memory before
 sending
 it and

 Is this possible with mod_perl and, if so, how should I go about
 implementing it?


 Yes, it is possible -- instead of loading the entire contents of
 a file into RAM, just read blocks in a loop and keep sending them until you
 reach EoF (End of File).

 You can also use $r-flush along the way if you like, but as I
 understand it this isn't necessary because Apache HTTPd will send the data
 as soon as its internal buffers contain enough data.  Of course, if you can
 tune your block size in your loop to match Apache's output buffer size,
 then that will probably help.  (I don't know much about the details of
 Apache's output buffers because I've not read up too much on them, so I
 hope my assumptions about this are correct.)

 One of the added benefits you get from using a loop is that you
 can also implement rate limiting if that becomes useful.  You can certainly
 also implement access controls as well by cross-checking the file being
 sent with whatever internal database queries you'd normally use to ensure
 it's okay to send the file first.


 You can also :
 1) write the data to a file
 2) $r-sendfile(...);
 3) add a cleanup handler, to delete the file when the request has been
 served.
 See here for details : http://perl.apache.org/docs/2.
 0/api/Apache2/RequestIO.html#C_sendfile_

 For this to work, there is an Apache configuration directive which must be
 set to on. I believe it is called UseSendFile.
 Essentially what senfile() does, is to delegate the actual reading and
 sending of the file to Apache httpd and the underlying OS, using code which
 is specifically optimised for this purpose.  It is much kore efficient than
 doing this in a read/write loop by yourself, at the cost of having less
 fine control over the operation.

Re: mod_perl for multi-process file processing?

[Perrin Harkins]

Cache::FastMmap is a great module for sharing read/write data, but it can't
compete with the speed of loading it all into memory before forking as Alan
said he plans to do.

- Perrin

On Tue, Feb 3, 2015 at 2:05 AM, Cosimo Streppone cos...@streppone.it
wrote:

 Alan Raetz wrote:

  So I have a perl application that upon startup loads about ten perl
 hashes (some of them complex) from files. This takes up a few GB of
 memory and about 5 minutes. It then iterates through some cases and
 reads from (never writes) these perl hashes. To process all our cases,
 it takes about 3 hours (millions of cases). We would like to speed up
 this process. I am thinking this is an ideal application of mod_perl
 because it would allow multiple processes but share memory.


 Sure you could use modperl for this.
 I would also consider at least these alternatives:

 - use Cache::FastMmap, https://metacpan.org/pod/Cache::FastMmap
   Load up your data with a loader script, and forget about it.
   Cache::FastMmap also works with modperl.

 - use a network server, like memcached or redis to store your
   read-only data, and use a lightweight network protocol (on localhost)
   to get the data.

 In both cases, reading from multiple processes will not be a problem.
 The cheapest solution for the consumer part (the cases above)
 would be to use a command like parallel to fire up as many copies
 of your consumer script as you can afford.

 Hope this helps,

 --
 Cosimo

Re: mod_perl for multi-process file processing?

[Perrin Harkins]

I agree, either threads or Parallel::ForkManager, depending on your
platform and your perl, will be a lot faster than mod_perl for this. Of
course there might be other reasons to use mod_perl, e.g. it's useful to
have this available as a remote service, or you want to call this
frequently for small jobs throughout the day without needing to reload the
data.

- Perrin

On Tue, Feb 3, 2015 at 7:42 AM, Patton, Billy N billy.pat...@h3net.com
wrote:

 Unless I’m entirely wrong it appears that you want to use shared threaded
 memory.
 This would allow you to keep out of apache altogether.
 Here is an example of using threads that I worked out using shared memory.
 We took a 4 hour task, serial, and turned it into 5 minutes with threads.
 This worked extremely well for me.  I wasn’t using large hashes, but I was
 hundreds of files, per thread, with 30k lines per file.
 #!/usr/bin/env perl -w
 use strict;
 use warnings;
 use Data::Dumper;
 $Data::Dumper::Indent = 1;
 $Data::Dumper::Sortkeys = 1;
 $Data::Dumper::Deepcopy = 1;
 use threads;
 use threads::shared;
 use constant  MAX_TRIES = 5;

 sub sub_threads($$$);

 my $switch= undef;
 my $hash  = undef;
 my $gsx   = undef;
 my $cnt   = 5;
 my %switches= ( 'A' = { 'b' = undef , 'c' = undef, 'd' = undef },
 'B' = { 'b' = undef , 'c' = undef, 'd' = undef },
 'C' = { 'b' = undef , 'c' = undef, 'd' = undef },
 'D' = { 'b' = undef , 'c' = undef, 'd' = undef },
 'E' = { 'b' = undef , 'c' = undef, 'd' = undef },
);
 my %threads : shared  = ();

 ##
 ## create the threads
 #
 while (($switch,$hash) = each %switches) {
   unless (exists $threads{$switch}) {
 my %h : shared;
 $threads{$switch} = \%h;
   }
   while (($gsx, $_) = each %$hash) {
 unless (exists $threads{$switch}{$gsx}) {
   my %h : shared;
   $threads{$switch}{$gsx} = \%h;
 }
 unless (exists $threads{$switch}{$gsx}{'messages'}) {
   my @h : shared;
   $threads{$switch}-{$gsx}-{'messages'} = \@h;
 }
 $hash-{$gsx}-{'thread'} =
 threads-create(\sub_threads,\$switch,\$gsx,\$cnt);
 $hash-{$gsx}-{'tries'}  = 1;
 $cnt += 5;
   }
 }

 #print Dumper \%threads;
 #print Dumper \%switches;

 ##
 ## endless loop to run while threads are still running
 ##
 $cnt = 1;
 while ($cnt) {
   $cnt = 0;
   while (($switch,$hash) = each %switches) {
 while (($gsx, $_) = each %$hash) {
   if ($hash-{$gsx}-{'thread'}-is_running()) {
 $cnt = 1;
 #print $switch-$gsx is running\n;
   } else {
 #print $switch-$gsx is NOT running\n;
 #print   Reason for failure : \n;
 #print ' ' .  join('\n' ,
 @{$threads{$switch}-{$gsx}-{'messages'}}) . \n;
 if ($hash-{$gsx}-{'tries'}  MAX_TRIES) {
 #  print   max tries not reached for $switch-$gsx, will be
 trying again!\n;
   $hash-{$gsx}-{'tries'}++;
   $hash-{$gsx}-{'thread'} =
 threads-create(\sub_threads,\$switch,\$gsx,\$cnt);
 } else {
   print send email! $switch-$gsx\n;
 }
   }
 }
 sleep 2;
   }
 }

 #print Dumper \%threads;
 #print Dumper \%switches;


 sub sub_threads($$$) {
   my $ptr_switch  = shift;
   my $ptr_gsx = shift;
   my $ptr_tNum= shift;
   sleep $$ptr_tNum;
   {
 lock(%threads);
 push @{$threads{$$ptr_switch}-{$$ptr_gsx}-{'messages'}} , Leaving
 thread $$ptr_switch-$$ptr_gsx;
 #$threads{$$ptr_switch}-{$ptr_gsx}-{'messages'} = Leaving thread
 $$ptr_switch-$$ptr_gsx;
 # locke freed at end oc scope
   }
   return 0;
 }

 On Feb 2, 2015, at 10:11 PM, Alan Raetz alanra...@gmail.commailto:
 alanra...@gmail.com wrote:

 So I have a perl application that upon startup loads about ten perl hashes
 (some of them complex) from files. This takes up a few GB of memory and
 about 5 minutes. It then iterates through some cases and reads from (never
 writes) these perl hashes. To process all our cases, it takes about 3 hours
 (millions of cases). We would like to speed up this process. I am thinking
 this is an ideal application of mod_perl because it would allow multiple
 processes but share memory.

 The scheme would be to load the hashes on apache startup and have a master
 program send requests with each case and apache children will use the
 shared hashes.

 I just want to verify some of the details about variable sharing.  Would
 the following setup work (oversimplified, but you get the idea…):

 In a file Data.pm, which I would use() in my Apache startup.pl
 http://startup.pl/, I would load the perl hashes and have hash
 references that would be retrieved with class methods:

 package Data;

 my %big_hash;

 open(FILE,file.txt);

 while ( FILE ) {

   … code ….

   $big_hash{ $key } = $value;
 }

 sub get_big_hashref {   return \%big_hash; }

 snip

 And so in the apache request handler, the code would be something like:

 

Re: Prototype mismatch

[Perrin Harkins]

Hi Worik,

This happens when you reload code that uses prototypes. In this case, those
constants are using prototypes. It's similar to this:
http://perl.apache.org/docs/1.0/guide/troubleshooting.html#Constant_subroutine_XXX_redefined

Not really a problem, but it will not happen if you stop and start your
server instead of reloading code in a running interpreter.

- Perrin

On Mon, Nov 17, 2014 at 6:26 PM, Worik Stanton worik.stan...@gmail.com
wrote:

 I get the following messages in my apache2/error.log when I restart.
 Why is that?  What steps should I take to diagnose the problem?

 The line in the script where this ocures says:

 use FileHandle;


 cheers
 Worik


 Prototype mismatch: sub Apache::AdMachine::Track::LOCK_UN: none vs ()
 Prototype mismatch: sub Apache::AdMachine::Track::LOCK_NB: none vs ()
 Prototype mismatch: sub Apache::AdMachine::Track::SEEK_END: none vs ()
 Prototype mismatch: sub Apache::AdMachine::Track::LOCK_EX: none vs ()
 Prototype mismatch: sub Apache::AdMachine::Track::LOCK_SH: none vs ()
 Prototype mismatch: sub Apache::AdMachine::Track::_IOFBF: none vs ()
 Prototype mismatch: sub Apache::AdMachine::Track::_IOLBF: none vs ()
 Prototype mismatch: sub Apache::AdMachine::Track::_IONBF: none vs ()
 Prototype mismatch: sub Apache::AdMachine::Track::FD_CLOEXEC: none vs ()
 Prototype mismatch: sub Apache::AdMachine::Track::F_DUPFD: none vs ()
 Prototype mismatch: sub Apache::AdMachine::Track::F_EXLCK: none vs ()
 Prototype mismatch: sub Apache::AdMachine::Track::F_GETFD: none vs ()
 Prototype mismatch: sub Apache::AdMachine::Track::F_GETFL: none vs ()
 Prototype mismatch: sub Apache::AdMachine::Track::F_GETLK: none vs ()
 Prototype mismatch: sub Apache::AdMachine::Track::F_GETLK64: none vs ()
 Prototype mismatch: sub Apache::AdMachine::Track::F_GETOWN: none vs ()

 --
 Why is the legal status of chardonnay different to that of cannabis?
worik.stan...@gmail.com 021-1680650, (03) 4821804
   Aotearoa (New Zealand)
  I voted for love

Re: Errors in the wrong place and wrong format

[Perrin Harkins]

I think that only redirects errors logged with Apche2::Log, not all of
STDERR. There might be a workaround for capturing STDERR. I'd try searching
the mailing list archives.

- Perrin

On Mon, Nov 17, 2014 at 6:28 PM, worik worik.stan...@gmail.com wrote:

 My virtual host has errors redirected to:

 Worik.errors.log

 (ErrorLog ${APACHE_LOG_DIR}/Worik.error.log)

 It is in this virtueal host that I have set-up mod-perl.  Why then are
 errors sent to error.log?

 And why do errors emanating from mod_perl not have time stamps?


 --
 Why is the legal status of chardonnay different to that of cannabis?
worik.stan...@gmail.com 021-1680650, (03) 4821804
   Aotearoa (New Zealand)
  I voted for love

Re: Disconnect database connection after idle timeout

[Perrin Harkins]

On Fri, Nov 14, 2014 at 7:58 AM, Vincent Veyron vv.li...@wanadoo.fr wrote:

 If I understand correctly, I could also use Apache::DBI and a persistent
 connection for a similar result, modulo what you wrote in an earlier
 message regarding the connection being more explicit (which I don't quite
 grasp, I'll have to study on that)?


Right. Apache::DBI overrides DBI-connect() when loaded so that a normal
connect() call gets a cached handle if available.

What you're doing here looks fine. If you ever switch off AutoCommit in
your code, I'd suggest adding a cleanup handler that checks if the handle
is not currently in AutoCommit mode, and if not, issues a rollback and
switches it to AutoCommit. That protects against an exception leaving your
database handle in a bad state. (Apache::DBI tries to do that
automatically, but it's not very smart about it. It only happens if you
have AutoCommit off when you connect.)

- Perrin

Re: Disconnect database connection after idle timeout

[Perrin Harkins]

On Fri, Nov 14, 2014 at 1:32 PM, Xinhuan Zheng xzh...@christianbook.com
wrote:

 Are you implying that the performance will be suffered when using
 mod_perl-enabled server processes as the front tier servers?


Not performance, scalability. You can't handle as many requests per second
if you use mod_perl without a front-end proxy. That's why people on this
list with a large site use them, and why every similar system in Ruby,
Python, and PHP also recommend separating your web front-end from your
application server processes if you need to scale.

- Perrin

Re: Disconnect database connection after idle timeout

[Perrin Harkins]

Hi,

Can you explain what problem you're trying to solve? Apache processes don't
have the option of doing things when there is no request to serve, so you
can't easily have them disconnect. It may be possible with alarms or cron
jobs or something, but it's probably not a good idea.

If you tune your configuration to avoid leaving large numbers of servers
idle, you should not have problems with unused connections. Also, make sure
you are using a front-end proxy of some kind and not serving static HTTP
requests from your mod_perl server.

If your problem is that you need more active connections than your server
can handle, you could look at DBD::Gofer:
http://www.slideshare.net/Tim.Bunce/dbdgofer-200809

- Perrin

On Thu, Nov 13, 2014 at 9:39 AM, Xinhuan Zheng xzh...@christianbook.com
wrote:

  Your understanding is correct. It’s what I am looking for. However, due
 to the apache forking child nature, I don’t feel comfortable using SIGALARM.

  We use Apache::DBI. I would prefer having enhancement in this module.
 Currently the module is implementing apache process wide global cache for
 db connections, which we already use. But one missing piece in this module
 is to handle the TTL (time-to-live) for a cached db connection.  If TTL
 were implemented, the module would have to disconnect from db connection
 after a pre-defined timeout so the oracle server process could be shut down
 more gracefully. Would it be possible to implement that? Or is it too hard
 to implement?

  - xinhuan

   From: Paul Silevitch p...@silevitch.com
 Date: Wednesday, November 12, 2014 at 11:53 PM
 To: Xinhuan Zheng xzh...@christianbook.com, modperl 
 modperl@perl.apache.org
 Subject: Re: Disconnect database connection after idle timeout

   I don't fully understand your need here.  I'm going to give my best.

  You could set an alarm in the cleanup handler that calls the disconnect
 after a specified amount of time.  If a new request comes in, you could
 cancel the alarm in a postreadrequest handler (or something early in the
 cycle).  To cover the race condition where the disconnect happens right
 before the cancel, you could check to make sure the database connection is
 active right after the cancel is called.

  HTH,

 Paul

 On Wed, Nov 12, 2014 at 9:36 PM, Xinhuan Zheng xzh...@christianbook.com
 wrote:

  Hello,

  I am having a database connection management question. Our
 apache+mod_perl application initiates a database connection request when it
 needs to then do data processing. Afterwards, if there is no more requests
 coming to this apache process, the database connection basically will be
 sitting there in idle state for quite a while until the OS TCP/IP idle
 timeout has reached. At that point the database would send a message into
 its alert log, telling that a connection timeout has occurred and the
 server process will be cleaned. I would like to figure out if mod_perl
 application can implement keep-alive timeout mechanism. The mod_perl would
 maintain the database connection after it finishes some processing until an
 idle timeout defined in the application has reached, for example, 5
 minutes. Then the mod_perl would initiate a database disconnection request
 so the server process can be cleaned more gracefully. We are using Oracle
 11g database. I knew in 11G oracle has implemented the connection pooling.
 I would think the oracle side connection pooling would be the server side
 maintaining the connection idle timeout. Would it be possible on the client
 side the mod_perl implement something like that? I just don’t know which
 side is more appropriate and how on the client side it can implement
 something like that.

  Thanks,
 - xinhuan

Re: Disconnect database connection after idle timeout

[Perrin Harkins]

On Thu, Nov 13, 2014 at 10:29 AM, Xinhuan Zheng xzh...@christianbook.com
wrote:

 We don’t have any front end proxy.


I think I see the problem... ;)

If you use a front-end proxy so that your mod_perl servers are only
handling mod_perl requests, and tune your configuration so that idle
mod_perl servers don't sit around for long, that should avoid timeouts.
Apache::DBI should also re-connect with no problems if a request comes in
after a connection has timed out. If that isn't happening, make sure you
are using Apache::DBI properly.

Seriously, using a front-end proxy usually reduces the number of databases
connections about 10 times. It's the easiest fix here by far.

- Perrin

Re: Disconnect database connection after idle timeout

[Perrin Harkins]

On Thu, Nov 13, 2014 at 11:29 AM, Dr James Smith j...@sanger.ac.uk wrote:

 From experience - and having chatted with our DBAs at work, with modern
 Oracle and with MySQL keeping persistent connections around is no real gain
 and usually lots of risks


It's certainly good to know how long it takes to get a fresh connection and
consider whether you need persistent connections or not. Connecting tends
to be fast on MySQL and caching is probably not needed unless you're
running a very performance-sensitive site. The last time I worked with
Oracle, connections were too slow to run without caching them. That was
years ago though, and the situation may have improved.

- Perrin

Re: Disconnect database connection after idle timeout

[Perrin Harkins]

On Thu, Nov 13, 2014 at 12:21 PM, Vincent Veyron vv.li...@wanadoo.fr
wrote:

 Nobody suggested using connect_cached, where the documentations says :

 The cached database handle is replaced with a new
connection if it has been disconnected or if the ping method fails

 Would that not solve OP's problem?


Well, Apache::DBI should already be doing a ping and a successful
re-connect if needed. If that isn't working with Apache::DBI, there's no
reason to think it will work better with connect_cached. Also, Apache::DBI
has some additional things in it (avoiding caching connections during
startup, protection from dangling transactions at the end of a request)
that might be needed.

- Perrin

Re: Disconnect database connection after idle timeout

[Perrin Harkins]

On Thu, Nov 13, 2014 at 12:19 PM, Xinhuan Zheng xzh...@christianbook.com
wrote:

 Having another tier (like DBD::Gofer) looks like really messy in
 infrastructure plus it’s not certain who is going to maintain that module’s
 quality.


I'd only recommend trying it after you set up a front-end proxy, tune your
mod_perl configuration, and use any Oracle tools available to you.

- Perrin

Re: Disconnect database connection after idle timeout

[Perrin Harkins]

Yes, it's an HTTP proxy. It handles sending out the bytes to remote
clients, so that your mod_perl server doesn't have to. A popular
high-performance choice these days is nginx.

There's some discussion of why to use a front-end proxy here:
http://perl.apache.org/docs/1.0/guide/strategy.html

- Perrin

On Thu, Nov 13, 2014 at 2:12 PM, Xinhuan Zheng xzh...@christianbook.com
wrote:

  Hi Perrin,

  I don’t quite understand what you mean by setting up a front-end proxy.
 What would you expect this “proxy” do? Does it take HTTP request?

  Thanks,
 - xinhuan

   From: Perrin Harkins phark...@gmail.com
 Date: Thursday, November 13, 2014 at 12:50 PM
 To: Xinhuan Zheng xzh...@christianbook.com
 Cc: Dr James Smith j...@sanger.ac.uk, mod_perl list 
 modperl@perl.apache.org
 Subject: Re: Disconnect database connection after idle timeout

On Thu, Nov 13, 2014 at 12:19 PM, Xinhuan Zheng 
 xzh...@christianbook.com wrote:

 Having another tier (like DBD::Gofer) looks like really messy in
 infrastructure plus it’s not certain who is going to maintain that module’s
 quality.


 I'd only recommend trying it after you set up a front-end proxy, tune your
 mod_perl configuration, and use any Oracle tools available to you.

  - Perrin

Re: Disconnect database connection after idle timeout

[Perrin Harkins]

Yes, it has nothing to do with the version of mod_perl. It's a general
technique for any HTTP application server.

- Perrin

On Thu, Nov 13, 2014 at 2:33 PM, John Dunlap j...@lariat.co wrote:

 That link is from the mod_perl 1.x documentation. Is this information
 still valid in mod_perl 2.x?

 On Thu, Nov 13, 2014 at 2:23 PM, Perrin Harkins phark...@gmail.com
 wrote:

 Yes, it's an HTTP proxy. It handles sending out the bytes to remote
 clients, so that your mod_perl server doesn't have to. A popular
 high-performance choice these days is nginx.

 There's some discussion of why to use a front-end proxy here:
 http://perl.apache.org/docs/1.0/guide/strategy.html

 - Perrin

 On Thu, Nov 13, 2014 at 2:12 PM, Xinhuan Zheng xzh...@christianbook.com
 wrote:

  Hi Perrin,

  I don’t quite understand what you mean by setting up a front-end
 proxy. What would you expect this “proxy” do? Does it take HTTP request?

  Thanks,
 - xinhuan

   From: Perrin Harkins phark...@gmail.com
 Date: Thursday, November 13, 2014 at 12:50 PM
 To: Xinhuan Zheng xzh...@christianbook.com
 Cc: Dr James Smith j...@sanger.ac.uk, mod_perl list 
 modperl@perl.apache.org
 Subject: Re: Disconnect database connection after idle timeout

On Thu, Nov 13, 2014 at 12:19 PM, Xinhuan Zheng 
 xzh...@christianbook.com wrote:

 Having another tier (like DBD::Gofer) looks like really messy in
 infrastructure plus it’s not certain who is going to maintain that module’s
 quality.


 I'd only recommend trying it after you set up a front-end proxy, tune
 your mod_perl configuration, and use any Oracle tools available to you.

  - Perrin





 --
 John Dunlap
 *CTO | Lariat *

 *Direct:*
 *j...@lariat.co j...@lariat.co*

 *Customer Service:*
 877.268.6667
 supp...@lariat.co

Re: Disconnect database connection after idle timeout

[Perrin Harkins]

On Thu, Nov 13, 2014 at 2:48 PM, Vincent Veyron vv.li...@wanadoo.fr wrote:

 But then, what is the point of using connect_cached?


You can use it outside of mod_perl. You can also use instead of Apache::DBI
if you don't want the connection to be more explicit (instead of magically
overriding the connect() call).

Keep in mind, neither of these work unless you call
DBI-connect()/connect_cached() at the beginning of every request. You
can't just keep a DBI handle stashed somewhere and expect that to work.

- Perrin

Re: Disconnect database connection after idle timeout

[Perrin Harkins]

On Thu, Nov 13, 2014 at 2:41 PM, Xinhuan Zheng xzh...@christianbook.com
wrote:

 From the description of the document, the “proxy” server acts much like a
 memcache but it appears the difference is the “proxy” understands the HTTP
 protocol while memcache does not.


Not exactly. While it is possible to do some caching in a proxy like this,
the more important thing it does is buffering the content, and possibly
serving static requests for JavaScript and images. That frees up the
mod_perl processes to move on to another request. This is the same process
model used by most other systems, like FastCGI or nginx + Starman.


 Would you think this “proxy” server sitting in front of the front end
 server but behind the load balancer?


Yes, it goes between your load balancer and your mod_perl server.

- Perrin

Re: Disconnect database connection after idle timeout

[Perrin Harkins]

On Thu, Nov 13, 2014 at 5:29 PM, Vincent Veyron vv.li...@wanadoo.fr wrote:

 I stash a reference to a DBI handle in pnotes during the HeaderParser
 phase of my requests; I then refer to this handle for every request in my
 PerlResponseHandlers.

 This seems to have been working fine for several months. Am I doing
 something wrong?


No, that's perfect. Since pnotes gets cleaned up at the end of every
request, there's no danger of the handle sticking around. I assume you're
calling DBI-connect() to get the handle in the HeaderParser phase, and
using Apache::DBI?

The dangerous thing is to put a DBI handle in a global and just keep using
it on every request. If you put a DBI handle in a global (or a closure
variable) make sure you clean it up at the end of every request.

- Perrin

Re: Disconnect database connection after idle timeout

[Perrin Harkins]

On Thu, Nov 13, 2014 at 5:38 PM, Xinhuan Zheng xzh...@christianbook.com
wrote:

 We have load balancer cache that can cache images and JavaScripts. This
 functions seems a bit duplicate.


It's not about caching. Here's a quote from that link I sent earlier:

Another drawback of this approach is that when serving output to a client
with a slow connection, the huge mod_perl-enabled server process (with all
of its system resources) will be tied up until the response is completely
written to the client. While it might take a few milliseconds for your
script to complete the request, there is a chance it will be still busy for
some number of seconds or even minutes if the request is from a slow
connection client.

You might think everyone has fast connections now so this won't matter, but
it does. It's especially bad if you have keep-alive on for the apache
server running mod_perl, since that means your large mod_perl processes sit
around for some extra time doing nothing, while holding onto their database
connections. Install a front-end proxy, turn off keep-alive  on your
mod_perl and reduce your max idle servers, and watch what happens.

- Perrin

Re: Shared Dirty memory of Apache parent process

[Perrin Harkins]

On Thu, Sep 18, 2014 at 5:35 AM, Nageswara rao Gurram 
nageshgurra...@gmail.com wrote:

 As the traffic is increasing shared memory in parent is increasing , thats
 what puzzling me ! If childs write into shared of parent, then it should
 come into private of child(copy on write) but why shared of parent is
 increasing? (correct me if i am wrong)
 OR is it work like, OS always shows copied pages of parent's shared in its
 shared dirty memory(in smaps) even if it internally maintains private of
 each child in that ?


I still don't think that writing to that memory in the child process is
increasing the size of the parent process. Instead, I think that as the
parent serves more requests, it grows in size, and new child processes that
get forked from the now larger parent process are also larger.

The question is, what is making your parent process grow? It might be
another HTTPD module (i.e. not mod_perl), or it might be something in your
configuration that is causing it to run some code in the parent. I think
others have reported growth in the parent from certain compile options as
well, but that may have been just on mod_perl 1.

I notice that the private_dirty on your active site is not much higher than
on your backup site, which means the child processes are not really using
much more memory. Most of the increased size is shared, so not using up
real RAM.

- Perrin

Re: Shared Dirty memory of Apache parent process

[Perrin Harkins]

On Wed, Sep 17, 2014 at 1:18 PM, Mark Hedges mark.hed...@ticketmaster.com
wrote:

 For example, you could use a tied DBM/MLDBM hash, DBD::SQLite
 or another file-based database with access locking for your
 cache, and save it in a shared memory filesystem like /dev/shm.


I would suggest that too, if it fits the use case. BerkeleyDB,
Cache::FastMmap, Reddis, or Memcached are all common solutions. They don't
work for everything though. Declaring a global variable in the parent
process and loading it with data in the child processes won't share
anything. It means you're loading the entire data into unshared memory in
each child. If you want to share that data you have to load it before
forking. After a child process modifies it, changes are not shared back to
the parent or to other children.

As a side note, it's quite difficult to know how much memory is really
shared. If you want to know, look at Linux::Smaps, not top or ps.

I don't see any way that loading data in the child process could increase
the size of the parent process, but memory sharing behavior is complicated
and I could be missing something.

- Perrin

Re: Shared Dirty memory of Apache parent process

[Perrin Harkins]

Hi,

Loading data in the parent process is a common strategy for data that you
won't modify. Do you need to change this data from the child processes? If
so, does it matter if the other child processes see the changes?

- Perrin

On Tue, Sep 16, 2014 at 10:28 AM, Nageswara rao Gurram 
nageshgurra...@gmail.com wrote:

 Hi,
I am new to mod perl environment so this may looks naive.
   Recenlty I observed my apache processes are getting huge.. When I tried
 to dig this down I found apache parent process (rss memory , mainly shared
 dirty) itself is increasing with number of requests it is serving, so when
 everytime it is creating a new child it is sharing all its memory with
 childs and they are getting even more bigger and so on..
  I am thinking this might be one reason. I decalred a global variable in
 apache startup (so it'll be shared on across all childs), this variable
 will be loaded with huge DB data in one of perl modules(after request
 comes). Will this increase shared dirty of parent ? (After child writes
 into it, it will no longer shared between parent and child., and parent
 shared dirty should get decreased .. right ?)
  At start parent is only 100MB and after some days it's going above 250MB
 and so childs are going above 400M..
 Can anybody help ?

 Thanks,
 Rao.

Re: ModPerl::RegistryCooker default_handler returned status

[Perrin Harkins]

HI Pavel,

Your messages include a lot of good details, but they are a bit long. I'd
suggest trying to keep things as short as you can.  Remember that everyone
working on open source is a volunteer, spending their own personal time to
read and answer your messages.

On Sun, Aug 31, 2014 at 4:15 PM, Pavel V. pavel2...@ngs.ru wrote:
 I completely misunderstand, why we need to return HTTP_* codes from
ModPerl handlers.
 Why Apache::OK or Apache::SERVER_ERROR are not enough?

Returning HTTP status code constants from a handler is legal and expected.
 See Return Values here:
http://httpd.apache.org/docs/2.4/developer/modguide.html

 Can anybody explain, why this was needed with mp1?

Sure.  CGI scripts don't have return values, since they are standalone
scripts and not subroutines.  They set status by sending the headers  out.
 A proper handler can do this with the return code, but if you do that,
your script won't be correct under mod_cgi anymore.  Many people needed
their scripts to run under both, so the workaround was to set $r-status()
if you detect a mod_perl environment, and Registry would pick that up and
use it as the return code.

It might have been better to do something different that would have left
status untouched, like set an $ENV variable, but I suspect people were
already trying to set $r-status() directly and Doug just wanted to make
that work.  Registry has always been a balance between porting CGI scripts
and allowing people to do more mod_perl-specific things.

 Why we are unable to left $r-status, the new value set by script,
untouched?

I believe this is because we want the server to get the response code from
the handler and set the status itself.

 After reading mailing list I understand what nothing will change in
nearest  century.
 Sorry to bother you.

There's no need to be rude.  It's not a simple situation with a simple
answer.  This is why the recent versions of Registry were designed to be
easy to override.  It's impossible to please everyone with one set of
behavior.  Most people who are trying to do things involving status codes
end up switching over to writing handlers, which is a better approach if
you're not concerned about your code working under mod_cgi.

- Perrin

Re: ModPerl::RegistryCooker default_handler returned status

[Perrin Harkins]

Hi Pavel,

You might get more interesting answers on the users mailing list, but
there's lots of discussion about this in the archives.  For example:
http://www.gossamer-threads.com/lists/modperl/modperl/45559?search_string=registry%20status;#45559
http://www.gossamer-threads.com/lists/modperl/modperl/88690?search_string=registry%20status;#88690

I think the gist of it is that some people make their Registry scripts set
$r-status because Registry doesn't scan the script output for a status
header like mod_cgi does.  However, that means Apache doesn't know what the
real status is.  In order to tell Apache what the status is, Registry
checks to see if you set $r-status and uses that as the return code, which
may affect what status Apache decides to send.

I may be reading your use case incorrectly, but you seem to be trying to
tell the server you have a 200 OK but really return a 404.  That's
discussed a little bit here:
http://perl.apache.org/docs/2.0/api/Apache2/RequestRec.html#C_status_.
 That is somewhat sophisticated for a Registry script, but you can always
just make your own subclass and override that method in RegistryCooker to
do what you want.  It's intended to be customizable in that way.

- Perrin



On Sun, Aug 31, 2014 at 7:50 AM, Pavel V. pavel2...@ngs.ru wrote:


   Hi all.

   (sorry for unfinished letter I sent before)

   Can anybody explain historical reasons of ModPerl::RegistryCooker
 default_handler() implementation?
   We can see the following piece of code there:

 # handlers shouldn't set $r-status but return it, so we reset the
 # status after running it
 my $old_status = $self-{REQ}-status;
 my $rc = $self-run;
 my $new_status = $self-{REQ}-status($old_status);
 return ($rc == Apache2::Const::OK  $old_status != $new_status)
 ? $new_status
 : $rc;

   The goal of ModPerl modules is Run unaltered CGI scripts under
 mod_perl.
   If scripts are unaltered, how they can change $r-status? I think,
 answer should be: 'in no way'.
   Why we need to reset status then? I see no reasons. But this solution
 makes us impossible to set
   r-status from 'altered' scripts correctly and produces strange effects
 by headers parser on
   responses  8k size.

   Did anyone can explain, why this code appear and for which scenarios it
 come?

   Apache's mod_cgi.c module returns OK regardless of the r-status value,
 which was set from script output.
   Why ModPerl handler should behave differently?

   One good example about problems, introduced by this solution, is 404
 status.
   Let`s look into small script:

   #!/usr/bin/perl -w
   use strict;
   use CGI;

   my $q = CGI-new;
    Variant 1
   print $q-header(-charset = windows-1251, -type = text/html,
 -status='404 Not Found');
   print SMALL RESPONSE;
    Variant 2
   #print $q-header(-charset = windows-1251, -type = text/html,
 -status='404 Not Found');
   #print 'BIG RESPONSE:' . '*' x 8192;

   Under CGI it prints only SMALL/BIG RESPONSE strings with 404 status
 code into browser.
   Under mod_perl, browser get status 200 instead of 404 and script
 response content is appended by
   default Apache error-handler content (like 'Status: OK \n /path/to/
 script.pl was not found on this
   server') for 'small response'.
   If we print 'big response', then browser get status 404 and apache
 error-handler appended content
   changes to 'Not Found \n The requested URL /perl/test.pl was not found
 on this server.'.

   The reasons for such behavior is what script uses CGI.pm which detects
 mod_perl.
   So, these scripts are practically 'altered' and can set $r-status inside
 of them. After script
   executes, we have r-status == 404. But ModPerl::RegistryCooker set
 r-status to 200 back and return
   404 as handler status - so apache ErrorDocument directive begin to work.
   (under mod_cgi handler status is OK, so no error processing occurs).
   Also, due to r-status changed, we can get 200 response code in browser
 if response was not send
   yet.
   Most interesting thing, what we get 404 status logged into Apache access
 log anyway, so it is harder to
   detect this problem.

   Ok, maybe this is completely CGI.pm problem? Let's look into next
 example - fully unaltered script:

   #!/usr/bin/perl -w
   use strict;
   print Status: 404 Not Found\n;
   print Content-Type: text/html; charset=windows-1251\n\n;
   #Small response
   print 'NOTFOUND:' . '*' x 81;
   #Big response
   #print 'NOTFOUND:' . '*' x 8192;

   When response is small, then 8k buffer is not filled, and r-status does
 not changed internally
   before cgi headers parsed on buffer flush. So, we get r-status == 200,
 and handler status is 200
   too. After headers are parsed, r-status become 404, but mod_perl
 handler returns status 200 to
   apache.

   When response is big, then 8k buffer is filled and flushed before
 handler is done.
   So, we have r-status == 404, and handler status is 404 too. As result,
 we have ErrorDocument
   

Re: Something wrong with CPAN ?

[Perrin Harkins]

The address for reporting technical issues is cpansea...@perl.org,
although I'm sure they've heard about it by now.

There's a good alternate search at https://metacpan.org/.

- Perrin


On Mon, Jun 9, 2014 at 7:23 AM, André Warnier a...@ice-sa.com wrote:
 Hi guys.

 Is it only me, or does the search box in www.cpan.org not work anymore ?

 503 Service Unavailable
 No server is available to handle this request.

 (been like that for a couple of days now)

 If it is not me alone, and if someone here knows where to send a message
 about this, could you advise ?

 Thanks
 André

Re: Trouble with script execution

[Perrin Harkins]

On Sun, May 18, 2014 at 9:47 PM, Worik Stanton worik.stan...@gmail.com wrote:
 I am sure I read somewhere that mod_perl monitors scripts and reloads
 them if the modification date changes.

You're probably thinking of Apache::Registry/ModPerl::Registry.  They
do that with your CGI scripts.

- Perrin

Re: API Docs

[Perrin Harkins]

The $r there is the Apache2::RequestRec object.  It gets passed to
your handler.  See the handler example in the overview:
https://perl.apache.org/docs/2.0/user/intro/start_fast.html

- Perrin

On Wed, May 14, 2014 at 7:52 PM, Worik Stanton worik.stan...@gmail.com wrote:
 I am still trying to fathom the API.

 I have struck a snag with the documentation.


 https://perl.apache.org/docs/2.0/api/Apache2/Connection.html#Synopsis
 says

   use Apache2::Connection ();
   use Apache2::RequestRec ();

   my $c = $r-connection;


 What is $r?

 I am missing something obvious, what is the context here I do not get?

 cheers
 Worik
 --
 The only true evil is turning people into things
  Granny Weatherwax
worik.stan...@gmail.com 021-1680650, (03) 4821804
   Aotearoa (New Zealand)

Re: Trouble with mod_perl API: E.g., Apache2::Directive::conftree()

[Perrin Harkins]

Hi Worik,

Your ErrorLog question isn't really a mod_perl issue, but I'd suggest
you check for warnings at startup.  You may not have APACHE_LOG_DIR
defined, etc.

I don't use Apache2::Directive, so I can't vouch for it.  I prefer
simpler things like PerlSetVar.  There is an automated test for this
though:
http://cpansearch.perl.org/src/PHRED/mod_perl-2.0.8/t/response/TestApache/conftree.pm

I'd suggest doing some more basic debugging, e.g.

  use Data::Dumper;
  print Dumper $tree-as_hash;

- Perrin

On Tue, May 13, 2014 at 8:24 PM, Worik Stanton worik.stan...@gmail.com wrote:
 I am having a lot of trouble with mod_perl APIs.

 The short of it is that when I say:

 use Apache2::Directive ();
 my $tree = Apache2::Directive::conftree();
 my $documentroot = $tree-lookup('DocumentRoot');
 warn \$documentroot '$documentroot';

 I get:
 [Wed May 14 12:13:43 2014] -e: Use of uninitialized value $documentroot
 in concatenation (.) or string at /home/worik/Worik.org/cgi/track line 11.
 [Wed May 14 12:13:43 2014] -e: $documentroot '' at
 /home/worik/Worik.org/cgi/track line 11.

 in the logs.

 I copied the code from
 https://perl.apache.org/docs/2.0/api/Apache2/Directive.html

 The longer of it is:

 In the past (15 years ago) I installed mod_perl and forgot about it. I
 set up my perl cgi script handler to be mod_perl in httpd.conf, and the
 only real effect it had on my development cycle was I often had to
 restart Apache to get it to notice I had changed a script.

 These days I am more ambitious and am returning to Web programming after
 a long break.

 I expect mod_perl to still be useful to cut the overhead of starting
 Perl. I expect a high throughput this time around, each request will be
 worth very little to me so I want lots and lots of requests.  SO far so
 good.

 I have set up my server:

 ## mod_perl2 configuration
 Alias /cgi/  /home/worik/Worik.org/cgi/
 PerlModule ModPerl::Registry
 ## PerlModule Apache::Registry
 PerlSwitches -T
 ErrorLog ${APACHE_LOG_DIR}/Worik.error.log
 Location /cgi/
SetHandler perl-script
PerlResponseHandler ModPerl::Registry
PerlOptions +ParseHeaders
Options +ExecCGI
Order allow,deny
Allow from all
 /Location

 Now I would like to use the API.  In the first instance I want to find
 what the directory is I am running in.  So I want to say:

 use Apache2::Directive ();
 my $tree = Apache2::Directive::conftree();
 my $pwd = $tree-lookup(cgi);

 But that fails just like the example at the start of this email.

 Also my errors are coming out in the wrong log.  When I say warn(foo)
 I expect it to come out in the file specified by the ErrorLog directive
 but it comes as:

 [Wed May 14 12:22:53 2014] -e: foo at /home/worik/Worik.org/cgi/track
 line 12.

 in /var/log/apache2/error.log which is not what I specified.

 I am at a loss.  Is the documentation up to date?  Is mod_perl still
 being used out there?

 cheers
 Worik

 --
 The only true evil is turning people into things
  Granny Weatherwax
worik.stan...@gmail.com 021-1680650, (03) 4821804
   Aotearoa (New Zealand)

Re: Apache::DBI and Pgbouncer

[Perrin Harkins]

Interesting.  Why did you have to install PgBouncer?  Can't Postgres
handle remote connections from your web server?

I don't use Postgres, but reading the description of PgBouncer I can
see some things you'd want to consider.

First, Apache::DBI prevents you from making persistent connections
before the parent process forks.  If you don't use it, you should
check your code to make sure that it closes any handles it opens
during server startup.

Second, there's the issue of what happens when your code throws an
exception.  Apache::DBI will issue a rollback on any active handles
that aren't in autocommit mode after each request.  If you don't use
it, I'd suggest adding your own cleanup handler to do a rollback.

Finally, there's the issue of performance.  It's not clear whether DBI
connects faster when using PgBouncer.  You should probably benchmark
that yourself.  You may still get a significant speed boost from
caching the connections (with Apache::DBI) on the client side.

- Perrin

On Tue, Apr 22, 2014 at 10:02 AM, jbiskofski jbiskof...@gmail.com wrote:
 I just want to confirm something with all you smart folks.

 I recently separated my web servers from my database servers, before I was
 using Apache::DBI to maintain persistent connections between Apache and
 Postgres. With this new setup I had to install PgBouncer. Can I now safely
 remove Apache::DBI from my application and use regular DBI ??

 Thank you.

Re: Apache::DBI and Pgbouncer

[Perrin Harkins]

Thanks John.  Were you using Apache::DBI before PgBouncer?
Apache::DBI would also eliminate the overhead of establishing new
connections.

- Perrin

On Tue, Apr 22, 2014 at 12:23 PM, John Dunlap j...@lariat.co wrote:
 I can speak to your final point. I recently deployed PGBouncer into our
 production environment and, like the OP, we have separate web and database
 servers. With PGBouncer running on the web server(you could also run it on
 the database server if you wanted to) we noticed a dramatic increase in
 performance. I haven't looked into it in detail but my best guess is that
 running the pool on the web server eliminates the overhead of establishing
 new connections(DNS lookups, establishing TCP connections, authentication,
 waiting for the database to spool up a new process, etc).


 On Tue, Apr 22, 2014 at 12:18 PM, Perrin Harkins phark...@gmail.com wrote:

 Interesting.  Why did you have to install PgBouncer?  Can't Postgres
 handle remote connections from your web server?

 I don't use Postgres, but reading the description of PgBouncer I can
 see some things you'd want to consider.

 First, Apache::DBI prevents you from making persistent connections
 before the parent process forks.  If you don't use it, you should
 check your code to make sure that it closes any handles it opens
 during server startup.

 Second, there's the issue of what happens when your code throws an
 exception.  Apache::DBI will issue a rollback on any active handles
 that aren't in autocommit mode after each request.  If you don't use
 it, I'd suggest adding your own cleanup handler to do a rollback.

 Finally, there's the issue of performance.  It's not clear whether DBI
 connects faster when using PgBouncer.  You should probably benchmark
 that yourself.  You may still get a significant speed boost from
 caching the connections (with Apache::DBI) on the client side.

 - Perrin

 On Tue, Apr 22, 2014 at 10:02 AM, jbiskofski jbiskof...@gmail.com wrote:
  I just want to confirm something with all you smart folks.
 
  I recently separated my web servers from my database servers, before I
  was
  using Apache::DBI to maintain persistent connections between Apache and
  Postgres. With this new setup I had to install PgBouncer. Can I now
  safely
  remove Apache::DBI from my application and use regular DBI ??
 
  Thank you.
 

Re: Apache::DBI and Pgbouncer

[Perrin Harkins]

Apache::DBI overrides DBI's connect() method so that you're using
persistent connections when you use DBI directly.  It may be that your
performance improvement came from better management of Pg resources
with PgBouncer than from reducing connection overhead.  You could test
it be removing Apache::DBI and benchmarking.

- Perrin

On Tue, Apr 22, 2014 at 12:33 PM, John Dunlap j...@lariat.co wrote:
 use Apache::DBI (); appears in our startup.pl but the application code uses
 DBI directly.


 On Tue, Apr 22, 2014 at 12:30 PM, Perrin Harkins phark...@gmail.com wrote:

 Thanks John.  Were you using Apache::DBI before PgBouncer?
 Apache::DBI would also eliminate the overhead of establishing new
 connections.

 - Perrin

 On Tue, Apr 22, 2014 at 12:23 PM, John Dunlap j...@lariat.co wrote:
  I can speak to your final point. I recently deployed PGBouncer into our
  production environment and, like the OP, we have separate web and
  database
  servers. With PGBouncer running on the web server(you could also run it
  on
  the database server if you wanted to) we noticed a dramatic increase in
  performance. I haven't looked into it in detail but my best guess is
  that
  running the pool on the web server eliminates the overhead of
  establishing
  new connections(DNS lookups, establishing TCP connections,
  authentication,
  waiting for the database to spool up a new process, etc).
 
 
  On Tue, Apr 22, 2014 at 12:18 PM, Perrin Harkins phark...@gmail.com
  wrote:
 
  Interesting.  Why did you have to install PgBouncer?  Can't Postgres
  handle remote connections from your web server?
 
  I don't use Postgres, but reading the description of PgBouncer I can
  see some things you'd want to consider.
 
  First, Apache::DBI prevents you from making persistent connections
  before the parent process forks.  If you don't use it, you should
  check your code to make sure that it closes any handles it opens
  during server startup.
 
  Second, there's the issue of what happens when your code throws an
  exception.  Apache::DBI will issue a rollback on any active handles
  that aren't in autocommit mode after each request.  If you don't use
  it, I'd suggest adding your own cleanup handler to do a rollback.
 
  Finally, there's the issue of performance.  It's not clear whether DBI
  connects faster when using PgBouncer.  You should probably benchmark
  that yourself.  You may still get a significant speed boost from
  caching the connections (with Apache::DBI) on the client side.
 
  - Perrin
 
  On Tue, Apr 22, 2014 at 10:02 AM, jbiskofski jbiskof...@gmail.com
  wrote:
   I just want to confirm something with all you smart folks.
  
   I recently separated my web servers from my database servers, before
   I
   was
   using Apache::DBI to maintain persistent connections between Apache
   and
   Postgres. With this new setup I had to install PgBouncer. Can I now
   safely
   remove Apache::DBI from my application and use regular DBI ??
  
   Thank you.
  
 
 

Re: support for Apache 2.4

[Perrin Harkins]

On Fri, Feb 14, 2014 at 8:01 PM, Jie Gao j@sydney.edu.au wrote:
 The link http://svn.apache.org/snapshots/modperl-2.0/ on page
 http://perl.apache.org/download/source.html#Development_mod_perl_2_0_Source_Distribution
 returns 404; http://perl.apache.org/dist/ is full of broken links.

Apologies for the disruptions on the site.  We have had to make some
changes in how it is generated in order to catch up to current Apache
policies, and that has exposed some parts of the site generation
process that were only understood by developers who are no longer
around.  We're working on it.

- Perrin

Re: Singleton Persistence

[Perrin Harkins]

On Sat, Feb 1, 2014 at 8:14 PM, John Dunlap j...@lariat.co wrote:
 In mod_perl, can instantiated singletons survive requests?

Sure, anything you put in a global will survive until the httpd child
process is shut down.  How many requests each child serves depends on
your configuration.  When a new child starts up, it will have the same
state as the parent process, so if you put something into a global in
the parent, it will be there in the children.  There is no sharing
between processes though, so changes in a child process will not be
reflected in the other processes or in newly spawned ones.

- Perrin

Re: Singleton Persistence

[Perrin Harkins]

On Mon, Feb 3, 2014 at 5:33 PM, Chris Bennett
ch...@bennettconstruction.us wrote:
 Right now I am using pg_1_.pl and pg_2.pl on the different hosts, but
 the code is identical, except that some data is pulled in from a config
 file for the different databases, etc used.

 Can I safely use pg.pl on both VirtualHosts or do I need to do something
 else also to do this?

I'm sure that you can use the same code, but the question is how do
you know which data to pull in and how do you store it in perl data
structures?  One common approach in this situation is to use
PerlSetVar inside your virtualhost config to set something you can
read from perl that will tell you which virtualhost you're in.  Then
you can use that as a key into a data structure or for triggering
conditional behavior.

- Perrin

Re: Returning 200 status from my handler

[Perrin Harkins]

Hi John,

 I notice, in the getting started documentation, that I am supposed to return
 Apache2::Const::OK to indicate an HTTP 200 success status

Actually, that's not about sending a HTTP 200 success status.  You
return Apache2::Const::OK to tell httpd that your handler ran
correctly.

 When I print out other values of
 constants from the same package they seem to have numeric values which
 directly correspond to their respective HTTP status codes.

True, but it's just due to a lack of imagination on the part of the
httpd developers.  The only constants that intentionally contain valid
HTTP status codes are the Apache2::Const::HTTP_* ones.

 However, it strikes me as a extremely bizarre,
 especially when returning Apache2::Const::HTTP_OK, which does evaluate to
 200, causes Apache2 to response with a canned error response page

Right because you returned something other than HTTP::Const::OK,
meaning your handler failed and httpd should sound the alarm.

I hope that clears it up for you.  This is a common confusion, so
nothing to feel bad about.

- Perrin

Re: Interaction with ORM software

[Perrin Harkins]

Hi Ted,

DBIx::DataModel doesn't use memcached and isn't really related to it.
You can certainly store things in memcached yourself, but your
DBIx::DataModel won't know anything about it.  You'd have to add a
caching layer on top yourself.  If having that built into your ORM is
important to you, you should look at DBIx::Class.  I think it's the
only popular one that supports this feature.

BTW, if you're concerned about speed, I'd recommend using Tim Bunce's
memcached client module, Cache::Memcached::libmemcached.  It should be
quite a bit faster than Cache::Memcached.  Or use the cache
abstraction layer, CHI, so you switch to a different cache module when
a faster one comes along.

- Perrin


On Tue, Jan 14, 2014 at 3:18 PM, Ted Byers r.ted.by...@gmail.com wrote:
 I write web code using Perl, running on Apache's web server.

 I am beginning to use the perl package DBIx::Datamodel, which looks to be a
 capable ORM package.  I know there is a perl package, Cache::Memcached, that
 provides in interface to memcached.  But what isn't clear to me is whether
 the two packages will place nicely with each other, or whether memcached has
 its own ORM.

 And can it handle cases in which there are multiple different databases, in
 different RDBMS, possibly on different machines?

 Thanks

 Ted

 --

 ---
 You received this message because you are subscribed to the Google Groups
 memcached group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to memcached+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/groups/opt_out.

-- 

--- 
You received this message because you are subscribed to the Google Groups 
memcached group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to memcached+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Please delete old releases from mirroring system

[Perrin Harkins]

FYI.  I deleted the -current stuff and fixed the other issues.  The
links on our download page on perl.apache.org are currently pretty
broken, but they were broken long before I made these changes.  Looks
like they were never fixed after hosting of releases moved off
perl.apache.org.  I'll see if I can fix them soon.  My feeling is that
everyone gets mod_perl from CPAN, so it doesn't make much difference,
but it looks bad.

- Perrin

On Wed, Jan 1, 2014 at 7:22 PM, Perrin Harkins per...@elem.com wrote:
 On Tue, Dec 31, 2013 at 1:05 PM, sebb seb...@gmail.com wrote:
 Are the -current artifacts and directories really needed?

 No.  Deleted.

 BTW, the .htaccess file contains a broken (obsolete) redirect

 Removed.

 Also the image link in HEADER.html does not work; I think you'll need
 to use an absolute URL.

 More artifacts from when this lived on perl.apache.org.  Fixed.

 Thanks!

 - Perrin

-
To unsubscribe, e-mail: dev-unsubscr...@perl.apache.org
For additional commands, e-mail: dev-h...@perl.apache.org

Re: Please delete old releases from mirroring system

[Perrin Harkins]

The source bundle releases are there.  It's just the links on
perl.apache.org that are broken.  They will be fixed.

I don't see anything on that page about pointing to releases with a mirror
script.  Do you have more info on that?

- Perrin


On Wed, Jan 1, 2014 at 8:41 PM, sebb seb...@gmail.com wrote:

 On 2 January 2014 00:26, Perrin Harkins per...@elem.com wrote:
  FYI.  I deleted the -current stuff and fixed the other issues.  The
  links on our download page on perl.apache.org are currently pretty
  broken, but they were broken long before I made these changes.  Looks
  like they were never fixed after hosting of releases moved off
  perl.apache.org.  I'll see if I can fix them soon.  My feeling is that
  everyone gets mod_perl from CPAN, so it doesn't make much difference,
  but it looks bad.

 Even if people generally get the code from CPAN, it is an ASF
 requirement to make the source bundle release available from the ASF
 mirror system.
 And the download page must use the appropriate mirror script to point
 to the source release.

 http://www.apache.org/dev/release-publishing.html#distribution

  - Perrin
 
  On Wed, Jan 1, 2014 at 7:22 PM, Perrin Harkins per...@elem.com wrote:
  On Tue, Dec 31, 2013 at 1:05 PM, sebb seb...@gmail.com wrote:
  Are the -current artifacts and directories really needed?
 
  No.  Deleted.
 
  BTW, the .htaccess file contains a broken (obsolete) redirect
 
  Removed.
 
  Also the image link in HEADER.html does not work; I think you'll need
  to use an absolute URL.
 
  More artifacts from when this lived on perl.apache.org.  Fixed.
 
  Thanks!
 
  - Perrin
 
  -
  To unsubscribe, e-mail: dev-unsubscr...@perl.apache.org
  For additional commands, e-mail: dev-h...@perl.apache.org
 

Re: Please delete old releases from mirroring system

[Perrin Harkins]

Hi,

I've deleted 2.0.7 from https://dist.apache.org/repos/dist/release/perl/.
 Please let us know if anything further is required.

Sorry for the delay on this.  We've only just switched to svnpubsub and are
still figuring it out.

- Perrin


On Tue, Dec 31, 2013 at 9:55 AM, sebb seb...@gmail.com wrote:

 On 31 December 2013 04:41, Fred Moyer f...@redhotpenguin.com wrote:
  Anyone here able to do this on people.apache.org? Just reset my
  password, but I left an ssh login session open and now appear to be
  temporarily banned at the firewall level.

 It cannot be done on people, because the project is now using svnpubsub.

 Please just delete the files from SVN at

 https://dist.apache.org/repos/dist/release/perl/

 Anyone on the PMC should be able to do that.

 You can reset your password using

 https://id.apache.org/


  Will try later unless someone else can.
 
  On Mon, Dec 30, 2013 at 5:17 PM, sebb seb...@gmail.com wrote:
  PING
 
  On 16 December 2013 23:44, sebb seb...@gmail.com wrote:
  There does not seem to be an issue tracker for the project, so posting
  to the dev list.
 
  To reduce the load on the ASF mirrors, projects are required to delete
  old releases [1]
 
  Please can you remove all non-current releases?
 
  i.e. 2.0.7
 
  Thanks!
 
  [Note that older releases are always available from the ASF archive
 server]
 
  [1] http://www.apache.org/dev/release.html#when-to-archive
 
  -
  To unsubscribe, e-mail: dev-unsubscr...@perl.apache.org
  For additional commands, e-mail: dev-h...@perl.apache.org
 

 -
 To unsubscribe, e-mail: dev-unsubscr...@perl.apache.org
 For additional commands, e-mail: dev-h...@perl.apache.org

Re: Apache::DBI connect

[Perrin Harkins]

On Fri, Nov 8, 2013 at 2:58 PM, Xinhuan Zheng xzh...@christianbook.comwrote:

 The query is from the application code not the 'select 1' test query.


OK, then it seems like Apache::DBI may not have a problem.


 While most of time I saw Apache::DBI-connect is called but sometime from
 a process, it will print out it finds a cached connection and ping is ok
 but sometimes from the same process it didn't find a cached connection.
 Since it's the same process, it doesn't make sense it can find a cached one
 then it didn't find any. Even after it says it didn't find a cached one
 then after that it prints out it still has a cached one and can use that
 one. It sounds like in same process sometimes the Apache:DBI connection
 cache can be accessed but sometimes it cannot.


The cache can always be accessed.  It's just a hash in a global variable.
 There are a couple of possible explanations for what you're seeing, off
the top of my head.  One is that you might be sending in slightly different
connection parameters, which means you can't use the cached connection.
 Another is that your code might not be calling DBI-connect() or
DBI-connect_cached().


 The base class is similar to a class in Java world. Basically it has set
 and get methods. Among other things, it sets $db and gets $db. The class
 has its own fetch method and it'll use $self-db method call to do the
 fetch/insert,etc.


It's ok to keep a database handle during the length of one request, but you
shouldn't keep it in your own global between requests, so if objects of
this class stay around for longer than one request you should not store a
db handle in them.  The safest thing would be for your $self-db() method
to just call DBI-connect_cached() every time.


 BTW, the class has a method AUTOLOAD, although I don't quite understand
 how AUTOLOAD works.


There's a lot of good documentation on AUTOLOAD in the Perl man pages and
books.  It's not relevant to your database issues though.

- Perrin

Re: Apache::DBI connect

[Perrin Harkins]

On Thu, Nov 7, 2013 at 11:06 PM, Xinhuan Zheng xzh...@christianbook.comwrote:

 As I turned on more debugging, when the problem occurs, the Apache cached
 connection reference is different than the database handle reference the
 query is using.


Which query are you talking about?  The ping in Apache::DBI?  Or something
else in your own code?


 We use Apache::DBI and DBI and use Apache::DBI is before DBI. We actually
 call DBI-connect_cached and based on DBI document, I guess this would be
 handled by Apache::DBI connect.


Yes, it should be.


 However, I guess we also defined a base class that can store a database
 handle. There are subclasses inherited from this base class. Is that good
 idea to store a database handle in a base class?


No, don't do that.  You should pick one approach for managing database
handles, whether it's Apache::DBI or something else, and only use that.  If
you keep a database handle around in a global somewhere, it may time out
and nothing will ping it because you're not calling connect() on each
request.

- Perrin

Re: Apache::DBI connect

[Perrin Harkins]

Have you used the DRCP settings in DBD::Oracle?
http://search.cpan.org/~pythian/DBD-Oracle-1.66/lib/DBD/Oracle.pm#Oracle_DRCP

From the description of DRCP, it sounds like you should never lose the
connection.  You may want to check the configuration.

From your debugging output, it looks like the line that tries to re-connect
is failing.  I'd add some debugging statements to confirm that, and then
try writing a simple script outside of mod_perl that tries to re-connect to
Oracle after you drop the connection from the Oracle side.  To figure out
more, you'll probably need to talk to whoever maintains DBD::Oracle.

- Perrin


On Wed, Nov 6, 2013 at 8:55 PM, Xinhuan Zheng xzh...@christianbook.comwrote:

  Hi Perrin,

  I am using Oracle Resident Connection Pool feature. The application
 randomly got ORA-03114 not connect to oracle database error. As per this
 document:

  http://search.cpan.org/~phred/Apache-DBI-1.12/lib/Apache/DBI.pm

  I have changed the DBD::Oracle to use the following ping instead of
 ora_ping:

  sub ping {
 my ($dbh) = @_;
 my $ret = 0;
 eval {
 local $SIG{__DIE__} = sub { return (0); };
 local $SIG{__WARN__} = sub { return (0); };
 # adapt the select statement to your database:
 $ret = $dbh-do('select 1');
 };
 return ($@) ? 0 : $ret;
 }

  After this change, the application can detect end-of-file
 communication error then after the application establishes new connection,
 the application still throws ORA-03114 not connect to oracle database
 error.

  below is the errors when Apache::DBI debug is set:

  3861 Apache::DBI need ping: yes
 DBD::Oracle::db ping failed: ORA-03113: end-of-file on communication
 channel
 Process ID: 29671
 Session ID: 3219 Serial number: 544 (DBD ERROR: OCIStmtExecute/Describe)
 at /usr/local/lib/perl5/site_perl/5.8.3/Apache/DBI.pm line 200.
 3861 Apache::DBI new connect to '...'
 3861 Apache::DBI need ping: yes
 3861 Apache::DBI already connected to '...'
 3861 Apache::DBI need ping: yes
 3861 Apache::DBI already connected to '...'
 3861 Apache::DBI need ping: yes
 3861 Apache::DBI already connected to '...'
 3861 Apache::DBI need ping: yes
 3861 Apache::DBI already connected to '...'
 3861 Apache::DBI need ping: yes
 3861 Apache::DBI already connected to '...'
 3861 Apache::DBI need ping: yes
 3861 Apache::DBI already connected to '...'
 3861 Apache::DBI need ping: yes
 DBD::Oracle::st execute failed: ORA-03114: not connected to ORACLE (DBD
 ERROR: OCIStmtExecute)
 3861 Apache::DBI need ping: yes
 3861 Apache::DBI already connected to ''
 3861 Apache::DBI disconnect (overloaded)
  [Wed Nov  6 16:49:50 2013] [crit] ORA-03114: not connected to ORACLE
 (DBD ERROR: OCIStmtExecute).

  Process 3861 establishes a new connection. and it says already
 connected to database. While the query execution failed with ORA-03114
 and the httpd process is in CLOSE_WAIT state. How does it thinks it's
 already connected while it's actually not?


- xinhuan


 

  From:  Perrin Harkins phark...@gmail.com
 Date:  Wednesday, November 6, 2013 5:54 PM
 To:  Xinhuan Zheng xzh...@christianbook.com
 Cc:  mod_perl list modperl@perl.apache.org
 Subject:  Re: Apache::DBI connect



  On Wed, Nov 6, 2013 at 4:07 PM, Xinhuan Zheng
 xzh...@christianbook.com wrote:

  I wonder this line of code should be changed to 'DBI-connect(@args) if
 ($@)'. If ping failed, that means the connection is already closed, $drh
 may be no longer valid, will $drh-connect always return a valid new
 connection? What's the difference between DBI-connect
 vs $drh-connect?



  The $drh there is a DBI driver, not a database handle.  It should return
 a new connection every time.  You can try putting in DBI-connect() there
 and see if it fixes the problem for you though.  If not, I'd suggest
 reporting the problem as a DBD::Oracle issue.

  By the way, which Oracle pooling solution are you using and how is it
 working out for you, aside from this problem?

  - Perrin

Re: Apache::DBI connect

[Perrin Harkins]

On Thu, Nov 7, 2013 at 9:46 AM, Xinhuan Zheng xzh...@christianbook.com
wrote:
The $ok is undef. In the case if the test does succeed (like the first
select), $ok returns 0E0.

That all sounds good.  0E0 is a true value in Perl.  It means zero but
true.  And undef is a false value.  You don't need to test for undef.

 Since I changed DBD::Oracle subroutine ping to use 'select 1 from dual',

I don't actually understand why you did that.  What was wrong with the
normal ping?

In any case, there's no need to change the Apache::DBI code, even with your
select 1 from dual test.  It returns a true value (0E0) if it succeeds
and a false value (undef) if it fails.

Did your script succeed in reconnecting after it lost the connection?

 I have another request. The Apache::DBI cached a dead database handle for
apache version 1.3.42 if startup.pl create a database handle. The apache
child processes inherits this dead handle. It doesn't cause application
error but it does take memory space. If there is many apache processes,
that's not good. Can you please identify and change the code for this
problem?

Yes, I haven't forgotten about that, but I haven't had time to work on it
yet.  You can try fixing it yourself by looking at the code in Apache::DBI
that checks if the server is starting under apache 1.x.  Otherwise, I will
eventually get to it.

- Perrin

Re: Apache::DBI connect

[Perrin Harkins]

It is in the DBI documentation.  Search for 0E0.

- Perrin


On Thu, Nov 7, 2013 at 12:41 PM, Xinhuan Zheng xzh...@christianbook.comwrote:

 So it returns string '0E0'? The document didn't say that.

 - xinhuan

 On 11/7/13 11:44 AM, Adam Prime adam.pr...@utoronto.ca wrote:

 perl -e if ('0E0') { print qq[hi\n] }
 hi
 
 OE0 as a string evaluates to true.  If you use it as a bareword /
 numeric then it's false, which is what your eval example below is doing.
 
 Adam
 
 
 On 13-11-07 11:29 AM, Xinhuan Zheng wrote:
  one correct - In both cases, the return value is evaluated to false.
 
  How do you distinguish?
 
  - xinhuan
 
  From: Xinhuan Zheng xzh...@christianbook.com
  mailto:xzh...@christianbook.com
  Date: Thursday, November 7, 2013 11:12 AM
  To: Perrin Harkins phark...@gmail.com mailto:phark...@gmail.com
  Cc: mod_perl list modperl@perl.apache.org
 mailto:modperl@perl.apache.org
  Subject: Re: Apache::DBI connect
 
  I don't actually understand why you did that.  What was wrong with the
 normal ping?
 
  With Oracle DRCP, even though ping succeeds, the connection to the
  server process is actually terminated. Or ora_ping() may return 0E0
  zero but true and undef. I don't know. ora_ping() is foreign to me. I
  made the change based on what Apache::DBI the document said.
 
  In any case, there's no need to change the Apache::DBI code, even with
 your select 1 from dual test.   It returns a true value (0E0) if it
 succeeds and a false value (undef) if it fails.
 
  In both cases, the return value is evaluated to true:
 
  200: if ($Connected{$Idx} and (!$needping or
 eval{$Connected{$Idx}-ping}))
 
  eval{0E0} and eval{undef} both return true. I did test that. How do you
  distinguish?
 
  Did your script succeed in reconnecting after it lost the connection?
 
  Yes.
 
  Yes, I haven't forgotten about that, but I haven't had time to work on
 it yet.  You can try fixing it yourself by looking at the code in
 Apache::DBI that checks if the server is starting under apache 1.x.
 Otherwise, I will eventually get to it.
 
  I don't understand that piece of code. I can't do the change. Hope you
  can help.
 
  - xinhuan
 
  From: Perrin Harkins phark...@gmail.com mailto:phark...@gmail.com
  Date: Thursday, November 7, 2013 11:00 AM
  To: Xinhuan Zheng xzh...@christianbook.com
  mailto:xzh...@christianbook.com
  Cc: mod_perl list modperl@perl.apache.org
 mailto:modperl@perl.apache.org
  Subject: Re: Apache::DBI connect
 
  On Thu, Nov 7, 2013 at 9:46 AM, Xinhuan Zheng xzh...@christianbook.com
  mailto:xzh...@christianbook.com wrote:
 The $ok is undef. In the case if the test does succeed (like the first
 select), $ok returns 0E0.
 
  That all sounds good.  0E0 is a true value in Perl.  It means zero but
  true.  And undef is a false value.  You don't need to test for undef.
 
  Since I changed DBD::Oracle subroutine ping to use 'select 1 from
 dual',
 
  I don't actually understand why you did that.  What was wrong with the
  normal ping?
 
  In any case, there's no need to change the Apache::DBI code, even with
  your select 1 from dual test.  It returns a true value (0E0) if it
  succeeds and a false value (undef) if it fails.
 
  Did your script succeed in reconnecting after it lost the connection?
 
  I have another request. The Apache::DBI cached a dead database handle
 for apache version 1.3.42 if  startup.pl http://startup.pl create a
 database handle. The apache
  child processes inherits this dead handle. It doesn't cause application
  error but it does take memory space. If there is many apache processes,
  that's not good. Can you please identify and change the code for this
  problem?
 
  Yes, I haven't forgotten about that, but I haven't had time to work on
  it yet.  You can try fixing it yourself by looking at the code in
  Apache::DBI that checks if the server is starting under apache 1.x.
Otherwise, I will eventually get to it.
 
  - Perrin
 
 

Re: Apache::DBI connect

[Perrin Harkins]

Sorry, I can't determine the problem from your log.  You'll need to either
run it in the debugger or add some debugging print statements to figure out
where it's having trouble.  All I can say from that output is that it it's
not succeeding in making a new connection after the ping fails, because it
doesn't say new connect... after that.  I also wonder where that
disconnect() call is coming from.

- Perrin


On Thu, Nov 7, 2013 at 1:07 PM, Xinhuan Zheng xzh...@christianbook.comwrote:

  I only looked at Apache::DBI not DBI document.

  My test program works. It reconnects to database OK. I ran it multiple
 times and every time it reconnects OK. But Apache::DBI doesn't work. You
 saw the previous debugging info. Where is the problem?

  - xinhuan

   From: Perrin Harkins phark...@gmail.com
 Date: Thursday, November 7, 2013 1:00 PM
 To: Xinhuan Zheng xzh...@christianbook.com
 Cc: Adam Prime adam.pr...@utoronto.ca, modperl@perl.apache.org 
 modperl@perl.apache.org

 Subject: Re: Apache::DBI connect

   It is in the DBI documentation.  Search for 0E0.

  - Perrin


 On Thu, Nov 7, 2013 at 12:41 PM, Xinhuan Zheng 
 xzh...@christianbook.comwrote:

 So it returns string '0E0'? The document didn't say that.

 - xinhuan

 On 11/7/13 11:44 AM, Adam Prime adam.pr...@utoronto.ca wrote:

 perl -e if ('0E0') { print qq[hi\n] }
 hi
 
 OE0 as a string evaluates to true.  If you use it as a bareword /
 numeric then it's false, which is what your eval example below is doing.
 
 Adam
 
 
 On 13-11-07 11:29 AM, Xinhuan Zheng wrote:
  one correct - In both cases, the return value is evaluated to false.
 
  How do you distinguish?
 
  - xinhuan
 
  From: Xinhuan Zheng xzh...@christianbook.com
  mailto:xzh...@christianbook.com
  Date: Thursday, November 7, 2013 11:12 AM
  To: Perrin Harkins phark...@gmail.com mailto:phark...@gmail.com
  Cc: mod_perl list modperl@perl.apache.org
 mailto:modperl@perl.apache.org
  Subject: Re: Apache::DBI connect
 
  I don't actually understand why you did that.  What was wrong with the
 normal ping?
 
  With Oracle DRCP, even though ping succeeds, the connection to the
  server process is actually terminated. Or ora_ping() may return 0E0
  zero but true and undef. I don't know. ora_ping() is foreign to me. I
  made the change based on what Apache::DBI the document said.
 
  In any case, there's no need to change the Apache::DBI code, even with
 your select 1 from dual test.   It returns a true value (0E0) if it
 succeeds and a false value (undef) if it fails.
 
  In both cases, the return value is evaluated to true:
 
  200: if ($Connected{$Idx} and (!$needping or
 eval{$Connected{$Idx}-ping}))
 
  eval{0E0} and eval{undef} both return true. I did test that. How do you
  distinguish?
 
  Did your script succeed in reconnecting after it lost the connection?
 
  Yes.
 
  Yes, I haven't forgotten about that, but I haven't had time to work on
 it yet.  You can try fixing it yourself by looking at the code in
 Apache::DBI that checks if the server is starting under apache 1.x.
 Otherwise, I will eventually get to it.
 
  I don't understand that piece of code. I can't do the change. Hope you
  can help.
 
  - xinhuan
 
  From: Perrin Harkins phark...@gmail.com mailto:phark...@gmail.com
  Date: Thursday, November 7, 2013 11:00 AM
  To: Xinhuan Zheng xzh...@christianbook.com
  mailto:xzh...@christianbook.com
  Cc: mod_perl list modperl@perl.apache.org
 mailto:modperl@perl.apache.org
  Subject: Re: Apache::DBI connect
 
  On Thu, Nov 7, 2013 at 9:46 AM, Xinhuan Zheng 
 xzh...@christianbook.com
  mailto:xzh...@christianbook.com wrote:
 The $ok is undef. In the case if the test does succeed (like the first
 select), $ok returns 0E0.
 
  That all sounds good.  0E0 is a true value in Perl.  It means zero but
  true.  And undef is a false value.  You don't need to test for undef.
 
  Since I changed DBD::Oracle subroutine ping to use 'select 1 from
 dual',
 
  I don't actually understand why you did that.  What was wrong with the
  normal ping?
 
  In any case, there's no need to change the Apache::DBI code, even with
  your select 1 from dual test.  It returns a true value (0E0) if it
  succeeds and a false value (undef) if it fails.
 
  Did your script succeed in reconnecting after it lost the connection?
 
  I have another request. The Apache::DBI cached a dead database handle
 for apache version 1.3.42 if  startup.pl http://startup.pl create a
 database handle. The apache
  child processes inherits this dead handle. It doesn't cause application
  error but it does take memory space. If there is many apache processes,
  that's not good. Can you please identify and change the code for this
  problem?
 
  Yes, I haven't forgotten about that, but I haven't had time to work on
  it yet.  You can try fixing it yourself by looking at the code in
  Apache::DBI that checks if the server is starting under apache 1.x.
Otherwise, I will eventually get to it.
 
  - Perrin
 
 

Re: Apache::DBI connect

[Perrin Harkins]

That line stops it from running the ping if $needping is false.  The or
and || operators in Perl are sometimes called short circuit operators
because of this.

- Perrin


On Thu, Nov 7, 2013 at 4:18 PM, Xinhuan Zheng xzh...@christianbook.comwrote:

  That's fine. I can do more debugging.

  But I have hard time to understand this line of code:

  200 if ($Connected{$Idx} and (!$needping or
 eval{$Connected{$Idx}-ping})) {

  Should this be:

  200 if ($Connected{$Idx} and ($needping and
 eval{$Connected{$Idx}-ping})) {

  What's the difference between the two? Why the code in favor of or
 instead of and?

  - xinhuan

   From: Perrin Harkins phark...@gmail.com
 Date: Thursday, November 7, 2013 3:05 PM

 To: Xinhuan Zheng xzh...@christianbook.com
 Cc: Adam Prime adam.pr...@utoronto.ca, modperl@perl.apache.org 
 modperl@perl.apache.org
 Subject: Re: Apache::DBI connect

   Sorry, I can't determine the problem from your log.  You'll need to
 either run it in the debugger or add some debugging print statements to
 figure out where it's having trouble.  All I can say from that output is
 that it it's not succeeding in making a new connection after the ping
 fails, because it doesn't say new connect... after that.  I also wonder
 where that disconnect() call is coming from.

  - Perrin


 On Thu, Nov 7, 2013 at 1:07 PM, Xinhuan Zheng xzh...@christianbook.comwrote:

  I only looked at Apache::DBI not DBI document.

  My test program works. It reconnects to database OK. I ran it multiple
 times and every time it reconnects OK. But Apache::DBI doesn't work. You
 saw the previous debugging info. Where is the problem?

  - xinhuan

   From: Perrin Harkins phark...@gmail.com
 Date: Thursday, November 7, 2013 1:00 PM
 To: Xinhuan Zheng xzh...@christianbook.com
 Cc: Adam Prime adam.pr...@utoronto.ca, modperl@perl.apache.org 
 modperl@perl.apache.org

 Subject: Re: Apache::DBI connect

   It is in the DBI documentation.  Search for 0E0.

  - Perrin


 On Thu, Nov 7, 2013 at 12:41 PM, Xinhuan Zheng 
 xzh...@christianbook.comwrote:

 So it returns string '0E0'? The document didn't say that.

 - xinhuan

 On 11/7/13 11:44 AM, Adam Prime adam.pr...@utoronto.ca wrote:

 perl -e if ('0E0') { print qq[hi\n] }
 hi
 
 OE0 as a string evaluates to true.  If you use it as a bareword /
 numeric then it's false, which is what your eval example below is doing.
 
 Adam
 
 
 On 13-11-07 11:29 AM, Xinhuan Zheng wrote:
  one correct - In both cases, the return value is evaluated to false.
 
  How do you distinguish?
 
  - xinhuan
 
  From: Xinhuan Zheng xzh...@christianbook.com
  mailto:xzh...@christianbook.com
  Date: Thursday, November 7, 2013 11:12 AM
  To: Perrin Harkins phark...@gmail.com mailto:phark...@gmail.com
  Cc: mod_perl list modperl@perl.apache.org
 mailto:modperl@perl.apache.org
  Subject: Re: Apache::DBI connect
 
  I don't actually understand why you did that.  What was wrong with
 the
 normal ping?
 
  With Oracle DRCP, even though ping succeeds, the connection to the
  server process is actually terminated. Or ora_ping() may return 0E0
  zero but true and undef. I don't know. ora_ping() is foreign to me.
 I
  made the change based on what Apache::DBI the document said.
 
  In any case, there's no need to change the Apache::DBI code, even
 with
 your select 1 from dual test.   It returns a true value (0E0) if it
 succeeds and a false value (undef) if it fails.
 
  In both cases, the return value is evaluated to true:
 
  200: if ($Connected{$Idx} and (!$needping or
 eval{$Connected{$Idx}-ping}))
 
  eval{0E0} and eval{undef} both return true. I did test that. How do
 you
  distinguish?
 
  Did your script succeed in reconnecting after it lost the connection?
 
  Yes.
 
  Yes, I haven't forgotten about that, but I haven't had time to work
 on
 it yet.  You can try fixing it yourself by looking at the code in
 Apache::DBI that checks if the server is starting under apache 1.x.
 Otherwise, I will eventually get to it.
 
  I don't understand that piece of code. I can't do the change. Hope you
  can help.
 
  - xinhuan
 
  From: Perrin Harkins phark...@gmail.com mailto:phark...@gmail.com
  Date: Thursday, November 7, 2013 11:00 AM
  To: Xinhuan Zheng xzh...@christianbook.com
  mailto:xzh...@christianbook.com
  Cc: mod_perl list modperl@perl.apache.org
 mailto:modperl@perl.apache.org
  Subject: Re: Apache::DBI connect
 
  On Thu, Nov 7, 2013 at 9:46 AM, Xinhuan Zheng 
 xzh...@christianbook.com
  mailto:xzh...@christianbook.com wrote:
 The $ok is undef. In the case if the test does succeed (like the first
 select), $ok returns 0E0.
 
  That all sounds good.  0E0 is a true value in Perl.  It means zero
 but
  true.  And undef is a false value.  You don't need to test for undef.
 
  Since I changed DBD::Oracle subroutine ping to use 'select 1 from
 dual',
 
  I don't actually understand why you did that.  What was wrong with the
  normal ping?
 
  In any case, there's no need to change the Apache::DBI code, even

Re: Apache::DBI connect

[Perrin Harkins]

On Wed, Nov 6, 2013 at 4:07 PM, Xinhuan Zheng xzh...@christianbook.comwrote:

 I wonder this line of code should be changed to 'DBI-connect(@args) if
 ($@)'. If ping failed, that means the connection is already closed, $drh
 may be no longer valid, will $drh-connect always return a valid new
 connection? What's the difference between DBI-connect vs $drh-connect?


The $drh there is a DBI driver, not a database handle.  It should return a
new connection every time.  You can try putting in DBI-connect() there and
see if it fixes the problem for you though.  If not, I'd suggest reporting
the problem as a DBD::Oracle issue.

By the way, which Oracle pooling solution are you using and how is it
working out for you, aside from this problem?

- Perrin

Re: mysql connection pooling

[Perrin Harkins]

Hi,

Are you already using a reverse proxy in front of mod_perl?  That should be
your first step, and solves this for most people.

The next thing I'd recommend is looking at Oracle's connection pooling
solutions.  I haven't used Oracle recently, but I remember that they had
multiple ways to do pooling.  If they have something you can use, that's
probably more efficient than doing it at the Perl level and wouldn't
require code changes.

Finally, you could use the DBD::Gofer approach described in those slides.
 If you read the DBD::Gofer documentation you'll see that the HTTP proxying
is handled by a separate distribution here:
http://search.cpan.org/dist/GoferTransport-http/

- Perrin


On Thu, Oct 24, 2013 at 3:44 PM, Xinhuan Zheng xzh...@christianbook.comwrote:

  Hi Perrin,

  I am facing a similar issue with Oracle database. Tim's slides doesn’t
 explain details of implementation. I have Apache httpd with mod_perl and
 DBI. Without many code changes, what does the proxy server look like to
 achieve the database connection pooling? What is Apache running DBI Gofer
 as shown in the slides page 26?

  Thanks,
 - xinhuan

   From: Perrin Harkins phark...@gmail.com
 Date: Thursday, June 27, 2013 3:05 PM
 To: xiaolan practicalp...@gmail.com
 Cc: modperl@perl.apache.org modperl@perl.apache.org
 Subject: Re: mysql connection pooling

   Hi,

  There is nothing exactly like the database pool in JDBC.  However, there
 are solutions for this problem.

  The first thing you should do is run a reverse proxy in front of your
 mod_perl server.  That typically reduces the number of mod_perl processes
 by a factor of 10, i.e. 1000 mod_perl processes for 10,000 front-end proxy
 processes.  See
 http://perl.apache.org/docs/1.0/guide/strategy.html#Adding_a_Proxy_Server_in_http_Accelerator_Mode

  Note that there are many lightweight proxy servers now that can act as a
 front-end for mod_perl, not just Apache httpd.

  That's definitely what you should do first, but if you've done that and
 it's still not enough, you can also use DBD::Gofer.  There is an
 explanation here:
 http://search.cpan.org/~timb/DBI-1.627/lib/DBD/Gofer.pm#Connection_Pooling_and_Throttling

  Tim's slides explaining how this was used in a real-world scenario are
 here:
 http://www.slideshare.net/Tim.Bunce/dbdgofer-200809

  - Perrin


 On Thu, Jun 27, 2013 at 2:26 AM, xiaolan practicalp...@gmail.com wrote:

 Hi,

  Currently I have been using Apache::DBI for long connections to Mysql.
 But having the problem that, if the apache processes are 1, they open
 1 connections to mysql, this make the DB crashed.
 Is there any software for modperl working like Java's JDBC for connection
 pooling?

  Thanks.

Re: Qu: installing mod_perl2 outside of apache2 and perl install directories

[Perrin Harkins]

I suspect there's a way to fix the issue, but my general advice is this:
don't be a slave to running the tests if you already have some way to know
if the module is working.  It's fine to force install rather than spend
hours fighting with a test setup.

- Perrin


On Tue, Sep 10, 2013 at 8:11 PM, Thomas M. Payerle paye...@umd.edu wrote:

 Hi,

 Our site uses networked file systems and the same code base is available
 on many systems.
 We have multiple versions of e.g. apache and perl installed, and e.g.
 init.d scripts will
 look up the version of apache to use for a particular server and start
 that version, out
 of the several installed on the system.  For perl, users/scripts would
 specify the version
 of perl they wanted.

 For the most part, this works well.  Apache modules e.g. would go into the
 apache build
 tree for that particular apache.  Same with perl modules.

 Except mod_perl depends on both apache and perl version, and wants to
 stick stuff in
 both apache and perl trees.  Which means how do I make sure I get the
 mod_perl built for
 the particular perl/apache combination I am using?

 I thought I had figured this out; put mod_perl in its own tree, with
 subdirectories for
 mod_perl version, apache version, and perl version.  During the mod_perl
 install, in
 the perl Makefile.PL phase, I specified PREFIX to where I wanted the
 perl bits to go,
 and used  MP_AP_DESTDIR to specify where I wanted the apache bits to go.

 To use it, I had to specify the full path to mod_perl.so, and use a
 PerlSwitches configuration
 directive so that the mod_perl related *.pm files, etc. would be found,
 but otherwise
 everything seemed happy.  At least some simple test scripts using
 ModPerl::Registry and a
 simple handler module of my own, and they worked.

 Then I tried installing Apache2::Request.  OK, after a little work I
 managed to get it to
 find the apache2 - mod_perl header files from my non-standard location,
 but tests were
 still failing.  It looks like Apache::TestConfig is not finding my
 mod_perl.so, and so
 the test httpd configs don't get mod_perl, which understandably causes
 things to fail.  So
 it looks like I need some more hacks to the mod_perl code.  It looked like
 I might be
 able to add a path to find_apache_module, which is fine if that is more or
 less all it is.
 But I am also concerned that I might be headed very quickly into some very
 deep stuff, and
 as I know relatively little re the Apache::Test* suite, was wondering if
 anyone had
 any advice or suggestions?

 Thanks in advance for any assistance.




 Tom Payerle
 IT-ETI-EUS  paye...@umd.edu
 University of Maryland  (301) 405-6135
 College Park, MD 20742-4111

Re: best way to copy a innodb table

[Perrin Harkins]

On Thu, Dec 1, 2011 at 5:54 PM, Miguel Angel Nieto miguel.ni...@percona.com
 wrote:

 You should check pt-archiver.


+1.  It works very well for this type of job.

- Perrin

Re: httpd24 branch now works with both httpd-2.2 and httpd-2.4

[Perrin Harkins]

That's great.  Thank you, Jan.

- Perrin


On Thu, Jun 27, 2013 at 6:55 AM, Jan Kaluža jkal...@redhat.com wrote:

 Hi,

 I have done some work on httpd24 branch today.

 Already done:
 - xs/tables are now divided into two separate directories:
 xs/tables/current and xs/tables/current24 (2.2 vs. 2.4)
 - xs_generate generates output to the proper directory according to httpd
 version.
 - everything should compile fine with both httpd versions
 - tests should work properly with both versions. there are currently some
 failing tests, but they fail even in trunk for me.

 What's missing (TODO):
 - xs/maps are still httpd-2.4 only, so if you try to run make
 xs_generate with httpd-2.2, it will break.

 We have to use #ifdefs in the xs/maps and merge xs/maps from trunk with
 xs/maps from httpd24 branch. I will do that, but probably not this week.

 I think it could be ready for merge after some more testing and when the
 things on TODO list above gets fixed.

 Regards,
 Jan Kaluza

 --**--**-
 To unsubscribe, e-mail: 
 dev-unsubscribe@perl.apache.**orgdev-unsubscr...@perl.apache.org
 For additional commands, e-mail: dev-h...@perl.apache.org

Re: mysql connection pooling

[Perrin Harkins]

Hi,

There is nothing exactly like the database pool in JDBC.  However, there
are solutions for this problem.

The first thing you should do is run a reverse proxy in front of your
mod_perl server.  That typically reduces the number of mod_perl processes
by a factor of 10, i.e. 1000 mod_perl processes for 10,000 front-end proxy
processes.  See
http://perl.apache.org/docs/1.0/guide/strategy.html#Adding_a_Proxy_Server_in_http_Accelerator_Mode

Note that there are many lightweight proxy servers now that can act as a
front-end for mod_perl, not just Apache httpd.

That's definitely what you should do first, but if you've done that and
it's still not enough, you can also use DBD::Gofer.  There is an
explanation here:
http://search.cpan.org/~timb/DBI-1.627/lib/DBD/Gofer.pm#Connection_Pooling_and_Throttling

Tim's slides explaining how this was used in a real-world scenario are here:
http://www.slideshare.net/Tim.Bunce/dbdgofer-200809

- Perrin


On Thu, Jun 27, 2013 at 2:26 AM, xiaolan practicalp...@gmail.com wrote:

 Hi,

 Currently I have been using Apache::DBI for long connections to Mysql.
 But having the problem that, if the apache processes are 1, they open
 1 connections to mysql, this make the DB crashed.
 Is there any software for modperl working like Java's JDBC for connection
 pooling?

 Thanks.

Re: Apache::DBI connection lost contact error

[Perrin Harkins]

That's unfortunate.  Are you using it the same way, with a DBI connection
in a Perl block?  I haven't seen anyone else mention a problem with it in
Apache 1.

- Perrin


On Wed, Jun 12, 2013 at 5:05 PM, Xinhuan Zheng xzh...@christianbook.comwrote:

  Hi Perrin,

  Today I tested the Apache-DBI 1.12 with Apache 1.3.42 and mod_perl 1. We
 have to use mod_perl 1 in some situations. There is same issue when
 startup.pl has created database handle. This database handle gets cached
 and forked across child processes. With debugging turned on, there isn't
 skipping connection during server startup, read the docu !! message at
 all. It appears this new version only works with Apache2 but not Apache1.
 Is that possible to fix it with Apache1?

  Thanks,
 - xinhuan

   From: Perrin Harkins phark...@gmail.com
 Date: Wednesday, June 12, 2013 5:53 AM
 To: Fred Moyer f...@redhotpenguin.com
 Cc: Xinhuan Zheng xzh...@christianbook.com, modperl@perl.apache.org 
 modperl@perl.apache.org

 Subject: Re: Apache::DBI connection lost contact error

   Thanks Fred!


 On Wed, Jun 12, 2013 at 3:29 AM, Fred Moyer f...@redhotpenguin.comwrote:

 Apache-DBI 1.12 was just pushed to CPAN with this update. Thanks for
 the great work on the fix Perrin.

 On Thu, Jun 6, 2013 at 2:53 PM, Perrin Harkins phark...@gmail.com
 wrote:
  That's great!  I'll commit the patch and see about getting a new
 release out
  to CPAN.
 
  - Perrin
 
 
  On Thu, Jun 6, 2013 at 5:02 PM, Xinhuan Zheng xzh...@christianbook.com
 
  wrote:
 
  Hi Perrin,
 
  I did a testing with debugging. I don't see the connection lost
 contact
  error anymore. The patch looks good to me.
 
  Thanks,
  - xinhuan
 
  From: Perrin Harkins phark...@gmail.com
  Date: Thursday, June 6, 2013 3:02 PM
  To: Xinhuan Zheng xzh...@christianbook.com
  Cc: modperl@perl.apache.org modperl@perl.apache.org
 
  Subject: Re: Apache::DBI connection lost contact error
 
  On Thu, Jun 6, 2013 at 12:22 PM, Xinhuan Zheng 
 xzh...@christianbook.com
  wrote:
   The database handle that is created in startup.pl needs to be really
   disconnected (not overloaded disconnect) so that won't leave an idle
 server
   process running on the database side. Once it's really disconnected,
 the
   server process can be cleaned up on the server side.
 
  Right, that's what the bug is preventing.
 
  Because my flight was delayed last night, I had time to make a patch.
  Please try this on your system and send the debug, like you did before.
 
  - Perrin
 
 

Re: Apache::DBI connection lost contact error

[Perrin Harkins]

Thanks Fred!


On Wed, Jun 12, 2013 at 3:29 AM, Fred Moyer f...@redhotpenguin.com wrote:

 Apache-DBI 1.12 was just pushed to CPAN with this update. Thanks for
 the great work on the fix Perrin.

 On Thu, Jun 6, 2013 at 2:53 PM, Perrin Harkins phark...@gmail.com wrote:
  That's great!  I'll commit the patch and see about getting a new release
 out
  to CPAN.
 
  - Perrin
 
 
  On Thu, Jun 6, 2013 at 5:02 PM, Xinhuan Zheng xzh...@christianbook.com
  wrote:
 
  Hi Perrin,
 
  I did a testing with debugging. I don't see the connection lost
 contact
  error anymore. The patch looks good to me.
 
  Thanks,
  - xinhuan
 
  From: Perrin Harkins phark...@gmail.com
  Date: Thursday, June 6, 2013 3:02 PM
  To: Xinhuan Zheng xzh...@christianbook.com
  Cc: modperl@perl.apache.org modperl@perl.apache.org
 
  Subject: Re: Apache::DBI connection lost contact error
 
  On Thu, Jun 6, 2013 at 12:22 PM, Xinhuan Zheng 
 xzh...@christianbook.com
  wrote:
   The database handle that is created in startup.pl needs to be really
   disconnected (not overloaded disconnect) so that won't leave an idle
 server
   process running on the database side. Once it's really disconnected,
 the
   server process can be cleaned up on the server side.
 
  Right, that's what the bug is preventing.
 
  Because my flight was delayed last night, I had time to make a patch.
  Please try this on your system and send the debug, like you did before.
 
  - Perrin
 
 

Re: svn commit: r1490648 - in /perl/Apache-DBI/trunk: Changes lib/Apache/DBI.pm

[Perrin Harkins]

Thanks!  Do you know what needs to happen to roll a new release of
Apache::DBI?  It's a separate CPAN dist from mod_perl.

- Perrin


On Fri, Jun 7, 2013 at 2:17 PM, Fred Moyer f...@redhotpenguin.com wrote:

 Perrin++
 On Jun 7, 2013 6:46 AM, per...@apache.org wrote:

 Author: perrin
 Date: Fri Jun  7 13:46:30 2013
 New Revision: 1490648

 URL: http://svn.apache.org/r1490648
 Log:
 Update changes and version

 Modified:
 perl/Apache-DBI/trunk/Changes
 perl/Apache-DBI/trunk/lib/Apache/DBI.pm

 Modified: perl/Apache-DBI/trunk/Changes
 URL:
 http://svn.apache.org/viewvc/perl/Apache-DBI/trunk/Changes?rev=1490648r1=1490647r2=1490648view=diff

 ==
 --- perl/Apache-DBI/trunk/Changes (original)
 +++ perl/Apache-DBI/trunk/Changes Fri Jun  7 13:46:30 2013
 @@ -1,6 +1,10 @@
  Revision history for ApacheDBI.

 -1.12-dev
 +1.12
 +
 +  - Fix detection of server startup in mod_perl 2, to avoid caching
 +connections in the parent process. Perrin Harkins
 +per...@elem.com

  1.11 October 7, 2011


 Modified: perl/Apache-DBI/trunk/lib/Apache/DBI.pm
 URL:
 http://svn.apache.org/viewvc/perl/Apache-DBI/trunk/lib/Apache/DBI.pm?rev=1490648r1=1490647r2=1490648view=diff

 ==
 --- perl/Apache-DBI/trunk/lib/Apache/DBI.pm (original)
 +++ perl/Apache-DBI/trunk/lib/Apache/DBI.pm Fri Jun  7 13:46:30 2013
 @@ -11,6 +11,7 @@ BEGIN {
  require Apache2::Module;
  require Apache2::RequestUtil;
  require Apache2::ServerUtil;
 +require ModPerl::Util;
  }
  elsif (defined $modperl::VERSION  $modperl::VERSION  1 
   $modperl::VERSION  1.99) {
 @@ -22,7 +23,7 @@ use Carp ();

  require_version DBI 1.00;

 -$Apache::DBI::VERSION = '1.12-dev';
 +$Apache::DBI::VERSION = '1.12';

  # 1: report about new connect
  # 2: full debug output

Re: svn commit: r1490648 - in /perl/Apache-DBI/trunk: Changes lib/Apache/DBI.pm

[Perrin Harkins]

I'd appreciate any help in getting things started.

- Perrin


On Fri, Jun 7, 2013 at 3:08 PM, Fred Moyer f...@redhotpenguin.com wrote:

 Yes, I can start the wheels rolling there unless you would rather take
 the helm on this one. Most of it is in the RELEASE document.

 On Fri, Jun 7, 2013 at 11:54 AM, Perrin Harkins per...@elem.com wrote:
  Thanks!  Do you know what needs to happen to roll a new release of
  Apache::DBI?  It's a separate CPAN dist from mod_perl.
 
  - Perrin
 
 
  On Fri, Jun 7, 2013 at 2:17 PM, Fred Moyer f...@redhotpenguin.com
 wrote:
 
  Perrin++
 
  On Jun 7, 2013 6:46 AM, per...@apache.org wrote:
 
  Author: perrin
  Date: Fri Jun  7 13:46:30 2013
  New Revision: 1490648
 
  URL: http://svn.apache.org/r1490648
  Log:
  Update changes and version
 
  Modified:
  perl/Apache-DBI/trunk/Changes
  perl/Apache-DBI/trunk/lib/Apache/DBI.pm
 
  Modified: perl/Apache-DBI/trunk/Changes
  URL:
 
 http://svn.apache.org/viewvc/perl/Apache-DBI/trunk/Changes?rev=1490648r1=1490647r2=1490648view=diff
 
 
 ==
  --- perl/Apache-DBI/trunk/Changes (original)
  +++ perl/Apache-DBI/trunk/Changes Fri Jun  7 13:46:30 2013
  @@ -1,6 +1,10 @@
   Revision history for ApacheDBI.
 
  -1.12-dev
  +1.12
  +
  +  - Fix detection of server startup in mod_perl 2, to avoid caching
  +connections in the parent process. Perrin Harkins
  +per...@elem.com
 
   1.11 October 7, 2011
 
 
  Modified: perl/Apache-DBI/trunk/lib/Apache/DBI.pm
  URL:
 
 http://svn.apache.org/viewvc/perl/Apache-DBI/trunk/lib/Apache/DBI.pm?rev=1490648r1=1490647r2=1490648view=diff
 
 
 ==
  --- perl/Apache-DBI/trunk/lib/Apache/DBI.pm (original)
  +++ perl/Apache-DBI/trunk/lib/Apache/DBI.pm Fri Jun  7 13:46:30 2013
  @@ -11,6 +11,7 @@ BEGIN {
   require Apache2::Module;
   require Apache2::RequestUtil;
   require Apache2::ServerUtil;
  +require ModPerl::Util;
   }
   elsif (defined $modperl::VERSION  $modperl::VERSION  1 
$modperl::VERSION  1.99) {
  @@ -22,7 +23,7 @@ use Carp ();
 
   require_version DBI 1.00;
 
  -$Apache::DBI::VERSION = '1.12-dev';
  +$Apache::DBI::VERSION = '1.12';
 
   # 1: report about new connect
   # 2: full debug output
 
 
 

Re: Apache::DBI connection lost contact error

[Perrin Harkins]

On Thu, Jun 6, 2013 at 12:22 PM, Xinhuan Zheng xzh...@christianbook.com
wrote:
 The database handle that is created in startup.pl needs to be really
disconnected (not overloaded disconnect) so that won't leave an idle server
 process running on the database side. Once it's really disconnected, the
server process can be cleaned up on the server side.

Right, that's what the bug is preventing.

Because my flight was delayed last night, I had time to make a patch.
 Please try this on your system and send the debug, like you did before.

- Perrin


apache-dbi.patch
Description: Binary data

Re: Apache::DBI connection lost contact error

[Perrin Harkins]

That's great!  I'll commit the patch and see about getting a new release
out to CPAN.

- Perrin


On Thu, Jun 6, 2013 at 5:02 PM, Xinhuan Zheng xzh...@christianbook.comwrote:

  Hi Perrin,

  I did a testing with debugging. I don't see the connection lost
 contact error anymore. The patch looks good to me.

  Thanks,
 - xinhuan

   From: Perrin Harkins phark...@gmail.com
 Date: Thursday, June 6, 2013 3:02 PM
 To: Xinhuan Zheng xzh...@christianbook.com
 Cc: modperl@perl.apache.org modperl@perl.apache.org

 Subject: Re: Apache::DBI connection lost contact error

   On Thu, Jun 6, 2013 at 12:22 PM, Xinhuan Zheng xzh...@christianbook.com
 wrote:
  The database handle that is created in startup.pl needs to be really
 disconnected (not overloaded disconnect) so that won't leave an idle server
  process running on the database side. Once it's really disconnected, the
 server process can be cleaned up on the server side.

  Right, that's what the bug is preventing.

  Because my flight was delayed last night, I had time to make a patch.
  Please try this on your system and send the debug, like you did before.

  - Perrin

Re: Apache::DBI connection lost contact error

[Perrin Harkins]

On Tue, Jun 4, 2013 at 2:32 PM, Xinhuan Zheng xzh...@christianbook.com
wrote:
 I have seen in other web servers when apachectl starts, there is no
connect lost contact error but it did happen when apachectl stops.

If your server does not receive constant requests, you may see this error
on shutdown because cached database handles are being destroyed but the
connection to the database has already timed out.

 If child process uses ping the inherited database handle and it is
ping-able, then the child process may use it. That'll be a problem.

The debugging log you showed us makes me think that won't happen.  It looks
like the handle fails immediately when used after a fork.

 When child process exits, it calls rollback if AutoCommit is off, but
when another new child process is just born and uses its connection to send
 database updates, will that rollback interfere with new child process
attempt to update database?

If the same connection was being used in multiple processes, many things
could go wrong, including that.  However, Since the connections die
immediately when you use them after a fork, this won't happen.

 Is the rollback necessary?

It's very necessary.  Otherwise, if your code throws an uncaught exception
in the middle of some database work, the next request would reuse the same
handle with uncommitted changes left on it.  In my opinion, this is the
most important thing about Apache::DBI.

I understand you're worried about it.  I should be able to send you a patch
to try within a few days, so you can wait for that if you want.  Or you can
try to fix it yourself.  My plan is to replace that line of code I showed
with some code that checks current_callback() to make sure we're not in a
startup phase.

- Perrin

Re: Apache::DBI connection lost contact error

[Perrin Harkins]

On Mon, Jun 3, 2013 at 4:36 PM, Dave Morgan dave.mor...@100.com wrote:
 As an administrator I still rely and depend on Apache::DBI, even if it is
unsupported.

Can we kill this rumor please?  Apache::DBI is supported.

- Perrin

Re: Apache::DBI connection lost contact error

[Perrin Harkins]

On Tue, Jun 4, 2013 at 9:59 AM, Xinhuan Zheng xzh...@christianbook.com
wrote:
 Will the new connection that's created be cached after then and the
connection that's not ping-able will be discarded?

Yes.

 Will this cached the new connection last until the child process exit?

Yes, unless it times out on the Oracle side due to inactivity.

 Will rollback be called when child process discovers the connection
handle not ping-able?

No, the connection is already closed at that point.

 Will rollback be called when child process exit?

Yes.

Also, I think I see how to fix this, so when I have a few minutes I'll send
you a patch to try.

- Perrin

Apache::DBI connection lost contact error

[Perrin Harkins]

On Monday, June 3, 2013, Xinhuan Zheng wrote:
 What is the check to see if the server is restarting? Is that new child
processes spawning?

It's a test for whether or not we're running in the parent process, used to
skip caching connections during startup. It's this, line 128:
if (Apache2::ServerUtil::restart_count() == 1) {

 If there is a bug in how it checks to see if the server is restarting,
does it affect putting it into production use?

It looks to me like there's no danger here, just an annoying error message.
 Your connection from the parent process is being passed across the fork,
but then when each child process tries to check it with ping() it fails and
a new connection is opened to replace it.

I'll investigate the possible bug further when I get back from the
conference I'm at right now (YAPC::NA in Austin).

- Perrin

Re: Apache::DBI connection lost contact error

[Perrin Harkins]

On Fri, May 31, 2013 at 5:45 PM, Xinhuan Zheng xzh...@christianbook.com
wrote:
 2520 Apache::DBI skipping connection during server startup,
read the docu !!
 2520 Apache::DBI skipping connection during server startup,
read the docu !!

That's good.

 2521 Apache::DBI push PerlChildExitHandler
 2521 Apache::DBI need ping: yes
 2521 Apache::DBI new connect to '….'
 2521 Apache::DBI disconnect (overloaded)
 2521 Apache::DBI need ping: yes
 2521 Apache::DBI already connected to '….'
 2521 Apache::DBI disconnect (overloaded)

That's not good.  If this is still part of the startup process, that
shouldn't be caching the database handle.

If you don't use Apache::DBI, do you still see these Oracle errors?

 Are children using same Database Handle?

It looks like that first handle is being copied across the fork from the
parent process.  They don't really use it, because it fails the ping test
and they just open new handles.  It should not cause any real problems,
just the error in your log.

 Does Apache::DBI work right with Apache2+mod_perl2?

Yes, but there may be a bug in how it checks to see if the server is
restarting.

- Perrin

Re: Apache::DBI connection lost contact error

[Perrin Harkins]

Try an explicit disconnect() call.

- Perrin


On Thu, May 30, 2013 at 7:46 PM, Xinhuan Zheng xzh...@christianbook.comwrote:

 The db handle is declared local and once it's out of scope, the destroy
 call will disconnect. But it appears even though variable is out of scope,
 we still get that error. Don't know why.
 - xinhuan

 On 5/30/13 8:31 AM, Jim Schueler jschue...@eloquency.com wrote:

 Did this solve your problem?
 
   -Jim
 
 On Wed, 29 May 2013, Perrin Harkins wrote:
 
  Hi,
  Apache::DBI is supposed to skip caching if you connect during startup.
 You
  should just need to disconnect your database handle after you finish
 with
  it.  It sounds like you're opening it and then leaving it open.
 
  - Perrin
 
 
  On Wed, May 29, 2013 at 3:24 PM, Xinhuan Zheng
 xzh...@christianbook.com
  wrote:
Hi,
 
  I have apache 2.2.23 statically compiled with mod_perl2 (prefork).
  perl binary is 5.10.1. In startup.pl file there is call
  Apache::DBI-connect_on_init.
 
  code
  use Apache::DBI;
  Apache::DBI-connect_on_init( $DB_DRIVER, $DB_USER, $DB_PASSWORD );
 
  use DBI;
  /code
 
  I need to call DBI-connect to load some data during server startup
  stage. There is problem with this setup. Whenever apachectl
  startup/shutdown, we got connection error like this:
 
  DBD::Oracle::db DESTROY failed: ORA-03135: connection lost contact
  Process ID: 0
  Session ID: 3252 Serial number: 15131 (DBD ERROR: OCISessionEnd) at
  /usr/local/lib/perl5/site_perl/5.10.1/Apache/DBI.pm line 228.
 
  I am trying to fix this error. I think it's related to DBI-connect
  in startup.pl. My question is:
   1. How do I accomplish loading data into database during server
  startup using Apache::DBI?
   2. Once data is loaded during server startup, how do I safely destroy
  this database handle but not affect the children instantiate their
  database handles?
  Thanks in advance,
 
  Xinhuan
 
 
 

Re: Apache::DBI connection lost contact error

[Perrin Harkins]

Well, you are on the modperl list, so that means you. :)

Xinhuan, the error is harmless, but if you're concerned about it I would
try turning on debugging to make sure the connection is not being cached.
 Do this:
$Apache::DBI::DEBUG = 2;

And then watch for a message like this in your log:
skipping connection during server startup

- Perrin


On Fri, May 31, 2013 at 10:33 AM, Jim Schueler jschue...@eloquency.comwrote:

 I'm afraid I'm out of my league.  I just noticed the following comment on
 the Apache::DBI man page:

   Edmund Mergl was the original author of Apache::DBI. It is now supported
   and maintained by the modperl mailinglist, see the mod_perl documentation
   for instructions on how to subscribe.

  -Jim


 On Fri, 31 May 2013, Xinhuan Zheng wrote:

  I believe I am using my declaration rather than local. I also tried
 explicitly disconnect but still have same issue. Since it only happens in
 parent/child processes, I don't know a good way to debug parent/child, nor
 reproducing the same error using a simple program. Can you guys help me
 with that?

 Thanks,
 - xinhuan

 On 5/31/13 9:02 AM, Jim Schueler jschue...@eloquency.com wrote:

  Perrin is right.  But fundamentally, I'd say that you're confusing
 'local' and 'my' variable scoping:


 http://www.perlmonks.org/?**node_id=94007http://www.perlmonks.org/?node_id=94007

  -Jim

 On Fri, 31 May 2013, Perrin Harkins wrote:

  Try an explicit disconnect() call.
 - Perrin


 On Thu, May 30, 2013 at 7:46 PM, Xinhuan Zheng
 xzh...@christianbook.com
 wrote:
   The db handle is declared local and once it's out of scope, the
   destroy
   call will disconnect. But it appears even though variable is out
   of scope,
   we still get that error. Don't know why.
   - xinhuan

   On 5/30/13 8:31 AM, Jim Schueler jschue...@eloquency.com
   wrote:

  Did this solve your problem?
  
-Jim
  
  On Wed, 29 May 2013, Perrin Harkins wrote:
  
   Hi,
   Apache::DBI is supposed to skip caching if you connect during
   startup.
  You
   should just need to disconnect your database handle after you
   finish
  with
   it.  It sounds like you're opening it and then leaving it
   open.
  
   - Perrin
  
  
   On Wed, May 29, 2013 at 3:24 PM, Xinhuan Zheng
  xzh...@christianbook.com
   wrote:
 Hi,
  
   I have apache 2.2.23 statically compiled with mod_perl2
   (prefork).
   perl binary is 5.10.1. In startup.pl file there is call
   Apache::DBI-connect_on_init.
  
   code
   use Apache::DBI;
   Apache::DBI-connect_on_init( $DB_DRIVER, $DB_USER,
   $DB_PASSWORD );
  
   use DBI;
   /code
  
   I need to call DBI-connect to load some data during server
   startup
   stage. There is problem with this setup. Whenever apachectl
   startup/shutdown, we got connection error like this:
  
   DBD::Oracle::db DESTROY failed: ORA-03135: connection lost
   contact
   Process ID: 0
   Session ID: 3252 Serial number: 15131 (DBD ERROR:
   OCISessionEnd) at
   /usr/local/lib/perl5/site_**perl/5.10.1/Apache/DBI.pm line 228.
  
   I am trying to fix this error. I think it's related to
   DBI-connect
   in startup.pl. My question is:
1. How do I accomplish loading data into database during
   server
   startup using Apache::DBI?
2. Once data is loaded during server startup, how do I
   safely destroy
   this database handle but not affect the children
   instantiate their
   database handles?
   Thanks in advance,
  
   Xinhuan
  
  
  

Re: Apache::DBI

[Perrin Harkins]

Hi Jim,

I appreciate the thought, but I'm not the mod_perl list.  If you look at
who has done the most support around here recently, it's probably Torsten.
 (Thanks Torsten!)  More to the point, there are many people on the list
who know enough perl to help with a question about Apache::DBI.  It's a
common practice to point people here for support on mod_perl modules.

What are you getting at?  Is there a module that you're having trouble with
and can't get support for?

- Perrin


On Fri, May 31, 2013 at 10:56 AM, Jim Schueler jschue...@eloquency.comwrote:

 There's an existing thread with an Apache::DBI question.  But since I want
 to post a separate question to this list, I decided to start a new thread.

 Just got done reading the Man page for Apache::DBI.  One of the last notes
 suggests that this package is obsolete (having been replaced by Class::DBI
 or DBIx::CLASS).  Beyond that is the following:

   Edmund Mergl was the original author of Apache::DBI. It is now supported
   and maintained by the modperl mailinglist, see the mod_perl documentation
   for instructions on how to subscribe.

 Unless Perrin Harkins agreed to take over support for this module, then
 that statement is not true.  Otherwise, out of respect for Perrin, I'll try
 to be general.

 (Aside:  Am I the only developer that comes across 'unless () {} else {}'
 constructions?)

 It seems very few distros on CPAN are actually supported.  For my part, I
 still monitor this list to support my own contributions from *many* years
 ago.  And I k

Re: Apache::DBI

[Perrin Harkins]

The mailing list has been the official place for support of all bundled
mod_perl modules for as long as I can remember.  I don't think there's a
rule about perl core modules being passed between individuals either,
although I could be wrong.  Sending people to a mailing list for support is
a common practice with any widely used module, e.g. DBI.

Apache::DBI is obsolete in the sense that most people are using an ORM
framework that handles database persistence for them, so they have no use
for Apache::DBI.  It's not broken, and people should feel free to use it if
it fits their use case.  It has certain advantages over
DBI-connect_cached() which have been discussed here many times.

I'm not sure why you're concluding that Apache::DBI is unsupported.  To me,
getting multiple responses on a mailing list is pretty much Platinum Level
Support for open source.  If no one stepping forward to say I own X, let
me debug it for you means that X is unsupported and obsolete, then I think
we're all in trouble.

Open source support has always been laissez-faire (unless you choose to
hire someone for it).  The only thing that has changed recently is that
with more viable web runtime options to choose from (PSGI, FastCGI's
comeback, etc.), the perl web dev world is more splintered and there are
fewer people on the mod_perl list than there used to be.  That's a little
sad for me to see, but the new stuff is pretty nice too, and lots of people
are still using mod_perl and answering questions on this list.

- Perrin


On Fri, May 31, 2013 at 11:50 AM, Jim Schueler jschue...@eloquency.comwrote:

 I still use Alpine.  And they never fixed the bug where ctrl-c (to cancel
 a message) and ctrl-x (to send) are so easily confused.  Oops.  Maybe it's
 time to start using a mouse.

 Having wasted so much time, I'll try to be succinct:

   Most modules on CPAN are bascially throwaways and not supported at all.
   Use them at your own risk.

   There are some modules that are just obsolete.  Good intentions aside,
   the developers lost interest and moved on.  These are less risky if
   there's an established user base.

   There are some very good modules, widely used, that are fully supported
   and perfectly safe for a production environment.

 Most mod_perl modules, especially the core modules, fall into that last,
 gold standard, category.  In many cases, support is transferred from one
 individual to another.  And so that commitment is documented.  But if a
 module is no longer supported, don't lie about it.  Support forums are an
 incredible resource.  But if commercial software developers similarly
 blurred this distinction, every p.o.s. would be advertising free 24x7 tech
 support.

 Apache::DBI seems like a #2 pretending to be a #3.  On the basis of your
 response, I've concluded that Apache::DBI is no longer supported and has
 been superceded by newer modules.  Especially if no one responds and
 explicitly accepts the responsibility, this seems like the most appropriate
 answer for the poster of the original thread.

 I owe you a :) from a couple posts ago.  :)

  -Jim


 On Fri, 31 May 2013, Perrin Harkins wrote:

  Hi Jim,
 I appreciate the thought, but I'm not the mod_perl list.  If you look at
 who
 has done the most support around here recently, it's probably Torsten.
  (Thanks Torsten!)  More to the point, there are many people on the list
 who
 know enough perl to help with a question about Apache::DBI.  It's a common
 practice to point people here for support on mod_perl modules.

 What are you getting at?  Is there a module that you're having trouble
 with
 and can't get support for?

 - Perrin


 On Fri, May 31, 2013 at 10:56 AM, Jim Schueler jschue...@eloquency.com
 wrote:
   There's an existing thread with an Apache::DBI question.  But
   since I want to post a separate question to this list, I decided
   to start a new thread.

   Just got done reading the Man page for Apache::DBI.  One of the
   last notes suggests that this package is obsolete (having been
   replaced by Class::DBI or DBIx::CLASS).  Beyond that is the
   following:

 Edmund Mergl was the original author of Apache::DBI. It is now
   supported
 and maintained by the modperl mailinglist, see the mod_perl
   documentation
 for instructions on how to subscribe.

   Unless Perrin Harkins agreed to take over support for this
   module, then that statement is not true.  Otherwise, out of
   respect for Perrin, I'll try to be general.

   (Aside:  Am I the only developer that comes across 'unless () {}
   else {}' constructions?)

   It seems very few distros on CPAN are actually supported.  For
   my part, I still monitor this list to support my own
   contributions from *many* years ago.  And I k

Re: [OT] Apache::DBI

[Perrin Harkins]

 on.  These are less risky if
   there's an established user base.

   There are some very good modules, widely used, that are fully
 supported
   and perfectly safe for a production environment.

 Most mod_perl modules, especially the core modules, fall into that
 last,
 gold standard, category.  In many cases, support is transferred from
 one
 individual to another.  And so that commitment is documented.  But if a
 module is no longer supported, don't lie about it.  Support forums are
 an
 incredible resource.  But if commercial software developers similarly
 blurred this distinction, every p.o.s. would be advertising free 24x7
 tech
 support.

 Apache::DBI seems like a #2 pretending to be a #3.  On the basis of
 your
 response, I've concluded that Apache::DBI is no longer supported and
 has
 been superceded by newer modules.  Especially if no one responds and
 explicitly accepts the responsibility, this seems like the most
 appropriate
 answer for the poster of the original thread.

 I owe you a :) from a couple posts ago.  :)

  -Jim

 On Fri, 31 May 2013, Perrin Harkins wrote:

  Hi Jim,
 I appreciate the thought, but I'm not the mod_perl list.  If you look
 at
 who
 has done the most support around here recently, it's probably Torsten.
  (Thanks Torsten!)  More to the point, there are many people on the
 list
 who
 know enough perl to help with a question about Apache::DBI.  It's a
 common
 practice to point people here for support on mod_perl modules.

 What are you getting at?  Is there a module that you're having trouble
 with
 and can't get support for?

 - Perrin


 On Fri, May 31, 2013 at 10:56 AM, Jim Schueler 
 jschue...@eloquency.com
 wrote:
   There's an existing thread with an Apache::DBI question.  But
   since I want to post a separate question to this list, I decided
   to start a new thread.

   Just got done reading the Man page for Apache::DBI.  One of the
   last notes suggests that this package is obsolete (having been
   replaced by Class::DBI or DBIx::CLASS).  Beyond that is the
   following:

 Edmund Mergl was the original author of Apache::DBI. It is now
   supported
 and maintained by the modperl mailinglist, see the mod_perl
   documentation
 for instructions on how to subscribe.

   Unless Perrin Harkins agreed to take over support for this
   module, then that statement is not true.  Otherwise, out of
   respect for Perrin, I'll try to be general.

   (Aside:  Am I the only developer that comes across 'unless () {}
   else {}' constructions?)

   It seems very few distros on CPAN are actually supported.  For
   my part, I still monitor this list to support my own
   contributions from *many* years ago.  And I k

Re: Apache::DBI connection lost contact error

[Perrin Harkins]

Hi,

Apache::DBI is supposed to skip caching if you connect during startup.  You
should just need to disconnect your database handle after you finish with
it.  It sounds like you're opening it and then leaving it open.

- Perrin


On Wed, May 29, 2013 at 3:24 PM, Xinhuan Zheng xzh...@christianbook.comwrote:

  Hi,

  I have apache 2.2.23 statically compiled with mod_perl2 (prefork). perl
 binary is 5.10.1. In startup.pl file there is call
 Apache::DBI-connect_on_init.

  code
  use Apache::DBI;
 Apache::DBI-connect_on_init( $DB_DRIVER, $DB_USER, $DB_PASSWORD );

  use DBI;
 /code

  I need to call DBI-connect to load some data during server startup
 stage. There is problem with this setup. Whenever apachectl
 startup/shutdown, we got connection error like this:

  DBD::Oracle::db DESTROY failed: ORA-03135: connection lost contact
 Process ID: 0
 Session ID: 3252 Serial number: 15131 (DBD ERROR: OCISessionEnd) at
 /usr/local/lib/perl5/site_perl/5.10.1/Apache/DBI.pm line 228.

  I am trying to fix this error. I think it's related to DBI-connect in
 startup.pl. My question is:

1. How do I accomplish loading data into database during server
startup using Apache::DBI?
2. Once data is loaded during server startup, how do I safely destroy
this database handle but not affect the children instantiate their database
handles?

 Thanks in advance,

  Xinhuan

Re: Time for 2.0.8?

[Perrin Harkins]

+1


On Mon, Mar 18, 2013 at 5:32 AM, Steve Hay steve@verosoftware.com wrote:
 The mod_perl trunk now contains a bunch of fixes for recent versions of
 perl, especially with respect to changes to rehashing. These changes
 will be necessary for mod_perl to build with the forthcoming perl
 5.18.0. That is due to be released on 18th May, and the perl repo is in
 full code freeze from Wednesday (20th March) so in theory no further
 changes will be forthcoming which will require fixes to mod_perl.

 Therefore, it would be good to roll a new release of mod_perl very soon
 so that we have a release which works with 5.18.0 when that gets
 released.

 -
 To unsubscribe, e-mail: dev-unsubscr...@perl.apache.org
 For additional commands, e-mail: dev-h...@perl.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@perl.apache.org
For additional commands, e-mail: dev-h...@perl.apache.org

Re: Website update?

[Perrin Harkins]

Hi Gerald,

Sorry for the delay.  I haven't done any work on the website, but I think I
can figure out what we need to do in order to update it.  I'll try to
investigate it this weekend.

- Perrin


On Sun, Nov 11, 2012 at 8:43 AM, rich...@ecos.de wrote:

 Hi,

 I like to update perl.apache.org/embperl which I didn't done for the last
 6 years. In this time Embperl development has continued, so the site is
 quite outdated.

 I know this is not quite the right place to ask this question, but since I
 didn't found any better place, I hope to find somebody on this list who can
 point me in the right direction.

 When I last updated the Embperl part, I had to rsync/ssh the content to
 perl.apache.org . Of course my ssh key does not work anymore, so I am not
 able to verify, if the website update is still the same.

 Can anybody let me know who is currently in charge for the website?

 Thanks  Regards

 Gerald


 -
 To unsubscribe, e-mail: dev-unsubscr...@perl.apache.org
 For additional commands, e-mail: dev-h...@perl.apache.org

Re: How best to get my mod_perl programs to run under CGI for non-mod_perl users?

[Perrin Harkins]

 Hi Chris,

One approach would be to modify your applications so they use some kind of
abstraction layer, like Plack, that runs on top of both CGI and mod_perl.
 There's more on Plack here: http://plackperl.org/

Another option, since you used Registry, would be to change your use of
Apache::Request to something like CGI.pm which works on both.  You may not
need to change much else.

- Perrin


On Sat, Dec 15, 2012 at 3:07 PM, Chris Bennett ch...@bennettconstruction.us
 wrote:

 Everything I use/write is strictly mod_perl (mod_perl-1.31 under
 OpenBSD).

 I am planning on releasing some of this open-source, but many users will
 not be running mod_perl.

 What might be the easiest way to go backwards to CGI without a huge
 amount of work?
 Should I write something that makes $r-print(); work as print?
 I have read that CGI.pm and Apache::Request are similar but not the
 same.
 Is something already written to do this out there? A guide?

 These applications use:
 use Apache::Request();
 use DBI();

 All of these modules are using Apache::Registry, not handler.

 Thanks,
 Chris Bennett

Re: Does mod_perl-2.0.4 support threads?

[Perrin Harkins]

All of the 2.x series supports threads.  Your Perl has to be compiled for
threads though.

- Perrin


On Fri, Dec 14, 2012 at 2:58 AM, Grant emailgr...@gmail.com wrote:

 I'm stuck on mod_perl-2.0.4 for now and I'm seeing something in Gentoo
 which makes me think it doesn't support threads.  Is that true?

 - Grant

Re: use global data in handler (was : custom proxy setup with mod_perl)

[Perrin Harkins]

It sounds like you have it right.  If you don't define the package variable
$URLS during startup, it will be undef when the child process handles the
first request and will keep whatever data you put into it after that.

- Perrin

On Sat, Nov 24, 2012 at 8:09 AM, André Warnier a...@ice-sa.com wrote:

 First, I may be wrong in my understanding. (that's a question).

 Second, if I am right in the above, then does this same understanding
 extend to all possible Apache MPMs ?  Or am I creating a horrible risk of
 some race condition, or memory leak here ?

Re: Questions about memcached in general and failover

[Perrin Harkins]

On Mon, Oct 22, 2012 at 5:09 AM, SAE simsimil...@googlemail.com wrote:
 But as explained above. If one of the memcached servers goes down. Some
 users experience the problem, that they not only get logged out but also
 have problems browsing the site at all or logging back in. Every page needs
 minutes to load then and it seems like the client is still trying to reach
 the server that got down and he's not trying that for only the first few
 clicks after one server went down, but for every click after that.

That sounds like an issue with your memcached client, rather than the
server.  See if there's a mailing list or other support for the client
you're using.

- Perrin

Re: Questions about memcached in general and failover

[Perrin Harkins]

On Tue, Oct 16, 2012 at 12:59 PM, Les Mikesell lesmikes...@gmail.com wrote:
 If you are going to be stuck with this restriction - and have to build
 your own fail-over, is there any advantage to using memcache compared
 to redis with its much larger feature set?

I agree.  If you need durability (i.e. if it would be a big problem to
lose some of the data in memcached), then memcached is the wrong tool
for you.  It aims to provide a level of failover that allows memcached
requests to continue to work, but does not aim to maintain data.

- Perrin

Re: Can I measure the use of index?

[Perrin Harkins]

On Mon, Oct 15, 2012 at 11:24 AM, Carlos Eduardo Caldi
ce_ca...@hotmail.com wrote:
 Somebody knows how can I log or measure the index use ?

http://www.percona.com/doc/percona-toolkit/2.1/pt-index-usage.html

- Perrin

-- 
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe:http://lists.mysql.com/mysql

Re: Coupling Plack/PSGI with mod_perl, or an alternate architecture?

[Perrin Harkins]

On Sun, Oct 14, 2012 at 2:56 PM, Tatsuhiko Miyagawa miyag...@gmail.com wrote:
 I think Lars meant no proxy required with *mod_perl* rather than plack
 stuff, but yes, Starman is recommended to put behind proxy otherwise
 your precious worker process is bound to slow networked clients, and
 gets even worse if you run Starman with keepalive support (which you
 can turn off with the options).

I would just add that a proxy is always recommended in front of
mod_perl for the same reasons.  It also cuts down on the number of DBI
handles you have open if you're using persistent connections.

- Perrin

Re: X-Forwarded-For

[Perrin Harkins]

On Sat, Oct 13, 2012 at 7:41 AM, pangj pa...@riseup.net wrote:
 In nginx's config file we have added the x-forwarded-for header.
 With modperl (MP2) how to get this header?

Use headers_in():
http://perl.apache.org/docs/2.0/api/Apache2/RequestRec.html#C_headers_in_

- Perrin

Re: Coupling Plack/PSGI with mod_perl, or an alternate architecture?

[Perrin Harkins]

On Wed, Oct 10, 2012 at 3:57 PM, Lars Dɪᴇᴄᴋᴏᴡ 迪拉斯 da...@cpan.org wrote:
 reasons to pick a mod_perl approach over a plack runner approach
 Requires no proxying.

Isn't Starman normally run with a proxy in front of it?  If not, it
should be.  Otherwise, you'd be tying up large processes sending bytes
to slow remote clients, just like you would with a standard mod_perl
app.

- Perrin