MP2: Second easy question: DB connection pooling
I've seen conflicting information in docs, websites, lists archives, etc. Is there currently a way to do database connection pooling with mod_perl2? I have tried using Apache::DBI with a use statement in my startup.pl but that doesn't seem to work. Using PerlModule Apache::DBI in my httpd.conf doesn't help either. Do I need to turn on mod_perl1 compatibility mode and then use Apache::DBI in the startup script? Is there something else entirely? =) An example or pointer to further information would be appreciated. --TWH -- Report problems: http://perl.apache.org/bugs/ Mail list info: http://perl.apache.org/maillist/modperl.html List etiquette: http://perl.apache.org/maillist/email-etiquette.html
strange apache restart problem
Hello! :) I've compile apache_1.3.31 with mod_perl-1.29 : perl Makefile.PL DO_HTTPD=1 USE_APACI=1 APACHE_PREFIX=/home/httpd EVERYTHING=1 Everything is ok and scripts run perfect, but when I upload new script or modify existing one and run "apachectl restart" i recieve /home/httpd/bin/apachectl restart: httpd restarted, but the old script is loaded!!! Only if I do "apachectl stop" and then "apachectl start" the new script is loaded. Please give me some advice. I try it on Suse,RedHat and Fedora with the same success. What I do wrong. Kosyo __ Do you Yahoo!? New and Improved Yahoo! Mail - Send 10MB messages! http://promotions.yahoo.com/new_mail -- Report problems: http://perl.apache.org/bugs/ Mail list info: http://perl.apache.org/maillist/modperl.html List etiquette: http://perl.apache.org/maillist/email-etiquette.html
write results of subrequest to file
I want to publish the result of a dynamic request to a file. So within a handler I create a subrequest that gives me the result of the dynamic request. Something like: $sub = $r->lookup_uri("$page?id=$id"); Now I want to get the results of this request and stream it into a file. Would using Apache::File be a possibillity? The main handler will continue and print if the write has been successfull. Cheers, Marc -- Report problems: http://perl.apache.org/bugs/ Mail list info: http://perl.apache.org/maillist/modperl.html List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: write results of subrequest to file
MJW Lambrichs wrote: > I want to publish the result of a dynamic request to a file. So within a > handler I create a subrequest that gives me the result of the dynamic > request. > > Something like: $sub = $r->lookup_uri("$page?id=$id"); > > Now I want to get the results of this request and stream it into a file. you can't do that with mod_perl 1.0 - you can $sub->run() but that result goes right to the client. basically, you need to issue a full request to the server using lwp or some similar tool. see the discussion in recipe 5.7 in the mod_perl developer's cookbook, including this code http://www.modperlcookbook.org/code/ch05/Cookbook/SubRequestContent.pm which is probably overly complex for simple needs but includes the LWP and uri-manipulating code you would want. HTH --Geoff -- Report problems: http://perl.apache.org/bugs/ Mail list info: http://perl.apache.org/maillist/modperl.html List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: write results of subrequest to file
On Fri, 2004-08-13 at 11:06, Geoffrey Young wrote: > you can't do that with mod_perl 1.0 - you can $sub->run() but that result > goes right to the client. For comparison, how would you do this in mp2? With a filter? - Perrin -- Report problems: http://perl.apache.org/bugs/ Mail list info: http://perl.apache.org/maillist/modperl.html List etiquette: http://perl.apache.org/maillist/email-etiquette.html
RE: Modperl and Distance Education
Looks like PHP has quite a lock on this market. Many of the online collaborative systems are PHP/mysql based. Top open-source contenders in this market would be: Moodle (moodle.org) ATutor (atutor.ca) Ilias (www.ilias.uni-koeln.de/ios/index-e.html) Note that these are collaborative learning management systems attempted to go head-to-head with commercial LMS's. I'm sure that there are some that are perl-based, but these are getting a lot of attention and they're all PHP-based. Todd From: Alfred Vahau [mailto:[EMAIL PROTECTED] Sent: Thursday, August 12, 2004 4:46 PMTo: ModperlSubject: Modperl and Distance Education Hi,Are there any major distance education sites powered by modperl technology?I work for a University with 7 campuses scattered around the country. Provision of distanceeducation through online access to teaching material and academic transcripts are among the longterm goals of the university.I find it challenging to develop a site to meet the university's requirements. It has been proposedthat I consider Java or PHP applications to develop the site to include online directory for allstaff and students connected to the university. However, I am confident that I can meet the sameobjectives using modperl technology. Somewhere I think I can use Bricolage or Mason and I amwondering whether these are appropriate technologies to consider. Or are there other applications ofmodperl relating to delivery of distance education that I am not aware of.Coming from a traditional Unix background in connection with academic research, I turned to Perl in2000 to survive in an environment dominated by Windows operating system. I continue to work on myown Breeze Project for which an early account was posted on the Oreilly site in 2002. Since then I havebeen following developments and have tested modperl and apache on trial basis.Any pointers to help in broadening my options would be highly appreciated.Many thanks in advance,Alfred VahauDirectorInformation Technology ServicesUniversity of Papua New GuineaInformation Resources Centre
Re: strange apache restart problem
On Fri, 2004-08-13 at 07:14, Konstantin Yotov wrote: > Everything is ok and scripts run perfect, but when I > upload new script or modify existing one and run > "apachectl restart" i recieve > /home/httpd/bin/apachectl restart: httpd restarted, > but the old script is loaded!!! Only if I do > "apachectl stop" and then "apachectl start" the new > script is loaded. That's normal. The restart command from apachectl just sends a SIGHUP to the server, which does not shut down and restart the perl interpreter. You have to stop the server and restart it to do that. You can use PerlFreshRestart if you want to, but I don't recommend it. Please check the docs for more info on this. - Perrin -- Report problems: http://perl.apache.org/bugs/ Mail list info: http://perl.apache.org/maillist/modperl.html List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: MP2: Second easy question: DB connection pooling
On Thu, 2004-08-12 at 14:05, Tim Howell wrote: > Is there currently a way to do database connection pooling with > mod_perl2? No. > I have tried using Apache::DBI with a use statement in my > startup.pl but that doesn't seem to work. It should. Make sure it comes before anything that uses DBI. > Do I need to turn on mod_perl1 > compatibility mode and then use Apache::DBI in the startup script? No, it should work exactly the same as it does on mp1. - Perrin -- Report problems: http://perl.apache.org/bugs/ Mail list info: http://perl.apache.org/maillist/modperl.html List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: write results of subrequest to file
Perrin Harkins wrote: > On Fri, 2004-08-13 at 11:06, Geoffrey Young wrote: > >>you can't do that with mod_perl 1.0 - you can $sub->run() but that result >>goes right to the client. > > > For comparison, how would you do this in mp2? With a filter? pretty much. I believe that subrequests in apache2 have the same limitations as before wrt calling run(). but if the main reason you want to capture a subrequest is to post-process dynamic content then you can simply insert an output filter to do that for you, as that is what they were designed for. --Geoff -- Report problems: http://perl.apache.org/bugs/ Mail list info: http://perl.apache.org/maillist/modperl.html List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Apache::DBI and DBIx::*
What does a DBIx::* module have to do in order to correctly call Apache::DBI's no-op version of disconnect() when running under mod_perl with Apache::DBI? Below is an example that illustrates a bug I found when using a particular DBIx:: module. --- package MyTest; use strict; use Apache::DBI; use DBIx::ContextualFetch; sub handler { my($r) = shift; my $dbh = DBI->connect('dbi:...', 'myuser', 'mypass', { RootClass => 'DBIx::ContextualFetch' }); $r->send_http_header('text/plain'); $dbh->do('...any valid sql...') ? print 'OK' : print 'Failed'; $dbh->disconnect; # This calls the wrong disconnect()! return 200; } 1; --- Run it under httpd -X to further isolate the problem and you'll see that the first request works, but all subsequent requests fail. This happens because the disconnect() call actually calls the "real" disconnect method instead of the Apache::DBI implementation, which is a no-op. On the next request, Apache::DBI happily provides what is now a disconnected $dbh. Any attempt to use that $dbh fails, obviously. If you enable Apache::DBI's debugging output ($Apache::DBI::DEBUG = 1), you'll see that the correct connect() method is called. That is, only one connection is made and then re-used. So, getting back to my original question, who is at fault here? Is this an Apache::DBI bug, a DBIx::ContextualFetch bug, or something else? One easy "fix" is to add a disconnect() method to DBIx::ContextualFetch that detects when Apache::DBI is in use. Example: sub DBIx::ContextualFetch::disconnect { if($ENV{'MOD_PERL'} && $Apache::DBI::VERSION) { # Do nothing when Apache::DBI is in use return 1; } shift->SUPER::disconnect(@_); } But isn't Apache::DBI supposed to be "transparent" and do all this stuff for me? Or are all bets off when using a DBIx module? If so, what is a DBIx author supposed to do to be a "good Apache::DBI citizen"? -John -- Report problems: http://perl.apache.org/bugs/ Mail list info: http://perl.apache.org/maillist/modperl.html List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: Apache::DBI and DBIx::*
On Fri, 2004-08-13 at 15:25, John Siracusa wrote: > What does a DBIx::* module have to do in order to correctly call > Apache::DBI's no-op version of disconnect() when running under mod_perl with > Apache::DBI? It needs to not set RootClass. Apache::DBI essentially subclasses DBI, which is what RootClass does as well. > Run it under httpd -X to further isolate the problem and you'll see that the > first request works, but all subsequent requests fail. This happens because > the disconnect() call actually calls the "real" disconnect method instead of > the Apache::DBI implementation, which is a no-op. Why call disconnect at all? Are you trying to run this as normal CGI too? - Perrin -- Report problems: http://perl.apache.org/bugs/ Mail list info: http://perl.apache.org/maillist/modperl.html List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: Apache::DBI and DBIx::*
while perrin addressed your other issues, I thought I'd bring this up as a learning point :) > > return 200; this is wrong - you should be returning OK (0) not HTTP_OK (200) from handlers if everything is swell. while 200 works in mp1 (due to mod_perl "guessing" what you meant) you cannot do this in mod_perl 2.0. so, if you do it correctly now you'll have one less thing to migrate later :) --Geoff -- Report problems: http://perl.apache.org/bugs/ Mail list info: http://perl.apache.org/maillist/modperl.html List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: Apache::DBI and DBIx::*
On Fri, 13 Aug 2004 15:32:03 -0400, Perrin Harkins <[EMAIL PROTECTED]> wrote: > On Fri, 2004-08-13 at 15:25, John Siracusa wrote: > > What does a DBIx::* module have to do in order to correctly call > > Apache::DBI's no-op version of disconnect() when running under mod_perl with > > Apache::DBI? > > It needs to not set RootClass. Apache::DBI essentially subclasses DBI, > which is what RootClass does as well. Er, yeah, "essentially" :) All the crazy re-blessing that goes on is quite a mess, IMO. So what you're saying is that DBIx::* and Apache::DBI will simply never get along because they both want to do the same thing, but use different (and apparently incompatible) methods to do so? > > Run it under httpd -X to further isolate the problem and you'll see that the > > first request works, but all subsequent requests fail. This happens because > > the disconnect() call actually calls the "real" disconnect method instead of > > the Apache::DBI implementation, which is a no-op. > > Why call disconnect at all? Are you trying to run this as normal CGI > too? The first answer is that I'm calling disconnect because it's The Right Thing To Do. Maybe some people think just letting things go out of scope is more "(mod_)perl-ish", but I always wonder if those people don't call close(FILE) either ;) Anyway, I'm not one of those people. The second answer is that it's common (and usually desirable) for certain DBI-using modules to be used both in both web and offline contexts. (The example was heavily simplified, obviously.) -John -- Report problems: http://perl.apache.org/bugs/ Mail list info: http://perl.apache.org/maillist/modperl.html List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: Apache::DBI and DBIx::*
On Fri, 13 Aug 2004 15:38:41 -0400, Geoffrey Young wrote: > > return 200; > > this is wrong - you should be returning OK (0) not HTTP_OK (200) from > handlers if everything is swell. while 200 works in mp1 (due to mod_perl > "guessing" what you meant) you cannot do this in mod_perl 2.0. so, if you > do it correctly now you'll have one less thing to migrate later :) Whoops, I was just trying to reduce the length of the example code by not use()ing Apache::Constants :) -John -- Report problems: http://perl.apache.org/bugs/ Mail list info: http://perl.apache.org/maillist/modperl.html List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: Apache::DBI and DBIx::*
Perrin Harkins wrote: On Fri, 2004-08-13 at 15:25, John Siracusa wrote: What does a DBIx::* module have to do in order to correctly call Apache::DBI's no-op version of disconnect() when running under mod_perl with Apache::DBI? It needs to not set RootClass. Apache::DBI essentially subclasses DBI, which is what RootClass does as well. Also make sure that you read: http://perl.apache.org/docs/1.0/guide/databases.html#Opening_Connections_With_Different_Parameters -- __ Stas BekmanJAm_pH --> Just Another mod_perl Hacker http://stason.org/ mod_perl Guide ---> http://perl.apache.org mailto:[EMAIL PROTECTED] http://use.perl.org http://apacheweek.com http://modperlbook.org http://apache.org http://ticketmaster.com -- Report problems: http://perl.apache.org/bugs/ Mail list info: http://perl.apache.org/maillist/modperl.html List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: Apache::DBI and DBIx::*
John Siracusa wrote: On Fri, 13 Aug 2004 15:38:41 -0400, Geoffrey Young wrote: return 200; this is wrong - you should be returning OK (0) not HTTP_OK (200) from handlers if everything is swell. while 200 works in mp1 (due to mod_perl "guessing" what you meant) you cannot do this in mod_perl 2.0. so, if you do it correctly now you'll have one less thing to migrate later :) Whoops, I was just trying to reduce the length of the example code by not use()ing Apache::Constants :) Apache::Constants::OK == 0, not 200 -- __ Stas BekmanJAm_pH --> Just Another mod_perl Hacker http://stason.org/ mod_perl Guide ---> http://perl.apache.org mailto:[EMAIL PROTECTED] http://use.perl.org http://apacheweek.com http://modperlbook.org http://apache.org http://ticketmaster.com -- Report problems: http://perl.apache.org/bugs/ Mail list info: http://perl.apache.org/maillist/modperl.html List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: Apache::DBI and DBIx::*
On Fri, 13 Aug 2004 12:51:32 -0700, Stas Bekman <[EMAIL PROTECTED]> wrote: > John Siracusa wrote: > > On Fri, 13 Aug 2004 15:38:41 -0400, Geoffrey Young wrote: > > > >>> return 200; > >> > >>this is wrong - you should be returning OK (0) not HTTP_OK (200) from > >>handlers if everything is swell. while 200 works in mp1 (due to mod_perl > >>"guessing" what you meant) you cannot do this in mod_perl 2.0. so, if you > >>do it correctly now you'll have one less thing to migrate later :) > > > > Whoops, I was just trying to reduce the length of the example code by not > > use()ing Apache::Constants :) > > Apache::Constants::OK == 0, not 200 Thus the "whoops" :) I was just saying that I got it wrong because I always use Apache::Constants in actual code. -John -- Report problems: http://perl.apache.org/bugs/ Mail list info: http://perl.apache.org/maillist/modperl.html List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: Apache::DBI and DBIx::*
On Fri, 13 Aug 2004 12:50:44 -0700, Stas Bekman <[EMAIL PROTECTED]> wrote: > Also make sure that you read: > http://perl.apache.org/docs/1.0/guide/databases.html#Opening_Connections_With_Di fferent_Parameters Unfortunately, setting RootClass after the connect doesn't have the desired effect (and isn't even valid, as far as I can tell; local()izing it throws an exception) So, what is the final answer here? Is this not a bug at all? If not, what do people do when they need to create code that uses DBIx:: modules and runs in both Apache::DBI and offline contexts? Call $dbh->disconnect() unless $ENV{'MOD_PERL'}? Ick... -John -- Report problems: http://perl.apache.org/bugs/ Mail list info: http://perl.apache.org/maillist/modperl.html List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: Apache::DBI and DBIx::*
> Er, yeah, "essentially" :) All the crazy re-blessing that goes on is quite > a mess, IMO. Agreed. I would like to replace Apache::DBI with something oriented towards connection management rather than porting old CGI scripts. I don't like the disabling of disconnect. The old Apache::DBI could stick around just for that purpose. > So what you're saying is that DBIx::* and Apache::DBI will simply > never get > along because they both want to do the same thing, but use different > (and > apparently incompatible) methods to do so? They both subclass, and multiple inheritance might not work here, but you can try it. Put this code somewhere after you load Apache::DBI: psuh @Apache::DBI::st::ISA, 'DBIx::ContextualFetch'; Note that this has nothing to do with DBIx::* as a whole. This one particular module that you are using happens to subclass DBI. Most of them don't. > The first answer is that I'm calling disconnect because it's The Right > Thing To Do. I can't see how. You know it's a no-op. You don't actually want it to disconnect. Why call it? The whole disabling-disconnect thing exists only for the purpose of porting old CGI scripts. > Maybe some people think just letting things go out of scope is more > "(mod_)perl-ish", but I always wonder if those people don't call > close(FILE) either ;) Now that Perl has lexically scoped filehandles, there isn't any reason to explicitly call close() in most cases. ;) > The second answer is that it's common (and usually desirable) for > certain DBI-using modules to be used both in both web and offline > contexts. I share a lot of code that way too, but I typically keep the connection open until the end of the command-line script and then close it there. - Perrin -- Report problems: http://perl.apache.org/bugs/ Mail list info: http://perl.apache.org/maillist/modperl.html List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: Apache::DBI and DBIx::*
Perrin Harkins wrote: Er, yeah, "essentially" :) All the crazy re-blessing that goes on is quite a mess, IMO. Agreed. I would like to replace Apache::DBI with something oriented towards connection management rather than porting old CGI scripts. I don't like the disabling of disconnect. The old Apache::DBI could stick around just for that purpose. IMHO, that should be folded into DBI::Pool, whose prototype was written about 2 years ago. I wasn't able to work on it since then. It's possible that Tim Bunce will work on it, or may be someone else, or me once mp2 is released. -- __ Stas BekmanJAm_pH --> Just Another mod_perl Hacker http://stason.org/ mod_perl Guide ---> http://perl.apache.org mailto:[EMAIL PROTECTED] http://use.perl.org http://apacheweek.com http://modperlbook.org http://apache.org http://ticketmaster.com -- Report problems: http://perl.apache.org/bugs/ Mail list info: http://perl.apache.org/maillist/modperl.html List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: Apache::DBI and DBIx::*
On Fri, 13 Aug 2004 16:22:00 -0400, Perrin Harkins <[EMAIL PROTECTED]> wrote: >> So what you're saying is that DBIx::* and Apache::DBI will simply never get >> along because they both want to do the same thing, but use different (and >> apparently incompatible) methods to do so? > > They both subclass, and multiple inheritance might not work here, but > you can try it. Put this code somewhere after you load Apache::DBI: > > psuh @Apache::DBI::st::ISA, 'DBIx::ContextualFetch'; Well, I'm actually using a class that is a subclass of that DBIx:: class, so I just added the disconnect() method (posted earlier) to the subclass. It just seemed odd to me to be doing stuff in service of Apache::DBI, which was totally transparent until then. > Note that this has nothing to do with DBIx::* as a whole. This one > particular module that you are using happens to subclass DBI. Most of > them don't. Ah, so I just got lucky, eh? :) > Now that Perl has lexically scoped filehandles, there isn't any reason > to explicitly call close() in most cases. ;) Tsk, for shame. Just wait until you forget to call close() after writing to a file... ;) > > The second answer is that it's common (and usually desirable) for > > certain DBI-using modules to be used both in both web and offline > > contexts. > > I share a lot of code that way too, but I typically keep the connection > open until the end of the command-line script and then close it there. That assumes the connection was opened by (or is at least accessible from) the script itself. If the connection is opened and managed by a module (which may abstract things such that you aren't even supposed to know whether or not it has to hit a database at all) rather than by your script, that's not an option. I tend to like abstractions like that, and I certainly want "model"-type objects to be able to exist happily in both mod_perl and offline contexts. I also wrap DBI rather than subclass it when I have a choice. Subclassing DBI in general is way too hard, IMO, and it wasn't my choice to use this DBIx:: module, but such is life. I'm mostly just trying to determine if/where to send a bug report. The answer seems to be no/nowhere. -John -- Report problems: http://perl.apache.org/bugs/ Mail list info: http://perl.apache.org/maillist/modperl.html List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: Apache::DBI and DBIx::*
On Fri, 2004-08-13 at 16:37, Stas Bekman wrote: > IMHO, that should be folded into DBI::Pool I'm not really interested in tackling pooling, since I don't plan to run threads. All that I really want my module to do is extend the DBI->connect_cached() method by adding the cleanup handler stuff from Apache::DBI. - Perrin -- Report problems: http://perl.apache.org/bugs/ Mail list info: http://perl.apache.org/maillist/modperl.html List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: Apache::DBI and DBIx::*
On Fri, 13 Aug 2004 13:37:55 -0700, Stas Bekman wrote: > Perrin Harkins wrote: > > I would like to replace Apache::DBI with something oriented > > towards connection management rather than porting old CGI scripts. I > > don't like the disabling of disconnect. The old Apache::DBI could stick > > around just for that purpose. > > IMHO, that should be folded into DBI::Pool, whose prototype was written > about 2 years ago. I wasn't able to work on it since then. It's possible > that Tim Bunce will work on it, or may be someone else, or me once mp2 is > released. I like the fact that both connect() and disconnect() are overridden. Of course, a proper server-wide DBI connection pool would be nice, but I wouldn't want to give up the ability to have the same code "magically" get its connection from the pool by calling connect() and return it to the pool by calling disconnect() when running in a certain context. Yeah, Yet Another DBI Wrapper could do that for me too, but I don't think the semantics described above are too crazy. Maybe just modify DBI and add "smart" methods named retain() and release() that call connect() and disconnect() when offline, but do the right thing when connection pooling is in effect? Hrm, something tells me Tim wouldn't go for that :) Anyway, especially in a future server-wide connection pooling context, some sort of method of returning connections to the pool prior to the end of the request cycle is very useful. Why hog it if you don't need it? Calling disconnect() is as good a signal as any the the connection is up for grabs again. -John -- Report problems: http://perl.apache.org/bugs/ Mail list info: http://perl.apache.org/maillist/modperl.html List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: Apache::DBI and DBIx::*
On Fri, 2004-08-13 at 16:43, John Siracusa wrote: > It > just seemed odd to me to be doing stuff in service of Apache::DBI, which was > totally transparent until then. That's one of the things I don't like about Apache::DBI. I don't want it to be magical. > > Now that Perl has lexically scoped filehandles, there isn't any reason > > to explicitly call close() in most cases. ;) > > Tsk, for shame. Just wait until you forget to call close() after writing to > a file... ;) That's the point actually -- perl closes it for me when it goes out of scope. > That assumes the connection was opened by (or is at least accessible from) > the script itself. If the connection is opened and managed by a module > (which may abstract things such that you aren't even supposed to know > whether or not it has to hit a database at all) rather than by your script, > that's not an option. True, you have to structure your code for this. Making all calls to open a db connection go through one utility class is my approach, but may not work for everyone. Hmm, I wonder why my Class::DBI unit test scripts never get the disconnect warning? I should look into that. Class::DBI uses connect_cached. > I also wrap DBI rather than subclass it when I have a choice. Me too. Subclassing DBI (or Apache or Apache::Request for that matter) is nearly always a bad idea IMO. In this one case it makes some sense though, since that module is literally all about changing the behavior of DBI methods. - Perrin -- Report problems: http://perl.apache.org/bugs/ Mail list info: http://perl.apache.org/maillist/modperl.html List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: Apache::DBI and DBIx::*
On Fri, 13 Aug 2004 16:45:59 -0400, Perrin Harkins <[EMAIL PROTECTED]> wrote: > On Fri, 2004-08-13 at 16:37, Stas Bekman wrote: > > IMHO, that should be folded into DBI::Pool > > I'm not really interested in tackling pooling, since I don't plan to run > threads. All that I really want my module to do is extend the > DBI->connect_cached() method by adding the cleanup handler stuff from > Apache::DBI. Even without threads, wouldn't you rather have each apache child pulling connections from a shared, server-wide pool rather than holding onto at least one connection a piece whether they need it or not? With the Apache::DBI system on a typical db-backed web site, the max number of simultaneous (mod_perl) requests is pretty much limited by the max number of simultaneous open database connections (since, long term, every apache child will be hanging onto at least one db connection). A server-wide pool removes that limitation, allowing each request to retain, use, and then release any number of connections (possibly zero) within the course of a single request. Maybe that'd lead to too much contention, I don't know, but I assumed something like that was a long-term mod_perl2/apache2/perl6 goal. Or maybe it's just a personal fantasy... :) -John -- Report problems: http://perl.apache.org/bugs/ Mail list info: http://perl.apache.org/maillist/modperl.html List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: Apache::DBI and DBIx::*
On Fri, 13 Aug 2004 17:02:27 -0400, Perrin Harkins <[EMAIL PROTECTED]> wrote: > > > Now that Perl has lexically scoped filehandles, there isn't any reason > > > to explicitly call close() in most cases. ;) > > > > Tsk, for shame. Just wait until you forget to call close() after writing to > > a file... ;) > > That's the point actually -- perl closes it for me when it goes out of > scope. Ah, but perl doesn't check the return value! :) Something tells me you haven't been bitten by this bug yet, but let me save you the grief. When, say, your disk fills up and you let that file handle go out of scope, close() will silently fail when trying to flush the buffers to disk. The moral of the story: always call close() explicitly *and check the return value* when writing to a file. -John -- Report problems: http://perl.apache.org/bugs/ Mail list info: http://perl.apache.org/maillist/modperl.html List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: Apache::DBI and DBIx::*
On Fri, 2004-08-13 at 17:05, John Siracusa wrote: > Even without threads, wouldn't you rather have each apache child pulling > connections from a shared, server-wide pool rather than holding onto at > least one connection a piece whether they need it or not? Sure. Can you figure out a way to do it? It is possible on unix systems to pass filehandles between processes, but that kind of C programming is not something I'm likely to tackle myself. > With the Apache::DBI system on a typical db-backed web site, the max number > of simultaneous (mod_perl) requests is pretty much limited by the max number > of simultaneous open database connections (since, long term, every apache > child will be hanging onto at least one db connection). You actually have a site where you can run more apache children than database connections? I have never seen such a beast. IMO the only real advantage of having pooling over the way things are now (assuming you run a front-end proxy for static stuff) is that sites which use different database logins for each user will be able to use persistent connections. Right now, that would result in a very large number of connections. > Maybe that'd lead to too much contention, I don't know, but I assumed > something like that was a long-term mod_perl2/apache2/perl6 goal. Or maybe > it's just a personal fantasy... :) Perl6 would presumably mean mod_perl 3. Anyway, the pooling thing is mostly driven by how much Java people hype it. The usefulness of pooling is pretty limited for a site that actually uses database connections on every request. - Perrin -- Report problems: http://perl.apache.org/bugs/ Mail list info: http://perl.apache.org/maillist/modperl.html List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: Apache::DBI and DBIx::*
On Fri, 2004-08-13 at 17:15, John Siracusa wrote: > Something tells me you haven't been bitten by this bug yet, but let me save > you the grief. When, say, your disk fills up and you let that file handle > go out of scope, close() will silently fail when trying to flush the buffers > to disk. If you're concerned about that, you should actually check the return value of print() as well. - Perrin -- Report problems: http://perl.apache.org/bugs/ Mail list info: http://perl.apache.org/maillist/modperl.html List etiquette: http://perl.apache.org/maillist/email-etiquette.html
[PATCH] Apache::DBI support for subclassed DBI connections
This patch should make Apache::DBI work with subclassed DBI connections. The only downside is that it makes ALL of your cached connections subclassed if you give it one that is subclassed. Given how most people use DBI subclasses (for universal functionality on all DBI connections), this is probably not an issue. - Perrin --- /usr/lib/perl5/site_perl/5.8.3/Apache/DBI.pm 2004-02-17 19:18:50.0 -0500 +++ ./DBI.pm 2004-08-13 17:41:41.045930048 -0400 @@ -130,6 +130,12 @@ # return the new database handle print STDERR "$prefix new connect to '$Idx'\n" if $Apache::DBI::DEBUG; + +# support subclassed DBI connections +if (ref $Connected{$Idx} ne 'DBI::st') { +push @Apache::DBI::st::ISA, ref $Connected{$Idx}; +} + return (bless $Connected{$Idx}, 'Apache::DBI::db'); } -- Report problems: http://perl.apache.org/bugs/ Mail list info: http://perl.apache.org/maillist/modperl.html List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: Apache::DBI and DBIx::*
Perrin Harkins wrote: On Fri, 2004-08-13 at 16:37, Stas Bekman wrote: IMHO, that should be folded into DBI::Pool I'm not really interested in tackling pooling, since I don't plan to run threads. Support for threads is only partial functionality of DBI::Pool. Otherwise it should be a ground-up rewrite based on the experiences with Apache::DBI and other wrappers so far. Apache::DBI should become just a thin wrapper on top of DBI::Pool, to handle things like connect_on_init. All that I really want my module to do is extend the DBI->connect_cached() method by adding the cleanup handler stuff from Apache::DBI. Sure, just thought that you are planning to work on yet another Apache::DBI version, so I've suggested to direct the effort into something new that (will) solves it all. -- __ Stas BekmanJAm_pH --> Just Another mod_perl Hacker http://stason.org/ mod_perl Guide ---> http://perl.apache.org mailto:[EMAIL PROTECTED] http://use.perl.org http://apacheweek.com http://modperlbook.org http://apache.org http://ticketmaster.com -- Report problems: http://perl.apache.org/bugs/ Mail list info: http://perl.apache.org/maillist/modperl.html List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: Apache::DBI and DBIx::*
Stas Bekman wrote: Support for threads is only partial functionality of DBI::Pool. Otherwise it should be a ground-up rewrite based on the experiences with Apache::DBI and other wrappers so far. Apache::DBI should become just a thin wrapper on top of DBI::Pool, to handle things like connect_on_init. The thin wrapper is actually the only part I'm talking about building here. Persistent connections are already available in standard DBI, so I don't need to do anything for that. My goal is just to make something that adds the mod_perl-specific stuff like connect_on_init and a cleanup handler, but doesn't do any of the annoying magical stuff, like automatically making connections persistent (just call connect_cached() for that) or disabling disconnect. The magic stuff could stick around in Apache::DBI for backwards compatibility and for people who want to port CGI scripts with it. I don't have any time to work on this right now, so it's sort of a moot point. When I do get time for it, I'll post here and we can discuss the design in more detail. - Perrin -- Report problems: http://perl.apache.org/bugs/ Mail list info: http://perl.apache.org/maillist/modperl.html List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Checking the return value of close() (was: Apache::DBI and DBIx::*)
On 8/13/04 5:23 PM, Perrin Harkins wrote: > On Fri, 2004-08-13 at 17:15, John Siracusa wrote: >> Something tells me you haven't been bitten by this bug yet, but let me save >> you the grief. When, say, your disk fills up and you let that file handle >> go out of scope, close() will silently fail when trying to flush the buffers >> to disk. > > If you're concerned about that, you should actually check the return > value of print() as well. Believe me, everyone should be "concerned about that" :) Also, the return value from print() is not always useful. In the situation I described, for example, print will happily return 1 so long as what it's printing fits in the output buffer, regardless of whether or not the data is actually going to make it to the disk. Checking the return value from close() gives a more reliable answer, and you only have to do it once at the end. -John -- Report problems: http://perl.apache.org/bugs/ Mail list info: http://perl.apache.org/maillist/modperl.html List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: Apache::DBI and DBIx::*
On 8/13/04 5:22 PM, Perrin Harkins wrote: > On Fri, 2004-08-13 at 17:05, John Siracusa wrote: >> Even without threads, wouldn't you rather have each apache child pulling >> connections from a shared, server-wide pool rather than holding onto at >> least one connection a piece whether they need it or not? > > Sure. Can you figure out a way to do it? Probably not in Perl 5 and apache 1.x, but I hope Perl 6, apache 2, and mod_(perl6|parrot) will be flexible enough... > You actually have a site where you can run more apache children than > database connections? I have never seen such a beast. Maybe your database is just too beefy? :) I've worked on sites that would routinely bump into a database connection limit of ~200 during traffic spikes. (And yes, of course static content was served elsewhere :) >> Maybe that'd lead to too much contention, I don't know, but I assumed >> something like that was a long-term mod_perl2/apache2/perl6 goal. Or maybe >> it's just a personal fantasy... :) > > Perl6 would presumably mean mod_perl 3. ...or mod_parrot or something, especially if this Parrot DBI-like-core thing takes off (I forget the name they're using) > Anyway, the pooling thing is mostly driven by how much Java people hype it. > The usefulness of pooling is pretty limited for a site that actually uses > database connections on every request. I've worked on sites that use a "session" (MySQL) database connection on every request, but don't use a "main" (Postgres) database connection nearly that often. If that main database is shared by several sites/apps, it can become the bottleneck. Maybe read-only replication would help, but it still seems wasteful. -John -- Report problems: http://perl.apache.org/bugs/ Mail list info: http://perl.apache.org/maillist/modperl.html List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: Checking the return value of close() (was: Apache::DBI and DBIx::*)
On 8/13/04 10:34 PM, John Siracusa wrote: > In the situation I described, for example, print will happily return 1 so long > as what it's printing fits in the output buffer, regardless of whether or not > the data is actually going to make it to the disk. Er, that should read "regardless of whether or not the disk is full", since "making it to disk" is outside the scope of Perl's core functions (and may be outside the scope of the OS if the hardware lies :) -John -- Report problems: http://perl.apache.org/bugs/ Mail list info: http://perl.apache.org/maillist/modperl.html List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: [PATCH] Apache::DBI support for subclassed DBI connections
On 8/13/04 5:43 PM, Perrin Harkins wrote: > This patch should make Apache::DBI work with subclassed DBI > connections. The only downside is that it makes ALL of your cached > connections subclassed if you give it one that is subclassed. Given how > most people use DBI subclasses (for universal functionality on all DBI > connections), this is probably not an issue. Cool, I'll try it out on Monday. If it works, you should submit it to the Apache::DBI maintainer (if you haven't already) -John -- Report problems: http://perl.apache.org/bugs/ Mail list info: http://perl.apache.org/maillist/modperl.html List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: Apache::DBI and DBIx::*
John Siracusa wrote: Even without threads, wouldn't you rather have each apache child pulling connections from a shared, server-wide pool rather than holding onto at least one connection a piece whether they need it or not? Sure. Can you figure out a way to do it? Probably not in Perl 5 and apache 1.x, but I hope Perl 6, apache 2, and mod_(perl6|parrot) will be flexible enough... It's not a matter of flexibility, but rather one of difficult coding and potential issues with closed source database client libraries. The coding would be in C, and it would involve passing data structures and open sockets between running processes. It's way beyond my level of unix-foo, but if someone out there wants to try it, there is nothing needed for it that hasn't already been there for years. Making DBI::Pool work for threads is probably easier, and that could also be done today if someone has both the incentive and the C-coding skills to deal with the threading and XS issues. I've worked on sites that use a "session" (MySQL) database connection on every request, but don't use a "main" (Postgres) database connection nearly that often. In that situation, if a significant percentage of requests need sessions but not the main database, and the difference is determined by URL, I would move the session-only requests to a separate server. Pooling sounds nice, but is a non-trivial problem, and only helps in a few specific situations. In the common case, where every request to the app-server needs a database handle, it is not very useful. - Perrin -- Report problems: http://perl.apache.org/bugs/ Mail list info: http://perl.apache.org/maillist/modperl.html List etiquette: http://perl.apache.org/maillist/email-etiquette.html