Re: BerkeleyDB with mod perl
On Thu, 2004-04-22 at 15:55, Chris Ochs wrote: I'm using the latest BerkeleyDB perl module on freebsd 5.2 with mod perl 1.xx. Following is the BerkeleyDB code I am using. When I start running lots of concurrent processes that run db_put I get the error 'Locker does not exist' in $BerkeleyDB::Error and db_put returns 'Invalid argument'. Any chance you might be opening the db handle before forking, i.e. in your startup.pl? I suspect that would cause problems. - Perrin -- Report problems: http://perl.apache.org/bugs/ Mail list info: http://perl.apache.org/maillist/modperl.html List etiquette: http://perl.apache.org/maillist/email-etiquette.html
[OT] Re: BerkeleyDB with mod perl
Hi Chris, I sympathize with you. I've been through my share of problems with BerkeleyDB as well. So far, I haven't had problems using it with mod_perl. I wrote a stress test tool[1] to prove to myself that BerkeleyDB was indeed safe to use in concurrent processes. You can set it to run as many processes doing as many inserts and reads as you would like. You may want to run that against a copy of your database to see if it exposes any errors. My mod_perl setup is on a Windows 2000 system and seems to be stable. I setup my environment in the same way that you are doing except that I pass in -ErrFile to the BerkeleyDB::Env setup. You may want to try that to see if it provides any more details for you. The only other suggestion I would make is to try without the tied interface if you don't have to change too much code. Since you're using db_put, you could setup just the handle via the following: my $bdb = new BerkeleyDB::Btree -Flags= DB_CREATE, -Filename = $dbfile, -Env = $env, or die Cannot open file $dbfile: $! - '$BerkeleyDB::Error'\n; Good luck and let us know if you find a solution, William [1] http://perlmonks.org/index.pl?node_id=330510 -- Knowmad Services Inc. http://www.knowmad.com -- Report problems: http://perl.apache.org/bugs/ Mail list info: http://perl.apache.org/maillist/modperl.html List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: BerkeleyDB with mod perl
On Thu, Apr 22, 2004 at 04:12:51PM -0400, Perrin Harkins wrote: Any chance you might be opening the db handle before forking, i.e. in your startup.pl? I suspect that would cause problems. Hi Perrin, My gut reaction is Yes, this makes sense. However, I'm scratching my head trying to understand why. It seems that all the children would share the same locked environment. But perhaps, they shouldn't be sharing the same handle (is this the equivalent of a connection?) to that environment. Is that where the problem lies? Does each child need to have a separate handle? Thanks, William -- Knowmad Services Inc. http://www.knowmad.com -- Report problems: http://perl.apache.org/bugs/ Mail list info: http://perl.apache.org/maillist/modperl.html List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: BerkeleyDB with mod perl
Sheesh yes I forgot to say that I set the environment and open the db handle in startup.pl. If I set the environment in startup.pl but open the db handle in my perl handler every time the page is accessed the performance drops by about 300%, but it does make the errors go away. Chris - Original Message - From: Perrin Harkins [EMAIL PROTECTED] To: Chris Ochs [EMAIL PROTECTED] Cc: Modperl List [EMAIL PROTECTED] Sent: Thursday, April 22, 2004 1:12 PM Subject: Re: BerkeleyDB with mod perl On Thu, 2004-04-22 at 15:55, Chris Ochs wrote: I'm using the latest BerkeleyDB perl module on freebsd 5.2 with mod perl 1.xx. Following is the BerkeleyDB code I am using. When I start running lots of concurrent processes that run db_put I get the error 'Locker does not exist' in $BerkeleyDB::Error and db_put returns 'Invalid argument'. Any chance you might be opening the db handle before forking, i.e. in your startup.pl? I suspect that would cause problems. - Perrin -- Report problems: http://perl.apache.org/bugs/ Mail list info: http://perl.apache.org/maillist/modperl.html List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: BerkeleyDB with mod perl
On Thu, 2004-04-22 at 16:27, William McKee wrote: My gut reaction is Yes, this makes sense. However, I'm scratching my head trying to understand why. There are file handles involved. You can't fork with an open file and expect to be able to do things with it from each process, at least not without awareness and coordination between them. I don't know enough about BDB internals to say for sure what would happen, but this sort of thing is often a problem with forking servers. - Perrin -- Report problems: http://perl.apache.org/bugs/ Mail list info: http://perl.apache.org/maillist/modperl.html List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: BerkeleyDB with mod perl
On Thu, 2004-04-22 at 16:34, Chris Ochs wrote: If I set the environment in startup.pl but open the db handle in my perl handler every time the page is accessed the performance drops by about 300%, but it does make the errors go away. There is no need to open the handle every time. You can open it once and put it in a global. Just do it after the fork. It's the same idea as Apache::DBI. - Perrin -- Report problems: http://perl.apache.org/bugs/ Mail list info: http://perl.apache.org/maillist/modperl.html List etiquette: http://perl.apache.org/maillist/email-etiquette.html
Re: BerkeleyDB with mod perl
Yep sticking it in a global and handling it like Apache::DBI works. - Original Message - From: Perrin Harkins [EMAIL PROTECTED] To: Chris Ochs [EMAIL PROTECTED] Cc: Modperl List [EMAIL PROTECTED] Sent: Thursday, April 22, 2004 1:36 PM Subject: Re: BerkeleyDB with mod perl On Thu, 2004-04-22 at 16:34, Chris Ochs wrote: If I set the environment in startup.pl but open the db handle in my perl handler every time the page is accessed the performance drops by about 300%, but it does make the errors go away. There is no need to open the handle every time. You can open it once and put it in a global. Just do it after the fork. It's the same idea as Apache::DBI. - Perrin -- Report problems: http://perl.apache.org/bugs/ Mail list info: http://perl.apache.org/maillist/modperl.html List etiquette: http://perl.apache.org/maillist/email-etiquette.html -- Report problems: http://perl.apache.org/bugs/ Mail list info: http://perl.apache.org/maillist/modperl.html List etiquette: http://perl.apache.org/maillist/email-etiquette.html
