ithreads-enabled Perl cannot share nested (Perl/C) datastructures. If it could the issue of dbi-pool would be a no-brainer since all we needed to do is $dbh ||= connect();
An attempted solution:
if Perl doesn't share nested datastructures, we will implement this sharing ourselves, using an items pool. After struggling with DBI guts and implementing this solution the result wasn't good: perl context can't be easily switched, so this doesn't work. I suppose that we could always create $dbh in the parent perl, and switch the context to it every time we needed to work with it, but that would result in serialization of all threads wanting to work with any $dbh, resulting in really bad performance, defeating the purpose.
Final (semi-working) solution:
as discussed with Tim and Hugo back at TPC (Jul 2002), the only possible solution is to share only the driver's private part of dbh. This is only possible if that private part has nothing to do with Perl, i.e. the implementation is in C and the threads implementation shares all the data between all threads (dunno if it's through for all threads libs, most likely is).
I didn't want to mess directly with DBI, so I've created DBI::Pool, which is a hack which should eventually fold into DBI itself.
Currently DBI::Pool is actually 3 things:
1. XS implementation of a basic free/busy lists management this is used to store and retrieve the data structures. I believe that this can be moved into Thread::Pool so it'll support sharing of C data-structures as well. But for now, it's here.
it includes several functions for a more general Thread::Pool (like grow()), which is unused in DBI::Pool.
2. XS functions imp_dbh2dbh() and dbh2imp_dbh() which extract and restore the private dbh data from/to $dbh
3. overriden DBI's connect/disconnect to work with the pool (pure Perl mess)
As mentioned before, this is just a proof of concept which has lots of loose ends. Though I can't continue any further, without getting some help from DBI gurus. The open issues are listed next.
Please notice that I've chosen to work with DBD::mysql (because that's the only db that I have here) and if you want to try my hack at work, you need to have it as well. Also you need to have a database 'test' and be able to connect with empty l/p, or change test.pl to adjust to your setup.
Also you need the ithreads-enabled perl-5.8.0 or higher. It'll build with ithreads-disabled perl as well, but I haven't adjusted the Perl code to work without the threads.
These are the open issues:
1. Because I needed to access the private data of the dbd driver, I needed to include the mysql header and struct imp_dbh_st. So please adjust the path to mysql.h if it's in a different location.
once the logic folds into DBI, this won't be necessary
2. I need a support from DBI to help me access the *really* private data in struct imp_dbh_st, because the following is a hack:
D_imp_dbh(dbh);
imp_dbh->mysql = ((imp_dbh_t *)imp_dbh_new)->mysql;When I re-install the stored dbh, I must not break the ->com structure, but overwrite the rest. So I guess the right approach is to copy away the original ->com, overwrite the whole imp_dbh and then copy back the original ->com. Also I'd prefer to store in the pool only the really private data. I guess all I need is to know the size of ->com struct with its sub-structs, preferrably at compile time.
3. $dbh->DESTROY. Currently I had to:
SvREFCNT_inc(dbh);
so imp_dbh won't lose it's data when $dbh goes out of scope, I have tried copying it but wasn't very successful. Neither playing with DBIc_FLAGS(imp_dbh) helped, but that's probably because I'm not very familiar with DBI guts. You help is needed here.
4. If you run 'make test' there could be segfaults when perl_destruct is called because DBI is getting confused about what to destroy (probably due to #3 above)
5. Finally, the most important issue is that if a thread logged in for real and created imp_dbh, it must not exit while other threads use the same data. The solution to this is either have to switch context to the parent's perl when doing the real login(), or we can try to memcpy() the private datastructure while being in the parent's context (e.g. by creating an SV), but this all goes back to #3 above. this could be a bottleneck, but I'm not sure.
(as a result if you arrange for the first thread to quit early in test.pl, it'll segfault)
6. Notice that the parent thread creates $obj, which serves as a warehouse for all items (imp_dbh). So as long as DBI::Pool is loaded in the parent thread, this is not an issue. we could easily test for that:
die "must be loaded in the parent thread"
unless threads->self->tid() == 0;7. I think this approach should work for non-thread perl all the same, just need to do some minor tweaks, like installing a NOOP for share() and lock(), or move these into XS protected by #ifdef USE_ITHREADS / #endif
8. and the overriding of connect/disconnect could be done in a better way (I think the current implementation might have problems), but it's good enough for getting started.
9. The current implementation disregards the login/passw/attr and assumes that there is only one pool. Let's tackle each problem at a time. Once previous 8 issues are resolved we can move onto this one.
The DBI::Pool package is attached. To run it, do:
perl Makefile.PL make make test
if 'make' fails, adjust the hardcoded mysql bits in Pool.xs.
'make test' may segfault, that's ok.
__________________________________________________________________ Stas Bekman JAm_pH ------> Just Another mod_perl Hacker http://stason.org/ mod_perl Guide ---> http://perl.apache.org mailto:[EMAIL PROTECTED] http://use.perl.org http://apacheweek.com http://modperlbook.org http://apache.org http://ticketmaster.com
DBI-Pool-0.01.tar.gz
Description: application/gzip
