Tim Bunce wrote: [...]
[...]The pool stores passive objects. This pool can be manipulated by any thread (with proper mutex locking).
1. The pool can act as a library, where objects are borrowed (moved from the free list to the busy list) and then returned (moved from busy list to the free list)
2. The pool can act as a shop which sells objects (objects are removed from the free list) and which buys the objects back (at the same price!) (objects are placed on the free list). In this scenario there is no busy list, as the shop manages only the inventory of the items it has for sale and doesn't care about items that were purchased.
#1 pros:
- we know how many total connections are open at any given moment
- if the thread dies and we have GC we can recover the connection
- we can free() the pool memory cleanly (since we control all the memory that was allocated)
#1 cons: ???
#2 pros: - no need to manage used connections (simpler pool management/a bit faster?)
#2 cons:
- we don't know how many total connections are open at any given moment
- if the thread unexpectedly dies, we need to figure out how to free() the memory allocated for the item.
I'd go for #1. I think the pros for #2 are insignificant.
Fine with me.
Now, back to memory management...
As I've mentioned recently, I figure the 'thing' that's placed into the pool is a scalar string that holds a copy of the entire imp data structure. Doing that, rather than passing an integer pointer around, means we avoid issues over which interpreter owns and frees that memory.
That just leaves two other related issues... a. How database API data gets cleaned up. b. Who (which interpreter) owns and frees any SVs _within_ the imp data.
The best way to handle (a) would be to create a new dbh using the imp data and then let the dbh get destroyed 'normally' (not pass the imp data back to the pool).
For (b) we may be able to duck the issue to a certain extent because all we're really trying to pass around is the pointers from the database API. There should be no need to use any SV pointers. On the other hand, any SVs need to get freed at some point. Only the driver know where they are so the driver has to be involved.
Seems like the $dbh->take_imp_data method will need driver involvement to 'cleanse' the imp data by freeing everything not essential to the connection - and hopefully that'll include all the SVs.
If SVs are required, for some reason, can they be made shared, or replaced with a shared copy? (My perl threads api knowledge is a bit rusty.)
There should be no perl datastructures involved at all in the pool storage. If we go with #1, the moment the thread connects for real, it first takes its $dbh to the cloning factory, where the imp_data is cloned (malloc+memcpy) and deposited into the storage (on the busy list). The the thread continues normally. When the threads disconnect()s, it has to do two things:
1. tell the storage that the item is now free. (put_back)
2. destroy everything it has created, but the private imp_data. So imp_data->com should be destroyed. Everything that comes after it inside imp_dbh_st should be left intact.
If this is done the pool management can safely free() the allocated memory of any items on its storage's free list. So we can set the MaxItems and destroy connections if there are more than MaxItems (again only those that are on the free list).
The interesting side effect is that now, DBI can be configured to block, if there aren't enough connection, but no more connections are allowed to be created (e.g., because of db limitations). So the thread that wants a connection can periodically poll the pool, to see if any of the connections became available, rather than return an error that there are no connections available (well, it can fallback to the error, after several attempts/timeout period).
Finally the only fishy thing is the following: Since we memcpy the whole imp_dbh, including the com struct, the memory used by that struct (first n bytes) is invalid after the original dbh has been destroyed and should never be accessed. In fact we may want to null it all. Only the memory past the com struct is valid. We could avoid that (and probably save some memory too) if the driver could provide us a copy of imp_dbh sans the com struct.
In retrospect I wanted to ask why DBI tells driver to define struct imp_xxh_st as:
struct imp_dbh_st {
dbih_dbc_t com; /* MUST be first element in structure */
private element1;
...
private elementN;
}and not:
typedef struct private_data imp_t;
struct imp_dbh_st {
dbih_dbc_t com; /* MUST be first element in structure */
imp_t imp; /* MUST be the second element and the only one */
}where imp_t could have any number of elements, while staying opaque to the DBI frontend.
__________________________________________________________________ Stas Bekman JAm_pH ------> Just Another mod_perl Hacker http://stason.org/ mod_perl Guide ---> http://perl.apache.org mailto:[EMAIL PROTECTED] http://use.perl.org http://apacheweek.com http://modperlbook.org http://apache.org http://ticketmaster.com
