On Fri, Feb 28, 2003 at 10:03:07AM +1100, Stas Bekman wrote:
>
> >But only the driver knows what's in that data and thus what needs to be
> >re-initialised for use in a different thread - like SVs.
>
> Cool, so you suggest that we don't bother to save only the really private
> data, but copy the whole imp_dbh as is and the driver has to supply the
> following function:
>
> int reinstall_imp_dbh(live_imp_dbh, saved_imp_dbh);
>
> which will do the equivalent of:
> live_imp_dbh->mysql = ((imp_dbh_t *)saved_imp_dbh)->mysql;
> as it was in my prototype.
>
> Is that correct?
Nearly. The DBI manages handle creation for the drivers, including
allocating the imp data structure. So if the code that does that
is passed a copy of a 'template' imp data structure it can just
copy it in. In terms of driver changes that just requires one line:
my $dbh = DBI::_new_dbh($drh, {
Name => $dbname,
+ UseImpData => $attr->{UseImpData},
});
It could then set a flag so that when the _login (or whatever it's
called) is invoked it can check for the flag and, if set, just
'tweak' the existing structure n whatever way is neded to make it
thread safe, like clone SVs:
if ( flag not set) {
/* this is the existing _login code: */
imp_dbh->foo_api_thingy = foo_connect(...);
imp_dbh->foo_sv = newSVpv(some_data);
}
else {
/* reuse pre-initialised imp_dbh->foo_api_thingy */
/* clone the sv to create thread safe copy in our interpreter */
imp_dbh->foo_sv = newSVsv(imp_dbh->foo_sv);
}
> >>in my design, each thread does its own connect and then deposit the
> >>object to the pool. No bottleneck whatsoever. Other than the pool
> >>management blocking.
> >
> >
> >Umm [thinks] okay, there's no reason why the connections in the pool
> >can't themselves be created via
> > $dbh = DBI->connect(..., { UseID => $id })
> >where the $id has come from a connection created outside the pool.
> >
> >We just need a way to 'take' imp data instead of 'borrow' it, so
> >the handle that's left behind is effectively dead and won't warn
> >when it's destroyed. That may be as simple as also setting
> >InactiveDestroy.
>
> That will work. What bothers me is that { UseID => $id } in the top level
> API.
>
> This should be transparent to the DBI user. Other than the user telling DBI
> that it wants to use the pooling.
Sure. That's no problem. Apache::DBI is transparent to the user.
I'd expect this to be as well and there's lots of ways to do it.
Not a big issue, but lets get the basics working first.
> Internally the connect should remove the item from the pool and pass it to
> the driver. If the item is not passed the driver does the real connect().
> During $dbh->destroy, the item is deposited to the pool.
Could be done the same was as Apache::DBI. Rebless the handle and
have a custom DESTROY. Umm, interestingly, the imp data struct
itself is blessed by the DBI into a DBD::_mem::db (::dr/::st) class.
Since it's the imp data we're after here we could just rebless the
imp data struct and use our own DESTROY there. Cute.
> >Let's see if I can summarize where we're at (from the DBI prespective)...
> >
> > my $id = $dbh->borrow_imp_data;
> >
> > Sets a flag to inidcate that the imp data is in use by
> > another thread. Returns a token for the imp data, possibly
> > simply the address, but could also be the whole imp data
> > structure. Using the whole struct is appealing because it
> > avoids a race hazard - if borrow_id sets a flag before
> > returning then whichever thread uses the id will see that
> > flag already set.
> >
> > my $id = $dbh->take_imp_data;
> >
> > As above but indicates that the imp data won't be returned
> > and the handle is effectively dead.
> >
> > $dbh->restore_imp_data;
> >
> > Indicates that the thread that borrowed the imp data has
> > finished using it.
> >
> > $dbh = DBI->connect(..., { UseImpData => $id })
> >
> > Creates a new dbh and initialises it with a copy of the given imp
> > data.
> > Also sets a flag to indicate that the imp data is borrowed.
> >
> >Does that give you all the mechanisms you need to build on?
>
> why do we need both, borrow and take? I thought that take will work better,
> because it's both atomic and will handle automatically the case when the
> thread dies before it gets the chance to deposit the item back. The only
> problem I see with take() is the memory leak in the case of dieing thread.
> So perhaps borrow is safer.
>
> borrow is fine too. Currently my implementation does something like:
>
> lock $obj;
> $item = $obj->get if $obj->free_count;
>
> so it's atomic as well.
It dawned on me early this morning that we hadn't been quite on the
same page thus far...
I'd been presuming the existance of a 'pool management thread' that
would manage a pool of real dbh's from which it would 'borrow' the
imp data to 'loan' to other threads requesting connections.
I now realize that the pool is a passive data object and is manipulated
directly by the requesting threads. In that scenario the 'thing' to
put into the pool is just the copy of the imp data structure.
(Think of it like a self-service shop full of brains in jars, you
come along, pick the brain you want, and take it to another shop,
the DBI, where you ask for it to be wrapped up in a new body: $dbh.
When you've finished with the body the brain can go back to the shop.
Umm, methinks I need some coffee!)
So, given the 'passive pool' scenario there's no need for 'borrow'
or 'restore' because there are no handles in the pool to borrow'
from or restore to. (When a newly created dbh has died the hospital,
DESTROY, performs a brain removal and gives it to the shop :)
So, we just need $h->remove_imp_data which can be a pretty savage
*removal* not a copy, leaving $h brain dead.
Are we on the same (slightly blood stained) page now? :-)
Tim.
