[ disclaimer: I am being funded to do this work ]
I've been looking into bottlenecks in a tight fetch loop with millions of
rows, e.g.
$sth->bind_columns(...);
while ($sth->fetch) {
# do a small amount of work
}
In the particular case I've been investigating, the method lookup in
XS_DBI_dispatch() contributed a measurable part of the total loop time;
in particular, when as a test I added very crude and hacky caching for the
lookup of the inner fetch method, the execution time for the loop dropped
from 23.0s to 19.2s.
So, I would like to attempt to add a method cache to XS_DBI_dispatch()
(and possibly see if anything else can be tweaked).
My questions are:
* Would such a patch be welcome in principle?
* Any gotchas that I might not be aware of (I'm not familiar with DBI/DBD
internals apart from what I've played with for the last few days).
* What's the earliest perl we have to maintain compatibility with?
* Does the test suite already have good coverage for method lookup
(especially where methods are redefined or stashes modified, i.e. the
sort of thing that would invalidate caches), or should I add some?
* can I rely on _install_method() being the only way that XS CVs can get
to point to XS_DBI_dispatch()?
* To what extent (if any) is DBI allowed to cheat when it comes to
bypassing the perl API (in particular I'm thinking of the method
invalidation stuff which I don't think has always been part of the API).
Thanks,
Dave
--
Please note that ash-trays are provided for the use of smokers,
whereas the floor is provided for the use of all patrons.
-- Bill Royston
