On Sun, Jun 19, 2005 at 06:16:25AM -0700, Peter Scott wrote:
> On Sat, 18 Jun 2005 15:34:36 +0100, Tim Bunce wrote:
>
> > On Fri, Jun 17, 2005 at 05:49:01PM -0700, Peter Scott wrote:
> >> The connected() method of subclasses doesn't get all the attribute
> >> that were set by the caller; three of them are deleted first:
> [snip]
> >> I'd rather like to get those attributes in my subclass. Would a
> >> patch be accepted?
> >
> > They're only deleted as they're applied to the handle, so connected()
> > can get the values from the handle.
> >
> >> Likewise - and this is more debatable - the $dsn passed to
> >> connected() has had the driver prefix stripped off so you can't
> >> tell which driver was used.
> >
> > You can get it from $dbh->{Driver}->{Name}
> > (unless you're using DBI_AUTOPROXY, I think).
> >
> >> That information would be most valuable to my application.
> >
> > Why?
>
> I'm writing a subclass designed to ensure that inserts coming in a
> continuous stream get to the database even if the connection is down and
> they have to be buffered on local disk pending retries. (It only has to
> handle inserts and the transaction issues are understood.) So my subclass
> is going to fire off a subprocess that does the actual connection to the
> database and reconnects whenever necessary, while the parent writes insert
> statements to files, looking to the user like a regular DBI interface
> (although it will claim that all do()s succeed, will signal fatal errors
> through a SIGCHLD handler croaking, and will probably croak on a fetch).
> (Furthermore, I'll have to timeout connects and do()s in a couple of
> seconds due to the data rate.) It's my first DBI subclass and I've only
> just started, so I may have to revise that design some, but that's the
> current idea. Therefore I'm planning on capturing the connect info from
> connected() before the fork.
Seems a little over the top to try to keep the DBI abstraction,
especially as data volumes and performance are significant here.
I'd probably just modify the application to write delimited files
and have a separate process handle the loading.[*]
> >> It does seem from reading the documentation
> >> as though the expectation is that connected() will get the same
> >> arguments as connect(), so would a patch for this be accepted also?
> >
> > A documentation patch? :)
>
> That wasn't what I had in mind :-) What was the original intent? Do
> you not want to change the behavior because of legacy expectations, or
> have I missed something in the reasoning behind providing connected()?
I'm reluctant to change, but the legacy expectations of the connected()
method are low so I'm open to persuasion.
Tim.
[*] Which is exactly what I've done at work on a system handling
thousands of records per second. The files have a header giving all the
info required by the loader (including connection details and the
schema for the table to create if it doesn't already exist).
The data is then loaded into MySQL using LOAD DATA ... IGNORE n LINES
where n is the number of lines in the header. Using bulk loading
this way is naturally much faster than the DBI to load each row.
The loader deamon (or daemons as there's a pool of them) are autonomous
and can load any data into any table in any database so long as the data
file has the right format and header and is placed into the spool
directory.
