First edition misdirected in error.
---------- Forwarded message ----------
From: Jonathan Leffler <[EMAIL PROTECTED]>
Date: Thu, 9 Sep 2004 00:07:12 -0700
Subject: Re: interface ideas for non-blocking mode
To: David Nicol <[EMAIL PROTECTED]>
Cc: [EMAIL PROTECTED]
On Wed, 8 Sep 2004 14:39:45 -0500, David Nicol <[EMAIL PROTECTED]> wrote:
> Right. And when your driver is providing a DBI interface to one of
> them, to provide
> a fileno() function that works, you may need to open a pipe and write
> to one end of it when you are expecting some data. [...]
We're talking past each other and not connecting. Probably too much
asynchronous communication between incompatible database systems :-(
Or we have different plans for what DBI is and/or should be -- I've
had my disagreements with Tim over this in the past, and can see that
it may happen again in the future.
I've gone back to what I believe is the first of your postings (27th
August - though the email header has 28th August just before 01:00 in
US/Eastern or maybe US/Central time zone; -05:00). It's a long
posting. In it, you say:
==> In the application I am currently working on, I want prepare and
==> execute to return as soon as they can, even if error reports will get
==> deferred, and I would like to know if data is ready before attempting
==> to fetch a row, so I don't have to wait for the server to send it.
These perceived requirements obviously colour your opinion on what is
required - that's OK; Open Source software gets written when it
scratches someone's itch.
==> Yes this class of issues can be trivially solved by demanding threading,
==> but that does not help when a(n unrealistic?) design constraint limits
==> you to one thread.
This worries me. I really think that asynchronous support via
threading is more sensible. It is also more likely to be widely
achievable - IMNSHO. However, it is your itch, so you get to lay out
your desires. However, be aware that not everyone suffers from the
same sources of irritation as you do.
==> If pointed to the simplest TCP-based driver, I will cheerfully
==> add this stuff to demonstrate that its practical.
Simple TCP drivers aren't the problem. It is the non-simple drivers
with closed source where there is *NO* access to the file handles that
are used internally, or where no file handles are used at all, and
whether or not they are used is irrelevant since there is no access to
the underlying transport layers regardless of what is used.
Demonstrate how it will be done in DBD::Oracle -- I'd prefer it to be
DBD::Informix, but that's probably a tad too partisan to be fair.
What I don't see in your discussion is how you would scope
asynchronicity. Is it at the $dbh level - so that the DBI->connect
method takes a new attribute that says { Asynchronous => 1 }. Or is
it something that is at the statement level, so $dbh->prepare method
has to take the flag. Somewhere along the line, you have to indicate
your desire for asynchronicity -- you cannot add it to everything
without breaking a lot of happily working applications.
==> Working out the implementation is not our problem yet (well it is our
==> problem if we're maintaining a driver, but it's not our problem if we're
==> wearing "Lords Of The DBI Specification" hats)
The Lords of the DBI Specification may NOT mandate features that
cannot be readily be supported by the vast majority of DBD authors
without impacting their entire lives for a year or two. It won't
work. I have zero time to work on DBD::Informix as it is. Putting
massive obstacles in the way of compliance with a revised
specification is nonsense.
I proposed a scroll cursor mechanism a couple of years ago (at least),
and outlined a DBMS-agnostic fallback mechanism written in pure Perl
that, although not particularly pretty or fast, would have worked.
But, I lacked the time to prosecute it through the DBI spec. However,
it was made clear to me that the feature would not become a part of
DBI without a fallback mechanism in place that DBI could provide for
those drivers not capable of providing an override mechanism. I'd be
happier if you showed how the mechanism would be provided
automatically for those drivers that cannot provide it themselves --
then you'll get my blessing. But while the imposition is on the DBD
authors to jump through your hurdles, you will get resistance all the
way from me.
> The idea of a common interface is that it provides a standard way for
> the code that uses the interface to request the feature.
I'm not ready to sign up for that. I disagree with the starting
point. DBI should enable access to databases, but it should not
demand that DBD authors write gobs of code to meet requirements that
their DBMS does not support. I prefer a 'more-or-less lowest common
denominator' approach. Everyone else seems to prefer a "let's drag
the whole world through contortions to make everything work everywhere
approach". Granted, Microsoft does this -- I don't regard that as a
good model for Open Source projects, though.
> When a query comes back empty, ready should return 0but_true because
> there are zero rows to fetch. That's something new.
>
> When a driver is an interface around one or more streams, the file
> descriptor number(s) of
> that (those) streams can be passed back with the fileno method, which
> will be a wrapper
> around the core C<fileno> function. In other situations such as the
> ones you list, creating
> a pipe that will have a file descriptor number, just for communication
> between the driver
> and the application, is not difficult.
Just for the record, even though Informix ESQL/C uses sockets or TLI
widgets or pipes or other file descriptors internally, there is no
access at the programming level to them. You are confusing Open
Source applications - where the library details are available to you -
with Closed Source where the library details are not available to you.
DBD::Informix, and other drivers such as DBD::Oracle, are constrained
by the API which the closed source company provides.
I have no intention whatsoever of providing a bolt-on piece of
gizmotronnery, opening pipes etc.
If the DBI default mechanisms provide such support, great. If not,
then support won't be forthcoming from the current maintainer of
DBD::Informix -- alternative maintainers are sought but have been for
about 5 years now with no-one showing any signs of interest.
> Originally I thought, all polling would be done in the ready() method,
> which would not
> block. A request was made to provide a file descriptor for use in an
> external event loop,
> and this resulted in the extension of the fileno() method.
In my totally biassed and not desparately cooperative opinion, it
can't be done sensibly for Informix. Or, I don't want to do it. Or
something along those lines.
> A fully asynchronous driver obviously doesn't need any CPU from the
> application, which
> could result in a situation where ready() would return true but there
> has never been a
> request for an event. So, we define the fileno() method in such a way
> that a fully
> asynchronous driver can mock-up a ready event to send to applications
> that are doing
> event loops, to maintain the interface. In the code I submitted to
> the list, this was
> accomplished by writing a lower-case 'a' to the pipe.
>
> The C language, AFAIK, does not distringuish between file handles and
> socket handles.
Nope; you're right - they're just small integers (for varying
definitions of small - but the values normally fit in a 16-bit signed
quantity).
> Just to be clear, if my proposal to support nonblocking modes by
> defining "ready"
> and "fileno" methods into the DBI specification is accepted, a driver
> that is providing
> a deferred asynchronous interface to a database that does not use any
> sort of file handle in the communication process will have to create
> an anonymous pipe for the purpose of indicating the desire to be
> polled to an application with an event loop. A
> fully asynchronous driver will have to do that too.
Then I'm not ready to accept your proposal. I may lose the battle -
we have AutoCommit in the product and it still stinks in my opinion,
so I've lost before, too. However, this could be the one that breaks
the camel's back. Quite easily.
> The semantics of the fileno FD set are as follows:
>
> * it MAY be readable at any time
>
> * it MAY become unreadable during readiness
>
> * it MUST become readable after C<execute>, and remain readable until
> C<ready> has been called before ready returns a true value
>
> * it MUST become readable after C<ready> has returned false, and
> remain readable until C<ready> has been called to return true.
>
> * readability DOES NOT IMPLY readiness
>
> * readiness DOES NOT IMPLY readability
I'm not at all sure I understand this. It appears to be English, but
the meanings of the words aren't (or don't seem to be) the meanings I
normally associate with them, even in a programming context. You talk
about an 'FD fileno set' in the singular, but at some point, you need
to specify what it means in terms of individual members of the set --
unless you are assuming that the set always has at most one member.
> The reference model is a defered asynchronous driver to a database connected
> via a stream. Drivers to databases that do not use streams for the
> communication,
> and fully asynchronous drivers, are required to pretend that they are
> deferred-asynchronmous drivers that connect via a stream, by providing one or
> more stream descriptors that can be used in an event loop. [...snip...]
Midnight - the witching hour. Time for good little DBD authors to go
to sleep. (Walk the dog, potty the kids, shut up shop, attempt to get
some sleep).
I'm sorry this sounds so curmudgeonly -- it probably is.
Nevertheless, I do feel that you are imposing unnecessary workloads on
driver writers because you have a particular constraint imposed on you
that is not especially necessary to the larger world.
If you come up with a proposal that has a viable default mechanism
that can be provided by DBI and overridden by DBD authors who have
more capable DBMS APIs, then that's one thing -- you can do that as
far as I am concerned. But if DBD authors have to be overloaded with
fiendish work just to meet the requirements of an extreme minority, I
will be obliged to use my (probably extremely limited) powers to veto
the suggestion. That which is easy is acceptable; that which is hard
is not.
--
Jonathan Leffler <[EMAIL PROTECTED]> #include <disclaimer.h>
Guardian of DBD::Informix - v2003.04 - http://dbi.perl.org
"I don't suffer from insanity - I enjoy every minute of it."
--
Jonathan Leffler <[EMAIL PROTECTED]> #include <disclaimer.h>
Guardian of DBD::Informix - v2003.04 - http://dbi.perl.org
"I don't suffer from insanity - I enjoy every minute of it."
