Re: execute_array enhancement to DBI and request for help for someone who knows DBD::Oracle or XS
What I need assistance with is to change DBD::Oracle's execute_for_fetch to understand array context so it can return the tuples executed and the rows affected (which it already has). At the same time we could fix the bug I reported in DBD::Oracle so in scalar context it returns the tuples executed instead of rows affected. Martin, I'm not familiar with the particular code in question, but the construct you're probably interested in is the 'GIMME' macro. It evaluates to the constant G_SCALAR in scalar or void context and G_ARRAY in array context. If you need to differentiate between void and scalar, use GIMME_V. These are documented in perlapi(1). Steve -- Opinions expressed in this message are mine personally, not those of my employer.
Problem with 'Username' attribute
Tim,
I'm working on qualifying perl-5.8.3 and DBI-1.46 for a critical
application which uses DBI. We've been running with perl-5.8.0 + DBI-1.35
for quite a while (color me conservative when my job's at stake).
After fighting my way through some broken logic in 5.8.0 Storable, which
prevents it from ever being able to accept uplevel data (bug persists in
5.8.3, btw), I'm running into clientside complaints of:
DBD::Proxy::db STORE failed: Server returned error:
Failed to execute method CallMethod:
Can't set DBI::ProxyServer::db=HASH(0x303d1cd8)-{Username}:
unrecognised attribute or invalid value at
/afs/btv.ibm.com/data/a29v/test/contrib/noarch/lib/perl5/site_perl/5.8.0/RPC/PlServer.pm
line 332.
I found a cryptic reference in the DBI sources entreating developers to
update their drivers in order to accomodate 'Username'. Unfortunately,
it's not clear to me precisely what needs to get upgraded nor why.
Given your general advocacy for backwards compatibility, I'm surprised by
this situation.
What needs to be updated to get rid of the message? Is there really no
way to interoperate with the older DBI code on the remote end without this
error message (which seems to slow things down to boot)?
Steve
p.s. - To make Storable do what it claims and properly obey the
$Storable::accept_future_minor variable, this needs to happen (apparently
not fixed in 5.8.3 either):
Fix problem where forward compat checking logic never tries to read
its setting and always barfs when newer client comes knocking...
--- perl-5.8.0/ext/Storable/Storable.xs.origSat Nov 13 17:27:53 2004
+++ perl-5.8.0/ext/Storable/Storable.xs Sat Nov 13 17:28:11 2004
@@ -1092,6 +1092,7 @@
cxt-netorder = 0; /* true if network order used */
cxt-forgive_me = -1; /* whether to be forgiving... */
+cxt-accept_future_minor = -1; /* would otherwise occur too late */
}
/*
Re: DBI version 2
On Mon, 12 Jan 2004, Tim Bunce wrote: Plans: 1. Move DBI source code to http://sourceforge.net/projects/dbi/ 2. Add a few dbi-dev people as developers who can modify the source. Tim, Pending permission from my employer, I'd like to volunteer as maintainer for the DBI::Proxy. Let me know your thoughts? If you think it makes sense, I'll start the process of getting approval (things are a bit tense with regard to Open Source participation, as you might imagine). Steve -- Steven N. Hirsch tie-line: 446-6557 ext: 802-769-6557 Staff Engineer Methodology Integration Team ASIC Product Development IBM Microelectronics
Re: Question re: signal handling
On Sat, 10 Jan 2004, Lincoln A. Baxter wrote: While this text is also in the perl 5.6.1 perlvar man page, I think that perl 5.6.0 does not infact use SA_RESTART (or at least for some reason our build of it did not). Running truss -v all on Solaris 5.8 (sun4u hardware), reveals that our perl 5.6 does not pass SA_RESTART to the sigaction routine, and perl 5.8.x does. Ok, the light just went on... One must qualify the value of 'x' when speaking of perl 5.8.x! In 5.8.0, use of SA_RESTART is a compile-time choice (not supported, apparently, by the Configure scripts), which explains why I'm not seeing the desired semantics. In 5.8.1 and 5.8.2 it is, as you've inferred, a function of your runtime environment (PERL_SIGNALS=unsafe). This begs the further question of why the notion of restartable system calls is tied so closely to Perl's newer safe signal semantics. To me, these are orthogonal notions. The principle of least surprise would suggest that they be decoupled, unless I'm overlooking the obvious. Steve
Question re: signal handling
Tim, I've run into an odd problem using the DBI::Proxy. In Programming Perl - 3rd Edition, the claim is made that Perl will always try to restart interrupted slow system calls if POSIX sigaction() is supported. Our experiences are suggesting that this is not correct. We have an application which connects over the Proxy and forks children (which also establish their own connections - this is another can-of-worms for future discussion). Unfortunately, the PlClient dies (with EINTR) if the DBI is in read() or write() at the time a child terminates. I can see clearly that the code (Comm.pm) is not trying to handle this case, believing (as I did) the claim of Perl doing the right thing and restarting the call. Before I tear into it, I want to make sure I have come to the correct conclusion. Certainly, perl-5.8.0 is not using SA_RESTART anymore, but there is other magic going on in support of safe interrupts which I do not fully understand. What is your take on this? Regards, Steve -- Steven N. Hirsch tie-line: 446-6557 ext: 802-769-6557 Staff Engineer Methodology Integration Team ASIC Product Development IBM Microelectronics
Re: RC2 (was: Announce: DBI 1.38 release candidate)
On Thu, 21 Aug 2003, Tim Bunce wrote:
On Wed, Aug 20, 2003 at 03:49:18PM +0100, Tim Bunce wrote:
I'd be grateful if you could take a moment to try out this
release candidate of DBI 1.38:
Thanks for all the feedback. Here's another. I've changed a few
small things including enabling better testing of Driver.xst.
I'm doing another release candidate because I suspect that might
generate some compiler warings on some platforms.
Same routine...
http://homepage.eircom.net/~timbunce/DBI-1.38-rc2-20030821.tar.gz
Tim,
I plan to beat on this over the next few days. However, here are two
small changes which I've found to be necessary when working with a DB2
backend database. Since making these mods all ugly messages are a thing
of the past.
In my opinion, it's time to throw in the towel and report the true
'Active' status back to the proxy client with every call. I'm getting
hammered at work, but will attempt to code this up Real Soon Now g.
Steve
--- DBI-1.38/lib/DBD/Proxy.pm.orig Tue Aug 19 20:15:28 2003
+++ DBI-1.38/lib/DBD/Proxy.pm Thu Aug 28 09:44:22 2003
@@ -352,8 +352,10 @@
$sth-{'proxy_sth'} = $rsth;
# If statement is a positioned update we do not want any readahead.
$sth-{'RowCacheSize'} = 1 if $stmt =~ /\bfor\s+update\b/i;
-# Since resources are used by prepared remote handle, mark us active.
-$sth-SUPER::STORE(Active = 1);
+ # In a perfect world, this would be correct. However, until a clean
+ # universal solution is implemented for keeping the client in sync
+ # with the backend database, refrain from making a leap of faith.
+ # $sth-SUPER::STORE(Active = 1);
}
$sth;
}
@@ -577,7 +579,9 @@
sub finish ($) {
my($sth) = @_;
-return 1 unless $sth-SUPER::FETCH('Active');
+# For reasons remarked upon in disconnect() above, let the
+# user decide when to hit the wire.
+# return 1 unless $sth-SUPER::FETCH('Active');
my $rsth = $sth-{'proxy_sth'};
$sth-SUPER::STORE('Active' = 0);
return 0 unless $rsth; # Something's out of sync
--
Steven N. Hirsch tie-line: 446-6557 ext: 802-769-6557
Staff Engineer Methodology Integration Team
ASIC Product Development IBM Microelectronics
Re: RC2 (was: Announce: DBI 1.38 release candidate)
On Thu, 28 Aug 2003, Tim Bunce wrote:
On Thu, Aug 28, 2003 at 09:50:21AM -0400, Steven N. Hirsch wrote:
Tim,
I plan to beat on this over the next few days. However, here are two
small changes which I've found to be necessary when working with a DB2
backend database. Since making these mods all ugly messages are a thing
of the past.
In my opinion, it's time to throw in the towel and report the true
'Active' status back to the proxy client with every call.
I'd *really* like to know under what circumstances the Active status
gets out of sync.
It seems related to stored procedure calls, IIRC. I fought with this
several months ago and ended up with the hacks below. My problems are
very likely due to inconsistancies between DB2 CLI behavior and what one
would think was the obvious g.
I'm getting
hammered at work, but will attempt to code this up Real Soon Now g.
Any patch to report the server-side active status needs to a) gracefully
handle client-server version mismatches, and b) ideally implement a more
flexible protocol in general so that new features can be added and version
mismatches more gracefully handled in future.
I agree 100%. The existing communications protocol is a fragile,
hard-to-maintain mess. This is why I've been putting off a permanent fix.
Do you agree that the move to something more elegant should involve a
different set of method names? That's the only way to avoid breaking
older code (unless I'm missing the obvious).
Steve
Tim.
Steve
--- DBI-1.38/lib/DBD/Proxy.pm.orig Tue Aug 19 20:15:28 2003
+++ DBI-1.38/lib/DBD/Proxy.pm Thu Aug 28 09:44:22 2003
@@ -352,8 +352,10 @@
$sth-{'proxy_sth'} = $rsth;
# If statement is a positioned update we do not want any readahead.
$sth-{'RowCacheSize'} = 1 if $stmt =~ /\bfor\s+update\b/i;
-# Since resources are used by prepared remote handle, mark us active.
-$sth-SUPER::STORE(Active = 1);
+ # In a perfect world, this would be correct. However, until a clean
+ # universal solution is implemented for keeping the client in sync
+ # with the backend database, refrain from making a leap of faith.
+ # $sth-SUPER::STORE(Active = 1);
}
$sth;
}
@@ -577,7 +579,9 @@
sub finish ($) {
my($sth) = @_;
-return 1 unless $sth-SUPER::FETCH('Active');
+# For reasons remarked upon in disconnect() above, let the
+# user decide when to hit the wire.
+# return 1 unless $sth-SUPER::FETCH('Active');
my $rsth = $sth-{'proxy_sth'};
$sth-SUPER::STORE('Active' = 0);
return 0 unless $rsth; # Something's out of sync
--
Steven N. Hirsch tie-line: 446-6557 ext: 802-769-6557
Staff Engineer Methodology Integration Team
ASIC Product Development IBM Microelectronics
--
Steven N. Hirsch tie-line: 446-6557 ext: 802-769-6557
Staff Engineer Methodology Integration Team
ASIC Product Development IBM Microelectronics
Fix for PlClient
Tim,
Not sure if you are currently maintaining this or not, but PlClient.pm
really needs to call RPC::Comm::Init() prior to any conversation with the
server. Without this patch, if you have specified 'compression = gzip',
the client will try to call Compress::Zlib::compress() without ever having
'require'-ed Compress::Zlib.
Steve
--- PlRPC-0.2016/lib/RPC/PlClient.pm.orig 2001-09-30 22:38:40.0 -0400
+++ PlRPC-0.2016/lib/RPC/PlClient.pm2003-03-09 13:34:37.0 -0500
@@ -52,6 +52,8 @@
my $self = [EMAIL PROTECTED];
bless($self, (ref($proto) || $proto));
+$self-RPC::PlServer::Comm::Init();
+
my $app = $self-{'application'} or
$self-Fatal(Missing application name);
my $version = $self-{'version'} or
@@ -86,7 +88,6 @@
die Refused by server: $msg unless $reply-[0];
$self-Debug(Logged in, server replies: $msg);
-$self-RPC::PlServer::Comm::Init();
return ($self, $msg) if wantarray;
$self;
}
Re: Taint tests disabled in DBI-1.31
On Sat, 30 Nov 2002, Bradley Baetz wrote:
Tim,
The tests for the new Taint stuff which was added appear to be disabled
in 1.31 - line 227 has
if (0 is_tainted($^X) !$DBI::PurePerl) {
Was there a reason for this? Thats not in the final pach which I sent,
and the tests pass for me.
Is 1.31 released? If not, where can I grab a preview?
Things missing from DBI-1.31
Tim,
The following bits of my Proxy patches seem to be missing from the
released DBI-1.31. I can assure you that localizing the 'die' handler is
quite necessary to keep client behavior the same between direct connect
and proxy connect. If you no longer have my message with the original
rationale, let me know and I can resend.
Regards,
Steve
--- DBI-1.31/lib/DBD/Proxy.pm.orig Fri Oct 11 12:13:45 2002
+++ DBI-1.31/lib/DBD/Proxy.pm Wed Sep 18 09:56:20 2002
@@ -234,6 +234,7 @@
'type' = $type,
'h' = DBI::_::$type
);
+local $SIG{__DIE__} = 'DEFAULT';
my $method_code = UNIVERSAL::can($expand{'h'}, $method) ?
q/package ~class~;
sub ~method~ {
@@ -283,6 +284,7 @@
}
if ($type eq 'remote' || $type eq 'cached') {
+local $SIG{__DIE__} = 'DEFAULT';
my $result = eval { $dbh-{'proxy_dbh'}-STORE($attr = $val) };
return DBD::Proxy::proxy_set_err($dbh, $@) if $@; # returns undef
$dbh-{$attr} = $val if $type eq 'cached';
@@ -302,6 +304,7 @@
return $dbh-SUPER::FETCH($attr) unless $type eq 'remote';
+local $SIG{__DIE__} = 'DEFAULT';
my $result = eval { $dbh-{'proxy_dbh'}-FETCH($attr) };
return DBD::Proxy::proxy_set_err($dbh, $@) if $@;
return $result;
@@ -320,6 +323,7 @@
if ( $proto_ver 1 ) {
$sth-{'proxy_attr_cache'} = {cache_filled = 0};
my $rdbh = $dbh-{'proxy_dbh'};
+ local $SIG{__DIE__} = 'DEFAULT';
my $rsth = eval { $rdbh-prepare($sth-{'Statement'}, $sth-{'proxy_attr'},
undef, $proto_ver) };
return DBD::Proxy::proxy_set_err($sth, $@) if $@;
return DBD::Proxy::proxy_set_err($sth, Constructor didn't return a handle:
$rsth)
@@ -355,7 +359,7 @@
#$dbh-{'proxy_quote'} = 'backslash_escaped';
# for example.
# Jochen
-
+local $SIG{__DIE__} = 'DEFAULT';
my $result = eval { $dbh-{'proxy_dbh'}-quote(@_) };
return DBD::Proxy::proxy_set_err($dbh, $@) if $@;
return $result;
@@ -365,6 +369,7 @@
my $dbh = shift;
my $rdbh = $dbh-{'proxy_dbh'};
#warn table_info(@_);
+local $SIG{__DIE__} = 'DEFAULT';
my($numFields, $names, $types, @rows) = eval { $rdbh-table_info(@_) };
return DBD::Proxy::proxy_set_err($dbh, $@) if $@;
my $sth = DBI::_new_sth($dbh, {
@@ -395,6 +400,7 @@
sub type_info_all {
my $dbh = shift;
+local $SIG{__DIE__} = 'DEFAULT';
my $result = eval { $dbh-{'proxy_dbh'}-type_info_all(@_) };
return DBD::Proxy::proxy_set_err($dbh, $@) if $@;
return $result;
@@ -450,6 +456,7 @@
my ($numRows, @outData);
+local $SIG{__DIE__} = 'DEFAULT';
if ( $proto_ver 1 ) {
($numRows, @outData) = eval { $rsth-execute($params, $proto_ver) };
return DBD::Proxy::proxy_set_err($sth, $@) if $@;
@@ -527,6 +534,7 @@
die Attempt to fetch row without execute;
}
my $num_rows = $sth-FETCH('RowCacheSize') || 20;
+ local $SIG{__DIE__} = 'DEFAULT';
my @rows = eval { $rsth-fetch($num_rows) };
return DBD::Proxy::proxy_set_err($sth, $@) if $@;
unless (@rows == $num_rows) {
@@ -559,6 +567,7 @@
? $sth-{'proxy_no_finish'}
: $sth-FETCH('Database')-{'proxy_no_finish'};
unless ($no_finish) {
+local $SIG{__DIE__} = 'DEFAULT';
my $result = eval { $rsth-finish() };
return DBD::Proxy::proxy_set_err($sth, $@) if $@;
return $result;
@@ -581,6 +590,7 @@
if ($type eq 'remote') {
my $rsth = $sth-{'proxy_sth'} or return undef;
+local $SIG{__DIE__} = 'DEFAULT';
my $result = eval { $rsth-STORE($attr = $val) };
return DBD::Proxy::proxy_set_err($sth, $@) if ($@);
return $result;
@@ -610,6 +620,7 @@
if ($type ne 'local') {
my $rsth = $sth-{'proxy_sth'} or return undef;
+local $SIG{__DIE__} = 'DEFAULT';
my $result = eval { $rsth-FETCH($attr) };
return DBD::Proxy::proxy_set_err($sth, $@) if $@;
return $result;
--- DBI-1.31/lib/DBI/ProxyServer.pm.origTue Oct 29 05:00:57 2002
+++ DBI-1.31/lib/DBI/ProxyServer.pm Tue Sep 17 13:20:02 2002
@@ -220,7 +220,7 @@
my $msg = $@;
undef $dbh-{'private_server'};
if ($msg) {
- $server-Error($msg);
+ $server-Debug(CallMethod died with: $@);
die $msg;
} else {
$server-Debug(CallMethod: = . join(,, @result));
Re: DBI 1.31 release?
On Thu, 21 Nov 2002, Tim Bunce wrote: On Thu, Nov 21, 2002 at 02:07:48AM +1100, Bradley Baetz wrote: Are there plans to have a 1.31 release sometime in the near future? I'd like to move Bugzilla over to using DBI properly, but that needs my Taint patch which was applied a couple of months back. It would be a lot simpler if I could point people to a release rather than having to get them to apply my patch manually. Hopefully before the end of next week. Maybe sooner. (Maybe not :) Tim, If a release of 1.31 is truly imminent, can we close on the last suggested patch I sent for Proxy? At the time you were too snowed under to look at it, but it's proved to be important. This is the change which localizes the __DIE__ handler around a number of remote calls on the client side. After applying it here, I have been using your strawman 1.31 release in production for the past month or so. Let me know if you want me to re-send the patch or rationale for same? Steve
Re: DBI 1.31 release?
Cool!
Is there somewhere I can grab a tarball to beat on prior to release?
Steve
On Thu, 21 Nov 2002, Tim Bunce wrote:
It's in. I've appended the current change list (but there's more to add yet).
Thanks for the feedback.
Tim.
The proxy now supports non-lazy (synchronous) prepare, positioned
updates (for selects containing 'for update'), PlRPC config set
via attributes, and accurate propagation of errors, all thanks
to Steven Hirsch (plus a minor fix from Sean McMurray and doc
tweaks from Michael A Chase).
The DBI_AUTOPROXY env var can now hold the full dsn of the proxy driver
plus attributes, like dbi:Proxy(proxy_foo=1):host=
Added TaintIn TaintOut attributes to give finer control over
tainting thanks to Bradley Baetz.
The fetchall_arrayref method, when called with a $maxrows parameter,
no longer gives an error if called again after all rows have been
fetched. This simplifies application logic when fetching in batches.
Also added batch-fetch while() loop example to the docs.
The connect() RaiseError/PrintError message now includes the username.
Changed last handle unknown or destroyed warning to be a trace message.
Updated dbish/DBI::Shell to Tom Lowery's much improved version.
Updated some minor doc issues thanks to H.Merijn Brand.
Updated Makefile.PL example in DBI::DBD thanks to KAWAI,Takanori.
Documented $dbh = $sth-{Database} attribute.
Documented $dbh-connected(...) post-connection call when subclassing.
Fixed execute_array() example thanks to Peter van Hardenberg.
The CursorName attribute now defaults to undef and not an error.
Removed forced requirement for drivers to implement DESTROY methods.
DBI::Profile changes:
Setup fixes thanks to Sam Tregar.
Added $DBI::err (etc) tied variable lookup time to profile
Added time for DESTROY method into parent handles profile (used to be ignored)
Changes for driver authors, not required but strongly recommended:
Change DBIS to DBIc_DBISTATE(imp_xxh) [or imp_dbh, imp_sth etc]
Change DBILOGFP to DBIc_LOGPIO(imp_xxh) [or imp_dbh, imp_sth etc]
Any function from which all instances of DBIS and DBILOGFP are
removed can also have dPERLINTERP removed (a good thing).
All use of the DBIh_EVENT* macros should be removed.
On Thu, Nov 21, 2002 at 07:50:47AM -0500, Steven N. Hirsch wrote:
On Thu, 21 Nov 2002, Tim Bunce wrote:
On Thu, Nov 21, 2002 at 02:07:48AM +1100, Bradley Baetz wrote:
Are there plans to have a 1.31 release sometime in the near future? I'd
like to move Bugzilla over to using DBI properly, but that needs my
Taint patch which was applied a couple of months back. It would be a lot
simpler if I could point people to a release rather than having to get
them to apply my patch manually.
Hopefully before the end of next week. Maybe sooner. (Maybe not :)
Tim,
If a release of 1.31 is truly imminent, can we close on the last suggested
patch I sent for Proxy? At the time you were too snowed under to look at
it, but it's proved to be important.
This is the change which localizes the __DIE__ handler around a number of
remote calls on the client side. After applying it here, I have been
using your strawman 1.31 release in production for the past month or so.
Let me know if you want me to re-send the patch or rationale for same?
Steve
--
Steven N. Hirsch tie-line: 446-6557 ext: 802-769-6557
Staff Engineer Methodology Integration Team
ASIC Product Development IBM Microelectronics
Re: Corrections to 1.31 Proxy new 'die' handling
On Wed, 18 Sep 2002, Steven N. Hirsch wrote:
Tim,
Did you ever have a chance to look over my postings relative to this?
Never heard a response.
Steve
I spent some time further quantifying the 'die' handling issue. It looks
as if it might be sufficient to locally clear the SIG{__DIE__} handler in
any blocks where where we eval calls through the proxy client. Let me
know what you think of the proposed patches? I'm not completely convinced
that it's necessary in the FETCH and STORE handlers, for example.
Also, there were two small items not present in your DBI-1.31 strawman,
one suppressed logging of failed calls on the server in the absence of
--debug. The other is a thinko that doesn't work (and is no longer
needed, given your fix to common_set_err()).
(snip)
--
Steven N. Hirsch tie-line: 446-6557 ext: 802-769-6557
Staff Engineer Methodology Integration Team
ASIC Product Development IBM Microelectronics
Re: Issues resulting from server-side 'die'
On Tue, 17 Sep 2002, Steven N. Hirsch wrote:
Ugh. That's:
# Global error handler for entire app.
$SIG{__DIE__} = sub { $dbh-rollback };
not ENV... (Too much coffee today)
--
Steven N. Hirsch tie-line: 446-6557 ext: 802-769-6557
Staff Engineer Methodology Integration Team
ASIC Product Development IBM Microelectronics
DBI::Proxy patches (summary)
Tim,
Just to be sure we are levelset, here is the current set of diffs for my
changes against DBI-1.30, followed by two of your fixes which are assumed
to also have been applied.
Regards,
Steve
-- Cut Here -
Index: ACT/hirschs/perl_modules/DBI/lib/DBD/Proxy.pm
diff -u lib/DBD/Proxy.pm:1.1.1.1 lib/DBD/Proxy.pm:1.11
--- lib/DBD/Proxy.pm:1.1.1.1Tue Sep 3 10:07:13 2002
+++ lib/DBD/Proxy.pmThu Sep 12 20:37:11 2002
-58,16 +58,23
$drh;
}
+sub proxy_set_err {
+ my ($h,$errmsg) = _;
+ my ($err,$state) =
+($errmsg =~ s/ \[err=(.*?),state=(.*?)\]//) ? ($1,$2) : (1,'');
+ return DBI::set_err($h, $err, $errmsg, $state);
+}
+
package DBD::Proxy::dr; # == DRIVER ==
$DBD::Proxy::dr::imp_data_size = 0;
sub connect ($$;$$) {
-my($drh, $dsn, $user, $auth)= _;
+my($drh, $dsn, $user, $auth, $attr)= _;
my($dsnOrig) = $dsn;
-my %attr;
+my %attr = %$attr;
my ($var, $val);
while (length($dsn)) {
if ($dsn =~ /^dsn=(.*)/) {
-107,30 +114,56
if ($) { $err .= Cannot create usercipher object: $@.; }
}
-return DBI::set_err($drh, 1, $err) if $err; # Returns undef
+return DBD::Proxy::proxy_set_err($drh, $err) if $err; # Returns undef
+my %client_opts = (
+ 'peeraddr' = $attr{'hostname'},
+ 'peerport' = $attr{'port'},
+ 'socket_proto' = 'tcp',
+ 'application'= $attr{dsn},
+ 'user' = $user || '',
+ 'password' = $auth || '',
+ 'version'= $DBD::Proxy::VERSION,
+ 'cipher' = $cipherRef,
+ 'debug' = $attr{debug} || 0,
+ 'timeout'= $attr{timeout} || undef,
+ 'logfile'= $attr{logfile} || undef
+ );
+# Options starting with 'proxy_rpc_' are forwarded to the RPC layer after
+# stripping the prefix.
+while (my($var,$val) = each %attr) {
+ if ($var =~ s/^proxy_rpc_//) {
+ $client_opts{$var} = $val;
+ }
+}
# Create an RPC::PlClient object.
-my($client, $msg) = eval { RPC::PlClient-new(
- 'peeraddr' = $attr{'hostname'},
- 'peerport' = $attr{'port'},
- 'socket_proto' = 'tcp',
- 'application' = $attr{dsn},
- 'user' = $user || '',
- 'password' = $auth || '',
- 'version' = $DBD::Proxy::VERSION,
- 'cipher'= $cipherRef,
- 'debug' = $attr{debug} || 0,
- 'timeout' = $attr{timeout} || undef,
- 'logfile' = $attr{logfile} || undef
-) };
+my($client, $msg) = eval { RPC::PlClient-new(%client_opts) };
-return DBI::set_err($drh, 1, Cannot log in to DBI::ProxyServer: $)
+return DBD::Proxy::proxy_set_err($drh, Cannot log in to DBI::ProxyServer: $)
if $; # Returns undef
-return DBI::set_err($drh, 1, Constructor didn't return a handle: $msg)
+return DBD::Proxy::proxy_set_err($drh, Constructor didn't return a handle: $msg)
unless ($msg =~ /^((?:\w+|\:\:)+)=(\w+)/); # Returns undef
$msg = RPC::PlClient::Object-new($1, $client, $msg);
+my $max_proto_ver;
+my ($server_ver_str) = eval { $client-Call('Version') };
+if ( $ ) {
+ # Server denies call, assume legacy protocol.
+ $max_proto_ver = 1;
+} else {
+ # Parse proxy server version.
+ my ($server_ver_num) = $server_ver_str =~ /^DBI::ProxyServer\s+([\d\.]+)/;
+ $max_proto_ver = $server_ver_num = 0.3 ? 2 : 1;
+}
+my $req_proto_ver;
+if ( exists $attr{proxy_lazy_prepare} ) {
+ $req_proto_ver = ($attr{proxy_lazy_prepare} == 0) ? 2 : 1;
+ return DBD::Proxy::proxy_set_err($drh,
+ DBI::ProxyServer does not support synchronous statement
+preparation.)
+ if $max_proto_ver $req_proto_ver;
+}
+
# Switch to user specific encryption mode, if desired
if ($userCipherRef) {
$client-{'cipher'} = $userCipherRef;
-141,7 +174,8
'Name' = $dsnOrig,
'proxy_dbh' = $msg,
'proxy_client' = $client,
- 'RowCacheSize' = $attr{'RowCacheSize'} || 20
+ 'RowCacheSize' = $attr{'RowCacheSize'} || 20,
+ 'proxy_proto_ver' = $req_proto_ver || 1
});
foreach $var (keys %attr) {
-168,6 +202,13
use vars qw(%ATTR $AUTOLOAD);
+# inherited: STORE / FETCH against this class.
+# local: STORE / FETCH against parent class.
+# cached:STORE to remote and local objects, FETCH from local.
+# remote:STORE / FETCH against remote object only (default).
+#
+# Note: Attribute names starting with 'proxy_' always treated as 'inherited'.
+#
%ATTR = ( # see also %ATTR in DBD::Proxy::st
'Warn' = 'local',
'Active' = 'local',
-198,7 +239,7
sub
Re: DBI::Proxy patches (minor cleanup)
On Thu, 12 Sep 2002, Tim Bunce wrote:
On Wed, Sep 11, 2002 at 08:50:51PM -0400, Steven N. Hirsch wrote:
Tim,
Two fixes and a suggestion follow. To be applied over my most recent
diffs.
The fixes ensure that $db_obj-state always gets called with a string of
exactly five chars (it complains otherwise.)
The suggested change to CallMethod prevents the logs from filling with
errors in the case where we will be triggering hundreds or thousands of
them quite knowingly (client app inserts in a loop where we suppress
RaiseError and simply continue past some expected failure conditions, e.g.
duplicate primary key.)
By feeding the death message to Debug, we can still choose to watch if
need be.
Yeap.
sub proxy_set_err {
my ($h,$errmsg) = _;
my ($err,$state) =
-($errmsg =~ s/ \[err=(.*?),state=(.*?)\]//) ? ($1,$2) : (1,5 x ' ');
+($errmsg =~ s/ \[err=(.*?),state=(.*?)\]//) ? ($1,$2) : (1,' ');
return DBI::set_err($h, $err, $errmsg, $state);
}
I think a better fix is to use ... : (1,'')
which is what I suggested originally. Was there a problem with that?
Yes, it causes an exception from DBI::set_err(), which carps about
expecting a five char string. My original use of a single space was a
typo, but I did try it with '' - same problem.
If state is '' but err is non-zero then the DBI will return state
as 'S1000' (general error).
I'll look into why this doesn't seem to be working for me.
--- ProxyServer.pm 2002/09/10 01:37:40 1.5
+++ ProxyServer.pm 2002/09/12 00:35:25 1.6
-195,7 +195,7
- my $state = $_[1]-state || '';
+ my $state = $_[1]-state || ' ';
I don't think that needs changing.
If the logic on the client is fixed per my patch, I agree. It's odd that
we're seeing different behavior from DBI::set_err().
Will advise...
Re: DBI::Proxy patches (minor cleanup)
On Thu, 12 Sep 2002, Tim Bunce wrote:
On Wed, Sep 11, 2002 at 08:50:51PM -0400, Steven N. Hirsch wrote:
Tim,
Two fixes and a suggestion follow. To be applied over my most recent
diffs.
The fixes ensure that $db_obj-state always gets called with a string of
exactly five chars (it complains otherwise.)
sub proxy_set_err {
my ($h,$errmsg) = _;
my ($err,$state) =
-($errmsg =~ s/ \[err=(.*?),state=(.*?)\]//) ? ($1,$2) : (1,5 x ' ');
+($errmsg =~ s/ \[err=(.*?),state=(.*?)\]//) ? ($1,$2) : (1,' ');
return DBI::set_err($h, $err, $errmsg, $state);
}
I think a better fix is to use ... : (1,'')
which is what I suggested originally. Was there a problem with that?
If state is '' but err is non-zero then the DBI will return state
as 'S1000' (general error).
Tim,
If state is 'undef', no error is raised. I think I may see what's going
on. There appear to be two 'set_err' routines in the DBI.xs. One of them
has:
if (state *state) {
if (strlen(state) != 5)
croak(set_err: state must be 5 character string);
sv_setpv(DBIc_STATE(imp_xxh), state);
}
else {
(void)SvOK_off(DBIc_STATE(imp_xxh));
}
which will tolerate a null string, since the outer if() conditional fails.
The second appearance looks like:
if (SvOK(state)) {
STRLEN len;
if (SvPV(state, len) len != 5)
croak(set_err: state must be 5 character string);
sv_setsv(DBIc_STATE(imp_xxh), state);
}
else {
(void)SvOK_off(DBIc_STATE(imp_xxh));
}
Here, a zero-length string will clearly be trapped as an error.
I'll leave the solution in your capable hands.. g.
Steve
--
Steven N. Hirsch tie-line: 446-6557 ext: 802-769-6557
Staff Engineer Methodology Integration Team
ASIC Product Development IBM Microelectronics
Re: DBI::Proxy patches (minor cleanup)
On Thu, 12 Sep 2002, Tim Bunce wrote:
Ah, I'd looked at the wrong set_err in DBI.xs (there are two).
Here's a patch:
***
*** 3630,3636
if (errstr==sv_no || !SvOK(errstr))
errstr = errval;
sv_setsv(DBIc_ERRSTR(imp_xxh), errstr);
! if (SvOK(state)) {
STRLEN len;
if (SvPV(state, len) len != 5)
croak(set_err: state must be 5 character string);
--- 3648,3654
if (errstr==sv_no || !SvOK(errstr))
errstr = errval;
sv_setsv(DBIc_ERRSTR(imp_xxh), errstr);
! if (SvTRUE(state)) {
STRLEN len;
if (SvPV(state, len) len != 5)
croak(set_err: state must be 5 character string);
Yeah, I already spotted that. My post must have crossed yours in the
wires g.
Steve
--
Steven N. Hirsch tie-line: 446-6557 ext: 802-769-6557
Staff Engineer Methodology Integration Team
ASIC Product Development IBM Microelectronics
Re: Proposed patch for DBI-1.30
On Tue, 10 Sep 2002, Tim Bunce wrote: I'm not sure. Wouldn't any fetch() method returning an array behave in the same manner? For whatever reason, we did not see this problem until Perl 5.8.0, so I'm not convinced it's a problem with the DB2 driver. fetch() is an alias for fetchrow_arrayref, so calling it in a scalar context should make no difference. Can you look into that? May a trace log would help. I'll let you know. Adding support for non-lazy prepare is a valuable addition - but it needs to be made configurable at the dbh level via an attribute. I'm not sure I agree with you on this. The Proxy should remain as transparent as possible to any client applications. By treating it as a another attribute in the DBI_AUTOPROXY dsn string, we can avoid the need to make any changes to the application. Unless I'm misunderstanding you, there would be no way to avoid this if it's only available through the dbh. Ah, I was forgetting that the DBI_AUTOPROXY env var setting currently forces a dbi:Proxy: prefix and so can't be used to set attributes. So I'll fix that [...later...] done. Now if the DBI_AUTOPROXY env var matches /^dbi:/i then the dbi will assume it has the full prefix. This has two advantages, first we can now set proxy attributes: DBI_AUTOPROXY=dbi:Proxy(proxy_lazy_prepare=1):host=... but also we can use DBI_AUTOPROXY to enable the use of any other driver that works in a similar way: DBI_AUTOPROXY=dbi:MyCustomProxy:... It also ensures that autoproxy mode works in the manner implied by the documentation. I wasted a lot of time trying to figure out why setting: DBI_AUTOPROXY='dbi:Proxy:hostname=foo;port=1234' caused a runtime error. Accepting the driver spec is definitely goodness. Any way I can get a patch for that? Can you grant r/o access to CVS or the like? Steve
Patches for DBI::Proxy (take 2)
= $sth-{TYPE};
+ }
+ ($handle, $NUM_OF_FIELDS, $sth-{'NUM_OF_PARAMS'},
+ $NAME, $TYPE, result);
}
-($handle, $NUM_OF_FIELDS, $sth-{'NUM_OF_PARAMS'},
- $NAME, $TYPE, result);
}
sub table_info {
-293,7 +307,8
# DBI::st and not DBI::ProxyServer::st. We could fix this by permitting
# the client to execute method DBI::st, but I don't like this.
my rows;
-while (my $row = $sth-fetch()) {
+while (my ($row) = $sth-fetch()) {
+last unless defined $row;
push(rows, [@$row]);
}
($numFields, $names, $types, rows);
-305,7 +320,7
DBI::ProxyServer::st::ISA = qw(DBI::st);
sub execute {
-my $sth = shift; my $params = shift;
+my $sth = shift; my $params = shift; my $proto_ver = shift;
my outParams;
if ($params) {
for (my $i = 0; $i $params;) {
-325,8 +340,18
}
}
}
-
my $rows = $sth-SUPER::execute();
+if ( $proto_ver and $proto_ver 1 and not $sth-{private_proxyserver_described}
+) {
+ my ($NAME, $TYPE);
+ my $NUM_OF_FIELDS = $sth-{NUM_OF_FIELDS};
+ if ($NUM_OF_FIELDS) {# is a SELECT
+ $NAME = $sth-{NAME};
+ $TYPE = $sth-{TYPE};
+ }
+ $sth-{private_proxyserver_described} = 1;
+ # First execution, we ship back description.
+ return ($rows, $NUM_OF_FIELDS, $sth-{'NUM_OF_PARAMS'}, $NAME, $TYPE,
+@outParams);
+}
($rows, outParams);
}
--
Steven N. Hirsch tie-line: 446-6557 ext: 802-769-6557
Staff Engineer Methodology Integration Team
ASIC Product Development IBM Microelectronics
Re: Proposed patch for DBI-1.30
On Tue, 10 Sep 2002, Tim Bunce wrote:
Can you grant r/o access to CVS or the like?
It's crossed my mind from time to time to go the sourceforge kind'a route
but it's never quite happened.
Tim.
Tim,
I'm sure this was a slip of the fingers, but the final dsn string must use
the real driver name (applies after your patch below):
--- DBI.pm.orig Tue Sep 10 19:50:19 2002
+++ DBI.pm Tue Sep 10 20:01:32 2002
-472,12 +472,13
.and DBI_DSN env var not set);
if ($ENV{DBI_AUTOPROXY} $driver ne 'Proxy' $driver ne 'Sponge' $driver
ne 'Switch') {
- $driver = 'Proxy';
+ my $proxy = 'Proxy';
if ($ENV{DBI_AUTOPROXY} =~ s/^dbi:(\w*?)(?:\((.*?)\))?://i) {
- $driver = $1;
+ $proxy = $1;
$driver_attrib_spec = ($driver_attrib_spec) ? $driver_attrib_spec,$2 :
$2;
}
$dsn = $ENV{DBI_AUTOPROXY};dsn=dbi:$driver:$dsn;
+ $driver = $proxy;
DBI-trace_msg( DBI_AUTOPROXY:
dbi:$driver($driver_attrib_spec):$dsn\n);
}
Steve
*** DBI.pm2002/07/18 14:23:44 11.18
--- DBI.pm2002/09/10 10:57:43
***
*** 471,480
or Carp::croak(Can't connect(_), no database driver specified
.and DBI_DSN env var not set);
! if ($ENV{DBI_AUTOPROXY} $driver ne 'Proxy' $driver ne 'Switch') {
! $dsn = $ENV{DBI_AUTOPROXY};dsn=dbi:$driver:$dsn;
$driver = 'Proxy';
! DBI-trace_msg( DBI_AUTOPROXY: dbi:$driver:$dsn\n);
}
my %attr; # take a copy we can delete from
--- 471,484
or Carp::croak(Can't connect(_), no database driver specified
.and DBI_DSN env var not set);
! if ($ENV{DBI_AUTOPROXY} $driver ne 'Proxy' $driver ne 'Sponge'
$driver ne 'Switch') {
$driver = 'Proxy';
! if ($ENV{DBI_AUTOPROXY} =~ s/^dbi:(\w*?)(?:\((.*?)\))?://i) {
! $driver = $1;
! $driver_attrib_spec = ($driver_attrib_spec) ? $driver_attrib_spec,$2 :
$2;
! }
! $dsn = $ENV{DBI_AUTOPROXY};dsn=dbi:$driver:$dsn;
! DBI-trace_msg( DBI_AUTOPROXY:
dbi:$driver($driver_attrib_spec):$dsn\n);
}
my %attr; # take a copy we can delete from
***
*** 2072,2085
to define a syntax for the C$data_source, it is recommended that
they follow the ODBC style, shown in the last example above.)
! If the environment variable CDBI_AUTOPROXY is defined (and the driver in
! C$data_source is not CProxy) then the connect request will
! automatically be changed to:
!
! dbi:Proxy:$ENV{DBI_AUTOPROXY};dsn=$data_source
!
! and passed to the DBD::Proxy module. CDBI_AUTOPROXY is typically set as
! Chostname=...;port= See the DBD::Proxy documentation for more details.
If C$username or C$password are undefined (rather than just empty),
then the DBI will substitute the values of the CDBI_USER and CDBI_PASS
--- 2077,2092
to define a syntax for the C$data_source, it is recommended that
they follow the ODBC style, shown in the last example above.)
! If the environment variable CDBI_AUTOPROXY is defined (and the
! driver in C$data_source is not CProxy) then the connect request
! will automatically be changed to:
!
! $ENV{DBI_AUTOPROXY};dsn=$data_source
!
! CDBI_AUTOPROXY is typically set as Cdbi:Proxy:hostname=...;port=
! If $ENV{DBI_AUTOPROXY} doesn't begin with 'Cdbi:' then dbi:Proxy:
! will be prepended to it first. See the DBD::Proxy documentation
! for more details.
If C$username or C$password are undefined (rather than just empty),
then the DBI will substitute the values of the CDBI_USER and CDBI_PASS
Re: Proposed patch for DBI-1.30
On Tue, 10 Sep 2002, Tim Bunce wrote:
or Carp::croak(Can't connect(_), no database driver specified
.and DBI_DSN env var not set);
! if ($ENV{DBI_AUTOPROXY} $driver ne 'Proxy' $driver ne 'Sponge'
$driver ne 'Switch') {
$driver = 'Proxy';
! if ($ENV{DBI_AUTOPROXY} =~ s/^dbi:(\w*?)(?:\((.*?)\))?://i) {
! $driver = $1;
! $driver_attrib_spec = ($driver_attrib_spec) ? $driver_attrib_spec,$2 :
$2;
! }
! $dsn = $ENV{DBI_AUTOPROXY};dsn=dbi:$driver:$dsn;
! DBI-trace_msg( DBI_AUTOPROXY:
dbi:$driver($driver_attrib_spec):$dsn\n);
}
Tim,
You are concatenating attributes which we know to be for the Proxy driver
to any existing attributes. However, if the user is setting up the
environment for autoproxy, isn't it more likely that the application is
unmodified and latter were truly intended for the remote database?
Perhaps they should become arguments to the remote database dsn instead?
Steve
Proposed patch for DBI-1.30
Tim, et al,
Here is a patch againt DBI-1.30 which adds some important
functionality and fixes a minor problem or two.
(Minor Highlights)
The notion of Proxy -- ProxyServer protocol level is introduced to
maintain sanity and interoperability with older versions. There is
almost certainly a better way to implement it, but I wanted to get
something that worked out there for comments and testing.
Support is added to Proxy.pm for passing in PlRPC specific parameters;
specifically I needed to set 'maxmessage' in the DBI_AUTOPROXY
environment string. Anything prefixed with 'rpc_' has this trimmed
off and passed along to the PlRPC object.
A bug in the ProxyServer table_info() method was fixed. With Perl
5.8.0 and DBD::DB2 driver v1.26, calling $sth-fetch() in scalar
context is a losing proposition, resulting in the dreaded Can't use
scalar '1' as an array ref.. exception.
I'm not sure if anyone has ever used 'dbish' to work with a proxied
database by setting DBI_AUTOPROXY, but as it stands the DBD::Sponge
pseudo driver gets invoked on the far end unless measures are taken.
This _should_ work, I suppose, but the dbish 'table_info' command
causes a dramatic core dump on the client side when run in this
manner. I suspect a bug in Storable, but have not been motivated to
track it down. Should be repeatable if anyone else wants to pursue
it. Forcibly clearing the environment variable after the initial
connection is setup ensures that Sponge runs on the client. Life is
then Good.
The dbish semantics for table_info also require that a Proxy
table_info call set the statement handle 'Active'. This now works
properly.
(Major Highlights)
Synchronous prepare / execute:
I remain unconvinced that the original lazy method (deferring
prepare until the first call to execute) was truly saving any
time. Further, although the DBI spec may legalize deferred prepare,
enforcing this on the proxy session when the underlying remote
database has meaningful and distinct prepare behavior violates the
principle of least surprise.
I make liberal use of positioned updates in my applications, and DB2
requires that a statement be prepared by the database before a valid
cursor name can be obtained. Also, fetch-ahead wreaks havoc in this
situation since we want the (remote) database cursor to remain on the
row being read on the client.
Propagation of remote database errors:
It is often desirable to suppress DBI exceptions and have the program
logic deal with common (or expected) cases. For this to work
transparently, the application must see the actual error information
generated by the remote database. My proposal is to always turn off
'RaiseError' on the ProxyServer and pass back the database-specific
codes on failure.
All this has been tested in a pre-production environment for a bit
over a week, using DB2 V7.2 as the remote database. I would greatly
appreciate any and all comments, criticisms and suggestions pertaining
to this code. If folks can throw it against a variety of other
databases, that would be great!
Regards,
Steve
(obligatory disclaimer: This message does not imply any official position
of IBM Corporation regarding its subject matter.)
--
Steven N. Hirsch tie-line: 446-6557 ext: 802-769-6557
Staff Engineer Methodology Integration Team
ASIC Product Development IBM Microelectronics
Cut Here -
Index: DBI-1.30/lib/DBD/Proxy.pm
diff -u DBI-1.30/lib/DBD/Proxy.pm:1.1.1.1 DBI-1.30/lib/DBD/Proxy.pm:1.7
--- DBI-1.30/lib/DBD/Proxy.pm:1.1.1.1 Tue Sep 3 10:07:13 2002
+++ DBI-1.30/lib/DBD/Proxy.pm Mon Sep 9 09:38:25 2002
-109,20 +109,28
return DBI::set_err($drh, 1, $err) if $err; # Returns undef
+my %client_opts = (
+ 'peeraddr' = $attr{'hostname'},
+ 'peerport' = $attr{'port'},
+ 'socket_proto' = 'tcp',
+ 'application'= $attr{dsn},
+ 'user' = $user || '',
+ 'password' = $auth || '',
+ 'version'= $DBD::Proxy::VERSION,
+ 'cipher' = $cipherRef,
+ 'debug' = $attr{debug} || 0,
+ 'timeout'= $attr{timeout} || undef,
+ 'logfile'= $attr{logfile} || undef
+ );
+# Options starting with 'rpc_' are forwarded to the RPC layer after
+# stripping the prefix.
+while (my($var,$val) = each %attr) {
+ if ($var =~ s/^rpc_//) {
+ $client_opts{$var} = $val;
+ }
+}
# Create an RPC::PlClient object.
-my($client, $msg) = eval { RPC::PlClient-new(
- 'peeraddr' = $attr{'hostname'},
- 'peerport' = $attr{'port
Re: Proposed patch for DBI-1.30
=(.*?)\]//) ? ($1,$2) : (1,''); return DBI::set_err($h, $err, $errmsg, $state); } That would be backwards compatible and would fix *every* remote call to pass back err errstr state properly. Cool. I'll play with that a bit. I'm very glad to see the proxy code getting more use and I'd love to see it improved. Thanks for helping out. You are quite welcome. Thank YOU for DBI in general! It's made development of database applications fun. Steve -- Steven N. Hirsch tie-line: 446-6557 ext: 802-769-6557 Staff Engineer Methodology Integration Team ASIC Product Development IBM Microelectronics
