Re: get_info (was: Common DBI Driver Test Suite - Requirements)

2014-02-06 Thread Tim Bunce
Hey Martin. Thanks for your recent work on this - much food for thought!

There's one more aspect of get_info that I'd appreciate your input on:

On Sat, Sep 28, 2013 at 05:01:09PM +0100, Tim Bunce wrote:
 
 Somewhat independant of that, I'd like to identify a set of get_info
 items that we recommend all drivers define.

That would be a handy thing to have, and to pass on to driver authors,
and then use to build tests for those items.

What I'm looking for here is mainly get_info items that we're likely to
need to be able to write generic tests that are self-configuring to what the
driver supports.

Could you try to put together such a list?

Tim.


Re: get_info

2014-02-06 Thread Martin J. Evans

On 06/02/14 12:59, Tim Bunce wrote:

Hey Martin. Thanks for your recent work on this - much food for thought!

There's one more aspect of get_info that I'd appreciate your input on:

On Sat, Sep 28, 2013 at 05:01:09PM +0100, Tim Bunce wrote:


Somewhat independant of that, I'd like to identify a set of get_info
items that we recommend all drivers define.


That would be a handy thing to have, and to pass on to driver authors,
and then use to build tests for those items.

What I'm looking for here is mainly get_info items that we're likely to
need to be able to write generic tests that are self-configuring to what the
driver supports.

Could you try to put together such a list?

Tim.



I am happy to do that but as you may already have noticed, I'm not being very 
active on the test discussions - just don't have the time right now for that. 
So if anything comes up in the test discussions please pass them on to me and 
I'll manage the list.

I should have a few tuits this weekend (more if the weather is bad) to take 
this a bit further.

Martin


FYI: Access methods - relative speed

2014-02-06 Thread H.Merijn Brand
I thought it might be time to re-check my performance graph(s) on
access methods in DBI now that most DBD's and engines have moved
forward since I did the previous tests:

 http://tux.nl/Talks/DBDc/prfx.html

$ cat test.pl
#!/pro/bin/perl

use 5.16.1;
use warnings;
use Time::HiRes qw( gettimeofday tv_interval );
use DBI;

my $n = 10;

# This test is about RELATIVE speed differences for access methods
# Not about speed differences between databases

my %db = (
Pg = {
dns = dbname=$ENV{LOGNAME},
n   = $n,
},
mysql  = {
dsn = database=$ENV{LOGNAME},
n   = $n / 100,
},
Oracle = {
dsn = host=$ENV{ORACLE_HOST};sid=$ENV{ORACLE_SID},
user= $ENV{ORACLE_USER},
pass= $ENV{ORACLE_PASS},
n   = $n / 50,
},
SQLite = {
dsn = dbname=sqlite.db,
n   = $n,
},
#Unify  = {
#   },
CSV= {
n   = $n / 100,
},
);

my %am = (
HR  = sub { my $sth = shift;
while (my $ref = $sth-fetchrow_hashref) { }
},

A   = sub { my $sth = shift;
while (my ($k, $c, $i, $v) = $sth-fetchrow_array) { }
},

AR  = sub { my $sth = shift;
while (my $ref = $sth-fetchrow_arrayref) { }
},

DAR = sub { my $sth = shift;
while (my $ref = DBI::st::fetchrow_arrayref ($sth)) { }
},

BC  = sub { my $sth = shift;
$sth-bind_columns (\my ($k, $c, $i, $v));
while ($sth-fetch) { }
},

DBC = sub { my $sth = shift;
$sth-bind_columns (\my ($k, $c, $i, $v));
while (DBI::st::fetchrow_arrayref ($sth)) { }
},
);

my %x;
foreach my $db (keys %db) {

print STDERR  $db ?  \r;

my $v;
eval require DBD::$db; \$v = DBD::$db-VERSION ();
$@ and next;

$db{$db}{vsn} = $v;
warn Testing for DBD::$db-$v\n;

my $dsn = dbi:$db:;
$db{$db}{dsn} and $dsn .= $db{$db}{dsn};
$db{$db}{$_} //= undef for qw( user pass );
$ENV{$_} = $db{$db}{env}{$_} for keys %{$db{$db}{env} // {}};

my $dbh = DBI-connect ($dsn, $db{$db}{user}, $db{$db}{pass}, {
RaiseError = 1,
PrintError = 1,
ChopBlanks = 1,
ShowErrorStatement = 1,
FetchHashKeyName   = NAME_lc,
}) or next;

$db eq CSV || $db eq mysql or $dbh-{AutoCommit} = 0;

$dbh-do (qq;
create table test10 (
k   integer not null primary key,
c   char (10),
i   integer,
v   varchar (10)
););
$dbh-{AutoCommit} or $dbh-commit;

my $sti = $dbh-prepare (insert into test10 values (?, ?, ?, ?));
$sti-execute ($_, $_, $_, $_) for 1 .. $db{$db}{n};
$sti-finish;
$dbh-{AutoCommit} or $dbh-commit;

foreach my $am (keys %am) {
my $sth = $dbh-prepare (select k, c, i, v from test10);
$sth-execute;
my $t0 = [gettimeofday];
$am{$am}-($sth);
$x{$db}{$am} = tv_interval ($t0);
}

$dbh-do (drop table test10);
$dbh-{AutoCommit} or $dbh-commit;
$dbh-disconnect;
}
warn Done!\n;

foreach my $db (keys %x) {
my $norm = 1000 * $x{$db}{A};
my %c = map { $_ = int ($norm / $x{$db}{$_}) } keys %am;

print $db-$db{$db}{vsn}\n;
printf %6s %5d\n, $_, $c{$_} for sort { $c{$a} = $c{$b} } keys %c;
}
$ perl speed.pl
Oracle-1.68
HR   639
   DBC   804
AR   913
BC   956
   DAR   967
 A   999
SQLite-1.40
HR   255
 A  1000
AR  1041
   DAR  1166
BC  1333
   DBC  1455
Pg-3.0.0
HR   198
 A  1000
AR  1143
   DAR  1186
   DBC  1326
BC  1332
CSV-0.41
HR   683
 A  1000
AR  1164
   DAR  1171
BC  1181
   DBC  1185
mysql-4.025
HR   102
 A   999
AR  1126
   DAR  1271
BC  1287
   DBC  1328

This clearly shows that having Oracle on a remote host causes the
network to have more influence than the access method

-- 
H.Merijn Brand  http://tux.nl   Perl Monger  http://amsterdam.pm.org/
using perl5.00307 .. 5.19   porting perl5 on HP-UX, AIX, and openSUSE
http://mirrors.develooper.com/hpux/http://www.test-smoke.org/
http://qa.perl.org   http://www.goldmark.org/jeff/stupid-disclaimers/


Re: FYI: Access methods - relative speed

2014-02-06 Thread H.Merijn Brand
On Thu, 6 Feb 2014 16:07:07 +0100, H.Merijn Brand
h.m.br...@xs4all.nl wrote:

Now with a second run on a database machine with Oracle local

laptop  server
perl-5.18.2-64int-ldperl-5.16.2-64all-ld
DBI-1.631   DBI-1.631

DBD::Oracle-1.68DBD::Oracle-1.68
Oracle 12.1.0.1.0   Oracle 11.2.0.3.0
- 11.2.0.3.0   - 11.2.0.3.0
HR   639HR   387
   DBC   804 A  1000
AR   913AR  1142
BC   956   DBC  1186
   DAR   967   DAR  1201
 A   999BC  1250

DBD::SQLite-1.40DBD::SQLite-1.40
HR   255HR   250
 A  1000 A  1000
AR  1041AR  1172
   DAR  1166   DAR  1244
BC  1333BC  1250
   DBC  1455   DBC  1313

DBD::Pg-3.0.0   DBD::Pg-3.0.0
PostgreSQL 9.3.2PostgreSQL 9.2.4
HR   198HR   179
 A  1000 A  1000
AR  1143AR  1279
   DAR  1186   DAR  1281
   DBC  1326BC  1417
BC  1332   DBC  1485

DBD::CSV-0.41   DBD::CSV-0.41
HR   683HR   791
 A  1000 A  1000
AR  1164BC  1153
   DAR  1171AR  1169
BC  1181   DAR  1176
   DBC  1185   DBC  1186

DBD::mysql-4.025DBD::mysql-4.026
MariaDB-5.5.33  MariaDB-5.5.33
HR   102HR95
 A   999 A  1000
AR  1126BC  1245
   DAR  1271AR  1438
BC  1287   DAR  1509
   DBC  1328   DBC  1612

-- 
H.Merijn Brand  http://tux.nl   Perl Monger  http://amsterdam.pm.org/
using perl5.00307 .. 5.19   porting perl5 on HP-UX, AIX, and openSUSE
http://mirrors.develooper.com/hpux/http://www.test-smoke.org/
http://qa.perl.org   http://www.goldmark.org/jeff/stupid-disclaimers/