RE: MySQL, blobs, GIS
> Hi all, >I have a MySQL-perl-cgi-dbi application which would like to store > images > and GIS info. Will dbi running with DBD:MySQL support these? Yes, I think it will. But people often frown on the idea of using an RDBMS to store binary data like images - there's a bit of a "religious war" on the topic. You might want to think seriously about it, and do some research, before you decide. Here is on article that a quick google turned up - I'm sure you can find many more: http://mike.kruckenberg.com/archives/2006/03/storing_binary.html It touches on some of the key concerns like performance and scalability, data management and backups, etc. HTH --edan
RE: possible leak in 1.52
> this doesn't apply to my needs, but are DB connects cached as well > with a separate 120 counter? I'm sort of out of my league here, but I think DBI caches the db handles as well as statement handles, and all of them are cleared every 120. So even though you are undef'ing them, they are cached. If you run your code in a loop for more than 120 connect/undef pairs, are you seeing the same behavior (i.e. all freed after 120 times)? If so, it sounds like it's expected and hunky-dory. Best, Ephraim
RE: possible leak in 1.52
I hope Tim will chime in and correct me, but since I was the last reporter of
leaks in the DBI, let me try my best:
I tried to reproduce your leak without success. Perhaps you are not letting
your test run for enough iterations. DBI does cache some handles that it only
releases every 120 calls. So you could see what looks like a 1 SV leak, but
you'll see that after 120 iterations they are all freed.
Try running 1000 iterations and see if you still experience a leak.
I recommend using DBD::Sponge (which comes with DBI) so that you are sure
you're not seeing a leak in your driver. Example code follows. Please let us
know what you find.
my $dbh = DBI->connect("dbi:Sponge:","","",{ RaiseError => 1 });
my $sth = $dbh->prepare(
"SELECT * FROM FOO",
{ rows => [[qw/1 2/],[qw/3 4/]], NAME => [qw/a b/]}
);
$sth->execute();
while ( my ($a,$b) = $sth->fetchrow_array() ) {
print "a:$a,b:$b\n";
}
$sth->finish;
$dbh->disconnect;
> -Original Message-
> From: Jonathan [mailto:[EMAIL PROTECTED]
> Sent: Wednesday, August 23, 2006 04:52
> To: dbi-users@perl.org
> Subject: Re: possible leak in 1.52
>
>
> it definitely seems to be a leak, and i've been able to reproduce it
> outside of mod_perl
>
> it seems that there are a few potential sources / behaviors
>
> a_ the statement handle created by a prepare never seems to
> expire
> in the lifetime of a db connection
> i could trace this as far back as
> DBI::_new_handle
>
> b_ if i connect, run 100 'update x where y=z' , then disconnect, no
> SV are released
>
> c_ if i then undef my db handle, a lot of SV are freed - but i seem
> to have a 138 SV which are not released
>
> d_ if i reconnect , i seem to use only 15 more SV than the original
> connect- whereas the original connect used 231 SV over my test
> platforms 'weight'
>
> e_ if i undef the reconnect, I seem to have leaked 1 SV since the
> first disconnect
>
> unfortunately, thats as far as my DBI knowledge can take me
>
RE: FW: memory leak in DBI ...
It certainly does appear to leak memory for you, too. My script is similar, as
are my results.
Thanks for checking.
Appears to be a confirmed memory leak to me. Does anyone have an idea of how
the weakref code can be fixed to solve this? It appears that Tim doesn't have
time...
Thanks,
Ephraim
> -Original Message-
> From: Tielman de Villiers [mailto:[EMAIL PROTECTED]
> Sent: Tuesday, July 18, 2006 15:38
> To: Ephraim Dan
> Subject: RE: FW: memory leak in DBI ...
>
> perl v5.8.7
>
> DBI ver 1.50:
> before | total pid size: 6024 kB
> 1 | total pid size: 9152 kB (+3128 kB)
>101 | total pid size: 9152 kB (+3128 kB)
>201 | total pid size: 9152 kB (+3128 kB)
>301 | total pid size: 9208 kB (+3184 kB)
>401 | total pid size: 9208 kB (+3184 kB)
>501 | total pid size: 9208 kB (+3184 kB)
>601 | total pid size: 9208 kB (+3184 kB)
>701 | total pid size: 9208 kB (+3184 kB)
>801 | total pid size: 9208 kB (+3184 kB)
>901 | total pid size: 9208 kB (+3184 kB)
> after | total pid size: 9332 kB
>
> DBI 1.51:
> before | total pid size: 5988 kB
> 1 | total pid size: 9116 kB (+3128 kB)
>101 | total pid size: 9116 kB (+3128 kB)
>201 | total pid size: 9116 kB (+3128 kB)
>301 | total pid size: 9172 kB (+3184 kB)
>401 | total pid size: 9172 kB (+3184 kB)
>501 | total pid size: 9172 kB (+3184 kB)
>601 | total pid size: 9172 kB (+3184 kB)
>701 | total pid size: 9172 kB (+3184 kB)
>801 | total pid size: 9172 kB (+3184 kB)
>901 | total pid size: 9172 kB (+3184 kB)
> after | total pid size: 9296 kB
>
>
>
>
>
>
>
>
>
> On Tue, 2006-07-18 at 05:31 -0700, Ephraim Dan wrote:
> > What results did this give you?
> >
> > Thanks for testing.
> > I'll send you my test script soon.
> >
> > > -Original Message-
> > > From: Tielman de Villiers [mailto:[EMAIL PROTECTED]
> > > Sent: Tuesday, July 18, 2006 15:29
> > > To: Ephraim Dan
> > > Subject: Re: FW: memory leak in DBI ...
> > >
> > > Hi Ephraim,
> > >
> > > I can have a look at this for you, could you pls send me your test
> > > script for the mem leaks? (I'll leave it up to you to post the results
> > > to the list).
> > >
> > > --tielman
> > >
> > > I've tested something like the below, but not sure if that gives the
> > > results you want (linux)?
> > >
> > >
> > > __START__
> > > my @statusArray;
> > >
> > > @statusArray = split (/\s+/, `grep Vm /proc/$$/status`);
> > > printf "before | total pid size: %5d kB\n", $statusArray[1];
> > >
> > > my $startmem = $statusArray[1];
> > >
> > > use DBI;
> > > ##my $dbh = DBI->connect( "dbi:Pg:dbname=test;host=xx", "user",
> > > "secret" );
> > > for my $i ( 1 .. 1000 ) {
> > > my $dbh = DBI->connect( "dbi:Pg:dbname=test;host=xx", "user",
> > > "secret" );
> > > my $sth = $dbh->prepare("SELECT * FROM test");
> > > $sth->execute();
> > > $sth->finish;
> > > $dbh->disconnect;
> > > @statusArray = split (/\s+/, `grep Vm /proc/$$/status`);
> > > if ( $i % 100 == 1 ) {
> > >printf "%6d | total pid size: %5d kB (+%3d kB)\n", $i,
> > > $statusArray[1], $statusArray[1] - $startmem;
> > > }
> > > }
> > > ##$dbh->disconnect;
> > >
> > > @statusArray = split (/\s+/, `grep Vm /proc/$$/status`);
> > > printf "after | total pid size: %5d kB\n", $statusArray[1];
> > >
> > > __END__
> > >
> > >
> > >
> > >
> > > On Mon, 2006-07-17 at 20:41 +0100, Tielman de Villiers wrote:
> > > >
> > > > > -Original Message-
> > > > > From: Ephraim Dan [mailto:[EMAIL PROTECTED]
> > > > > Sent: 17 July 2006 18:39
> > > > > To: Ephraim Dan; Tim Bunce
> > > > > Cc: dbi-users@perl.org
> > > > > Subject: RE: memory leak in DBI ...
> > > > >
> > > > >
> > > > > Tim,
> > > > >
> > > > > Any chance you've had a chance to look at this?
> > > > >
> > > > > Can someone else try to reproduce a memory leak in a simple
> > > > > connect/prepare/execute/disconnect loop using DBI 1.51 and
> > > > > a
RE: memory leak in DBI ...
Tim, Any chance you've had a chance to look at this? Can someone else try to reproduce a memory leak in a simple connect/prepare/execute/disconnect loop using DBI 1.51 and any DBD driver? Thanks... --edan From: Ephraim Dan [mailto:[EMAIL PROTECTED] Sent: Thu 7/13/2006 11:02 To: Tim Bunce Cc: dbi-users@perl.org Subject: RE: memory leak in DBI XS bootstrap code > I've just noticed that you're using 5.8.0. Try the latest 5.8.x. [edan] We can't upgrade perl very easily in our production environment, but I tested on another box running perl 5.8.8, DBI 1.50, and I get the same leak of 4 per loop. Any chance you could find the time to attempt to reproduce this yourself? Thanks, edan
RE: memory leak in DBI XS bootstrap code
> I've just noticed that you're using 5.8.0. Try the latest 5.8.x. [edan] We can't upgrade perl very easily in our production environment, but I tested on another box running perl 5.8.8, DBI 1.50, and I get the same leak of 4 per loop. Any chance you could find the time to attempt to reproduce this yourself? Thanks, edan
RE: memory leak in DBI XS bootstrap code
> Did you try the other suggestions?
> How much improvement did each give? (when testing by destroying the
> interpreter)
Yeah, sorry I didn't give more detail on that. Here goes:
The first change (Doru Petrescu):
No change, still leaks 4 per connect/prepare/execute/fetch/disconnect loop.
The (av_len(av) % 120 == 0) change:
That does reduce it to 3 per loop, but I am not sure what you mean that it is
not a leak - when it leaks 4, it's 4 every time - it's not stabilizing after
some time, or jumping back down once in a while, or only leaking 3 sometimes,
just going up and up by 4 every time. So how is that not a leak?
As I said, the "if (0)" on the whole chunk of code produces squeaky clean
iterations.
-edan
> > So... do I need to keep this patched myself, at the risk of breaking
> > when the code changes in some future release, or can you somehow make
> > this optional, or fix it,
>
> I'd probably accept a patch that adds an option to Makefile.PL that then
> passes a -DDBI_NO_WEAKREFS to the compiler which, with a suitable
> #ifndef in DBI.xs, would disable that code. Or a new per-handle
> flag could be added.
>
> But I'm reluctant to do either till I know more about the cause.
>
> > or explain to me what I can do to make the
> > leak go away using the current code?
>
> I need more info - see above.
>
> Tim.
>
> > Thanks so much,
> > edan
> >
> > > > > -----Original Message-
> > > > > From: Tim Bunce [mailto:[EMAIL PROTECTED]
> > > > > Sent: Tuesday, July 11, 2006 16:59
> > > > > To: Ephraim Dan
> > > > > Cc: Tim Bunce; dbi-users@perl.org
> > > > > Subject: Re: memory leak in DBI XS bootstrap code
> > > > >
> > > > > I'm sorry but I simply don't have the time at the moment.
> > > > >
> > > > > Some random suggestions:
> > > > > - try using different sets of methods to isolate the minimum
> > > > > requirements.
> > > > >for example:
> > > > > call fetchrow_array just once.
> > > > > don't call fetchrow_array at all.
> > > > > call fetchrow_array in loop but don't call finish
> > > > > don't execute, fetch, finish.
> > > > > etc etc etc
> > > > > - try with other drivers to see if it's the driver or the DBI
> that's
> > > > > leaking
> > > > >
> > > > > Tim.
> > > > >
> > > > > On Tue, Jul 11, 2006 at 03:59:05AM -0700, Ephraim Dan wrote:
> > > > > > Tim,
> > > > > >
> > > > > > I'm experiencing further memory leaks which are causing me more
> > > grief.
> > > > > I put together the following test script which illustrates the
> > > leakage:
> > > > > >
> > > > > > #/usr/bin/perl
> > > > > > # dbi.pl
> > > > > >
> > > > > > use DBI;
> > > > > > use Devel::Leak;
> > > > > > my $leak_handle;
> > > > > >
> > > > > > for ( 1 .. 1000 ) {
> > > > > >my $dbh = DBI->connect("DBI:mysql:MYDB","user","secret",
> > > > > {RaiseError=>1});
> > > > > >my $sth = $dbh->prepare("SELECT COUNT(*) FROM TABLE");
> > > > > >$sth->execute();
> > > > > >while ( my ($c) = $sth->fetchrow_array() ) {
> > > > > > # print "$c\n";
> > > > > >}
> > > > > >$sth->finish;
> > > > > >$dbh->disconnect;
> > > > > >
> > > > > >system("ps -aux | fgrep dbi.pl | fgrep -v fgrep");
> > > > > >test_leak();
> > > > > > }
> > > > > >
> > > > > > sub test_leak {
> > > > > > if ( $handle ) {
> > > > > > Devel::Leak::CheckSV($handle);
> > > > > > }
> > > > > > print STDERR "count: " . Devel::Leak::NoteSV($handle) . "\n";
> > > > > > }
> > > > > >
> > > > > > Devel::Leak shows that it is leaking 4 "things" every iteration:
> > > > > >
> > > > > > count: 12128
> > > > > > root 17720 17.8 0.3 6840 4220 pts/0 S 09:
RE: memory leak in DBI XS bootstrap code
> You could also try another change a few lines further up where it says
>
> #ifdef sv_rvweaken
> if (1) {
>
> Change the 1 to a 0 to disable the use of weakrefs which was added in
> 1.49.
[edan] This seems to have done it! It fixes all the leaks! Hurrah! So... do
I need to keep this patched myself, at the risk of breaking when the code
changes in some future release, or can you somehow make this optional, or fix
it, or explain to me what I can do to make the leak go away using the current
code?
Thanks so much,
edan
> > > -Original Message-
> > > From: Tim Bunce [mailto:[EMAIL PROTECTED]
> > > Sent: Tuesday, July 11, 2006 16:59
> > > To: Ephraim Dan
> > > Cc: Tim Bunce; dbi-users@perl.org
> > > Subject: Re: memory leak in DBI XS bootstrap code
> > >
> > > I'm sorry but I simply don't have the time at the moment.
> > >
> > > Some random suggestions:
> > > - try using different sets of methods to isolate the minimum
> > > requirements.
> > >for example:
> > > call fetchrow_array just once.
> > > don't call fetchrow_array at all.
> > > call fetchrow_array in loop but don't call finish
> > > don't execute, fetch, finish.
> > > etc etc etc
> > > - try with other drivers to see if it's the driver or the DBI that's
> > > leaking
> > >
> > > Tim.
> > >
> > > On Tue, Jul 11, 2006 at 03:59:05AM -0700, Ephraim Dan wrote:
> > > > Tim,
> > > >
> > > > I'm experiencing further memory leaks which are causing me more
> grief.
> > > I put together the following test script which illustrates the
> leakage:
> > > >
> > > > #/usr/bin/perl
> > > > # dbi.pl
> > > >
> > > > use DBI;
> > > > use Devel::Leak;
> > > > my $leak_handle;
> > > >
> > > > for ( 1 .. 1000 ) {
> > > >my $dbh = DBI->connect("DBI:mysql:MYDB","user","secret",
> > > {RaiseError=>1});
> > > >my $sth = $dbh->prepare("SELECT COUNT(*) FROM TABLE");
> > > >$sth->execute();
> > > >while ( my ($c) = $sth->fetchrow_array() ) {
> > > > # print "$c\n";
> > > >}
> > > >$sth->finish;
> > > >$dbh->disconnect;
> > > >
> > > >system("ps -aux | fgrep dbi.pl | fgrep -v fgrep");
> > > >test_leak();
> > > > }
> > > >
> > > > sub test_leak {
> > > > if ( $handle ) {
> > > > Devel::Leak::CheckSV($handle);
> > > > }
> > > > print STDERR "count: " . Devel::Leak::NoteSV($handle) . "\n";
> > > > }
> > > >
> > > > Devel::Leak shows that it is leaking 4 "things" every iteration:
> > > >
> > > > count: 12128
> > > > root 17720 17.8 0.3 6840 4220 pts/0S09:55 0:00 perl
> > > dbi.pl
> > > > new 0x8aad3e8 : SV = PVHV(0x8a68b30) at 0x8aad3e8
> > > > REFCNT = 1
> > > > FLAGS = (PADBUSY,PADMY,SHAREKEYS)
> > > > IV = 0
> > > > NV = 0
> > > > ARRAY = 0x0
> > > > KEYS = 0
> > > > FILL = 0
> > > > MAX = 7
> > > > RITER = -1
> > > > EITER = 0x0
> > > > new 0x8aad3f4 : SV = NULL(0x0) at 0x8aad3f4
> > > > REFCNT = 1
> > > > FLAGS = (PADBUSY,PADMY)
> > > > new 0x8aad400 : SV = NULL(0x0) at 0x8aad400
> > > > REFCNT = 1
> > > > FLAGS = (PADBUSY,PADMY)
> > > > new 0x8aad40c : SV = NULL(0x0) at 0x8aad40c
> > > > REFCNT = 1
> > > > FLAGS = (PADBUSY,PADMY)
> > > > count: 12132
> > > >
> > > > I think there might be several leaks here. If I move the connect()
> out
> > > of the loop, it's leaking, but less - just one of those NULL SV guys.
> > > >
> > > > Can you please help me track this down? I'm still a little scared
> of
> > > the DBI.xs internals, despite my recent exposure...
> > > >
> > > > Thanks a lot,
> > > > edan
> > > >
> > > >
> > > > > -Original Message-
> > > > > From: Tim Bunce [mailto:[EMAIL PROTECTED]
> > > > > Sent: Monday, July 03, 2006 12:16
> > >
RE: memory leak in DBI XS bootstrap code
I am 99% sure that it's DBI that's leaking. We were using DBI 1.32 previously,
and the same script does not leak when 1.32 is installed. Changing only DBI to
1.51 produces the leak. No DBDs were touched.
I think it's isolated to connect and prepare:
1. When the loop consists only of connect and disconnect, 1 SV is leaked.
2. When the loop is connect, prepare and disconnect, 4 SVs are leaked.
3. When the connect is done only once, and the loop is only prepare, 1 SV is
leaked.
4. When connect and prepare are outside the loop, and the loop consists of
execute, fetchrow_array, finish, 0 SVs are leaked.
I also tried it with DBD::Sponge, and the same numbers resulted.
I have tried various attempts at finding this leak, but to no avail. I am
pretty sure it's in the XS code, but I can't be sure. I tried using
Devel::Cycle to see if it's a circular reference, but it didn't show anything.
I don't know of other ways to make a memory leak in pure perl, which is why I
suspect XS.
I tried to use Devel::LeakTrace to find where the leaks are, but I'm not so
sure of its results. For what it's worth, it seems to report leaks within
DBI.pm in the following subroutines:
connect() (return $dbh)
install_driver() (eval qq{ DBI::_firesafe require $driver_class },
$DBI::installed_drh{$driver} = $drh)
setup_driver() (push @{"${class}_mem::ISA"}, $mem_class)
_new_handle() (DBI::_setup_handle($h, $imp_class, $parent, $imp_data))
Do you have any other pointers on how to track this down? I am more than
willing to try more things, but I've run out of ideas. I have tried valgrind
but I don't think it helps with perl refcount issues, since they are not true
memory leaks in the sense of lost pointers to malloc'ed buffers... In any
event this has been a dead end so far...
Thanks for any help you can give, Tim (or anyone else, please?)
-edan
> -Original Message-
> From: Tim Bunce [mailto:[EMAIL PROTECTED]
> Sent: Tuesday, July 11, 2006 16:59
> To: Ephraim Dan
> Cc: Tim Bunce; dbi-users@perl.org
> Subject: Re: memory leak in DBI XS bootstrap code
>
> I'm sorry but I simply don't have the time at the moment.
>
> Some random suggestions:
> - try using different sets of methods to isolate the minimum
> requirements.
>for example:
> call fetchrow_array just once.
> don't call fetchrow_array at all.
> call fetchrow_array in loop but don't call finish
> don't execute, fetch, finish.
> etc etc etc
> - try with other drivers to see if it's the driver or the DBI that's
> leaking
>
> Tim.
>
> On Tue, Jul 11, 2006 at 03:59:05AM -0700, Ephraim Dan wrote:
> > Tim,
> >
> > I'm experiencing further memory leaks which are causing me more grief.
> I put together the following test script which illustrates the leakage:
> >
> > #/usr/bin/perl
> > # dbi.pl
> >
> > use DBI;
> > use Devel::Leak;
> > my $leak_handle;
> >
> > for ( 1 .. 1000 ) {
> >my $dbh = DBI->connect("DBI:mysql:MYDB","user","secret",
> {RaiseError=>1});
> >my $sth = $dbh->prepare("SELECT COUNT(*) FROM TABLE");
> >$sth->execute();
> >while ( my ($c) = $sth->fetchrow_array() ) {
> > # print "$c\n";
> >}
> >$sth->finish;
> >$dbh->disconnect;
> >
> >system("ps -aux | fgrep dbi.pl | fgrep -v fgrep");
> >test_leak();
> > }
> >
> > sub test_leak {
> > if ( $handle ) {
> > Devel::Leak::CheckSV($handle);
> > }
> > print STDERR "count: " . Devel::Leak::NoteSV($handle) . "\n";
> > }
> >
> > Devel::Leak shows that it is leaking 4 "things" every iteration:
> >
> > count: 12128
> > root 17720 17.8 0.3 6840 4220 pts/0S09:55 0:00 perl
> dbi.pl
> > new 0x8aad3e8 : SV = PVHV(0x8a68b30) at 0x8aad3e8
> > REFCNT = 1
> > FLAGS = (PADBUSY,PADMY,SHAREKEYS)
> > IV = 0
> > NV = 0
> > ARRAY = 0x0
> > KEYS = 0
> > FILL = 0
> > MAX = 7
> > RITER = -1
> > EITER = 0x0
> > new 0x8aad3f4 : SV = NULL(0x0) at 0x8aad3f4
> > REFCNT = 1
> > FLAGS = (PADBUSY,PADMY)
> > new 0x8aad400 : SV = NULL(0x0) at 0x8aad400
> > REFCNT = 1
> > FLAGS = (PADBUSY,PADMY)
> > new 0x8aad40c : SV = NULL(0x0) at 0x8aad40c
> > REFCNT = 1
> > FLAGS = (PADBUSY,PADMY)
> > count: 12132
> >
> > I think there might be several leaks here. If I move the connect() out
> of the loop, it's leaking, but less - just one of th
RE: mysql installation problems
ql/mysql.so \ >-L/usr/local/lib > -L/usr/local/mysql-standard-5.0.22-linux-i686/lib -lmysqlclient -lz - > lcrypt > -lnsl -lm -lc -lnss_files -lnss_dns -lresolv -lc -lnss_files -lnss_dns > -lresolv \ > > chmod 755 blib/arch/auto/DBD/mysql/mysql.so > cp mysql.bs blib/arch/auto/DBD/mysql/mysql.bs > chmod 644 blib/arch/auto/DBD/mysql/mysql.bs > Manifying blib/man3/DBD::mysql.3pm > Manifying blib/man3/DBD::mysql::INSTALL.3pm > Manifying blib/man3/Mysql.3pm > Manifying blib/man3/Bundle::DBD::mysql.3pm > SERVER:/usr/src/perl/DBD-mysql-3.0006# make test > PERL_DL_NONLAZY=1 /usr/bin/perl "-MExtUtils::Command::MM" "-e" > "test_harness(0, 'blib/lib', 'blib/arch')" t/*.t > t/00base.install_driver(mysql) failed: Can't load > '/usr/src/perl/DBD-mysql-3.0006/blib/arch/auto/DBD/mysql/mysql.so' for > module DBD::mysql: > /usr/src/perl/DBD-mysql-3.0006/blib/arch/auto/DBD/mysql/mysql.so: > undefined > symbol: __pure_virtual at /usr/lib/perl/5.8/DynaLoader.pm line 225. > at (eval 3) line 3 > Compilation failed in require at (eval 3) line 3. > Perhaps a required shared library or dll isn't installed where expected > at t/00base.t line 38 > t/00base.dubious > > Test returned status 2 (wstat 512, 0x200) > DIED. FAILED tests 4-5 > Failed 2/5 tests, 60.00% okay > t/10dsnlist..install_driver(mysql) failed: Can't load > '/usr/src/perl/DBD-mysql-3.0006/blib/arch/auto/DBD/mysql/mysql.so' for > module DBD::mysql: > /usr/src/perl/DBD-mysql-3.0006/blib/arch/auto/DBD/mysql/mysql.so: > undefined > symbol: __pure_virtual at /usr/lib/perl/5.8/DynaLoader.pm line 225. > at (eval 3) line 3 > Compilation failed in require at (eval 3) line 3. > Perhaps a required shared library or dll isn't installed where expected > at t/10dsnlist.t line 45 > t/10dsnlist..dubious > > Test returned status 2 (wstat 512, 0x200) > DIED. FAILED tests 1-9 > Failed 9/9 tests, 0.00% okay > . > . > .and > .so > .on > . > . > Failed Test Stat Wstat Total Fail Failed List of Failed > -- > -- > --- > t/00base.t 2 512 54 80.00% 4-5 > t/10dsnlist.t 2 512 9 18 200.00% 1-9 > t/20createdrop.t 2 512 5 10 200.00% 1-5 > t/30insertfetch.t 2 51211 22 200.00% 1-11 > t/35limit.t2 512 113 226 200.00% 1-113 > t/35prepare.t 2 51234 68 200.00% 1-34 > t/40bindparam.t2 51228 56 200.00% 1-28 > t/40blobs.t2 51211 22 200.00% 1-11 > t/40listfields.t 2 51218 36 200.00% 1-18 > t/40nulls.t2 51211 22 200.00% 1-11 > t/40numrows.t 2 51225 50 200.00% 1-25 > t/41bindparam.t2 51211 22 200.00% 1-11 > t/41blobs_prepare.t2 51224 48 200.00% 1-24 > t/42bindparam.t2 51211 22 200.00% 1-11 > t/50chopblanks.t 2 51235 70 200.00% 1-35 > t/50commit.t 2 51230 60 200.00% 1-30 > t/dbdadmin.t 2 51221 42 200.00% 1-21 > t/insertid.t 2 51212 24 200.00% 1-12 > t/param_values.t 2 512 8 16 200.00% 1-8 > t/prepare_noerror.t2 512 36 200.00% 1-3 > t/texecute.t 2 512 9 18 200.00% 1-9 > 1 test skipped. > Failed 21/22 test scripts, 4.55% okay. 431/434 subtests failed, 0.69% > okay. > make: *** [test_dynamic] Error 2 > > > -- > - > > Seems to be a library problem but I don't know where to start looking... > > > -Ursprüngliche Nachricht- > Von: Ephraim Dan [mailto:[EMAIL PROTECTED] > Gesendet: Dienstag, 11. Juli 2006 13:36 > An: Whisky; mailing list > Betreff: RE: mysql installation problems > > You're overriding the cflags, so none of the mysql include directories are > present. Try leaving off the --cflags argument, so that mysql_config can > supply the cflags... > > > -Original Message- > > From: Whisky [mailto:[EMAIL PROTECTED] > > Sent: Tuesday, July 11, 2006 14:18 > > To: 'mailing list' > > Subject: DBD:mysql installation problems > > > > Hello List! > > > > I've got quite some trouble installing DBD:mysqli (3.0006) on a Debian > box > > running MySQL 5.0.22. It would be great if some of you had some idea on > > how > > to solve this. > > So here's what happens when I try doe build t
RE: mysql installation problems
You're overriding the cflags, so none of the mysql include directories are present. Try leaving off the --cflags argument, so that mysql_config can supply the cflags... > -Original Message- > From: Whisky [mailto:[EMAIL PROTECTED] > Sent: Tuesday, July 11, 2006 14:18 > To: 'mailing list' > Subject: DBD:mysql installation problems > > Hello List! > > I've got quite some trouble installing DBD:mysqli (3.0006) on a Debian box > running MySQL 5.0.22. It would be great if some of you had some idea on > how > to solve this. > So here's what happens when I try doe build the package: > > -- > -- > > > SERVER:/usr/src/perl/DBD-mysql-3.0006# perl Makefile.PL --cflags='-O3 > -march=pentium4 -mtune=pentium4' > --mysql_config=/usr/local/mysql-standard-5.0.22-linux- > i686/bin/mysql_config > I will use the following settings for compiling and testing: > > cflags(User's choice) = -O3 -march=pentium4 -mtune=pentium4 > embedded (mysql_config ) = > libs (mysql_config ) = -L/usr/local/lib > -L/usr/local/mysql-standard-5.0.22-linux-i686/lib -lmysqlclient -lz - > lcrypt > -lnsl -lm -lc -lnss_files -lnss_dns -lresolv -lc -lnss_files -lnss_dns > -lresolv > mysql_config (Users choice ) = > /usr/local/mysql-standard-5.0.22-linux-i686/bin/mysql_config > nocatchstderr (default ) = 0 > nofoundrows (default ) = 0 > ssl (guessed ) = 0 > testdb(default ) = test > testhost (default ) = > testpassword (default ) = > testsocket(default ) = > testuser (default ) = > > To change these settings, see 'perl Makefile.PL --help' and > 'perldoc INSTALL'. > > Checking if your kit is complete... > Looks good > Using DBI 1.51 (for perl 5.008008 on i486-linux-gnu-thread-multi) > installed > in /usr/lib/perl5/auto/DBI/ > Writing Makefile for DBD::mysql > SERVER:/usr/src/perl/DBD-mysql-3.0006# make > cp lib/DBD/mysql.pm blib/lib/DBD/mysql.pm > cp lib/DBD/mysql/GetInfo.pm blib/lib/DBD/mysql/GetInfo.pm > cp lib/Mysql.pm blib/lib/Mysql.pm > cp lib/DBD/mysql/INSTALL.pod blib/lib/DBD/mysql/INSTALL.pod > cp lib/Mysql/Statement.pm blib/lib/Mysql/Statement.pm > cp lib/Bundle/DBD/mysql.pm blib/lib/Bundle/DBD/mysql.pm > cc -c -I/usr/lib/perl5/auto/DBI -O3 -march=pentium4 -mtune=pentium4 > -DDBD_MYSQL_INSERT_ID_IS_GOOD -g -D_REENTRANT -D_GNU_SOURCE > -DTHREADS_HAVE_PIDS -DDEBIAN -fno-strict-aliasing -pipe - > I/usr/local/include > -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O2 -DVERSION=\"3.0006\" > -DXS_VERSION=\"3.0006\" -fPIC "-I/usr/lib/perl/5.8/CORE" dbdimp.c > In file included from dbdimp.c:20: > dbdimp.h:21:49: error: mysql.h: No such file or directory > dbdimp.h:22:45: error: mysqld_error.h: No such file or directory > dbdimp.h:23:49: error: errmsg.h: No such file or directory > In file included from dbdimp.c:20: > dbdimp.h:124: error: expected specifier-qualifier-list before 'MYSQL' > dbdimp.h:214: error: expected specifier-qualifier-list before 'MYSQL_RES' > In file included from dbdimp.c:20: > dbdimp.h:268: error: expected ')' before '*' token > dbdimp.h:271: error: expected '=', ',', ';', 'asm' or '__attribute__' > before > 'mysql_st_internal_execute' > dbdimp.h:292: error: expected '=', ',', ';', 'asm' or '__attribute__' > before > '*' token > dbdimp.c:398: error: expected ')' before '*' token > dbdimp.c:666: error: 'FIELD_TYPE_VAR_STRING' undeclared here (not in a > function) > dbdimp.c:673: error: 'FIELD_TYPE_DECIMAL' undeclared here (not in a > function) > dbdimp.c:680: error: 'FIELD_TYPE_TINY' undeclared here (not in a function) > dbdimp.c:687: error: 'FIELD_TYPE_SHORT' undeclared here (not in a > function) > dbdimp.c:694: error: 'FIELD_TYPE_LONG' undeclared here (not in a function) > dbdimp.c:701: error: 'FIELD_TYPE_FLOAT' undeclared here (not in a > function) > dbdimp.c:708: error: 'FIELD_TYPE_DOUBLE' undeclared here (not in a > function) > dbdimp.c:725: error: 'FIELD_TYPE_TIMESTAMP' undeclared here (not in a > function) > dbdimp.c:732: error: 'FIELD_TYPE_LONGLONG' undeclared here (not in a > function) > dbdimp.c:739: error: 'FIELD_TYPE_INT24' undeclared here (not in a > function) > dbdimp.c:746: error: 'FIELD_TYPE_DATE' undeclared here (not in a function) > dbdimp.c:753: error: 'FIELD_TYPE_TIME' undeclared here (not in a function) > dbdimp.c:760: error: 'FIELD_TYPE_DATETIME' undeclared here (not in a > function) > dbdimp.c:767: error: 'FIELD_TYPE_YEAR' undeclared here (not in a function) > dbdimp.c:774: error: 'FIELD_TYPE_NEWDATE' undeclared here (not in a > function) > dbdimp.c:781: error: 'FIELD_TYPE_ENUM' undeclared here (not in a function) > dbdimp.c:788: error: 'FIELD_TYPE_SET' undeclared here (not in a function) > dbdimp.c:795: error: 'FIELD_TYPE_BLOB' undeclared here (not in a function) > dbdimp.c:802: error: 'FIELD_TYPE_TINY_BLOB' undeclared here (not in a > function) > dbdimp.c:809: error: 'FIELD_TYPE_MEDIUM_BLO
RE: memory leak in DBI XS bootstrap code
Tim,
I'm experiencing further memory leaks which are causing me more grief. I put
together the following test script which illustrates the leakage:
#/usr/bin/perl
# dbi.pl
use DBI;
use Devel::Leak;
my $leak_handle;
for ( 1 .. 1000 ) {
my $dbh = DBI->connect("DBI:mysql:MYDB","user","secret", {RaiseError=>1});
my $sth = $dbh->prepare("SELECT COUNT(*) FROM TABLE");
$sth->execute();
while ( my ($c) = $sth->fetchrow_array() ) {
# print "$c\n";
}
$sth->finish;
$dbh->disconnect;
system("ps -aux | fgrep dbi.pl | fgrep -v fgrep");
test_leak();
}
sub test_leak {
if ( $handle ) {
Devel::Leak::CheckSV($handle);
}
print STDERR "count: " . Devel::Leak::NoteSV($handle) . "\n";
}
Devel::Leak shows that it is leaking 4 "things" every iteration:
count: 12128
root 17720 17.8 0.3 6840 4220 pts/0S09:55 0:00 perl dbi.pl
new 0x8aad3e8 : SV = PVHV(0x8a68b30) at 0x8aad3e8
REFCNT = 1
FLAGS = (PADBUSY,PADMY,SHAREKEYS)
IV = 0
NV = 0
ARRAY = 0x0
KEYS = 0
FILL = 0
MAX = 7
RITER = -1
EITER = 0x0
new 0x8aad3f4 : SV = NULL(0x0) at 0x8aad3f4
REFCNT = 1
FLAGS = (PADBUSY,PADMY)
new 0x8aad400 : SV = NULL(0x0) at 0x8aad400
REFCNT = 1
FLAGS = (PADBUSY,PADMY)
new 0x8aad40c : SV = NULL(0x0) at 0x8aad40c
REFCNT = 1
FLAGS = (PADBUSY,PADMY)
count: 12132
I think there might be several leaks here. If I move the connect() out of the
loop, it's leaking, but less - just one of those NULL SV guys.
Can you please help me track this down? I'm still a little scared of the
DBI.xs internals, despite my recent exposure...
Thanks a lot,
edan
> -Original Message-
> From: Tim Bunce [mailto:[EMAIL PROTECTED]
> Sent: Monday, July 03, 2006 12:16
> To: Ephraim Dan
> Cc: Tim Bunce; dbi-users@perl.org
> Subject: Re: memory leak in DBI XS bootstrap code
>
> On Sat, Jul 01, 2006 at 10:42:58PM -0700, Ephraim Dan wrote:
> > Thanks for the help Tim! You've really done me a great service.
> >
> > I have created the following patch, which appears to fix all of the
> leaks. Can you please review it, and give it your stamp of approval? I
> am willing to use this patch if you approve it, until a new DBI is
> available with the fixes.
>
> Looks okay at first sight, except that Newz returns memory that's been
> reset to zero bytes. You need to add the memzero I suggested previously.
> Might as well go into malloc_using_sv since it's not performance critical.
>
> Um, looks like CLONE plus _clone_dbis plus dbi_bootinit already do the
> right thing (your patch also fixes a leak on thread creation) so that's
> okay. INIT_PERINTERP is harder to be sure about, but I think it's okay.
>
> I'll apply the patch and add the memzero. Thanks edan.
>
> Tim.
>
> > Thanks again,
> > edan
> >
> > The patch:
> >
> > --- DBI.xs.orig 2006-06-30 10:20:10.0 -0400
> > +++ DBI.xs 2006-07-02 04:59:32.0 -0400
> > @@ -171,7 +171,7 @@
> > dPERINTERP_SV; dPERINTERP_PTR(PERINTERP_t *, PERINTERP)
> > # define INIT_PERINTERP \
> > dPERINTERP; \
> > - Newz(0,PERINTERP,1,PERINTERP_t); \
> > +PERINTERP = malloc_using_sv(sizeof(PERINTERP_t));\
> > sv_setiv(perinterp_sv, PTR2IV(PERINTERP))
> >
> > # undef DBIS
> > @@ -209,13 +209,29 @@
> > stc_s, sizeof(dbih_stc_t), fdc_s, sizeof(dbih_fdc_t),
> msg);
> > }
> >
> > +static void *
> > +malloc_using_sv(STRLEN len)
> > +{
> > +dTHX;
> > +SV *sv = newSV(len);
> > +return SvPVX(sv);
> > +}
> > +
> > +static char *
> > +savepv_using_sv(char *str)
> > +{
> > +char *buf = malloc_using_sv(strlen(str));
> > +strcpy(buf, str);
> > +return buf;
> > +}
> > +
> > static void
> > dbi_bootinit(dbistate_t * parent_dbis)
> > {
> > dTHX;
> > INIT_PERINTERP;
> >
> > -Newz(dummy, DBIS, 1, dbistate_t);
> > +DBIS = (struct dbistate_st*)malloc_using_sv(sizeof(struct
> dbistate_st));
> >
> > /* store version and size so we can spot DBI/DBD version mismatch
> */
> > DBIS->check_version = check_version;
> > @@ -3793,7 +3809,7 @@
> > ima->minargs = (U8)SvIV(*av_fetch(av, 0, 1));
> > ima->maxargs = (U8)SvIV(*av_fetch(av, 1, 1));
> > svp = av_fetch(av, 2, 0);
> > - ima->usage_msg = savepv( (svp) ? SvPV(*svp,lna) : "");
> > + ima->u
RE: memory leak in DBI XS bootstrap code
Thanks for the help Tim! You've really done me a great service.
I have created the following patch, which appears to fix all of the leaks. Can
you please review it, and give it your stamp of approval? I am willing to use
this patch if you approve it, until a new DBI is available with the fixes.
Thanks again,
edan
The patch:
--- DBI.xs.orig 2006-06-30 10:20:10.0 -0400
+++ DBI.xs 2006-07-02 04:59:32.0 -0400
@@ -171,7 +171,7 @@
dPERINTERP_SV; dPERINTERP_PTR(PERINTERP_t *, PERINTERP)
# define INIT_PERINTERP \
dPERINTERP; \
- Newz(0,PERINTERP,1,PERINTERP_t); \
+PERINTERP = malloc_using_sv(sizeof(PERINTERP_t));\
sv_setiv(perinterp_sv, PTR2IV(PERINTERP))
# undef DBIS
@@ -209,13 +209,29 @@
stc_s, sizeof(dbih_stc_t), fdc_s, sizeof(dbih_fdc_t), msg);
}
+static void *
+malloc_using_sv(STRLEN len)
+{
+dTHX;
+SV *sv = newSV(len);
+return SvPVX(sv);
+}
+
+static char *
+savepv_using_sv(char *str)
+{
+char *buf = malloc_using_sv(strlen(str));
+strcpy(buf, str);
+return buf;
+}
+
static void
dbi_bootinit(dbistate_t * parent_dbis)
{
dTHX;
INIT_PERINTERP;
-Newz(dummy, DBIS, 1, dbistate_t);
+DBIS = (struct dbistate_st*)malloc_using_sv(sizeof(struct dbistate_st));
/* store version and size so we can spot DBI/DBD version mismatch */
DBIS->check_version = check_version;
@@ -3793,7 +3809,7 @@
ima->minargs = (U8)SvIV(*av_fetch(av, 0, 1));
ima->maxargs = (U8)SvIV(*av_fetch(av, 1, 1));
svp = av_fetch(av, 2, 0);
- ima->usage_msg = savepv( (svp) ? SvPV(*svp,lna) : "");
+ ima->usage_msg = (svp) ? savepv_using_sv(SvPV(*svp, lna)) : "";
ima->flags |= IMA_HAS_USAGE;
if (trace_msg && DBIS_TRACE_LEVEL >= 11)
sv_catpvf(trace_msg, ",\nusage: min %d, max %d, '%s'",
> -Original Message-
> From: Tim Bunce [mailto:[EMAIL PROTECTED]
> Sent: Friday, June 30, 2006 16:33
> To: Ephraim Dan
> Cc: Tim Bunce; dbi-users@perl.org
> Subject: Re: memory leak in DBI XS bootstrap code
>
> On Fri, Jun 30, 2006 at 04:08:46AM -0700, Ephraim Dan wrote:
> >This is why I love the perl community - you're willing to help even
> though you don't have time, since
> >you care about your code, and that other people can benefit from it.
> >
> >But ... you sort of left a dangling "But" in your last post. Do you
> have a suggestion to fix the
> >_install_method problem?
> >
> >I just ran valgrind with your fix - there seems to still be a leak.
> It is reported on DBI.xs line 216:
> >INIT_PERINTERP;
> >That looks to also have a Newz in it - is that what leaks?
>
> Yes. It's basically a plain malloc and there's no mechanism to free it
> later.
>
> > Can you suggest how to fix that one?
>
> The trick is to use newSV to alloc memory in the form of an SV
> so that when the interpreter is destroyed the SV memory gets freed
> automatically.
>
> >The _install_method leak is reported on line 3800 (in my modified
> DBI.xs) which validates your theory:
> >ima->usage_msg = savepv( (svp) ? SvPV(*svp,lna) : "");
> >If you can fix these, I'll be forever in your debt...
>
> As above, and as outlined previously, use an SV.
> Something like this (completely untested):
>
> static void *
> malloc_using_sv(STRLEN len) {
>SV *sv = newSV(len);
>return SvPVX(sv);
> }
>
> static char *
> savepv_using_sv(char *str) {
> char *buf = malloc_using_sv(strlen(str));
> strcpy(buf, str);
> return buf;
> }
>
> Tim.
>
> >--edan
> >
> >---------
> ---
> >
> >From: Tim Bunce [mailto:[EMAIL PROTECTED]
> >Sent: Fri 6/30/2006 11:35
> >To: Ephraim Dan
> >Cc: Tim Bunce; dbi-users@perl.org
> >Subject: Re: memory leak in DBI XS bootstrap code
> >
> >On Fri, Jun 30, 2006 at 12:24:33AM -0700, Ephraim Dan wrote:
> >> > > I don't see what you mean in the "INSTALL" that comes with perl
> 5.8.0 (that's what we're using).
> >> > The file called INSTALL in the perl source code directory.
> >>
> >> That I knew. What are the special instructions that I'm supposed
> to
> >> find there? Am I supposed to build it with debugging? Can you
> >> specify what special configuration exactly you meant th
RE: memory leak in DBI XS bootstrap code
This is why I love the perl community - you're willing to help even though you don't have time, since you care about your code, and that other people can benefit from it. But ... you sort of left a dangling "But" in your last post. Do you have a suggestion to fix the _install_method problem? I just ran valgrind with your fix - there seems to still be a leak. It is reported on DBI.xs line 216: INIT_PERINTERP; That looks to also have a Newz in it - is that what leaks? Can you suggest how to fix that one? The _install_method leak is reported on line 3800 (in my modified DBI.xs) which validates your theory: ima->usage_msg = savepv( (svp) ? SvPV(*svp,lna) : ""); If you can fix these, I'll be forever in your debt... --edan From: Tim Bunce [mailto:[EMAIL PROTECTED] Sent: Fri 6/30/2006 11:35 To: Ephraim Dan Cc: Tim Bunce; dbi-users@perl.org Subject: Re: memory leak in DBI XS bootstrap code On Fri, Jun 30, 2006 at 12:24:33AM -0700, Ephraim Dan wrote: > > > I don't see what you mean in the "INSTALL" that comes with perl 5.8.0 > > > (that's what we're using). > > The file called INSTALL in the perl source code directory. > > That I knew. What are the special instructions that I'm supposed to > find there? Am I supposed to build it with debugging? Can you > specify what special configuration exactly you meant that I should use? I'm sorry. I should have checked myself first. You can build perl with -DPURIFY to enable more precise leak detection. > It looks like the leak in boot_DBI is on purpose: > /* Remember the last handle used. BEWARE! Sneaky stuff here! */ > /* We want a handle reference but we don't want to increment */ > /* the handle's reference count and we don't want perl to try */ > /* to destroy it during global destruction. Take care!*/ > DBI_UNSET_LAST_HANDLE;/* ensure setup the correct way */ > > Why is this being done, and does anyone have a way to fix it? Why don't we > want perl to destroy it? Me, that's exactly what I want. I don't believe that's a leak. It's just using an SV as a reference without telling perl its a reference. If perl knew it was a reference we'd get double-free warnings as the referenced value would be freed twice. The leak is probably the DBIS structure itself: Newz(dummy, DBIS, 1, dbistate_t); That should probably be using along the lines of: sv = newSV(sizeof(struct dbistate_st)); DBIS = (struct dbistate_st*)SvPVX(sv); memzero(DBIS, sizeof(struct dbistate_st)); So the memory is allocated within an SV so gets freed on global destruction. > Still looking into the other leak in _install_method... any pointers still > appreciated... Thanks for your help so far, Tim. I've taken a look at the code. I'd guess that it's due to savepnv: ima->usage_msg = savepv( (svp) ? SvPV(*svp,lna) : ""); But Tim. > -edan > > > > -Original Message- > > > From: Tim Bunce [mailto:[EMAIL PROTECTED] > > > Sent: Thursday, June 29, 2006 14:47 > > > To: Ephraim Dan > > > Cc: dbi-users@perl.org > > > Subject: Re: memory leak in DBI XS bootstrap code > > > > > > Try building perl with options to make valgrind leak tracing more > > > effective (see perl's INSTALL file). That may help you pinpoint > > > the problem. > > > > > > Tim. > > > > > > On Thu, Jun 29, 2006 at 04:33:40AM -0700, Ephraim Dan wrote: > > > > I am experiencing what I believe to be a memory leak in the DBI > > > bootstrap code. This is a problem for me because I am embedding perl in a > > > long-running program, and DBI is being loaded over and over, so my program > > > grows and grows. > > > > > > > > The problem appears to be in the following routines: > > > > > > > > boot_DBI (in /usr/lib/perl5/vendor_perl/5.8.0/i386-linux-thread- > > > multi/auto/DBI/DBI.so) > > > > XS_DBI__install_method (in /usr/lib/perl5/vendor_perl/5.8.0/i386-linux- > > > thread-multi/auto/DBI/DBI.so) > > > > > > > > I am using DBI 1.51 > > > > > > > > The tool "valgrind" (http://valgrind.org <http://valgrind.org/> > > > > <http://valgrind.org/> ) can be used to reproduce the > > > leak using the following code: > > > > > > > > --- > > > > File: embed_test.c > > > > --- > > > > > > > > #include/* from the Perl distribution */ > > > > #incl
RE: memory leak in DBI XS bootstrap code
> > I don't see what you mean in the "INSTALL" that comes with perl 5.8.0
> > (that's what we're using).
> The file called INSTALL in the perl source code directory.
That I knew. What are the special instructions that I'm supposed to find
there? Am I supposed to build it with debugging? Can you specify what special
configuration exactly you meant that I should use?
> I've no time, sorry. You'll have to scratch this itch yourself (or hope that
> someone else has the same itch.)
It looks like the leak in boot_DBI is on purpose:
/* Remember the last handle used. BEWARE! Sneaky stuff here!*/
/* We want a handle reference but we don't want to increment*/
/* the handle's reference count and we don't want perl to try */
/* to destroy it during global destruction. Take care! */
DBI_UNSET_LAST_HANDLE; /* ensure setup the correct way */
Why is this being done, and does anyone have a way to fix it? Why don't we
want perl to destroy it? Me, that's exactly what I want.
Still looking into the other leak in _install_method... any pointers still
appreciated... Thanks for your help so far, Tim.
-edan
> > -Original Message-
> > From: Tim Bunce [mailto:[EMAIL PROTECTED]
> > Sent: Thursday, June 29, 2006 14:47
> > To: Ephraim Dan
> > Cc: dbi-users@perl.org
> > Subject: Re: memory leak in DBI XS bootstrap code
> >
> > Try building perl with options to make valgrind leak tracing more
> > effective (see perl's INSTALL file). That may help you pinpoint
> > the problem.
> >
> > Tim.
> >
> > On Thu, Jun 29, 2006 at 04:33:40AM -0700, Ephraim Dan wrote:
> > > I am experiencing what I believe to be a memory leak in the DBI
> > bootstrap code. This is a problem for me because I am embedding perl in a
> > long-running program, and DBI is being loaded over and over, so my program
> > grows and grows.
> > >
> > > The problem appears to be in the following routines:
> > >
> > > boot_DBI (in /usr/lib/perl5/vendor_perl/5.8.0/i386-linux-thread-
> > multi/auto/DBI/DBI.so)
> > > XS_DBI__install_method (in /usr/lib/perl5/vendor_perl/5.8.0/i386-linux-
> > thread-multi/auto/DBI/DBI.so)
> > >
> > > I am using DBI 1.51
> > >
> > > The tool "valgrind" (http://valgrind.org <http://valgrind.org/> ) can be
> > > used to reproduce the
> > leak using the following code:
> > >
> > > ---
> > > File: embed_test.c
> > > ---
> > >
> > > #include/* from the Perl distribution */
> > > #include /* from the Perl distribution */
> > >
> > > static PerlInterpreter *my_perl; /***The Perl interpreter***/
> > > EXTERN_C void xs_init (pTHX); /***init dyn. loading ***/
> > >
> > > int main(int argc, char **argv, char **env)
> > > {
> > > char *embedding[] = { "", "-e", "0" };
> > > my_perl = perl_alloc();
> > > perl_construct(my_perl);
> > > perl_parse(my_perl, xs_init, 3, embedding, (char **)NULL);
> > > PL_exit_flags |= PERL_EXIT_DESTRUCT_END;
> > > perl_run(my_perl);
> > > eval_pv("use DBI", TRUE);
> > > perl_destruct(my_perl);
> > > perl_free(my_perl);
> > > }
> > >
> > > ---
> > > File: Makefile
> > > ---
> > >
> > > CC_OPTS = $(shell perl -MExtUtils::Embed -e ccopts)
> > > LD_OPTS = $(shell perl -MExtUtils::Embed -e ldopts)
> > >
> > > EXE = embed_test
> > >
> > > $(EXE): xsinit.o embed_test.o
> > > gcc -o $(EXE) embed_test.o xsinit.o $(LD_OPTS)
> > >
> > > embed_test.o: embed_test.c
> > > gcc -c embed_test.c $(CC_OPTS)
> > >
> > > xsinit.o: xsinit.c
> > > gcc -c xsinit.c $(CC_OPTS)
> > >
> > > xsinit.c:
> > > perl -MExtUtils::Embed -e xsinit -- -o xsinit.c
> > >
> > > clean:
> > > rm -f *.o xsinit.c $(EXE)
> > >
> > > ---
> > > EOF
> > > ---
> > >
> > > Can anyone suggest a fix for this? I'd be more than willing to take a
> > patch to DBI 1.51 as soon as someone has one.
> > >
> > > Thanks,
> > > Ephraim Dan
> > >
RE: memory leak in DBI XS bootstrap code
I don't see what you mean in the "INSTALL" that comes with perl 5.8.0 (that's
what we're using). Can you be more specific? Would I need to build perl in a
special way, or DBI, or both?
I was sort of hoping someone more familiar with the code might be able to
pinpoint it (of course, this is always preferable to doing it myself).
Any further pointers are appreciated...
-edan
> -Original Message-
> From: Tim Bunce [mailto:[EMAIL PROTECTED]
> Sent: Thursday, June 29, 2006 14:47
> To: Ephraim Dan
> Cc: dbi-users@perl.org
> Subject: Re: memory leak in DBI XS bootstrap code
>
> Try building perl with options to make valgrind leak tracing more
> effective (see perl's INSTALL file). That may help you pinpoint
> the problem.
>
> Tim.
>
> On Thu, Jun 29, 2006 at 04:33:40AM -0700, Ephraim Dan wrote:
> > I am experiencing what I believe to be a memory leak in the DBI
> bootstrap code. This is a problem for me because I am embedding perl in a
> long-running program, and DBI is being loaded over and over, so my program
> grows and grows.
> >
> > The problem appears to be in the following routines:
> >
> > boot_DBI (in /usr/lib/perl5/vendor_perl/5.8.0/i386-linux-thread-
> multi/auto/DBI/DBI.so)
> > XS_DBI__install_method (in /usr/lib/perl5/vendor_perl/5.8.0/i386-linux-
> thread-multi/auto/DBI/DBI.so)
> >
> > I am using DBI 1.51
> >
> > The tool "valgrind" (http://valgrind.org) can be used to reproduce the
> leak using the following code:
> >
> > ---
> > File: embed_test.c
> > ---
> >
> > #include/* from the Perl distribution */
> > #include /* from the Perl distribution */
> >
> > static PerlInterpreter *my_perl; /***The Perl interpreter***/
> > EXTERN_C void xs_init (pTHX); /***init dyn. loading ***/
> >
> > int main(int argc, char **argv, char **env)
> > {
> > char *embedding[] = { "", "-e", "0" };
> > my_perl = perl_alloc();
> > perl_construct(my_perl);
> > perl_parse(my_perl, xs_init, 3, embedding, (char **)NULL);
> > PL_exit_flags |= PERL_EXIT_DESTRUCT_END;
> > perl_run(my_perl);
> > eval_pv("use DBI", TRUE);
> > perl_destruct(my_perl);
> > perl_free(my_perl);
> > }
> >
> > ---
> > File: Makefile
> > ---
> >
> > CC_OPTS = $(shell perl -MExtUtils::Embed -e ccopts)
> > LD_OPTS = $(shell perl -MExtUtils::Embed -e ldopts)
> >
> > EXE = embed_test
> >
> > $(EXE): xsinit.o embed_test.o
> > gcc -o $(EXE) embed_test.o xsinit.o $(LD_OPTS)
> >
> > embed_test.o: embed_test.c
> > gcc -c embed_test.c $(CC_OPTS)
> >
> > xsinit.o: xsinit.c
> > gcc -c xsinit.c $(CC_OPTS)
> >
> > xsinit.c:
> > perl -MExtUtils::Embed -e xsinit -- -o xsinit.c
> >
> > clean:
> > rm -f *.o xsinit.c $(EXE)
> >
> > ---
> > EOF
> > ---
> >
> > Can anyone suggest a fix for this? I'd be more than willing to take a
> patch to DBI 1.51 as soon as someone has one.
> >
> > Thanks,
> > Ephraim Dan
> >
memory leak in DBI XS bootstrap code
I am experiencing what I believe to be a memory leak in the DBI bootstrap code.
This is a problem for me because I am embedding perl in a long-running
program, and DBI is being loaded over and over, so my program grows and grows.
The problem appears to be in the following routines:
boot_DBI (in
/usr/lib/perl5/vendor_perl/5.8.0/i386-linux-thread-multi/auto/DBI/DBI.so)
XS_DBI__install_method (in
/usr/lib/perl5/vendor_perl/5.8.0/i386-linux-thread-multi/auto/DBI/DBI.so)
I am using DBI 1.51
The tool "valgrind" (http://valgrind.org) can be used to reproduce the leak
using the following code:
---
File: embed_test.c
---
#include/* from the Perl distribution */
#include /* from the Perl distribution */
static PerlInterpreter *my_perl; /***The Perl interpreter***/
EXTERN_C void xs_init (pTHX); /***init dyn. loading ***/
int main(int argc, char **argv, char **env)
{
char *embedding[] = { "", "-e", "0" };
my_perl = perl_alloc();
perl_construct(my_perl);
perl_parse(my_perl, xs_init, 3, embedding, (char **)NULL);
PL_exit_flags |= PERL_EXIT_DESTRUCT_END;
perl_run(my_perl);
eval_pv("use DBI", TRUE);
perl_destruct(my_perl);
perl_free(my_perl);
}
---
File: Makefile
---
CC_OPTS = $(shell perl -MExtUtils::Embed -e ccopts)
LD_OPTS = $(shell perl -MExtUtils::Embed -e ldopts)
EXE = embed_test
$(EXE): xsinit.o embed_test.o
gcc -o $(EXE) embed_test.o xsinit.o $(LD_OPTS)
embed_test.o: embed_test.c
gcc -c embed_test.c $(CC_OPTS)
xsinit.o: xsinit.c
gcc -c xsinit.c $(CC_OPTS)
xsinit.c:
perl -MExtUtils::Embed -e xsinit -- -o xsinit.c
clean:
rm -f *.o xsinit.c $(EXE)
---
EOF
---
Can anyone suggest a fix for this? I'd be more than willing to take a patch to
DBI 1.51 as soon as someone has one.
Thanks,
Ephraim Dan
