On 22/02/2011 17:40, Tim Bunce wrote:
On Mon, Feb 21, 2011 at 05:59:34PM +0000, Martin J. Evans wrote:
It would appear this problem and possibly other leaks are attracting quite a
lot of attention now. Someone else just posted another cpan rating mentioning
leaks at:
http://cpanratings.perl.org/dist/DBD-Pg#8254
I narrowed down one of the problems (below) but it is beyond me now - sorry.
Using perl 5.12.2 I couldn't reproduce it with:
perl -MDBI -we '$a = DBI::_concat_hash_sorted({1..10000},"=",",",1,1)
while 1'
nor with:
perl -Mblib -MDBI -we
'$sth=DBI->connect("dbi:NullP:",0,0,{ShowErrorStatement=>1})->prepare("ERROR 1 err");
$sth->execute(1..10000) while 1' 2> /dev/null
The ticket says "Other DBI drivers do not seem to leak" so I suspect the
leak is in DBD::Pg's dbd_st_FETCH_attrib function (in dbdimp.c) where it
handles the ParamValues attribute by constructing a hash. (That's called
from DBI.xs by the mg_get(*svp);)
Thanks. I was not suggesting the problem was in DBI, just that by
ifdeffing some code out in DBI the problem goes away and that might help
identify the problem.
I've no time to dig deeper, sorry. (I'll copy the above to the ticket.)
Tim.
This may be useful:
http://www.perlmonks.org/?node_id=52609
as I noted, I use hv_store in DBD::ODBC and DBD::Pg uses hv_store_ent.
It appears if you use hv_store_ent you may need to look more closely at
the key - see the link above.
Hopefully this helps Greg. I don't have access to a postgres db right
now to test it out.
Martin
Martin
--
Martin J. Evans
Easysoft Limited
http://www.easysoft.com
On 13/01/11 18:53, Martin J. Evans wrote:
I briefly looked at the DBD::Pg rt ticket
https://rt.cpan.org/Ticket/Display.html?id=60863 and I'm not sure this is a bug
in DBD::Pg and think it might be in DBI. Since ShowErrorStatement set causes
the leak and unset the leak goes away I isolated it to some code in DBI.xs:
if ( DBIc_has(imp_xxh, DBIcf_ShowErrorStatement)
&& !is_unrelated_to_Statement
&& (DBIc_TYPE(imp_xxh) == DBIt_ST || ima_flags& IMA_SHOW_ERR_STMT)
&& (statement_svp = hv_fetch((HV*)SvRV(h), "Statement", 9, 0))
&& statement_svp&& SvOK(*statement_svp)
) {
SV **svp = 0;
sv_catpv(msg, " [for Statement \"");
sv_catsv(msg, *statement_svp);
/* fetch from tied outer handle to trigger FETCH magic */
/* could add DBIcf_ShowErrorParams (default to on?) */
#ifdef THIS_ONE_STOPS_LEAK
if (!(ima_flags& IMA_HIDE_ERR_PARAMVALUES)) {
svp = hv_fetch((HV*)DBIc_MY_H(imp_xxh),"ParamValues",11,FALSE);
if (svp&& SvMAGICAL(*svp))
mg_get(*svp); /* XXX may recurse, may croak. could use eval */
}
#endif
if (svp&& SvRV(*svp)&& SvTYPE(SvRV(*svp)) == SVt_PVHV&& HvKEYS(SvRV(*svp))>0
) {
#ifndef THESE_TWO_WITHOUT_ABOVE_DOES_NOT_STOP_LEAK
SV *param_values_sv = sv_2mortal(_join_hash_sorted((HV*)SvRV(*svp), "=",1, ",
",2, 1, -1));
#endif
sv_catpv(msg, "\" with ParamValues: ");
#ifndef THESE_TWO_WITHOUT_ABOVE_DOES_NOT_STOP_LEAK
sv_catsv(msg, param_values_sv);
#endif
sv_catpvn(msg, "]", 1);
}
else {
sv_catpv(msg, "\"]");
}
}
I was suspicious about _join_hash_sorted but it appears if you just take
THESE_TWO_WITHOUT_ABOVE_DOES_NOT_STOP_LEAK out it still leaks.
If you just take THIS_ONE_STOPS_LEAK (which obviously also takes out the other
2) the leak goes away.
I've not duplicated the problem with any other DBDs as yet but I'm running the
exact code in the RT.
I've no idea as yet what the actual problem is.
I was running Perl 5.10.1 on Linux (32bit) but no one who has tried this yet
(on multiple Perls in Linux) has failed to see the issue.
With THIS_ONE_STOPS_LEAK defined I get:
martin@bragi:/tmp/DBD-Pg-2.17.2$ perl -I blib/lib -I blib/arch leak.pl
Cycles: 2000 Proc size: 13700K
Cycles: 4000 Proc size: 13700K
Cycles: 6000 Proc size: 13700K
Cycles: 8000 Proc size: 13700K
Cycles: 10000 Proc size: 13700K
Cycles: 12000 Proc size: 13700K
Cycles: 14000 Proc size: 13700K
Cycles: 16000 Proc size: 13700K
Cycles: 18000 Proc size: 13700K
and a stock DBI from trunk) I get:
martin@bragi:/tmp/DBD-Pg-2.17.2$ perl -I blib/lib -I blib/arch leak.pl
Cycles: 2000 Proc size: 16792K
Cycles: 4000 Proc size: 19892K
Cycles: 6000 Proc size: 22980K
Cycles: 8000 Proc size: 26200K
Cycles: 10000 Proc size: 29296K
Cycles: 12000 Proc size: 32376K
Cycles: 14000 Proc size: 35472K
Cycles: 16000 Proc size: 38692K
Cycles: 18000 Proc size: 41796K
Martin
