Hello Martin,
Sorry for the late reply.
Basically what my script tries to do is to copy lots of records from one
database to another. To do this, I've come to the idea of using "execute
array" to speed up the process. I generally use NLS_LANG set to UTF-8
for Oracle, because it's the default encoding on my Linux distribution,
so it was not a choice I've made for that specific application.
It's when reviewing the code (after a little help from valgrind
memory-checker), that I got the idea to turn UTF-8 off. Specifically the
lines around ~ 3825 : ...
/* update the utf8_flgs for this value */
if (SvUTF8(sv)) {
utf8_flgs[i] |= ARRAY_BIND_UTF8;
if (SvTRUE(tuples_status)){
av_push(tuples_utf8_av,newSViv(j));
}
... made me think it was probably worth a try.
As so far I understand, the 'tuples_utf8_av' is not given back to perl
interpreter, I assumed that it was safe to decrement it's reference
count to allow it to be garbage-collected. But as I said in my previous
mail, I've no experience in Perl internals and that may be a false
assumption.
Thanks very much for your feedback.
Best regards,
Pierre-Alain Blanc
On 09/10/12 09:53, Martin J. Evans wrote:
On 08/10/12 13:46, Pierre-Alain Blanc wrote:
Hello,
I've had a problem when using 'execute_array' to insert (lots of)
records with DBD::Oracle (version 1.50): the script consumed too much
memory and finally crashed (killed by kernel). I tried to trigger the
garbage-collection with some code rewrite but it didn't help. But if
I told Oracle *not* to use an UTF-8 charset (changing NLS_LANG from
(for example) "german_germany.utf8" to "german_germany.we8dec"), the
problem disappeared.
After some investigations, I think the leak is in
'ora_st_execute_array' method of dbdimp.c. Please find the patch as
attachment. As I'm completly new to writing C for Perl, it may be
something I did not understand or did not correctly fixed. Sorry if
it would be the case.
Thanks & best regards, Pierre-Alain Blanc
Thanks for looking into this. The patch:
Index: dbdimp.c
===================================================================
--- dbdimp.c (revision 15435)
+++ dbdimp.c (working copy)
@@ -3839,6 +3839,7 @@
}
Safefree(phs);
Safefree(utf8_flgs);
+ SvREFCNT_dec(tuples_utf8_av);
/* Store array of bind typles, for use in OCIBindDynamic()
callback. */
imp_sth->bind_tuples = tuples_av;
imp_sth->rowwise = (columns_av == NULL);
looks good but it does not explain how changing chrset made the
problem go away. Perhaps you could give me a better idea of what you
were doing then I can replicate and test it.
Martin
