This seems to be a threading error with the linux kernel version. I am running this process on newer kernels (2.6.22.x) and the error never occurs. We also are experiencing a lot the "Futex WAIT" issues with Oracle and the 2.6.20 kernels.
Thanks for listening. On Fri, 2008-03-07 at 11:25 -0600, Scott T. Hildreth wrote: > Should have posted to users not dev. This is really a bizarre problem. > I can get it to fail about every fifth iteration otherwise the process > works. I ran it from another server connect to the same database and > it will intermittently fail. I run it from a third sever and I can't > get it to core dump. All 3 servers have the same kernel & Perl > versions. I've tried recompiling Perl, DBI, DBD::Oracle, still no luck. > I created a test case, which uses execute_array and of course I can't > get it to core dump. If anyone has any ideas on what might be going on > here, I would love to hear them! > > Thanks > STH > > On Wed, 2008-03-05 at 15:21 -0600, Scott T. Hildreth wrote: > > I am not sure how to describe this, my co-worker will run his process and > > get a core dump > > (I pasted the back trace below) and then run the process again with no core > > dumps. Sometimes > > it will core dump several times in a row and then the next run it finishes > > fine. I ran the process > > with DBI_TRACE=9 and this is what shows up at the end of the log, > > > > > > 1 -> execute_for_fetch for DBD::Oracle::st (DBI::st=HASH(0xN)~INNER > > CODE(0xN) undef) > > ora_st_execute_array UPDATE count=10 (ARRAY(0xN) undef undef)... > > > > OCIBindByName(112df38,1132188,10a9138,":p1",placeh_len=3,value_p=0,value_sz=-1517274788,dty=1,indp=0,alenp=0,rcodep=0,maxarr_len=0,curelep=0 > > (*=0),mode=2)=ERROR > > OCIErrorGet(10a9138,1,"<NULL>",7fff058d684c,"ORA-02005: implicit > > (-1) length not valid for this bind or define datatype > > ",1024,2)=SUCCESS > > OCIErrorGet after OCIBindByName (er1:ok): -1, 2005: ORA-02005: implicit > > (-1) length not valid for this bind or define datatype > > > > OCIErrorGet(10a9138,2,"<NULL>",7fff058d684c,"ORA-02005: implicit > > (-1) length not valid for this bind or define datatype > > ",1024,2)=NO_DATA > > > > At first I thought it was a 32bit library with a 64bit Perl problem, but > > Oracle.so & Perl are both linked > > with the correct 64 bit libs. The Oracle client is 10.2.0.3 and DBI > > versions are, > > > > Perl : 5.008008 (x86_64-linux) > > OS : linux (2.6.20.19) > > DBI : 1.602 > > DBD::mysql : 4.005 > > DBD::Sponge : 12.010002 > > DBD::SQLite : 1.13 > > DBD::Proxy : 0.2004 > > DBD::Oracle : 1.20 > > DBD::Multiplex : 2.04 > > DBD::Gofer : 0.010103 > > DBD::File : 0.35 > > DBD::ExampleP : 12.010007 > > DBD::DBM : 0.03 > > > > I am going to try to isolate a small test case, but right now I wanted to > > post what I > > have found so far. > > > > Thanks, > > STH > > > > ############## Back Trace > > ############################################################# > > > > (gdb) bt > > #0 0x00002b66ec7d9b95 in raise () from /lib64/libc.so.6 > > #1 0x00002b66ec7daf90 in abort () from /lib64/libc.so.6 > > #2 0x00002b66ec81035b in __libc_message () from /lib64/libc.so.6 > > #3 0x00002b66ec81534e in malloc_printerr () from /lib64/libc.so.6 > > #4 0x00002b66ec81695c in free () from /lib64/libc.so.6 > > #5 0x00002b66ef0ac102 in ora_st_execute_array () from > > /usr/local/perl-5.8.8/lib/site_perl/5.8.8/x86_64-linux/auto/DBD/Oracle/Oracle.so > > #6 0x00002b66ef0a62bf in XS_DBD__Oracle__st_ora_execute_array () > > from > > /usr/local/perl-5.8.8/lib/site_perl/5.8.8/x86_64-linux/auto/DBD/Oracle/Oracle.so > > #7 0x000000000046bc47 in Perl_pp_entersub () > > #8 0x000000000046a29e in Perl_runops_standard () > > #9 0x000000000041e82d in Perl_call_sv () > > #10 0x00002b66ec9ee038 in XS_DBI_dispatch () from > > /usr/local/perl-5.8.8/lib/site_perl/5.8.8/x86_64-linux/auto/DBI/DBI.so > > #11 0x000000000046bc47 in Perl_pp_entersub () > > #12 0x000000000046a29e in Perl_runops_standard () > > #13 0x000000000041e82d in Perl_call_sv () > > #14 0x00002b66ec9ee038 in XS_DBI_dispatch () from > > /usr/local/perl-5.8.8/lib/site_perl/5.8.8/x86_64-linux/auto/DBI/DBI.so > > #15 0x000000000046bc47 in Perl_pp_entersub () > > #16 0x000000000046a29e in Perl_runops_standard () > > #17 0x000000000041f1d1 in perl_run () > > #18 0x000000000041ba2c in main () > > > > ######################################################################################## > > > > *** glibc detected *** /usr/local/bin/perl: double free or corruption > > (!prev): 0x0000000001163e10 *** > > ======= Backtrace: ========= > > /lib64/libc.so.6[0x2ad39584e34e] > > /lib64/libc.so.6(__libc_free+0x6c)[0x2ad39584f95c] > > /usr/local/perl-5.8.8/lib/site_perl/5.8.8/x86_64-linux/auto/DBD/Oracle/Oracle.so(ora_st_execute_array+0xfa4)[0x2ad3980e6e94] > > /usr/local/perl-5.8.8/lib/site_perl/5.8.8/x86_64-linux/auto/DBD/Oracle/Oracle.so(XS_DBD__Oracle__st_ora_execute_array+0xef)[0x2ad3980e0f9f] > > /usr/local/bin/perl(Perl_pp_entersub+0x6b7)[0x46bae7] > > /usr/local/bin/perl(Perl_runops_standard+0xe)[0x46a13e] > > /usr/local/bin/perl(Perl_call_sv+0x49d)[0x41e80d] > > /usr/local/perl-5.8.8/lib/site_perl/5.8.8/x86_64-linux/auto/DBI/DBI.so(XS_DBI_dispatch+0x7a8)[0x2ad395a27068] > > /usr/local/bin/perl(Perl_pp_entersub+0x6b7)[0x46bae7] > > /usr/local/bin/perl(Perl_runops_standard+0xe)[0x46a13e] > > /usr/local/bin/perl(Perl_call_sv+0x49d)[0x41e80d] > > /usr/local/perl-5.8.8/lib/site_perl/5.8.8/x86_64-linux/auto/DBI/DBI.so(XS_DBI_dispatch+0x7a8)[0x2ad395a27068] > > /usr/local/bin/perl(Perl_pp_entersub+0x6b7)[0x46bae7] > > /usr/local/bin/perl(Perl_runops_standard+0xe)[0x46a13e] > > /usr/local/bin/perl(perl_run+0x2c1)[0x41f1b1] > > /usr/local/bin/perl(main+0xac)[0x41ba2c] > > /lib64/libc.so.6(__libc_start_main+0xf4)[0x2ad395800154] > > /usr/local/bin/perl[0x41b8e9] > > ======= Memory map: ======== > > 00400000-004fc000 r-xp 00000000 08:02 427095 > > /usr/local/perl-5.8.8/bin/perl > > 005fb000-00601000 rw-p 000fb000 08:02 427095 > > /usr/local/perl-5.8.8/bin/perl > > 00601000-01183000 rw-p 00601000 00:00 0 > > [heap] > > 2ad39511b000-2ad395136000 r-xp 00000000 08:02 4295 > > /lib64/ld-2.4.so > > 2ad395136000-2ad395137000 rw-p 2ad395136000 00:00 0 > > 2ad395147000-2ad395148000 rw-p 2ad395147000 00:00 0 > > 2ad395235000-2ad395237000 rw-p 0001a000 08:02 4295 > > /lib64/ld-2.4.so > > 2ad395237000-2ad39524a000 r-xp 00000000 08:02 4168 > > /lib64/libnsl-2.4.so > > 2ad39524a000-2ad395349000 ---p 00013000 08:02 4168 > > /lib64/libnsl-2.4.so > > 2ad395349000-2ad39534b000 rw-p 00012000 08:02 4168 > > /lib64/libnsl-2.4.so > > 2ad39534b000-2ad39534d000 rw-p 2ad39534b000 00:00 0 > > 2ad39534d000-2ad39534f000 r-xp 00000000 08:02 4163 > > /lib64/libdl-2.4.so > > 2ad39534f000-2ad39544f000 ---p 00002000 08:02 4163 > > /lib64/libdl-2.4.so > > 2ad39544f000-2ad395451000 rw-p 00002000 08:02 4163 > > /lib64/libdl-2.4.so > > 2ad395451000-2ad3954a5000 r-xp 00000000 08:02 4165 > > /lib64/libm-2.4.so > > 2ad3954a5000-2ad3955a4000 ---p 00054000 08:02 4165 > > /lib64/libm-2.4.so > > 2ad3955a4000-2ad3955a6000 rw-p 00053000 08:02 4165 > > /lib64/libm-2.4.so > > 2ad3955a6000-2ad3955a7000 rw-p 2ad3955a6000 00:00 0 > > 2ad3955a7000-2ad3955b0000 r-xp 00000000 08:02 4161 > > /lib64/libcrypt-2.4.so > > 2ad3955b0000-2ad3956af000 ---p 00009000 08:02 4161 > > /lib64/libcrypt-2.4.so > > 2ad3956af000-2ad3956b2000 rw-p 00008000 08:02 4161 > > /lib64/libcrypt-2.4.so > > 2ad3956b2000-2ad3956e0000 rw-p 2ad3956b2000 00:00 0 > > 2ad3956e0000-2ad3956e2000 r-xp 00000000 08:02 4191 > > /lib64/libutil-2.4.so > > 2ad3956e2000-2ad3957e1000 ---p 00002000 08:02 4191 > > /lib64/libutil-2.4.so > > 2ad3957e1000-2ad3957e3000 rw-p 00001000 08:02 4191 > > /lib64/libutil-2.4.so > > 2ad3957e3000-2ad39590a000 r-xp 00000000 08:02 4157 > > /lib64/libc-2.4.so > > 2ad39590a000-2ad395a0a000 ---p 00127000 08:02 4157 > > /lib64/libc-2.4.so > > 2ad395a0a000-2ad395a0d000 r--p 00127000 08:02 4157 > > /lib64/libc-2.4.so > > 2ad395a0d000-2ad395a0f000 rw-p 0012a000 08:02 4157 > > /lib64/libc-2.4.so > > 2ad395a0f000-2ad395a16000 rw-p 2ad395a0f000 00:00 0 > > 2ad395a16000-2ad395a30000 r-xp 00000000 08:02 442510 > > /usr/local/perl-5.8.8/lib/site_perl/5.8.8/x86_64-linux/auto/DBI/DBI.so > > 2ad395a30000-2ad395b30000 ---p 0001a000 08:02 442510 > > /usr/local/perl-5.8.8/lib/site_pzsh: 26245 abort (core dumped) DBI_TRACE=2