Re: segfault with rlm_perl
Anatoly Ivanov anatoly.iva...@gmail.com wrote: I am running freeradius (2.1.8) with rlm_perl (5.10.1, USE_ITHREADS) on a Debian-Lenny system. The problem is radius fails with segfault ??? periodically and intermittently. I have no way to reliably reproduce the problem ??? it happens only in production, and it is impossible to reliably predict when or backtrace why. Stab in the dark, is your Perl code actually thread safe...including *all* the Perl modules you use? Seems DBD::MySQL is safe (that actually surprises me): http://search.cpan.org/~capttofu/DBD-mysql-4.018/lib/DBD/mysql.pm#MULTITHREADING Are you using any global variables in your script, and if so have you protected them from concurrent access? The really quick test is to tell freeradius not to enable threading to your module and comment out 'use threads(::shared)?;' in your code. It seems that I am running into some kind of memory allocation error. Coredump type #3 (see below) is the most popular one; coredumps with backtrace going into perl seem to be rather random (it fails in different parts of libperl.so) - again, see below. Looks like if you install some more debugging libraries (libc6-dbg and any others you think might help, looks like you have already installed perl-debug) a few more blanks will be filled in. Cheers -- Alexander Clouter .sigmonster says: Avoid gunfire in the bathroom tonight. - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
Re: segfault with rlm_perl
On Jan 4, 2011, at 9:59 PM, Anatoly Ivanov wrote: Hi, I am running freeradius (2.1.8) with rlm_perl (5.10.1, USE_ITHREADS) on a Debian-Lenny system. The problem is radius fails with segfault – periodically and intermittently. I have no way to reliably reproduce the problem – it happens only in production, and it is impossible to reliably predict when or backtrace why. This is have to be thread problem. Use CLONE function in your perl code (this is executed per every new thread) to initialize per thread variables and DBI Connections. It seems that I am running into some kind of memory allocation error. Coredump type #3 (see below) is the most popular one; coredumps with backtrace going into perl seem to be rather random (it fails in different parts of libperl.so) - again, see below. If you want to debug - search this list or google using keywords debug rlm_perl I understand that freeradius has a newer version available - but I am hesitant to upgrade a production server without a very good reason. And I could not find such reason for an upgrade after reading the CHANGELOG for 2.1.10. But maybe I am wrong? Any ideas? Thank you! Best Regards, Boian Jordanov Head of Voice Department tel. +359 2 4004 723 tel. +359 2 4004 002 - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
segfault with rlm_perl
Hi, I am running freeradius (2.1.8) with rlm_perl (5.10.1, USE_ITHREADS) on a Debian-Lenny system. The problem is radius fails with segfault – periodically and intermittently. I have no way to reliably reproduce the problem – it happens only in production, and it is impossible to reliably predict when or backtrace why. It seems that I am running into some kind of memory allocation error. Coredump type #3 (see below) is the most popular one; coredumps with backtrace going into perl seem to be rather random (it fails in different parts of libperl.so) - again, see below. I understand that freeradius has a newer version available - but I am hesitant to upgrade a production server without a very good reason. And I could not find such reason for an upgrade after reading the CHANGELOG for 2.1.10. But maybe I am wrong? Any ideas? Thank you! PS: bt for coredump #1: #0 0xb47d1ab6 in mysql_st_execute (sth=0xaf13a1a8, imp_sth=0xadf537a8) at dbdimp.c:3209 #1 0xb47da215 in XS_DBD__mysql__st_execute (my_perl=0xb831a08, cv=0xb1e16858) at mysql.xsi:588 #2 0xb70740ab in XS_DBI_dispatch () from /opt/server/lib/perl5/i686-linux-thread-multi/auto/DBI/DBI.so #3 0xb76a841e in Perl_pp_entersub () from /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so #4 0xb76a6841 in Perl_runops_standard () from /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so #5 0xb7641500 in Perl_call_sv () from /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so #6 0xb7642284 in Perl_call_pv () from /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so #7 0xb72026ad in rlmperl_call () from /opt/server/radius/lib/rlm_perl.so #8 0x08063524 in modcall () #9 0x0805ffe7 in indexed_modcall () #10 0x080602fc in module_accounting () #11 0x0804e9d1 in rad_accounting () #12 0x0806df65 in radius_handle_request () #13 0x080661d0 in request_handler_thread () #14 0xb759c4c0 in start_thread () from /lib/i686/cmov/libpthread.so.0 #15 0xb734384e in clone () from /lib/i686/cmov/libc.so.6 bt for coredump #2: #0 0xb7690c80 in Perl_pad_undef () from /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so #1 0xb764db67 in Perl_cv_undef () from /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so #2 0xb76dcf5a in Perl_sv_clear () from /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so #3 0xb76dd288 in Perl_sv_free2 () from /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so #4 0xb76dd391 in Perl_sv_free () from /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so #5 0xb76dcd57 in Perl_sv_clear () from /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so #6 0xb76dd288 in Perl_sv_free2 () from /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so #7 0xb70af022 in XS_Sys__Syslog_closelog_xs () from /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/auto/Sys/Syslog/Syslog.so #8 0xb76c941e in Perl_pp_entersub () from /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so #9 0xb76c7841 in Perl_runops_standard () from /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so #10 0xb7662500 in Perl_call_sv () from /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so #11 0xb7663284 in Perl_call_pv () from /opt/server/lib/perl5/5.10.1/i686-linux-thread-multi/CORE/libperl.so #12 0xb72236ad in rlmperl_call () from /opt/server/radius/lib/rlm_perl.so #13 0x08063524 in modcall () #14 0x0805ffe7 in indexed_modcall () #15 0x0806037c in module_authenticate () #16 0x0804f908 in rad_authenticate () #17 0x0806df65 in radius_handle_request () #18 0x080661d0 in request_handler_thread () #19 0xb75bd4c0 in start_thread () from /lib/i686/cmov/libpthread.so.0 #20 0xb736484e in clone () from /lib/i686/cmov/libc.so.6 bt for coredump #3: #0 0xb77a2424 in __kernel_vsyscall () #1 0xb72d6640 in raise () from /lib/i686/cmov/libc.so.6 #2 0xb72d8018 in abort () from /lib/i686/cmov/libc.so.6 #3 0xb731348d in ?? () from /lib/i686/cmov/libc.so.6 #4 0x0005 in ?? () #5 0xb583f614 in ?? () #6 0x0400 in ?? () #7 0xb73e97c8 in ?? () from /lib/i686/cmov/libc.so.6 #8 0x0017 in ?? () #9 0xbfd8b66b in ?? () #10 0x0020 in ?? () #11 0xb73e97e1 in ?? () from /lib/i686/cmov/libc.so.6 #12 0x0002 in ?? () #13 0xb73e985c in ?? () from /lib/i686/cmov/libc.so.6 #14 0x0023 in ?? () #15 0xb73e97e5 in ?? () from /lib/i686/cmov/libc.so.6 #16 0x0004 in ?? () #17 0xb583fb43 in ?? () #18 0x0008 in ?? () #19 0xb73e97eb in ?? () from /lib/i686/cmov/libc.so.6 #20 0x0005 in ?? () #21 0xb72eb388 in vfprintf () from /lib/i686/cmov/libc.so.6 #22 0xb7319764 in ?? () from /lib/i686/cmov/libc.so.6 #23 0x0002 in ?? () #24 0xb73e97c8 in ?? () from /lib/i686/cmov/libc.so.6 #25 0xbfd8b66b in ?? () #26 0xb73e985c in ?? () from /lib/i686/cmov/libc.so.6 #27 0xb583fb43 in ?? () #28
Re: segfault with rlm_perl
On 2011/01/04 09:59 PM, Anatoly Ivanov wrote: Hi, I am running freeradius (2.1.8) with rlm_perl (5.10.1, USE_ITHREADS) on a Debian-Lenny system. The problem is radius fails with segfault – periodically and intermittently. I have no way to reliably reproduce the problem – it happens only in production, and it is impossible to reliably predict when or backtrace why. It seems that I am running into some kind of memory allocation error. Coredump type #3 (see below) is the most popular one; coredumps with backtrace going into perl seem to be rather random (it fails in different parts of libperl.so) - again, see below. I understand that freeradius has a newer version available - but I am hesitant to upgrade a production server without a very good reason. And I could not find such reason for an upgrade after reading the CHANGELOG for 2.1.10. But maybe I am wrong? Any ideas? A complete gut feel after reading this says you have a hardware problem - faulty ram. Has this happened from the beginning or suddenly now? You can try memtesting (http://www.memtest.org/) the server, or a trick that I've found works sometimes (if you can't take the server out of production) to show a ram problem, is to compile a kernel. I've seen compilation fail at different stages with faulty RAM. I realiuse the advice might sound ridiculous, but it has worked for me before. -- Johan Meiring Cape PC Services CC Tel: (021) 883-8271 Fax: (021) 886-7782 - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
segfault when rlm_perl used
Hi, i build FR 2.1.7 on slackware 13.0 (glibc-2.9) with perl 5.10.0. every time, i kill radiusd, message like radiusd[5754]: segfault at 0 ip b73e2213 sp bfb86dac error 4 in libc-2.9.so[b736b000+15a000] appears in the log. This is not good bad, but worst when radiusd crashed on run-time. Then in logs appears message like: Oct 13 21:58:53 rs kernel: radiusd[11441]: segfault at 20004 ip b7478636 sp b502bcb0 error 4 in rlm_perl-2.1.7.so[b73cb000+15b000] Oct 14 22:09:56 rs kernel: radiusd[17687]: segfault at 8 ip b731e35d sp b6f61ce0 error 4 in rlm_perl-2.1.7.so[b71fd000+15b000] Oct 14 22:14:06 rs kernel: radiusd[18374]: segfault at 8 ip b73d735d sp b701ace0 error 4 in rlm_perl-2.1.7.so[b72b6000+15b000] Oct 15 18:07:58 rs kernel: radiusd[23858]: segfault at 8 ip b745935d sp ae89cce0 error 4 in rlm_perl-2.1.7.so[b7338000+15b000] Oct 15 18:08:56 rs kernel: radiusd[23896]: segfault at c ip b74d400e sp ae919c90 error 4[b73b5000+15b000] Oct 15 18:09:55 rs kernel: radiusd[24042]: segfault at 8 ip b736935d sp b07acce0 error 4 in rlm_perl-2.1.7.so[b7248000+15b000] when i comment out all perl statement in config, any segfault (in rlm_perl-2.1.7.so and in libc-2.9.so) disappears. Regards, Michael. - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
Re: segfault when rlm_perl used
On Oct 16, 2009, at 2:57 PM, Michael Chernyakhovsky wrote: Oct 13 21:58:53 rs kernel: radiusd[11441]: segfault at 20004 ip b7478636 sp b502bcb0 error 4 in rlm_perl-2.1.7.so[b73cb000+15b000] Oct 14 22:09:56 rs kernel: radiusd[17687]: segfault at 8 ip b731e35d sp b6f61ce0 error 4 in rlm_perl-2.1.7.so[b71fd000+15b000] Oct 14 22:14:06 rs kernel: radiusd[18374]: segfault at 8 ip b73d735d sp b701ace0 error 4 in rlm_perl-2.1.7.so[b72b6000+15b000] Oct 15 18:07:58 rs kernel: radiusd[23858]: segfault at 8 ip b745935d sp ae89cce0 error 4 in rlm_perl-2.1.7.so[b7338000+15b000] Oct 15 18:08:56 rs kernel: radiusd[23896]: segfault at c ip b74d400e sp ae919c90 error 4[b73b5000+15b000] Oct 15 18:09:55 rs kernel: radiusd[24042]: segfault at 8 ip b736935d sp b07acce0 error 4 in rlm_perl-2.1.7.so[b7248000+15b000] Could you include the output of gdb, as suggested in doc/bugs? Best Regards, Boian Jordanov RD Expert Orbitel - Next Generation Telecom tel. +359 2 4004 723 tel. +359 2 4004 002 - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
Re[2]: segfault when rlm_perl used
Здравствуйте, FreeRadius! Вы писали 16 октября 2009 г., 18:03:17: On Oct 16, 2009, at 2:57 PM, Michael Chernyakhovsky wrote: Oct 13 21:58:53 rs kernel: radiusd[11441]: segfault at 20004 ip b7478636 sp b502bcb0 error 4 in rlm_perl-2.1.7.so[b73cb000+15b000] Oct 14 22:09:56 rs kernel: radiusd[17687]: segfault at 8 ip b731e35d sp b6f61ce0 error 4 in rlm_perl-2.1.7.so[b71fd000+15b000] Oct 14 22:14:06 rs kernel: radiusd[18374]: segfault at 8 ip b73d735d sp b701ace0 error 4 in rlm_perl-2.1.7.so[b72b6000+15b000] Oct 15 18:07:58 rs kernel: radiusd[23858]: segfault at 8 ip b745935d sp ae89cce0 error 4 in rlm_perl-2.1.7.so[b7338000+15b000] Oct 15 18:08:56 rs kernel: radiusd[23896]: segfault at c ip b74d400e sp ae919c90 error 4[b73b5000+15b000] Oct 15 18:09:55 rs kernel: radiusd[24042]: segfault at 8 ip b736935d sp b07acce0 error 4 in rlm_perl-2.1.7.so[b7248000+15b000] Could you include the output of gdb, as suggested in doc/bugs? Bug 31 created: https://bugs.freeradius.org/bugzilla/show_bug.cgi?id=31 - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html -- С уважением, Михаил Черняховский, НП Магинфоцентр, г. Магнитогорск. тел. (3519) 49-69-00, моб. +7 902 896 2872 - List info/subscribe/unsubscribe? See http://www.freeradius.org/list/users.html
