Bug#916375: apache2: Segmentation fault when mod_perl.so is loaded
Package: apache2 Version: 2.4.25-3+deb9u6 Severity: grave Tags: patch Justification: renders package unusable Dear Maintainer, For communication with our customers we use the OTRS Ticket System. The server was reinstalled about a year ago and worked fine until it was rebooted three days ago. Since then the Apache web server crashes reproducibly as soon as it has to process a request. Here is a stacktrace: cd /etc/apache2 . envvars gdb /usr/sbin/apache2 gdb> set args -X gdb> run # ... now use the webbrowser to send an http request to the apache2 server: http://otrs-testing/otrs/index.pl?Action=Admin gdb> Thread 1 "/opt/otrs/bin/cgi-bi" received signal SIGSEGV, Segmentation fault. gdb> bt #0 0x7fffdcd290c7 in free_defaults () from /usr/lib/x86_64-linux-gnu/libmariadbclient.so.18 #1 0x7fffdcd29422 in free_defaults () from /usr/lib/x86_64-linux-gnu/libmariadbclient.so.18 #2 0x7fffdcd29461 in free_defaults () from /usr/lib/x86_64-linux-gnu/libmariadbclient.so.18 #3 0x7fffdcd29637 in free_defaults () from /usr/lib/x86_64-linux-gnu/libmariadbclient.so.18 #4 0x7fffdccf5868 in mysql_options4 () from /usr/lib/x86_64-linux-gnu/libmariadbclient.so.18 #5 0x7fffdd2cabc8 in mysql_dr_connect () from /usr/lib/x86_64-linux-gnu/perl5/5.24/auto/DBD/mysql/mysql.so #6 0x7fffdd2ccc69 in ?? () from /usr/lib/x86_64-linux-gnu/perl5/5.24/auto/DBD/mysql/mysql.so #7 0x7fffdd2ccd71 in mysql_db_login () from /usr/lib/x86_64-linux-gnu/perl5/5.24/auto/DBD/mysql/mysql.so #8 0x7fffdd2d9651 in ?? () from /usr/lib/x86_64-linux-gnu/perl5/5.24/auto/DBD/mysql/mysql.so #9 0x73901950 in Perl_pp_entersub () from /usr/lib/x86_64-linux-gnu/libperl.so.5.24 #10 0x738f9e96 in Perl_runops_standard () from /usr/lib/x86_64-linux-gnu/libperl.so.5.24 #11 0x73879d5b in Perl_call_sv () from /usr/lib/x86_64-linux-gnu/libperl.so.5.24 #12 0x7fffea84c2e9 in XS_DBI_dispatch () from /usr/lib/x86_64-linux-gnu/perl5/5.24/auto/DBI/DBI.so #13 0x73901950 in Perl_pp_entersub () from /usr/lib/x86_64-linux-gnu/libperl.so.5.24 #14 0x738f9e96 in Perl_runops_standard () from /usr/lib/x86_64-linux-gnu/libperl.so.5.24 #15 0x7387a10e in Perl_call_sv () from /usr/lib/x86_64-linux-gnu/libperl.so.5.24 #16 0x73c32c68 in modperl_callback () from /usr/lib/apache2/modules/mod_perl.so #17 0x73c33606 in modperl_callback_run_handlers () from /usr/lib/apache2/modules/mod_perl.so #18 0x73c33d9f in modperl_callback_per_dir () from /usr/lib/apache2/modules/mod_perl.so #19 0x73c2e0fb in ?? () from /usr/lib/apache2/modules/mod_perl.so #20 0x73c2e32c in modperl_response_handler_cgi () from /usr/lib/apache2/modules/mod_perl.so #21 0x555abc40 in ap_run_handler () #22 0x555ac1d6 in ap_invoke_handler () #23 0x555c3e13 in ap_process_async_request () #24 0x555c3f20 in ap_process_request () #25 0x555bffdd in ?? () #26 0x555b5ab0 in ap_run_process_connection () #27 0x740686bf in ?? () from /usr/lib/apache2/modules/mod_mpm_prefork.so #28 0x740688f2 in ?? () from /usr/lib/apache2/modules/mod_mpm_prefork.so #29 0x74069e37 in ?? () from /usr/lib/apache2/modules/mod_mpm_prefork.so #30 0x5558f00e in ap_run_mpm () #31 0x55587c4d in main () We are using unattended upgrades (security only), and we suspect that an automatic system update has installed a new Perl version that now causes these problems. We have a second OTRS fallback system that hasn't been restarted for 94 days, and everything is still working fine there. If we compare the shared libraries loaded by apache on these two systems (/proc//maps) we can see that the following .so files have been renewed. 484 -rw-r--r-- 1 root root 489960 Nov 29 20:45 /usr/lib/x86_64-linux-gnu/libtiff.so.5.2.6 2008 -rw-r--r-- 1 root root 2049312 Nov 29 12:11 /usr/lib/x86_64-linux-gnu/libperl.so.5.24.1 92 -rw-r--r-- 1 root root88760 Nov 29 12:11 /usr/lib/x86_64-linux-gnu/perl/5.24.1/auto/B/B.so 20 -rw-r--r-- 1 root root187
Bug#916375: apache2: Segmentation fault when mod_perl.so is loaded
On Thu, Dec 13, 2018 at 06:44:32PM +0100, h.thien wrote: > Package: apache2 > Version: 2.4.25-3+deb9u6 > Severity: grave > Tags: patch > Justification: renders package unusable > gdb> Thread 1 "/opt/otrs/bin/cgi-bi" received signal SIGSEGV, > Segmentation fault. > gdb> bt > #0 0x7fffdcd290c7 in free_defaults () from > /usr/lib/x86_64-linux-gnu/libmariadbclient.so.18 > We are using unattended upgrades (security only), and we suspect that > an automatic system update has installed a new Perl version that now causes > these problems. Have you ruled out the MariaDB update? That one seems the most likely to have triggered this regression. https://www.debian.org/security/2018/dsa-4341 (Not sure how reliably you got the list of loaded shared libraries; you seem to be running the prefork mpm so presumably only some of your apache processes will have the libraries loaded by the actual Perl application.) -- Niko Tyni nt...@debian.org
Bug#916375: apache2: Segmentation fault when mod_perl.so is loaded
Thanks Niko, it seems that you where right regarding libmariadbclient18 ... downgrading libmariadbclient18 fixed the issue :-) dpkg -i libmariadbclient18_10.1.26-0+deb9u1_amd64.deb Niko Tyni wrote: On Thu, Dec 13, 2018 at 06:44:32PM +0100, h.thien wrote: Package: apache2 Version: 2.4.25-3+deb9u6 Severity: grave Tags: patch Justification: renders package unusable gdb> Thread 1 "/opt/otrs/bin/cgi-bi" received signal SIGSEGV, Segmentation fault. gdb> bt #0 0x7fffdcd290c7 in free_defaults () from /usr/lib/x86_64-linux-gnu/libmariadbclient.so.18 We are using unattended upgrades (security only), and we suspect that an automatic system update has installed a new Perl version that now causes these problems. Have you ruled out the MariaDB update? That one seems the most likely to have triggered this regression. https://www.debian.org/security/2018/dsa-4341 (Not sure how reliably you got the list of loaded shared libraries; you seem to be running the prefork mpm so presumably only some of your apache processes will have the libraries loaded by the actual Perl application.) -- Hans-Georg Thien | Software Engineer callas software GmbH | Schoenhauser Allee 6/7 | 10119 Berlin | Germany Tel +49.30.4439031-0 | Fax +49.30.4416402 | www.callassoftware.com Amtsgericht Charlottenburg, HRB 59615 Geschäftsführung: Olaf Drümmer, Ulrich Frotscher, Dietrich von Seggern
Bug#916375: apache2: Segmentation fault when mod_perl.so is loaded
Control: reassign -1 libmariadbclient18 10.1.37-0+deb9u1 This reportedly regressed with DSA-4341-1, reassigning to the correct package. -- Niko Tyni nt...@debian.org
Processed: Re: Bug#916375: apache2: Segmentation fault when mod_perl.so is loaded
Processing control commands: > reassign -1 libmariadbclient18 10.1.37-0+deb9u1 Bug #916375 [apache2] apache2: Segmentation fault when mod_perl.so is enabled Bug reassigned from package 'apache2' to 'libmariadbclient18'. No longer marked as found in versions apache2/2.4.25-3+deb9u6. Ignoring request to alter fixed versions of bug #916375 to the same values previously set Bug #916375 [libmariadbclient18] apache2: Segmentation fault when mod_perl.so is enabled Marked as found in versions mariadb-10.1/10.1.37-0+deb9u1. -- 916375: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=916375 Debian Bug Tracking System Contact ow...@bugs.debian.org with problems
Bug#916375: Fwd: Bug#916375: apache2: Segmentation fault when mod_perl.so is loaded
Cc: libapache2-mod-perl2 Message Subject : Bug#916375: apache2: Segmentation fault when mod_perl.so is loaded Date : Thu, 13 Dec 2018 19:09:02 + >From : h.thien To : debian-bugs-d...@lists.debian.org Cc : t...@security.debian.org, Debian Apache Maintainers Package: apache2 Version: 2.4.25-3+deb9u6 Severity: grave Tags: patch Justification: renders package unusable Dear Maintainer, For communication with our customers we use the OTRS Ticket System. The server was reinstalled about a year ago and worked fine until it was rebooted three days ago. Since then the Apache web server crashes reproducibly as soon as it has to process a request. Here is a stacktrace: cd /etc/apache2 . envvars gdb /usr/sbin/apache2 gdb> set args -X gdb> run # ... now use the webbrowser to send an http request to the apache2 server: http://otrs-testing/otrs/index.pl?Action=Admin gdb> Thread 1 "/opt/otrs/bin/cgi-bi" received signal SIGSEGV, Segmentation fault. gdb> bt #0 0x7fffdcd290c7 in free_defaults () from /usr/lib/x86_64-linux-gnu/libmariadbclient.so.18 #1 0x7fffdcd29422 in free_defaults () from /usr/lib/x86_64-linux-gnu/libmariadbclient.so.18 #2 0x7fffdcd29461 in free_defaults () from /usr/lib/x86_64-linux-gnu/libmariadbclient.so.18 #3 0x7fffdcd29637 in free_defaults () from /usr/lib/x86_64-linux-gnu/libmariadbclient.so.18 #4 0x7fffdccf5868 in mysql_options4 () from /usr/lib/x86_64-linux-gnu/libmariadbclient.so.18 #5 0x7fffdd2cabc8 in mysql_dr_connect () from /usr/lib/x86_64-linux-gnu/perl5/5.24/auto/DBD/mysql/mysql.so #6 0x7fffdd2ccc69 in ?? () from /usr/lib/x86_64-linux-gnu/perl5/5.24/auto/DBD/mysql/mysql.so #7 0x7fffdd2ccd71 in mysql_db_login () from /usr/lib/x86_64-linux-gnu/perl5/5.24/auto/DBD/mysql/mysql.so #8 0x7fffdd2d9651 in ?? () from /usr/lib/x86_64-linux-gnu/perl5/5.24/auto/DBD/mysql/mysql.so #9 0x73901950 in Perl_pp_entersub () from /usr/lib/x86_64-linux-gnu/libperl.so.5.24 #10 0x738f9e96 in Perl_runops_standard () from /usr/lib/x86_64-linux-gnu/libperl.so.5.24 #11 0x73879d5b in Perl_call_sv () from /usr/lib/x86_64-linux-gnu/libperl.so.5.24 #12 0x7fffea84c2e9 in XS_DBI_dispatch () from /usr/lib/x86_64-linux-gnu/perl5/5.24/auto/DBI/DBI.so #13 0x73901950 in Perl_pp_entersub () from /usr/lib/x86_64-linux-gnu/libperl.so.5.24 #14 0x738f9e96 in Perl_runops_standard () from /usr/lib/x86_64-linux-gnu/libperl.so.5.24 #15 0x7387a10e in Perl_call_sv () from /usr/lib/x86_64-linux-gnu/libperl.so.5.24 #16 0x73c32c68 in modperl_callback () from /usr/lib/apache2/modules/mod_perl.so #17 0x73c33606 in modperl_callback_run_handlers () from /usr/lib/apache2/modules/mod_perl.so #18 0x73c33d9f in modperl_callback_per_dir () from /usr/lib/apache2/modules/mod_perl.so #19 0x73c2e0fb in ?? () from /usr/lib/apache2/modules/mod_perl.so #20 0x73c2e32c in modperl_response_handler_cgi () from /usr/lib/apache2/modules/mod_perl.so #21 0x555abc40 in ap_run_handler () #22 0x555ac1d6 in ap_invoke_handler () #23 0x555c3e13 in ap_process_async_request () #24 0x555c3f20 in ap_process_request () #25 0x555bffdd in ?? () #26 0x555b5ab0 in ap_run_process_connection () #27 0x740686bf in ?? () from /usr/lib/apache2/modules/mod_mpm_prefork.so #28 0x740688f2 in ?? () from /usr/lib/apache2/modules/mod_mpm_prefork.so #29 0x74069e37 in ?? () from /usr/lib/apache2/modules/mod_mpm_prefork.so #30 0x5558f00e in ap_run_mpm () #31 0x55587c4d in main () We are using unattended upgrades (security only), and we suspect that an automatic system update has installed a new Perl version that now causes these problems. We have a second OTRS fallback system that hasn't been restarted for 94 days, and everything is still working fine there. If we compare the shared libraries loaded by apache on these two systems (/proc//maps) we can see that the following .so files have been renewed. 484 -rw-r--r-- 1 root root 489960 Nov 29 20:45 /usr/lib/x86_64-linux-gnu/libtiff.so.5.2