Hallo zusammen,
seit geraumer Zeit beobachte ich auf einem Debian Etch-System (64-Bit) mit
monolitischem
Kernel (2.6.30.5) Segfaults, die sich im Logfile wie folgt äussern:
[Mon Nov 23 20:22:15 2009] [notice] child pid 28065 exit signal Segmentation
fault (11), possible coredump in /tmp
[Mon Nov 23 21:03:11 2009] [notice] child pid 1070 exit signal Segmentation
fault (11), possible coredump in /tmp
Im Kernel-Logfile äusser sich der Segfault wie folgt:
httpd[28065]: segfault at 52bc2350 ip 00007f5552a08941 sp 00007fff38529c20
error 4 in libc-2.3.6.so[7f5552986000+121000]
httpd[1070]: segfault at 52bc2350 ip 00007f5552a08941 sp 00007fff3852de60 error
4 in libc-2.3.6.so[7f5552986000+121000]
Sieht auf den ersten Blick aus, wie wenn das Problem der glibc zugrunde liegt.
Um das Ganze genauer
eingrenzen zu können, habe ich mod_backtrace sowie die
CoreDumpDirectory-Direktive aktiviert. Der Apache
ist auf dem System wie folgt kompiliert:
"./configure" \
"--with-layout=Apache" \
"--enable-ssl" \
"--enable-rewrite" \
"--enable-expires" \
"--enable-headers" \
"--enable-unique-id" \
"--disable-imagemap" \
"--disable-userdir" \
"--disable-cgi" \
"--disable-asis" \
"--disable-autoindex" \
"--disable-version" \
"--disable-include" \
"--disable-filter" \
"--disable-actions" \
"--enable-so" \
"--enable-exception-hook" \
"$@"
Folgender Softwarestand nebst Apache:
Server Version: Apache/2.2.14 (Unix) mod_ssl/2.2.14 OpenSSL/0.9.8c PHP/5.2.11
modsecurity 5.2.10 (+Mlogc).
Ich bin etwas ratlos was die Segfaults angeht. Ich habe bereits alle
Komponenten nochmals
händisch neu kompiliert und installiert um etwaige Fehler ausschliessen zu
können. Auch
habe ich den Core-Dump mit gdb ausgelesen, der Output findet sich in unteren
Teil der
Nachricht. Weitere Infos wie Kernel-Konfiguration oder sonstiges liefere ich
bei Bedarf
sehr gerne.
Über jeden Hinweis/Kommentar bin ich dankbar und würde mich freuen eine Antwort
zu erhalten.
Schöne Grüße,
Werner
gdb /usr/local/apache2/bin/httpd /tmp/core
------------------------------------------
gdb /usr/local/apache2/bin/httpd /tmp/core
GNU gdb 6.4.90-debian
Copyright (C) 2006 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu"...Using host libthread_db library
"/lib/libthread_db.so.1".
Reading symbols from /usr/lib/libssl.so.0.9.8...done.
Loaded symbols for /usr/lib/libssl.so.0.9.8
Reading symbols from /usr/lib/libcrypto.so.0.9.8...done.
Loaded symbols for /usr/lib/libcrypto.so.0.9.8
Reading symbols from /lib/libm.so.6...done.
Loaded symbols for /lib/libm.so.6
Reading symbols from /usr/local/apache2/lib/libaprutil-1.so.0...done.
Loaded symbols for /usr/local/apache2/lib/libaprutil-1.so.0
Reading symbols from /usr/local/apache2/lib/libexpat.so.0...done.
Loaded symbols for /usr/local/apache2/lib/libexpat.so.0
Reading symbols from /usr/lib/libexpat.so.1...done.
Loaded symbols for /usr/lib/libexpat.so.1
Reading symbols from /usr/local/apache2/lib/libapr-1.so.0...done.
Loaded symbols for /usr/local/apache2/lib/libapr-1.so.0
Reading symbols from /lib/librt.so.1...done.
Loaded symbols for /lib/librt.so.1
Reading symbols from /lib/libcrypt.so.1...done.
Loaded symbols for /lib/libcrypt.so.1
Reading symbols from /lib/libpthread.so.0...done.
Loaded symbols for /lib/libpthread.so.0
Reading symbols from /lib/libdl.so.2...done.
Loaded symbols for /lib/libdl.so.2
Reading symbols from /lib/libc.so.6...done.
Loaded symbols for /lib/libc.so.6
Reading symbols from /usr/lib/libz.so.1...done.
Loaded symbols for /usr/lib/libz.so.1
Reading symbols from /lib/ld-linux-x86-64.so.2...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from /lib/libnsl.so.1...done.
Loaded symbols for /lib/libnsl.so.1
Reading symbols from /lib/libnss_compat.so.2...done.
Loaded symbols for /lib/libnss_compat.so.2
Reading symbols from /lib/libnss_nis.so.2...done.
Loaded symbols for /lib/libnss_nis.so.2
Reading symbols from /lib/libnss_files.so.2...done.
Loaded symbols for /lib/libnss_files.so.2
Reading symbols from /usr/local/apache2/modules/libphp5.so...done.
Loaded symbols for /usr/local/apache2/modules/libphp5.so
Reading symbols from /usr/lib/libc-client.so.2002edebian...done.
Loaded symbols for /usr/lib/libc-client.so.2002edebian
Reading symbols from /usr/lib/libmysqlclient.so.15...done.
Loaded symbols for /usr/lib/libmysqlclient.so.15
Reading symbols from /usr/lib/libmcrypt.so.4...done.
Loaded symbols for /usr/lib/libmcrypt.so.4
Reading symbols from /usr/lib/libltdl.so.3...done.
Loaded symbols for /usr/lib/libltdl.so.3
Reading symbols from /lib/libpam.so.0...done.
Loaded symbols for /lib/libpam.so.0
Reading symbols from /usr/lib/libgd.so.2...done.
Loaded symbols for /usr/lib/libgd.so.2
Reading symbols from /usr/lib/libt1.so.5...done.
Loaded symbols for /usr/lib/libt1.so.5
Reading symbols from /usr/lib/libfreetype.so.6...done.
Loaded symbols for /usr/lib/libfreetype.so.6
Reading symbols from /usr/lib/libX11.so.6...done.
Loaded symbols for /usr/lib/libX11.so.6
Reading symbols from /usr/lib/libXpm.so.4...done.
Loaded symbols for /usr/lib/libXpm.so.4
Reading symbols from /usr/lib/libpng12.so.0...done.
Loaded symbols for /usr/lib/libpng12.so.0
Reading symbols from /usr/lib/libjpeg.so.62...done.
Loaded symbols for /usr/lib/libjpeg.so.62
Reading symbols from /lib/libresolv.so.2...done.
Loaded symbols for /lib/libresolv.so.2
Reading symbols from /usr/lib/libgssapi_krb5.so.2...done.
Loaded symbols for /usr/lib/libgssapi_krb5.so.2
Reading symbols from /usr/lib/libkrb5.so.3...done.
Loaded symbols for /usr/lib/libkrb5.so.3
Reading symbols from /usr/lib/libk5crypto.so.3...done.
Loaded symbols for /usr/lib/libk5crypto.so.3
Reading symbols from /lib/libcom_err.so.2...done.
Loaded symbols for /lib/libcom_err.so.2
Reading symbols from /usr/lib/libcurl.so.3...done.
Loaded symbols for /usr/lib/libcurl.so.3
Reading symbols from /usr/lib/libkrb5support.so.0...done.
Loaded symbols for /usr/lib/libkrb5support.so.0
Reading symbols from /usr/lib/libidn.so.11...done.
Loaded symbols for /usr/lib/libidn.so.11
Reading symbols from /usr/lib/libxml2.so.2...done.
Loaded symbols for /usr/lib/libxml2.so.2
Reading symbols from /usr/lib/libXau.so.6...done.
Loaded symbols for /usr/lib/libXau.so.6
Reading symbols from /usr/lib/libXdmcp.so.6...done.
Loaded symbols for /usr/lib/libXdmcp.so.6
Reading symbols from /usr/local/apache2/modules/mod_backtrace.so...done.
Loaded symbols for /usr/local/apache2/modules/mod_backtrace.so
Reading symbols from /usr/lib/liblua5.1.so.0...done.
Loaded symbols for /usr/lib/liblua5.1.so.0
Reading symbols from /usr/local/apache2/modules/mod_security2.so...done.
Loaded symbols for /usr/local/apache2/modules/mod_security2.so
Reading symbols from /usr/lib/libpcre.so.3...done.
Loaded symbols for /usr/lib/libpcre.so.3
Reading symbols from
/usr/local/lib/php/extensions/no-debug-non-zts-20060613/eaccelerator.so...done.
Loaded symbols for
/usr/local/lib/php/extensions/no-debug-non-zts-20060613/eaccelerator.so
Reading symbols from
/usr/local/lib/php/extensions/no-debug-non-zts-20060613/memcache.so...done.
Loaded symbols for
/usr/local/lib/php/extensions/no-debug-non-zts-20060613/memcache.so
Reading symbols from /usr/lib/gconv/ISO8859-1.so...done.
Loaded symbols for /usr/lib/gconv/ISO8859-1.so
Failed to read a valid object file image from memory.
Core was generated by `/usr/local/apache2/bin/httpd -k start'.
Program terminated with signal 11, Segmentation fault.
#0 0x00007f5552a08941 in asctime_r () from /lib/libc.so.6
(gdb) bt full
#0 0x00007f5552a08941 in asctime_r () from /lib/libc.so.6
No symbol table info available.
#1 0x00007f5550016cf8 in bt_exception_hook (ei=0x7fff38529fb0) at
mod_backtrace.c:96
msg_len = <value optimized out>
logfd = <value optimized out>
msg_prefix = '\0' <repeats 59 times>
buffer = '\0' <repeats 89 times>,
"\020\000\000\000\000\000\...@ֻru\177\000\000????\000\000\000\000p{^ku\177\000\000?$\236ru\177\000\000\003\b\000\000\000\000\000\000?a?\001\000\000\000\000\001\000\000\000\000\000\000\000?$?ru\177\000\000?gd\000\000\000\000\000\000\000\000\024\000\000\000\000\020q\233ru\177\000\000\000\004\000\000\000\000\000\000\230?\000\000\000\000\000\000?d\nk",
'\0' <repeats 12 times>, "??\vK", '\0' <repeats 12 times>, "??\vK", '\0'
<repeats 36 times>,
"?\202?\001\000\000\000\...@ֻru\177\000\000@ֻru\177\000\000\f?\236ru\177\000\...@ֻru"...
now = 1259054586
using_errorlog = <value optimized out>
array = {0x4447d0, 0x400, 0xee98, 0x4b0a64c0, 0x0, 0x4b0ba5f8, 0x0,
0x4b0ba5f8, 0x0, 0x0, 0x0, 0x0, 0x1ab82a0, 0x7f5552bbd640, 0x7f5552bbd640,
0x7f55529ee20c, 0x7f5552bbd640, 0x7500000014000000, 0x7f55529b5110,
0x7f5552422e5d}
#2 0x0000000000444509 in ap_run_fatal_exception (ei=0x7fff38529fb0) at
mpm_common.c:67
n = 0
rv = 304
#3 0x000000000044484a in sig_coredump (sig=11) at mpm_common.c:1204
No locals.
#4 0x00007f55529b5110 in killpg () from /lib/libc.so.6
No symbol table info available.
#5 0x0000000000000000 in ?? ()
No symbol table info available.
seit der Aktualisierung des Kernels von Version 2.6.26.9 auf Version 2.6.28
segfaultet der Apache Webserver:
[Fri Jan 09 12:39:00 2009] [notice] child pid 11609 exit signal Segmentation
fault (11)
[Fri Jan 09 12:39:00 2009] [notice] child pid 11610 exit signal Segmentation
fault (11)
[Fri Jan 09 12:39:00 2009] [notice] child pid 11611 exit signal Segmentation
fault (11)
[Fri Jan 09 12:39:00 2009] [notice] child pid 11612 exit signal Segmentation
fault (11)
[Fri Jan 09 12:39:00 2009] [notice] child pid 11613 exit signal Segmentation
fault (11)
Der Webserver läuft auf einem Debian Etch System (64-Bit) und monolithischem
Kernel.
Folgende Versionen sind im Einsatz:
Apache/2.2.11 (Unix) mod_ssl/2.2.11 OpenSSL/0.9.8c PHP/5.2.8
Um die Ursache des Segfaults eingrenzen zu können, habe ich gemäß dem
Debugging-Instruktionen
CoreDumpDirectory in der Konfiguration aktiviert. Darüber hinaus habe ich noch
mod_backtrace
installiert und aktiviert - meinen Apache-Webserver habe ich mit folgendne
Optionen compiled:
"./configure" \
"--with-layout=Apache" \
"--enable-ssl" \
"--enable-rewrite" \
"--enable-expires" \
"--enable-headers" \
"--disable-cgi" \
"--disable-asis" \
"--disable-imagemap" \
"--disable-userdir" \
"--enable-so" \
"--enable-exception-hook" \
"$@"
/usr/local/apache2/logs/error.log
[Fri Jan 09 12:57:30 2009] [notice] child pid 25254 exit signal Segmentation
fault (11), possible coredump in /tmp
[Fri Jan 09 12:57:30 2009] [notice] child pid 25255 exit signal Segmentation
fault (11), possible coredump in /tmp
[Fri Jan 09 12:57:30 2009] [notice] child pid 25256 exit signal Segmentation
fault (11), possible coredump in /tmp
[Fri Jan 09 12:57:30 2009] [notice] child pid 25257 exit signal Segmentation
fault (11), possible coredump in /tmp
[Fri Jan 09 12:57:30 2009] [notice] child pid 25258 exit signal Segmentation
fault (11), possible coredump in /tmp
[Fri Jan 09 12:57:30 2009] [notice] child pid 25259 exit signal Segmentation
fault (11), possible coredump in /tmp
/var/log/kern.log
Jan 9 12:57:27 n2 kernel: httpd[25259]: segfault at 2c841350 ip
00007fa52c687941 sp 00007fff35652190 error 4 in
libc-2.3.6.so[7fa52c605000+121000]
Jan 9 12:57:27 n2 kernel: httpd[25260]: segfault at 2c841350 ip
00007fa52c687941 sp 00007fff35652190 error 4 in
libc-2.3.6.so[7fa52c605000+121000]
Jan 9 12:57:30 n2 kernel: httpd[25265]: segfault at 2c841350 ip
00007fa52c687941 sp 00007fff35652190 error 4 in
libc-2.3.6.so[7fa52c605000+121000]
Jan 9 12:57:31 n2 kernel: httpd[25266]: segfault at 2c841350 ip
00007fa52c687941 sp 00007fff35652190 error 4 in
libc-2.3.6.so[7fa52c605000+121000]
Ich bin ehrlich gesagt, etwas ratlos was die Segfaults angeht. Apache 2.0.63
hat keinerlei Probleme mit
Kernel 2.6.28, Apache 2.2.11 segfaultet fröhlich. Unter Kernel 2.6.26.9
bestehen die beschriebenen
Probleme nicht, alles funktioniert dort tadellos. Ich habe bereits alle
Komponenten wie PHP5, Apache, etc.
nochmals händisch recompiled um etwaige Fehler ausschliessen zu können. Den
CoreDump habe ich mittels
gdb ausgelesen, der Output befindet sich im unteren Teil dieser Nachricht.
Kernel-Config und weiter benötigte
Informationen liefere ich bei Bedarf sehr gern.
Über jeden Hinweis/Kommentar bin ich dankbar und würde mich freuen eine Antwort
zu erhalten.
Schöne Grüße,
Werner
gdb /usr/local/apache2/bin/httpd /tmp/core
------------------------------------------
... gekürzt ...
Reading symbols from /usr/local/apache2.2/modules/mod_backtrace.so...done.
Loaded symbols for /usr/local/apache2/modules/mod_backtrace.so
Reading symbols from
/usr/local/lib/php/extensions/no-debug-non-zts-20060613/eaccelerator.so...done.
Loaded symbols for
/usr/local/lib/php/extensions/no-debug-non-zts-20060613/eaccelerator.so
Reading symbols from
/usr/local/lib/php/extensions/no-debug-non-zts-20060613/memcache.so...done.
Loaded symbols for
/usr/local/lib/php/extensions/no-debug-non-zts-20060613/memcache.so
Failed to read a valid object file image from memory.
Core was generated by `/usr/local/apache2/bin/httpd -k start'.
Program terminated with signal 11, Segmentation fault.
#0 0x00007fa52c687941 in asctime_r () from /lib/libc.so.6
(gdb) bt full
#0 0x00007fa52c687941 in asctime_r () from /lib/libc.so.6
No symbol table info available.
#1 0x00007fa529c9ccf8 in bt_exception_hook (ei=0x7fff35652520) at
mod_backtrace.c:96
msg_len = <value optimized out>
logfd = <value optimized out>
msg_prefix = '\0' <repeats 59 times>
buffer =
"�\2164,�\177\000\000�\2164,�\177\000\000\f\000\000\000\000\000\000\000�qd,�\177\000\000�Q0\002\000\000\000\000@(e5�\177\000\000\220\000\000\000\000\000\000\000\2326g,�\177\000\000`#e5�\177\000\000�)e5�\177\000\000�\2164,�\177\000\000\220(e5�\177",
'\0' <repeats 18 times>, "
\000\000\000\000\000\000\000��\203,�\177\000\000�\000\000\000\000\000\000\000ph2\002\000\000\000\000\000�d-�\177\000\000�\024\225,�\177",
'\0'
<repeats 13 times>,
"\204����\020\024\225,�\177\000\000\000\000\000\000\000\000\000\000\001\000\000\000\000\000\000\0002<T-�\177\000\000\000\000\000"...
now = 1231502252
using_errorlog = <value optimized out>
array = {0x0, 0x0, 0x1, 0x7fa52d543c32, 0x0, 0x6, 0x7fff35652440,
0x40eae8, 0xd68c224, 0x41db1d, 0x0, 0x2a, 0x7fa52cc9c1f0, 0x7fa52cc9ec10,
0x0, 0x0, 0x0, 0x7fa584000000, 0x7fa52c951410,
0x5b2c18}
#2 0x0000000000444129 in ap_run_fatal_exception (ei=0x7fff35652520) at
mpm_common.c:67
n = 0
rv = 0
#3 0x000000000044446a in sig_coredump (sig=11) at mpm_common.c:1204
No locals.
#4 <signal handler called>
No symbol table info available.
#5 apr_pollset_add (pollset=0x0, descriptor=0x7fff356529e0) at
poll/unix/epoll.c:150
ev = {events = 1, data = {ptr = 0x0, fd = 0, u32 = 0, u64 = 0}}
ret = <value optimized out>
elem = <value optimized out>
rv = <value optimized out>
#6 0x000000000047e81d in child_main (child_num_arg=<value optimized out>) at
prefork.c:532
pfd = {p = 0x0, desc_type = APR_POLL_SOCKET, reqevents = 1, rtnevents =
0, desc = {f = 0x208d0b0, s = 0x208d0b0}, client_data = 0x208d078}
ptrans = (apr_pool_t *) 0x23245b8
allocator = (apr_allocator_t *) 0x23224b0
status = <value optimized out>
i = 2
lr = (ap_listen_rec *) 0x208d078
pollset = (apr_pollset_t *) 0x0
sbh = (ap_sb_handle_t *) 0x23226b0
bucket_alloc = <value optimized out>
last_poll_idx = <value optimized out>
#7 0x000000000047ec34 in make_child (s=0x2097ef8, slot=127) at prefork.c:746
pid = 0
#8 0x000000000047f4d6 in ap_mpm_run (_pconf=<value optimized out>, plog=<value
optimized out>, s=<value optimized out>) at prefork.c:881
pidfile = <value optimized out>
active_children = <value optimized out>
cutoff = <value optimized out>
index = <value optimized out>
remaining_children_to_start = 0
rv = <value optimized out>
#9 0x000000000042a157 in main (argc=3, argv=0x7fff35652df8) at main.c:740
c = 0 '\0'
configtestonly = 0
confname = 0x48884f "conf/httpd.conf"
def_server_root = 0x48885f "/usr/local/apache2"
temp_error_log = 0x0
error = <value optimized out>
process = (process_rec *) 0x208c220
server_conf = <value optimized out>
pglobal = (apr_pool_t *) 0x208c128
pconf = (apr_pool_t *) 0x208e138
plog = (apr_pool_t *) 0x20d4368
ptemp = (apr_pool_t *) 0x2098188
pcommands = (apr_pool_t *) 0x2090148
opt = (apr_getopt_t *) 0x2090240
rv = 0
optarg = 0x7fa52c609170 "�'"
(gdb)
--------------------------------------------------------------------------
Apache HTTP Server Mailing List "users-de"
unsubscribe-Anfragen an users-de-unsubscr...@httpd.apache.org
sonstige Anfragen an users-de-h...@httpd.apache.org
--------------------------------------------------------------------------
