Am 29.06.2016 um 11:58 schrieb Michael Diener:
I get occasional Apache 2 crashes being caused by mod_jk and I'm running
out of ideas about the cause of the problem. I hope somebody here can point
me in the right direction.
Can you reproduce? Does it also happen on a test system?
tomcat6 6.0.39-1
libapache2-mod-jk 1:1.2.37-3
Latest we provide in the project is 1.2.41. It is pretty easy to compile
yourself and would be an interesting check to see, whether it is just an
old already fixed problem.
apache2 2.4.7-1ubuntu4
java version "1.6.0_45"
Java(TM) SE Runtime Environment (build 1.6.0_45-b06)
Java HotSpot(TM) 64-Bit Server VM (build 20.45-b01, mixed mode)
/var/log/apache2/error.log
**** buffer overflow detected ***: /usr/sbin/apache2 terminated=======
Backtrace:
=========/lib/x86_64-linux-gnu/libc.so.6(+0x7329f)[0x7fe9aa7de29f]/lib/x86_64-linux-gnu/libc.so.6(__fortify_fail+0x5c)[0x7fe9aa875bbc]/lib/x86_64-linux-gnu/libc.so.6(+0x109a90)[0x7fe9aa874a90]/lib/x86_64-linux-gnu/libc.so.6(+0x10ab07)[0x7fe9aa875b07]/usr/lib/apache2/modules/mod_jk.so(jk_open_socket+0x8d8)[0x7fe9a7c60cb8]/usr/lib/apache2/modules/mod_jk.so(ajp_connect_to_endpoint+0x65)[0x7fe9a7c7bf75]/usr/lib/apache2/modules/mod_jk.so(+0x36422)[0x7fe9a7c7d422]/usr/lib/apache2/modules/mod_jk.so(+0x1674c)[0x7fe9a7c5d74c]/usr/sbin/apache2(ap_run_handler+0x40)[0x7fe9ab65fbe0]/usr/sbin/apache2(ap_invoke_handler+0x69)[0x7fe9ab660129]/usr/sbin/apache2(ap_process_async_request+0x20a)[0x7fe9ab6756ca]/usr/sbin/apache2(+0x69500)[0x7fe9ab672500]/usr/sbin/apache2(ap_run_process_connection+0x40)[0x7fe9ab669220]/usr/lib/apache2/modules/mod_mpm_event.so(+0x681b)[0x7fe9a783981b]/lib/x86_64-linux-gnu/libpthread.so.0(+0x8184)[0x7fe9aab38184]/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7fe9aa86537d]*
======= Memory map: ========
7fe688000000-7fe68806a000 rw-p 00000000 00:00 0
7fe68806a000-7fe68c000000 ---p 00000000 00:00 0
.......
7fffa6c27000-7fffa6c48000 rw-p 00000000 00:00 0 [stack]
7fffa6c86000-7fffa6c88000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]
[Wed Jun 29 05:01:50.052325 2016] [core:notice] [pid 1747:tid
140641581987712] AH00051: child pid 17018 exit signal Aborted (6), possible
coredump in /etc/apache2
The log indicates there might be a coredump, but there is not.
Configure your system so that it writes core dump files, look at the
core dump file with "gdb", e.g. running "thread apply all bt full" and
provide the output.
Likely your httpd is running as a suid process (started as root but the
children switch uid to something else, like www or similar). You have to
enable core dump for suid processes explicitly in your Linux.
There is no log in /var/log/apache2/mod_jk.log at the same time.
/var/log/tomcat6/catalina.out
Jun 29, 2016 5:01:49 AM org.apache.jk.common.ChannelSocket processConnection
WARNING: processCallbacks status 2
Jun 29, 2016 5:01:49 AM org.apache.jk.common.ChannelSocket processConnection
WARNING: processCallbacks status 2
The Tomcat log indicates AFAIK that the client connection has been lost.
/etc/libapache2-mod-jk/httpd-jk.conf
<IfModule jk_module>
JkWorkersFile /etc/libapache2-mod-jk/workers.properties
JkLogFile /var/log/apache2/mod_jk.log
JkLogLevel warn
JkShmFile /var/log/apache2/jk-runtime-status
</IfModule>
/etc/libapache2-mod-jk/workers.properties
The next lines look like they are coming from very old and partially
nonsense recipies. Your crash won't be a result of those but you should
probably start to create your config based on what's inside the conf
folder of our source distribution for mod_jk. The files can also be
found under http://svn.apache.org/viewvc/tomcat/jk/trunk/conf/.
workers.tomcat_home=/usr/share/tomcat6
workers.java_home=/usr/lib/jvm/java-6-sun
ps=/
worker.list=loadbalancer
worker.loadbalancer.type=lb
worker.loadbalancer.balance_workers=ajp13_worker,ajp13_worker2
worker.loadbalancer.sticky_session=0
worker.ajp13_worker.port=xxx
worker.ajp13_worker.host=localhost
worker.ajp13_worker.type=ajp13
worker.ajp13_worker.lbfactor=50
If you don't really need the increased max_packet_size, you should
comment it out here and the packetSize below and retry. Those settings
are not so common. There were also changes related to them in newer
versions.
worker.ajp13_worker.max_packet_size=65536
Not crash related, but I don't like the general socket_timeout. Look at
the configs proposed above. They use lots of good timeouts, but
socket_timeout is not one of them.
worker.ajp13_worker.socket_timeout=300
worker.ajp13_worker.ping_mode=A
worker.ajp13_worker.secret=xxx
worker.ajp13_worker.fail_on_status=503
worker.ajp13_worker.connection_pool_size=32768
#worker.ajp13_worker.activation=disabled
worker.ajp13_worker.redirect=ajp13_worker2
worker.ajp13_worker2.port=xxx
worker.ajp13_worker2.host=otherhost
worker.ajp13_worker2.type=ajp13
worker.ajp13_worker2.lbfactor=1
worker.ajp13_worker2.max_packet_size=65536
worker.ajp13_worker2.socket_timeout=300
worker.ajp13_worker2.ping_mode=A
worker.ajp13_worker2.secret=xxx
worker.ajp13_worker2.fail_on_status=503
worker.ajp13_worker2.connection_pool_size=32768
worker.ajp13_worker2.activation=disabled
#worker.ajp13_worker2.redirect=ajp13_worker
/etc/tomcat6/server.xml
<Connector
port="xxx" protocol="AJP/1.3" redirectPort="8443"
enableLookups="false" maxThreads="65536" minSpareThreads="25"
maxSpareThreads="75"
connectionTimeout="300000" packetSize="65536" request.secret="xxx"
/>
ls /etc/apache2/mods-enabled/
access_compat.load auth_basic.load authz_core.load autoindex.conf
deflate.load env.load headers.load mime.conf mpm_event.load rewrite.load
socache_shmcb.load status.conf
alias.conf authn_core.load authz_host.load autoindex.load dir.conf
expires.load jk.conf mime.load negotiation.conf setenvif.conf ssl.conf
status.load
alias.load authn_file.load authz_user.load deflate.conf dir.load
filter.load jk.load mpm_event.conf negotiation.load setenvif.load ssl.load
lsb_release -rd
Description: Ubuntu 14.04.4 LTS
Release: 14.04
Regards,
Rainer
