Package: apache2
Version: 2.4.56-1~deb11u2
Severity: important
Dear Maintainer,
I see many segmentation faults in apache2, for exemple in the last 24h I got:
Tue 2023-05-16 13:40:59 CEST 775740 33 33 11 present /usr/sbin/apache2
Tue 2023-05-16 13:52:44 CEST 798329 33 33 11 present /usr/sbin/apache2
Tue 2023-05-16 16:15:46 CEST 810709 33 33 11 present /usr/sbin/apache2
Tue 2023-05-16 16:28:55 CEST 817483 33 33 11 present /usr/sbin/apache2
Tue 2023-05-16 17:59:23 CEST 823129 33 33 11 present /usr/sbin/apache2
Tue 2023-05-16 18:35:50 CEST 826974 33 33 11 present /usr/sbin/apache2
Tue 2023-05-16 18:44:15 CEST 831974 33 33 11 present /usr/sbin/apache2
Tue 2023-05-16 18:44:56 CEST 836174 33 33 11 present /usr/sbin/apache2
Tue 2023-05-16 18:54:56 CEST 822618 33 33 11 present /usr/sbin/apache2
Tue 2023-05-16 21:12:28 CEST 836246 33 33 11 present /usr/sbin/apache2
Tue 2023-05-16 21:21:10 CEST 853959 33 33 11 present /usr/sbin/apache2
Tue 2023-05-16 22:04:42 CEST 858749 33 33 11 present /usr/sbin/apache2
Tue 2023-05-16 23:26:09 CEST 866610 33 33 11 present /usr/sbin/apache2
Wed 2023-05-17 00:08:42 CEST 865968 33 33 11 present /usr/sbin/apache2
Wed 2023-05-17 00:24:04 CEST 874807 33 33 11 present /usr/sbin/apache2
Wed 2023-05-17 00:47:25 CEST 878675 33 33 11 present /usr/sbin/apache2
Wed 2023-05-17 01:42:14 CEST 877580 33 33 11 present /usr/sbin/apache2
Wed 2023-05-17 09:21:02 CEST 949781 33 33 11 present /usr/sbin/apache2
Wed 2023-05-17 09:50:49 CEST 954784 33 33 11 present /usr/sbin/apache2
All crashes I looked into are at the same function: purge_consumed_buckets at
h2_bucket_beam.c:159
Here is the output of the "bt full" command from the core:
#0 0x00007ffb03778981 in purge_consumed_buckets
(beam=beam@entry=0x7ffae452c0a0) at h2_bucket_beam.c:159
b = 0x7ffae45ea108
#1 0x00007ffb03778aaf in beam_shutdown (how=APR_SHUTDOWN_READWRITE,
beam=<optimized out>) at h2_bucket_beam.c:258
No locals.
#2 beam_shutdown (how=APR_SHUTDOWN_READWRITE, beam=0x7ffae452c0a0) at
h2_bucket_beam.c:242
No locals.
#3 beam_cleanup (data=0x7ffae452c0a0) at h2_bucket_beam.c:265
beam = 0x7ffae452c0a0
#4 0x00007ffb03e6780e in run_cleanups (cref=0x7ffae452c098) at
./memory/unix/apr_pools.c:2629
c = <optimized out>
c = <optimized out>
#5 apr_pool_destroy (pool=0x7ffae452c028) at ./memory/unix/apr_pools.c:987
active = <optimized out>
allocator = <optimized out>
#6 0x00007ffb03e6782d in apr_pool_destroy (pool=0x7ffae4530028) at
./memory/unix/apr_pools.c:997
active = <optimized out>
allocator = <optimized out>
#7 0x00007ffb03e6782d in apr_pool_destroy (pool=0x7ffae4551028) at
./memory/unix/apr_pools.c:997
active = <optimized out>
allocator = <optimized out>
#8 0x00007ffb03e6782d in apr_pool_destroy (pool=0x7ffae45a1028) at
./memory/unix/apr_pools.c:997
active = <optimized out>
allocator = <optimized out>
#9 0x00007ffb03e6782d in apr_pool_destroy (pool=0x7ffae4606028) at
./memory/unix/apr_pools.c:997
active = <optimized out>
allocator = <optimized out>
#10 0x00007ffb037914c5 in h2_session_pre_close (session=0x7ffae46060a0,
async=<optimized out>) at h2_session.c:1988
status = 0
#11 0x00007ffb0377b745 in h2_c1_pre_close (ctx=<optimized out>, c=<optimized
out>) at h2_c1.c:180
status = <optimized out>
conn_ctx = <optimized out>
#12 0x000056438478c9b0 in ap_run_pre_close_connection
(c=c@entry=0x7ffae4614360) at connection.c:44
pHook = <optimized out>
n = 0
rv = 0
#13 0x000056438478cade in ap_prep_lingering_close (c=0x7ffae4614360) at
connection.c:101
No locals.
#14 ap_start_lingering_close (c=0x7ffae4614360) at connection.c:127
csd = 0x7ffae46140b0
#15 0x00007ffb03b08abe in process_lingering_close (cs=0x7ffae46142b0) at
event.c:1500
csd = 0x7ffae46140b0
dummybuf =
"\027\003\003\000\023\067\020\251\027\003\215Re\345\310{\f8\312X\332N\310\375",
'\000' <repeats 17385 times>...
nbytes = 0
rv = <optimized out>
q = <optimized out>
#16 0x00007ffb03b0a512 in process_socket (thd=thd@entry=0x7ffb037345c8,
p=<optimized out>, sock=<optimized out>, cs=<optimized out>,
my_child_num=my_child_num@entry=3, my_thread_num=my_thread_num@entry=16) at
event.c:1238
c = <optimized out>
conn_id = <optimized out>
clogging = <optimized out>
rv = <optimized out>
rc = <optimized out>
#17 0x00007ffb03b0b125 in worker_thread (thd=0x7ffb037345c8, dummy=<optimized
out>) at event.c:2199
csd = 0x7ffae46140b0
cs = 0x7ffae46142b0
te = 0x0
ptrans = 0x0
ti = <optimized out>
process_slot = -855667096
thread_slot = 16
rv = <optimized out>
is_idle = 0
#18 0x00007ffb03e2aea7 in start_thread (arg=<optimized out>) at
pthread_create.c:477
ret = <optimized out>
pd = <optimized out>
unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140715157853952,
-1517716079030320448, 140715846122926, 140715846122927, 140715157852032,
8396800, 1520638580441989824, 1520521782042673856}, mask_was_saved = 0}}, priv
= {pad = {0x0,
0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype =
0}}}
not_first_call = 0
#19 0x00007ffb03d4aa2f in clone () at
../sysdeps/unix/sysv/linux/x86_64/clone.S:95
It may - or not - be related with the fact I got hundreds (had been hundreds of
thousands) of persistent connections from random IR hosts to my https server, I
guess it can put more stress on h2 cleanup functions that the normal (quite
low) usage I had before.
I also seen that this function was corrected a few days ago :
https://github.com/icing/mod_h2/commit/ff00b3fdff368b225e70c61ca0fefdbd3d83f6de
I don't know enough of apache2's codebase to see if it may has an impact, but
it seems unlikely to me; as the AP_BUCKET_IS_EOR() check seems related to a
newly introduced member
Regards,
-- Package-specific info:
-- System Information:
Debian Release: 11.7
APT prefers stable-updates
APT policy: (500, 'stable-updates'), (500, 'stable-security'), (500,
'stable-debug'), (500, 'oldstable-updates'), (500, 'oldstable-debug'), (500,
'stable'), (500, 'oldstable')
Architecture: amd64 (x86_64)
Kernel: Linux 5.10.0-23-amd64 (SMP w/8 CPU threads)
Locale: LANG=fr_FR.UTF-8, LC_CTYPE=fr_FR.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
Versions of packages apache2 depends on:
ii apache2-bin 2.4.56-1~deb11u2
ii apache2-data 2.4.56-1~deb11u2
ii apache2-utils 2.4.56-1~deb11u2
ii dpkg 1.20.12
ii init-system-helpers 1.60
ii lsb-base 11.1.0
ii mime-support 3.66
ii perl 5.32.1-4+deb11u2
ii procps 2:3.3.17-5
Versions of packages apache2 recommends:
ii ssl-cert 1.1.0+nmu1
Versions of packages apache2 suggests:
pn apache2-doc <none>
ii apache2-suexec-pristine 2.4.56-1~deb11u2
ii chimera2 [www-browser] 2.0a19-8+b2
ii dillo [www-browser] 3.0.5-7
ii links2 [www-browser] 2.21-1+b1
ii lynx [www-browser] 2.9.0dev.6-3~deb11u1
Versions of packages apache2-bin depends on:
ii libapr1 1.7.0-6+deb11u2
ii libaprutil1 1.6.1-5+deb11u1
ii libaprutil1-dbd-sqlite3 1.6.1-5+deb11u1
ii libaprutil1-ldap 1.6.1-5+deb11u1
ii libbrotli1 1.0.9-2+b2
ii libc6 2.31-13+deb11u6
ii libcrypt1 1:4.4.18-4
ii libcurl4 7.74.0-1.3+deb11u7
ii libjansson4 2.13.1-1.1
ii libldap-2.4-2 2.4.57+dfsg-3+deb11u1
ii liblua5.3-0 5.3.3-1.1+b1
ii libnghttp2-14 1.43.0-1
ii libpcre3 2:8.44-2+0~20210301.9+debian11~1.gbpa278ad
ii libssl1.1 1.1.1n-0+deb11u4
ii libxml2 2.9.14+dfsg-0.1+0~20230421.14+debian11~1.gbpf14485
ii perl 5.32.1-4+deb11u2
ii zlib1g 1:1.2.11.dfsg-2+deb11u2
Versions of packages apache2-bin suggests:
pn apache2-doc <none>
ii apache2-suexec-pristine 2.4.56-1~deb11u2
ii chimera2 [www-browser] 2.0a19-8+b2
ii dillo [www-browser] 3.0.5-7
ii links2 [www-browser] 2.21-1+b1
ii lynx [www-browser] 2.9.0dev.6-3~deb11u1
Versions of packages apache2 is related to:
ii apache2 2.4.56-1~deb11u2
ii apache2-bin 2.4.56-1~deb11u2
-- Configuration Files:
/etc/apache2/apache2.conf changed [not included]
/etc/apache2/conf-available/security.conf changed [not included]
/etc/apache2/envvars changed [not included]
/etc/apache2/mods-available/cern_meta.load [Errno 2] Aucun fichier ou dossier
de ce type: '/etc/apache2/mods-available/cern_meta.load'
/etc/apache2/mods-available/ident.load [Errno 2] Aucun fichier ou dossier de ce
type: '/etc/apache2/mods-available/ident.load'
/etc/apache2/mods-available/imagemap.load [Errno 2] Aucun fichier ou dossier de
ce type: '/etc/apache2/mods-available/imagemap.load'
/etc/apache2/mods-available/proxy_hcheck.load [Errno 2] Aucun fichier ou
dossier de ce type: '/etc/apache2/mods-available/proxy_hcheck.load'
/etc/apache2/mods-available/proxy_html.load changed [not included]
/etc/apache2/mods-available/proxy_http2.load changed [not included]
/etc/apache2/mods-available/userdir.conf changed [not included]
/etc/apache2/ports.conf changed [not included]
/etc/apache2/sites-available/000-default.conf changed [not included]
/etc/apache2/sites-available/default-ssl.conf changed [not included]
/etc/cron.daily/apache2 changed [not included]
/etc/default/apache-htcacheclean [Errno 2] Aucun fichier ou dossier de ce type:
'/etc/default/apache-htcacheclean'
/etc/init.d/apache2 changed [not included]
/etc/logrotate.d/apache2 changed [not included]
-- no debconf information
