Package: apache2 Version: 2.4.56-1~deb11u2 Severity: important Dear Maintainer,
I see many segmentation faults in apache2, for exemple in the last 24h I got: Tue 2023-05-16 13:40:59 CEST 775740 33 33 11 present /usr/sbin/apache2 Tue 2023-05-16 13:52:44 CEST 798329 33 33 11 present /usr/sbin/apache2 Tue 2023-05-16 16:15:46 CEST 810709 33 33 11 present /usr/sbin/apache2 Tue 2023-05-16 16:28:55 CEST 817483 33 33 11 present /usr/sbin/apache2 Tue 2023-05-16 17:59:23 CEST 823129 33 33 11 present /usr/sbin/apache2 Tue 2023-05-16 18:35:50 CEST 826974 33 33 11 present /usr/sbin/apache2 Tue 2023-05-16 18:44:15 CEST 831974 33 33 11 present /usr/sbin/apache2 Tue 2023-05-16 18:44:56 CEST 836174 33 33 11 present /usr/sbin/apache2 Tue 2023-05-16 18:54:56 CEST 822618 33 33 11 present /usr/sbin/apache2 Tue 2023-05-16 21:12:28 CEST 836246 33 33 11 present /usr/sbin/apache2 Tue 2023-05-16 21:21:10 CEST 853959 33 33 11 present /usr/sbin/apache2 Tue 2023-05-16 22:04:42 CEST 858749 33 33 11 present /usr/sbin/apache2 Tue 2023-05-16 23:26:09 CEST 866610 33 33 11 present /usr/sbin/apache2 Wed 2023-05-17 00:08:42 CEST 865968 33 33 11 present /usr/sbin/apache2 Wed 2023-05-17 00:24:04 CEST 874807 33 33 11 present /usr/sbin/apache2 Wed 2023-05-17 00:47:25 CEST 878675 33 33 11 present /usr/sbin/apache2 Wed 2023-05-17 01:42:14 CEST 877580 33 33 11 present /usr/sbin/apache2 Wed 2023-05-17 09:21:02 CEST 949781 33 33 11 present /usr/sbin/apache2 Wed 2023-05-17 09:50:49 CEST 954784 33 33 11 present /usr/sbin/apache2 All crashes I looked into are at the same function: purge_consumed_buckets at h2_bucket_beam.c:159 Here is the output of the "bt full" command from the core: #0 0x00007ffb03778981 in purge_consumed_buckets (beam=beam@entry=0x7ffae452c0a0) at h2_bucket_beam.c:159 b = 0x7ffae45ea108 #1 0x00007ffb03778aaf in beam_shutdown (how=APR_SHUTDOWN_READWRITE, beam=<optimized out>) at h2_bucket_beam.c:258 No locals. #2 beam_shutdown (how=APR_SHUTDOWN_READWRITE, beam=0x7ffae452c0a0) at h2_bucket_beam.c:242 No locals. #3 beam_cleanup (data=0x7ffae452c0a0) at h2_bucket_beam.c:265 beam = 0x7ffae452c0a0 #4 0x00007ffb03e6780e in run_cleanups (cref=0x7ffae452c098) at ./memory/unix/apr_pools.c:2629 c = <optimized out> c = <optimized out> #5 apr_pool_destroy (pool=0x7ffae452c028) at ./memory/unix/apr_pools.c:987 active = <optimized out> allocator = <optimized out> #6 0x00007ffb03e6782d in apr_pool_destroy (pool=0x7ffae4530028) at ./memory/unix/apr_pools.c:997 active = <optimized out> allocator = <optimized out> #7 0x00007ffb03e6782d in apr_pool_destroy (pool=0x7ffae4551028) at ./memory/unix/apr_pools.c:997 active = <optimized out> allocator = <optimized out> #8 0x00007ffb03e6782d in apr_pool_destroy (pool=0x7ffae45a1028) at ./memory/unix/apr_pools.c:997 active = <optimized out> allocator = <optimized out> #9 0x00007ffb03e6782d in apr_pool_destroy (pool=0x7ffae4606028) at ./memory/unix/apr_pools.c:997 active = <optimized out> allocator = <optimized out> #10 0x00007ffb037914c5 in h2_session_pre_close (session=0x7ffae46060a0, async=<optimized out>) at h2_session.c:1988 status = 0 #11 0x00007ffb0377b745 in h2_c1_pre_close (ctx=<optimized out>, c=<optimized out>) at h2_c1.c:180 status = <optimized out> conn_ctx = <optimized out> #12 0x000056438478c9b0 in ap_run_pre_close_connection (c=c@entry=0x7ffae4614360) at connection.c:44 pHook = <optimized out> n = 0 rv = 0 #13 0x000056438478cade in ap_prep_lingering_close (c=0x7ffae4614360) at connection.c:101 No locals. #14 ap_start_lingering_close (c=0x7ffae4614360) at connection.c:127 csd = 0x7ffae46140b0 #15 0x00007ffb03b08abe in process_lingering_close (cs=0x7ffae46142b0) at event.c:1500 csd = 0x7ffae46140b0 dummybuf = "\027\003\003\000\023\067\020\251\027\003\215Re\345\310{\f8\312X\332N\310\375", '\000' <repeats 17385 times>... nbytes = 0 rv = <optimized out> q = <optimized out> #16 0x00007ffb03b0a512 in process_socket (thd=thd@entry=0x7ffb037345c8, p=<optimized out>, sock=<optimized out>, cs=<optimized out>, my_child_num=my_child_num@entry=3, my_thread_num=my_thread_num@entry=16) at event.c:1238 c = <optimized out> conn_id = <optimized out> clogging = <optimized out> rv = <optimized out> rc = <optimized out> #17 0x00007ffb03b0b125 in worker_thread (thd=0x7ffb037345c8, dummy=<optimized out>) at event.c:2199 csd = 0x7ffae46140b0 cs = 0x7ffae46142b0 te = 0x0 ptrans = 0x0 ti = <optimized out> process_slot = -855667096 thread_slot = 16 rv = <optimized out> is_idle = 0 #18 0x00007ffb03e2aea7 in start_thread (arg=<optimized out>) at pthread_create.c:477 ret = <optimized out> pd = <optimized out> unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140715157853952, -1517716079030320448, 140715846122926, 140715846122927, 140715157852032, 8396800, 1520638580441989824, 1520521782042673856}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}} not_first_call = 0 #19 0x00007ffb03d4aa2f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95 It may - or not - be related with the fact I got hundreds (had been hundreds of thousands) of persistent connections from random IR hosts to my https server, I guess it can put more stress on h2 cleanup functions that the normal (quite low) usage I had before. I also seen that this function was corrected a few days ago : https://github.com/icing/mod_h2/commit/ff00b3fdff368b225e70c61ca0fefdbd3d83f6de I don't know enough of apache2's codebase to see if it may has an impact, but it seems unlikely to me; as the AP_BUCKET_IS_EOR() check seems related to a newly introduced member Regards, -- Package-specific info: -- System Information: Debian Release: 11.7 APT prefers stable-updates APT policy: (500, 'stable-updates'), (500, 'stable-security'), (500, 'stable-debug'), (500, 'oldstable-updates'), (500, 'oldstable-debug'), (500, 'stable'), (500, 'oldstable') Architecture: amd64 (x86_64) Kernel: Linux 5.10.0-23-amd64 (SMP w/8 CPU threads) Locale: LANG=fr_FR.UTF-8, LC_CTYPE=fr_FR.UTF-8 (charmap=UTF-8), LANGUAGE not set Shell: /bin/sh linked to /bin/dash Init: systemd (via /run/systemd/system) Versions of packages apache2 depends on: ii apache2-bin 2.4.56-1~deb11u2 ii apache2-data 2.4.56-1~deb11u2 ii apache2-utils 2.4.56-1~deb11u2 ii dpkg 1.20.12 ii init-system-helpers 1.60 ii lsb-base 11.1.0 ii mime-support 3.66 ii perl 5.32.1-4+deb11u2 ii procps 2:3.3.17-5 Versions of packages apache2 recommends: ii ssl-cert 1.1.0+nmu1 Versions of packages apache2 suggests: pn apache2-doc <none> ii apache2-suexec-pristine 2.4.56-1~deb11u2 ii chimera2 [www-browser] 2.0a19-8+b2 ii dillo [www-browser] 3.0.5-7 ii links2 [www-browser] 2.21-1+b1 ii lynx [www-browser] 2.9.0dev.6-3~deb11u1 Versions of packages apache2-bin depends on: ii libapr1 1.7.0-6+deb11u2 ii libaprutil1 1.6.1-5+deb11u1 ii libaprutil1-dbd-sqlite3 1.6.1-5+deb11u1 ii libaprutil1-ldap 1.6.1-5+deb11u1 ii libbrotli1 1.0.9-2+b2 ii libc6 2.31-13+deb11u6 ii libcrypt1 1:4.4.18-4 ii libcurl4 7.74.0-1.3+deb11u7 ii libjansson4 2.13.1-1.1 ii libldap-2.4-2 2.4.57+dfsg-3+deb11u1 ii liblua5.3-0 5.3.3-1.1+b1 ii libnghttp2-14 1.43.0-1 ii libpcre3 2:8.44-2+0~20210301.9+debian11~1.gbpa278ad ii libssl1.1 1.1.1n-0+deb11u4 ii libxml2 2.9.14+dfsg-0.1+0~20230421.14+debian11~1.gbpf14485 ii perl 5.32.1-4+deb11u2 ii zlib1g 1:1.2.11.dfsg-2+deb11u2 Versions of packages apache2-bin suggests: pn apache2-doc <none> ii apache2-suexec-pristine 2.4.56-1~deb11u2 ii chimera2 [www-browser] 2.0a19-8+b2 ii dillo [www-browser] 3.0.5-7 ii links2 [www-browser] 2.21-1+b1 ii lynx [www-browser] 2.9.0dev.6-3~deb11u1 Versions of packages apache2 is related to: ii apache2 2.4.56-1~deb11u2 ii apache2-bin 2.4.56-1~deb11u2 -- Configuration Files: /etc/apache2/apache2.conf changed [not included] /etc/apache2/conf-available/security.conf changed [not included] /etc/apache2/envvars changed [not included] /etc/apache2/mods-available/cern_meta.load [Errno 2] Aucun fichier ou dossier de ce type: '/etc/apache2/mods-available/cern_meta.load' /etc/apache2/mods-available/ident.load [Errno 2] Aucun fichier ou dossier de ce type: '/etc/apache2/mods-available/ident.load' /etc/apache2/mods-available/imagemap.load [Errno 2] Aucun fichier ou dossier de ce type: '/etc/apache2/mods-available/imagemap.load' /etc/apache2/mods-available/proxy_hcheck.load [Errno 2] Aucun fichier ou dossier de ce type: '/etc/apache2/mods-available/proxy_hcheck.load' /etc/apache2/mods-available/proxy_html.load changed [not included] /etc/apache2/mods-available/proxy_http2.load changed [not included] /etc/apache2/mods-available/userdir.conf changed [not included] /etc/apache2/ports.conf changed [not included] /etc/apache2/sites-available/000-default.conf changed [not included] /etc/apache2/sites-available/default-ssl.conf changed [not included] /etc/cron.daily/apache2 changed [not included] /etc/default/apache-htcacheclean [Errno 2] Aucun fichier ou dossier de ce type: '/etc/default/apache-htcacheclean' /etc/init.d/apache2 changed [not included] /etc/logrotate.d/apache2 changed [not included] -- no debconf information