https://bz.apache.org/bugzilla/show_bug.cgi?id=68561
Bug ID: 68561 Summary: IDLE processes are not killed with mpm_event Product: Apache httpd-2 Version: 2.4.52 Hardware: PC OS: Linux Status: NEW Severity: normal Priority: P2 Component: mpm_event Assignee: bugs@httpd.apache.org Reporter: skogorev...@yandex.ru Target Milestone: --- After a few reloads, IDLE processes stop being killed. I am using Apache2=2.4.52 with patch 1901199 applied (from https://bz.apache.org/bugzilla/show_bug.cgi?id=66004), but it is still reproducible for me. I am using Debian GNU/Linux 10 (buster) mpm_event configuration: <IfModule mpm_event_module> StartServers 50 MinSpareThreads 350 MaxSpareThreads 400 ThreadLimit 7 ThreadsPerChild 7 MaxRequestWorkers 924 MaxConnectionsPerChild 0 ServerLimit 132 </IfModule> I get these messages in the logs: [Mon Jan 29 07:53:02.223009 2024] [mpm_event:trace5] [pid 29303:tid 139759872828544] event.c(2936): Not shutting down one child: active 130/132, total 131/132/132, idle threads 824, max workers 924 [Mon Jan 29 07:53:03.223613 2024] [mpm_event:trace5] [pid 29303:tid 139759872828544] event.c(2936): Not shutting down one child: active 130/132, total 131/132/132, idle threads 826, max workers 924 [Mon Jan 29 07:53:04.224772 2024] [mpm_event:trace5] [pid 29303:tid 139759872828544] event.c(2936): Not shutting down one child: active 130/132, total 131/132/132, idle threads 823, max workers 924 [Mon Jan 29 07:53:05.225888 2024] [mpm_event:trace5] [pid 29303:tid 139759872828544] event.c(2936): Not shutting down one child: active 130/132, total 131/132/132, idle threads 823, max workers 924 [Mon Jan 29 07:53:06.227033 2024] [mpm_event:trace5] [pid 29303:tid 139759872828544] event.c(2936): Not shutting down one child: active 130/132, total 131/132/132, idle threads 823, max workers 924 [Mon Jan 29 07:53:07.228175 2024] [mpm_event:trace5] [pid 29303:tid 139759872828544] event.c(2936): Not shutting down one child: active 130/132, total 131/132/132, idle threads 822, max workers 924 [Mon Jan 29 07:53:08.229332 2024] [mpm_event:trace5] [pid 29303:tid 139759872828544] event.c(2936): Not shutting down one child: active 130/132, total 131/132/132, idle threads 823, max workers 924 [Mon Jan 29 07:53:09.230462 2024] [mpm_event:trace5] [pid 29303:tid 139759872828544] event.c(2936): Not shutting down one child: active 130/132, total 131/132/132, idle threads 821, max workers 924 [Mon Jan 29 07:53:10.231594 2024] [mpm_event:trace5] [pid 29303:tid 139759872828544] event.c(2936): Not shutting down one child: active 130/132, total 131/132/132, idle threads 824, max workers 924 [Mon Jan 29 07:53:11.234644 2024] [mpm_event:trace5] [pid 29303:tid 139759872828544] event.c(2936): Not shutting down one child: active 130/132, total 131/132/132, idle threads 825, max workers 924 [Mon Jan 29 07:53:12.235627 2024] [mpm_event:trace5] [pid 29303:tid 139759872828544] event.c(2936): Not shutting down one child: active 130/132, total 131/132/132, idle threads 825, max workers 924 [Mon Jan 29 07:53:13.236948 2024] [mpm_event:trace5] [pid 29303:tid 139759872828544] event.c(2936): Not shutting down one child: active 130/132, total 131/132/132, idle threads 826, max workers 924 [Mon Jan 29 07:53:14.238283 2024] [mpm_event:trace5] [pid 29303:tid 139759872828544] event.c(2936): Not shutting down one child: active 130/132, total 131/132/132, idle threads 826, max workers 924 [Mon Jan 29 07:53:15.239535 2024] [mpm_event:trace5] [pid 29303:tid 139759872828544] event.c(2936): Not shutting down one child: active 130/132, total 131/132/132, idle threads 823, max workers 924 [Mon Jan 29 07:53:16.240305 2024] [mpm_event:trace5] [pid 29303:tid 139759872828544] event.c(2936): Not shutting down one child: active 130/132, total 131/132/132, idle threads 825, max workers 924 [Mon Jan 29 07:53:17.241504 2024] [mpm_event:trace5] [pid 29303:tid 139759872828544] event.c(2936): Not shutting down one child: active 130/132, total 131/132/132, idle threads 825, max workers 924 [Mon Jan 29 07:53:18.242678 2024] [mpm_event:trace5] [pid 29303:tid 139759872828544] event.c(2936): Not shutting down one child: active 130/132, total 131/132/132, idle threads 826, max workers 924 [Mon Jan 29 07:53:19.243689 2024] [mpm_event:trace5] [pid 29303:tid 139759872828544] event.c(2936): Not shutting down one child: active 130/132, total 131/132/132, idle threads 826, max workers 924 [Mon Jan 29 07:53:20.244875 2024] [mpm_event:trace5] [pid 29303:tid 139759872828544] event.c(2936): Not shutting down one child: active 130/132, total 131/132/132, idle threads 826, max workers 924 This problem does not reproduce immediately, but after some time. Stopping/starting apache2 recovers from this situation. This is only reproduced in a production environment where there is a lot of traffic. When I do a reboot, all child processes are terminated and re-created: [Mon Jan 29 08:03:03.902993 2024] [mpm_event:debug] [pid 29303:tid 139759872828544] event.c(741): Child 1 started: pid 24591, gen 30, active 15/132, total 63/132/132 [Mon Jan 29 08:03:03.903172 2024] [mpm_event:debug] [pid 29303:tid 139759872828544] event.c(716): Child 4 stopped: pid 20114, gen 29, active 15/132, total 62/132/132, quiescing 2 [Mon Jan 29 08:03:03.903282 2024] [mpm_event:debug] [pid 29303:tid 139759872828544] event.c(716): Child 40 stopped: pid 20488, gen 29, active 15/132, total 61/132/132, quiescing 2 [Mon Jan 29 08:03:03.903369 2024] [mpm_event:debug] [pid 29303:tid 139759872828544] event.c(716): Child 44 stopped: pid 20491, gen 29, active 15/132, total 60/132/132, quiescing 2 [Mon Jan 29 08:03:03.903449 2024] [mpm_event:debug] [pid 29303:tid 139759872828544] event.c(716): Child 49 stopped: pid 20495, gen 29, active 15/132, total 59/132/132, quiescing 2 [Mon Jan 29 08:03:03.903524 2024] [mpm_event:debug] [pid 29303:tid 139759872828544] event.c(716): Child 51 stopped: pid 20497, gen 29, active 15/132, total 58/132/132, quiescing 2 [Mon Jan 29 08:03:03.903619 2024] [mpm_event:debug] [pid 29303:tid 139759872828544] event.c(716): Child 53 stopped: pid 20499, gen 29, active 15/132, total 57/132/132, quiescing 2 [Mon Jan 29 08:03:03.903696 2024] [mpm_event:debug] [pid 29303:tid 139759872828544] event.c(716): Child 55 stopped: pid 20501, gen 29, active 15/132, total 56/132/132, quiescing 2 [Mon Jan 29 08:03:03.903778 2024] [mpm_event:debug] [pid 29303:tid 139759872828544] event.c(716): Child 56 stopped: pid 20502, gen 29, active 15/132, total 55/132/132, quiescing 2 [Mon Jan 29 08:03:03.903863 2024] [mpm_event:debug] [pid 29303:tid 139759872828544] event.c(716): Child 59 stopped: pid 20505, gen 29, active 15/132, total 54/132/132, quiescing 2 [Mon Jan 29 08:03:03.903940 2024] [mpm_event:debug] [pid 29303:tid 139759872828544] event.c(716): Child 60 stopped: pid 20506, gen 29, active 15/132, total 53/132/132, quiescing 2 ........ ........ ........ [Mon Jan 29 08:03:04.059700 2024] [mpm_event:trace1] [pid 24591:tid 139759720564480] event.c(1895): Idle workers: 0 [Mon Jan 29 08:03:04.908328 2024] [mpm_event:debug] [pid 29303:tid 139759872828544] event.c(741): Child 3 started: pid 24603, gen 30, active 16/132, total 18/132/132 [Mon Jan 29 08:03:04.908496 2024] [mpm_event:debug] [pid 29303:tid 139759872828544] event.c(716): Child 79 stopped: pid 20850, gen 29, active 16/132, total 17/86/132, quiescing 2 [Mon Jan 29 08:03:05.014824 2024] [mpm_event:debug] [pid 24603:tid 139759856973568] event.c(2395): AH02471: start_threads: Using epoll (wakeable) [Mon Jan 29 08:03:05.029307 2024] [mpm_event:debug] [pid 24603:tid 139759720564480] event.c(1892): Too many open connections (0), not accepting new conns in this process [Mon Jan 29 08:03:05.033225 2024] [mpm_event:trace1] [pid 24603:tid 139759720564480] event.c(1895): Idle workers: 3 [Mon Jan 29 08:03:05.910375 2024] [mpm_event:debug] [pid 29303:tid 139759872828544] event.c(741): Child 4 started: pid 24690, gen 30, active 17/132, total 18/86/132 [Mon Jan 29 08:03:05.975729 2024] [mpm_event:debug] [pid 24690:tid 139759856973568] event.c(2395): AH02471: start_threads: Using epoll (wakeable) [Mon Jan 29 08:03:06.006641 2024] [mpm_event:debug] [pid 24690:tid 139759720564480] event.c(1892): Too many open connections (6), not accepting new conns in this process [Mon Jan 29 08:03:06.007728 2024] [mpm_event:trace1] [pid 24690:tid 139759720564480] event.c(1895): Idle workers: 0 [Mon Jan 29 08:03:06.912213 2024] [mpm_event:debug] [pid 29303:tid 139759872828544] event.c(741): Child 5 started: pid 24709, gen 30, active 18/132, total 19/86/132 [Mon Jan 29 08:03:07.035438 2024] [mpm_event:debug] [pid 24709:tid 139759856973568] event.c(2395): AH02471: start_threads: Using epoll (wakeable) [Mon Jan 29 08:03:07.064820 2024] [mpm_event:debug] [pid 24709:tid 139759720564480] event.c(1892): Too many open connections (5), not accepting new conns in this process [Mon Jan 29 08:03:07.065639 2024] [mpm_event:trace1] [pid 24709:tid 139759720564480] event.c(1895): Idle workers: 1 .......... .......... .......... [Mon Jan 29 08:03:14.926642 2024] [mpm_event:info] [pid 29303:tid 139759872828544] AH00486: server seems busy, (you may need to increase StartServers, ThreadsPerChild or Min/MaxSpareThreads), spawning 8 children, there are around 0 idle threads, 29 active children, and 30 children that are shutting down [Mon Jan 29 08:03:14.927470 2024] [mpm_event:debug] [pid 29303:tid 139759872828544] event.c(741): Child 20 started: pid 24873, gen 30, active 30/132, total 31/86/132 [Mon Jan 29 08:03:14.927900 2024] [mpm_event:debug] [pid 29303:tid 139759872828544] event.c(741): Child 21 started: pid 24874, gen 30, active 31/132, total 32/86/132 [Mon Jan 29 08:03:14.928331 2024] [mpm_event:debug] [pid 29303:tid 139759872828544] event.c(741): Child 22 started: pid 24875, gen 30, active 32/132, total 33/86/132 [Mon Jan 29 08:03:14.928738 2024] [mpm_event:debug] [pid 29303:tid 139759872828544] event.c(741): Child 23 started: pid 24876, gen 30, active 33/132, total 34/86/132 [Mon Jan 29 08:03:14.929142 2024] [mpm_event:debug] [pid 29303:tid 139759872828544] event.c(741): Child 24 started: pid 24877, gen 30, active 34/132, total 35/86/132 [Mon Jan 29 08:03:14.929506 2024] [mpm_event:debug] [pid 29303:tid 139759872828544] event.c(741): Child 25 started: pid 24878, gen 30, active 35/132, total 36/86/132 [Mon Jan 29 08:03:14.929893 2024] [mpm_event:debug] [pid 29303:tid 139759872828544] event.c(741): Child 27 started: pid 24879, gen 30, active 36/132, total 37/86/132 [Mon Jan 29 08:03:14.930263 2024] [mpm_event:debug] [pid 29303:tid 139759872828544] event.c(741): Child 28 started: pid 24880, gen 30, active 37/132, total 38/86/132 [Mon Jan 29 08:03:15.095599 2024] [mpm_event:debug] [pid 24875:tid 139759856973568] event.c(2395): AH02471: start_threads: Using epoll (wakeable) [Mon Jan 29 08:03:15.107594 2024] [mpm_event:debug] [pid 24873:tid 139759856973568] event.c(2395): AH02471: start_threads: Using epoll (wakeable) [Mon Jan 29 08:03:15.108810 2024] [mpm_event:debug] [pid 24874:tid 139759856973568] event.c(2395): AH02471: start_threads: Using epoll (wakeable) [Mon Jan 29 08:03:15.109032 2024] [mpm_event:debug] [pid 24877:tid 139759856973568] event.c(2395): AH02471: start_threads: Using epoll (wakeable) [Mon Jan 29 08:03:15.117436 2024] [mpm_event:debug] [pid 24873:tid 139759720564480] event.c(1892): Too many open connections (0), not accepting new conns in this process [Mon Jan 29 08:03:15.117466 2024] [mpm_event:trace1] [pid 24873:tid 139759720564480] event.c(1895): Idle workers: 0 [Mon Jan 29 08:03:15.123705 2024] [mpm_event:debug] [pid 24877:tid 139759720564480] event.c(1892): Too many open connections (0), not accepting new conns in this process [Mon Jan 29 08:03:15.123761 2024] [mpm_event:trace1] [pid 24877:tid 139759720564480] event.c(1895): Idle workers: 0 ........ ........ ........ [Mon Jan 29 08:18:38.202850 2024] [mpm_event:trace5] [pid 29303:tid 139759872828544] event.c(2936): Not shutting down one child: active 131/132, total 132/132/132, idle threads 825, max workers 924 [Mon Jan 29 08:18:38.247684 2024] [mpm_event:trace6] [pid 25885:tid 139759720564480] event.c(1707): connections: 0 (clogged: 0 write-completion: 0 keep-alive: 0 lingering: 0 suspended: 0) [Mon Jan 29 08:18:38.628382 2024] [mpm_event:trace6] [pid 25564:tid 139759720564480] event.c(1707): connections: 0 (clogged: 0 write-completion: 0 keep-alive: 0 lingering: 0 suspended: 0) [Mon Jan 29 08:18:38.628469 2024] [mpm_event:trace6] [pid 25539:tid 139759720564480] event.c(1707): connections: 0 (clogged: 0 write-completion: 0 keep-alive: 0 lingering: 0 suspended: 0) [Mon Jan 29 08:18:38.910546 2024] [mpm_event:trace6] [pid 25180:tid 139759720564480] event.c(1707): connections: 0 (clogged: 0 write-completion: 0 keep-alive: 0 lingering: 0 suspended: 0) [Mon Jan 29 08:18:39.003809 2024] [mpm_event:trace6] [pid 25197:tid 139759720564480] event.c(1707): connections: 0 (clogged: 0 write-completion: 0 keep-alive: 0 lingering: 0 suspended: 0) [Mon Jan 29 08:18:39.003781 2024] [mpm_event:trace6] [pid 25886:tid 139759720564480] event.c(1707): connections: 0 (clogged: 0 write-completion: 0 keep-alive: 0 lingering: 0 suspended: 0) [Mon Jan 29 08:18:39.080928 2024] [mpm_event:trace6] [pid 25567:tid 139759720564480] event.c(1707): connections: 0 (clogged: 0 write-completion: 0 keep-alive: 0 lingering: 0 suspended: 0) [Mon Jan 29 08:18:39.083700 2024] [mpm_event:trace6] [pid 24982:tid 139759720564480] event.c(1707): connections: 0 (clogged: 0 write-completion: 0 keep-alive: 0 lingering: 0 suspended: 0) [Mon Jan 29 08:18:39.204016 2024] [mpm_event:trace5] [pid 29303:tid 139759872828544] event.c(2936): Not shutting down one child: active 131/132, total 132/132/132, idle threads 824, max workers 924 [Mon Jan 29 08:18:39.254909 2024] [mpm_event:trace6] [pid 25885:tid 139759720564480] event.c(1707): connections: 0 (clogged: 0 write-completion: 0 keep-alive: 0 lingering: 0 suspended: 0) [Mon Jan 29 08:18:39.409146 2024] [mpm_event:trace6] [pid 25577:tid 139759720564480] event.c(1707): connections: 0 (clogged: 0 write-completion: 0 keep-alive: 0 lingering: 0 suspended: 0) [Mon Jan 29 08:18:39.409132 2024] [mpm_event:trace6] [pid 25543:tid 139759720564480] event.c(1707): connections: 0 (clogged: 0 write-completion: 0 keep-alive: 0 lingering: 0 suspended: 0) [Mon Jan 29 08:18:39.720029 2024] [mpm_event:trace6] [pid 25883:tid 139759720564480] event.c(1707): connections: 0 (clogged: 0 write-completion: 0 keep-alive: 0 lingering: 0 suspended: 0) [Mon Jan 29 08:18:39.797372 2024] [mpm_event:trace6] [pid 25564:tid 139759720564480] event.c(1707): connections: 0 (clogged: 0 write-completion: 0 keep-alive: 0 lingering: 0 suspended: 0) [Mon Jan 29 08:18:40.016911 2024] [mpm_event:trace6] [pid 25887:tid 139759720564480] event.c(1707): connections: 0 (clogged: 0 write-completion: 0 keep-alive: 0 lingering: 0 suspended: 0) [Mon Jan 29 08:18:40.060063 2024] [mpm_event:trace6] [pid 25886:tid 139759720564480] event.c(1707): connections: 0 (clogged: 0 write-completion: 0 keep-alive: 0 lingering: 0 suspended: 0) [Mon Jan 29 08:18:40.194472 2024] [mpm_event:trace6] [pid 25539:tid 139759720564480] event.c(1707): connections: 0 (clogged: 0 write-completion: 0 keep-alive: 0 lingering: 0 suspended: 0) [Mon Jan 29 08:18:40.205143 2024] [mpm_event:trace5] [pid 29303:tid 139759872828544] event.c(2936): Not shutting down one child: active 131/132, total 132/132/132, idle threads 824, max workers 924 [Mon Jan 29 08:18:40.304130 2024] [mpm_event:trace6] [pid 25885:tid 139759720564480] event.c(1707): connections: 1 (clogged: 0 write-completion: 0 keep-alive: 0 lingering: 0 suspended: 0) [Mon Jan 29 08:18:40.565580 2024] [mpm_event:trace6] [pid 25567:tid 139759720564480] event.c(1707): connections: 0 (clogged: 0 write-completion: 0 keep-alive: 0 lingering: 0 suspended: 0) [Mon Jan 29 08:18:40.751005 2024] [mpm_event:trace6] [pid 25192:tid 139759720564480] event.c(1707): connections: 0 (clogged: 0 write-completion: 0 keep-alive: 0 lingering: 0 suspended: 0) [Mon Jan 29 08:18:40.865651 2024] [mpm_event:trace6] [pid 25564:tid 139759720564480] event.c(1707): connections: 0 (clogged: 0 write-completion: 0 keep-alive: 0 lingering: 0 suspended: 0) [Mon Jan 29 08:18:41.075927 2024] [mpm_event:trace6] [pid 25553:tid 139759720564480] event.c(1707): connections: 0 (clogged: 0 write-completion: 0 keep-alive: 0 lingering: 0 suspended: 0) [Mon Jan 29 08:18:41.206274 2024] [mpm_event:trace5] [pid 29303:tid 139759872828544] event.c(2936): Not shutting down one child: active 131/132, total 132/132/132, idle threads 825, max workers 924 -- You are receiving this mail because: You are the assignee for the bug. --------------------------------------------------------------------- To unsubscribe, e-mail: bugs-unsubscr...@httpd.apache.org For additional commands, e-mail: bugs-h...@httpd.apache.org