On 10/12/21 4:58 PM, Yann Ylavic wrote:
> On Mon, Oct 11, 2021 at 2:32 PM <bugzi...@apache.org> wrote:
>>
>> https://bz.apache.org/bugzilla/show_bug.cgi?id=65626
>>
>> Bug ID: 65626
>> Summary: MPM Event doesn't shutdown idle children after working
>> under high load
>> Product: Apache httpd-2
>> Version: 2.4.37
>> Hardware: PC
>> Status: NEW
>> Severity: normal
>> Priority: P2
>> Component: mpm_event
>> Assignee: b...@httpd.apache.org
>> Reporter: gregvoro...@gmail.com
>> Target Milestone: ---
>>
>> After running ab with concurrency equal or more than MaxRequestWorkers for
>> some
>> time, apache doesn't kill idle processes any longer. Following is logged
>> every
>> second:
>>
>> [Mon Oct 11 15:17:29.416964 2021] [mpm_event:trace5] [pid 71:tid
>> 140381265582400] event.c(2834): Not shutting down child: total daemons 16 /
>> active limit 16 / ServerLimit 16
>
> At https://github.com/apache/httpd/blob/trunk/server/mpm/event/event.c#L3181
> we handle MaxSpareThreads like this:
>
> if (idle_thread_count > max_spare_threads / num_buckets)
> {
> /*
> * Child processes that we ask to shut down won't die immediately
> * but may stay around for a long time when they finish their
> * requests. If the server load changes many times, many such
> * gracefully finishing processes may accumulate, filling up the
> * scoreboard. To avoid running out of scoreboard entries, we
> * don't shut down more processes when the total number of processes
> * is high.
> * [...]
> */
> if (retained->total_daemons <= active_daemons_limit &&
> retained->total_daemons < server_limit) {
> /* Kill off one child */
> ap_mpm_podx_signal(retained->buckets[child_bucket].pod,
> AP_MPM_PODX_GRACEFUL);
> retained->idle_spawn_rate[child_bucket] = 1;
> } else {
> ap_log_error(APLOG_MARK, APLOG_TRACE5, 0, ap_server_conf,
> "Not shutting down child: total daemons %d / "
> [...]);
> }
> }
>
> with active_daemons_limit = MaxRequestWorkers / ThreadsPerChild (thus
> a constant), and retained->total_daemons which is
> incremented/decremented when a child forks/exits.
> So it seems that besides MaxRequestsPerChild, there is nothing to
> recover after active_daemons_limit was reached at one point in time.
Good catch, I think we need to have a conditions where we start shuting down
again.
>
> Shouldn't we also compute a busy_thread_count and kill children
> if/when later things settle down and thus busy_thread_count <
> max_spare_threads (like in the attached patch)?
Hm. Wouldn't it be better instead to ensure that idle_thread_count is "far"
larger than the min_spare_threads something like.
idle_thread_count > active_daemons_limit/2/num_buckets*ThreadsPerChild +
min_spare_threads
(2 is arbitrary choice could be also larger to start earlier reducing daemons)
This would prevent the situation that we have a dying daemon and a rise of
requests would cause the start of a new daemon "soon".
?
Regards
Rüdiger
