Hello,

On apache 2.2.22, 2.2.23 and 2.4.4 I'm able to reproduce a problem
where graceful restart takes very long time. Linux 3.7.10, glibc 2.17 here.

keepalive is off, ServerLimit  600; StartServers  64; MinSpareServers 32;
MaxSpareServers 64; MaxClients 600; MaxRequestsPerChild  2000;
minimal number of apache modules

I'm running ab where 192.168.1.121 is my local ip:
/usr/bin/ab -n 100000 -c 64 http://192.168.1.121/
(ususally I'm interrupting ab in first 1-2 seconds of its run, starting it 
again and leaving running)

and then doing graceful restart while ab is still running:
[Thu Apr 11 13:27:32.624875 2013] [mpm_prefork:notice] [pid 10202] AH00171: 
Graceful restart requested, doing restart
[Thu Apr 11 13:28:15.836068 2013] [mpm_prefork:notice] [pid 10202] AH00163: 
Apache/2.4.4 (Unix) configured -- resuming normal operations

It takes almost 1 minute to resume operations. Now if I don't do
ab "magic" then resume is immediate (1-2s).

The whole time is spend in prefork ap_mpm_pod_killpg

        ap_log_error(APLOG_MARK, APLOG_NOTICE, 0, ap_server_conf, APLOGNO(00171)
                    "Graceful restart requested, doing restart");

        /* kill off the idle ones */
        ap_mpm_pod_killpg(pod, retained->max_daemons_limit);

where ap_mpm_pod_killpg iterates over childrens (?) and that takes ~1 minute.

I also did testing with StartServers 128. ap_mpm_pod_killpg had to iterate
over 128 childrens where 72 of them were done in the same second while
the rest in over minute - log here http://pastebin.com/3tP5hW1f.

Example strace of main httpd process while doing graceful restart:
http://pastebin.com/QFH5TjT6

The whole problem is that until ap_mpm_pod_killpg finishes apache
is not responding to new requests and the downtime is between 30s
to even 4 minutes.

-- 
Arkadiusz Miśkiewicz, arekm / maven.pl

Reply via email to