Package: apache2-mpm-prefork Version: 2.2.8-3 Severity: normal
I've been coming across issues lately with Apache child threads hanging in a wait state (and not serving connections) indefinitely with no recovery. Now, it may be php related or something, I'm not ruling that out. But at present I'm unable to debug it to the degree of finding the source (after much searching and doing things like stracing bad threads). What happens is the server is running fine in general, but certain clients seem to be able to cause a thread to lock up while serving a request. The majority of the offenders seem to be while serving jpg thumbnail images out of a gallery, although I have found other examples as well where threads do not recover. The bad cases are the same few files requested by one client, typically very rapidly in succession and usually from country TLDs like .pl or .ar (which makes me believe it's an intentional DoS, although I can't find a description of the attack anywhere). The following is an strace when the server was recieving these requests on a thread being accessed by the IP causing the lock ups: http://underhanded.org/13098.out The requests in that thread referencing URLs with "thumbnail" in them are the ones in question. I was unable to derive anything useful from that particular trace, but it's there if you should need it. Outside of the apparently deliberate attempts to lock up (or maybe just crawl) the server, each of these threads shares the common attribute of being flagged as (W)ait status in apache itself, lasting until the thread is killed (20 hours + on some). Here's an example of a grep of long wait status threads and what they were doing: [EMAIL PROTECTED]:/etc# /usr/sbin/apache2ctl fullstatus | /bin/grep -A 1 -E '[0-9]+/[0-9]+/[0-9]+[[:space:]]+W[[:space:]]+[0-9]+\.[0-9]+[[:space:]]+[0-9]{2,}' 77-0 6850 0/130/30864 W 1.19 24970 0 0.0 0.84 205.13 195.114.160.136 blond.website.com GET / HTTP/1.1 -- 111-0 12167 0/13/601 W 0.00 73426 0 0.0 0.04 3.51 91.106.219.88 blond.website.com GET -- 136-0 6870 0/146/26295 W 1.14 24810 0 0.0 1.06 175.10 195.114.160.136 blond.website.com GET / HTTP/1.1 -- 141-0 9486 0/0/24479 W 0.20 24970 0 0.0 0.00 156.45 195.114.160.136 blond.website.com GET /articles/category HTTP/1.1 -- 164-0 9496 0/26/21884 W 0.22 24804 0 0.0 0.07 141.98 195.114.160.136 blond.website.com GET / HTTP/1.1 -- 332-0 9607 0/19/5332 W 0.33 24809 0 0.0 0.19 30.86 195.114.160.136 blond.website.com GET /articles/category HTTP/1.1 -- 376-0 13058 0/186/655 W 0.75 72737 0 0.0 1.22 3.95 128.180.192.139 blond.website.com GET / HTTP/1.1 -- 405-0 7313 0/169/831 W 0.85 25003 0 0.0 0.69 4.55 195.114.160.136 blond.website.com GET /articles/category HTTP/1.1 -- 424-0 8346 0/78/1624 W 0.28 24939 0 0.0 0.30 10.43 195.114.160.136 blond.website.com GET / HTTP/1.1 -- 480-0 7355 1/134/175 W 0.75 24933 0 41.7 1.24 1.49 195.114.160.136 blond.website.com GET /articles/category HTTP/1.1 Any suggestions as to rectifying this would be GREATLY appreciated, as I am at my wits' end trying to track down what exactly is going on. Currently I am writing a script to take teh results frmo the above grep and kill off the processes. If you need any additional information or need me to test anything to help resolve this or reproduce, please do not hesitate to let me know. Here some relevant parts of the apache config (this happens with KeepAlives off as well): ------ HostnameLookups Off Timeout 5 KeepAlive On MaxKeepAliveRequests 1000 KeepAliveTimeout 4 <IfModule mpm_prefork_module> ServerLimit 700 StartServers 100 MinSpareServers 5 MaxSpareServers 10 MaxClients 700 MaxRequestsPerChild 10000 GracefulShutdownTimeout 15 </IfModule> ----- -- Package-specific info: List of enabled modules from 'apache2 -M': actions* alias auth_basic auth_mysql authn_alias authn_file authnz_ldap authz_default authz_groupfile authz_host authz_user autoindex cgi dav_fs dav_lock dav deflate dir env include ldap mime negotiation php5 rewrite setenvif status suexec unique_id vhost_alias (A * means that the .conf file for that module is not enabled in /etc/apache2/mods-enabled/) -- System Information: Debian Release: lenny/sid APT prefers unstable APT policy: (500, 'unstable') Architecture: i386 (i686) Kernel: Linux 2.6.22-1-686 (SMP w/4 CPU cores) Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/bash Versions of packages apache2-mpm-prefork depends on: ii apache2.2-common 2.2.8-3 Next generation, scalable, extenda ii libapr1 1.2.12-1 The Apache Portable Runtime Librar ii libaprutil1 1.2.12+dfsg-3 The Apache Portable Runtime Utilit ii libc6 2.7-10 GNU C Library: Shared libraries ii libpcre3 7.6-2 Perl 5 Compatible Regular Expressi apache2-mpm-prefork recommends no packages. -- no debconf information -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]