Hi again all. Hmm.. this is embarassing.. the problem has been found (and it was indeed the PHP-app).
The problem was, that I tested on a full copy of the live installation and the 404 handler and the index file had the error in common (and when going to "//" on the server one hits the index-file, which I had overlook). Anyways thanks for a couple of great debugging tips - and sorry for the noise. Svenne On 24-10-2011 18:02, Svenne Krap wrote: > Hi. > > I am currently investigating a misbehaving apache httpd 2.2.21 with apr > 1.4.5 on linux 64-bit. > > I have a virtual domain defined as (it runs behind a 1-on-1 nat, hence > the rfc 1918 address) : > > <VirtualHost 192.168.3.2:80> > ServerName sub.domain.tld > ServerAdmin [email protected] > ServerAlias www.sub.domain.tld > ServerPath /var/www/customer/sub.domain.tld > DocumentRoot /var/www/customer/sub.domain.tld/html > ErrorLog /var/www/customer/sub.domain.tld/log/error_log > CustomLog /var/www/customer/sub.domain.tld/log/access_log combined > php_value include_path /var/www/customer/sub.domain.tld/inc:. > </VirtualHost>h > > > <Directory "/var/www/customer/sub.domain.tld/html"> > Options Indexes FollowSymLinks MultiViews > AllowOverride All > Order allow,deny > Allow from all > </Directory> > > > That domain runs a custom php-application (originally believed to be the > culprint) on php 5.3.8. > > Inside the document root there is a .htaccess file with the following > one line of content: > > ErrorDocument 404 /404.php > > During test I found out, that even a plain html-file has the same > problem (i.e. ErrorDocument 404 /404.html ), so currently I think I am > ruling out PHP even tough it is easier to hit with a php script (or > maybe there are two different issues). > > To the problem: > > ** if anyone visits http://sub.domain.tld// (note double slash at end) > the server goes 100% cpu and seemingly never recovers until a apachectl > restart has been issued. ** > > If it is php the doubleslash only has to appear anywhere in the URI > (with a plain html it seems it needs to be in front). > > There is absolutely nothing in the log files (neither for the domain > itself nor the overall apache ones in /var/log/ nor the general syslogs) > > I have tried to triage it (including googling it) for days, but the only > thing I find are some messages from 2004 that seemingly describes a > different problem (and of course thousands of messages of how to get rid > of double slashes from a badly written CMS). > > My big problem is even bigger as seemingly some crawler bots for search > engines starts their requests with a double slash.. and as they are > firing of multiple request at once, they practically kills the server. > > I hope someone has some ideas, and I will be willing to run any > reasonable test (it is a production server tough) > > Thanks in advance > > Svenne
