I have a large sized project that runs lots of WWW::Mechanize processes.  
usually 60-100 processes at a time, hitting the net and visiting sites.  

Weve been having a problem for some time now and done just about all the 
Os or hardware analysis we can, and we are unsure what to suspect next but 
obviously Mechanize (or even perl 5.8.0) are suspects.

Basically, after a variable amount of time that seems to exponentially 
decrease as the number of threads increases, the processes will hang in a 
'D' state of uninterruptible sleep.  those of you who know the linux 
kernel will know this is a really rare and really bad situation.  only a 
reboot can free them up.  

the kernel scheduler at a low level is awaiting the availability of a 
resource that obviously is not available.  

it rarely happens in certain nfs situations or cd-rom situations... if you 
do a web search for the problem that is.  

we can easily reproduce it once a day just by firing up a lot of copies of 
our code, which is heavily Mechanized/libwww.  

We have solaris boxes, we have redhat boxes, both succumb to it.  we now 
are testing freeBSD and openBSD boxes to see if they have it as well.  

we are running a new 2.6 kernel with the new improved scheduler which we 
had hoped would solve the problem but hasnt.  

Can anyone think of situations that, under load, Mechanize could generate 
requests to the kernel that would actually send it into this rebooting 
situation?  this is a condition that the kernel should prevent obviously, 
but isnt.  day by day the evidence is mounting that the problem is either 
in Mechanize or possibly perl... but we are running 5.6.0 on solaris and 
5.8.0 on redhat-2.6 so it seems unlikely its perl.

any ideas of what to look for??

-- 
gedanken

Reply via email to