Re: mod_mbox spinning processes on ajax
* Joshua Slive [EMAIL PROTECTED] [2005-10-21 15:01:04]: By the way, if someone has a chance to update mod_mbox on ajax, that would be great. We just hit load average of 100 before I bounced httpd to clear out all the never-ending mod_mbox requests. Sorry, I don't have neither the level of expertise on the ASF Infrastructure nor the access rights for this operation. Paul ? Justin ? Regards, - Sam -- Maxime Petazzoni (http://www.bulix.org) -- gone crazy, back soon. leave message. signature.asc Description: Digital signature
Re: mod_mbox spinning processes on ajax
On Oct 21, 2005, at 12:50 PM, Maxime Petazzoni wrote: * Joshua Slive [EMAIL PROTECTED] [2005-10-21 15:01:04]: By the way, if someone has a chance to update mod_mbox on ajax, that would be great. We just hit load average of 100 before I bounced httpd to clear out all the never-ending mod_mbox requests. Sorry, I don't have neither the level of expertise on the ASF Infrastructure nor the access rights for this operation. Ajax is now running revision 327633 of mod_mbox. S. smime.p7s Description: S/MIME cryptographic signature
Re: mod_mbox spinning processes on ajax
Maxime Petazzoni wrote: There appear to be some httpd processes spinning on cpu on ajax. Attaching with gdb and asking for a backtrace gives me something like this: This email is 457 kbytes long, maybe it just take some time to iterate through 450k+ characters. Maybe we could find a better algorithm ? It may well be that the problem is just a poorly performing algorithm. But it should be a priority to be fixed. There are over 30 httpd processes on cpu on ajax at the moment, some having used several hundred minutes of cpu time (and mostly in the range of 25-50MB of RAM). And every one that I try is working on mbox_wrap_text, usually on an svn-commit notice email. This is probably triggered by a robot, but regardless, it needs to be fixed. I've dropped a few more core files in ajax:/raid1/httpd-cores. Joshua.
Re: mod_mbox spinning processes on ajax
Hi, * Joshua Slive [EMAIL PROTECTED] [2005-10-20 16:20:58]: It may well be that the problem is just a poorly performing algorithm. But it should be a priority to be fixed. As you can see in the latest commit (r327019), we've set up a small hack to avoid mod_mbox from wrapping messages larger than 50k. Thanks for the reports, - Sam -- Maxime Petazzoni (http://www.bulix.org) -- gone crazy, back soon. leave message. signature.asc Description: Digital signature
Re: mod_mbox spinning processes on ajax
--On October 21, 2005 12:49:48 AM +0200 Maxime Petazzoni [EMAIL PROTECTED] wrote: As you can see in the latest commit (r327019), we've set up a small hack to avoid mod_mbox from wrapping messages larger than 50k. I don't get it. Why is mod_mbox wrapping any text to begin with? This is the MUA's responsibility not ours. (Are we working around dumb MUA's that don't wrap?) Also, the strlen call in mbox_wrap_text's for loop is problematic. The string's length isn't changing. strlen on a 450,000-length array 450,000 times sounds like a problem. I'd also think that we'd be able to compute the length when get the MIME-ified body instead of relying on strlen again. (We have the escape'd length in mbox_mime_get_body; that'd greatly simplify the get_body call to eliminate *any* strlen calls.) -- justin
Re: mod_mbox spinning processes on ajax
Justin Erenkrantz wrote: --On October 21, 2005 12:49:48 AM +0200 Maxime Petazzoni [EMAIL PROTECTED] wrote: As you can see in the latest commit (r327019), we've set up a small hack to avoid mod_mbox from wrapping messages larger than 50k. I don't get it. Why is mod_mbox wrapping any text to begin with? This is the MUA's responsibility not ours. (Are we working around dumb MUA's that don't wrap?) We are the mua in the case of mod_mbox. If you mean the sender is responsible, it isn't really true any more in the era of format=flowed. Web mail-archives that don't wrap can be very irritating. Joshua.
Re: mod_mbox spinning processes on ajax
* Justin Erenkrantz [EMAIL PROTECTED] [2005-10-20 16:01:27]: --On October 21, 2005 12:49:48 AM +0200 Maxime Petazzoni [EMAIL PROTECTED] wrote: As you can see in the latest commit (r327019), we've set up a small hack to avoid mod_mbox from wrapping messages larger than 50k. I don't get it. Why is mod_mbox wrapping any text to begin with? This is the MUA's responsibility not ours. (Are we working around dumb MUA's that don't wrap?) Indeed we are. Also, the strlen call in mbox_wrap_text's for loop is problematic. The string's length isn't changing. strlen on a 450,000-length array 450,000 times sounds like a problem. I'd also think that we'd be able to compute the length when get the MIME-ified body instead of relying on strlen again. (We have the escape'd length in mbox_mime_get_body; that'd greatly simplify the get_body call to eliminate *any* strlen calls.) -- justin I've just commited a small changeset in order to compute the string length only once in the function. But yes, we could rely on the computed length earlier in the body processing. - Sam -- Maxime Petazzoni (http://www.bulix.org) -- gone crazy, back soon. leave message. signature.asc Description: Digital signature
Re: mod_mbox spinning processes on ajax
--On October 21, 2005 1:09:28 AM +0200 Maxime Petazzoni [EMAIL PROTECTED] wrote: I've just commited a small changeset in order to compute the string length only once in the function. But yes, we could rely on the computed length earlier in the body processing. I wonder if fixing the loop would mean that we could remove the 50k limitation. It should only have to iterate 450,000*2 times which isn't *that* expensive. =) -- justin
Re: mod_mbox spinning processes on ajax
* Justin Erenkrantz [EMAIL PROTECTED] [2005-10-20 16:16:56]: I've just commited a small changeset in order to compute the string length only once in the function. But yes, we could rely on the computed length earlier in the body processing. I wonder if fixing the loop would mean that we could remove the 50k limitation. It should only have to iterate 450,000*2 times which isn't *that* expensive. =) -- justin I had the same idea, but I must go to bed now. May you try if you have some spare time ? I'll take a look at it tomorrow if you did not. - Sam -- Maxime Petazzoni (http://www.bulix.org) -- gone crazy, back soon. leave message. signature.asc Description: Digital signature
Re: mod_mbox spinning processes on ajax
Maxime Petazzoni wrote: Salut, * Joshua Slive [EMAIL PROTECTED] [2005-10-16 15:54:18]: There appear to be some httpd processes spinning on cpu on ajax. Attaching with gdb and asking for a backtrace gives me something like this: [snip] Could you send us the URL that raised the problem ? We don't see anything that could be wrong in the mbox_wrap_text code :( This URL seems to give the problem: http://mail-archives.apache.org/mod_mbox/xmlgraphics-fop-commits/200012.mbox/[EMAIL PROTECTED] There are no spinning httpds on ajax at the moment -- they seem to come and go. I suspect that they are not permanently spinning, but rather are dying whenever the client gives up on waiting. Connecting to the above URL gives me instant 100% CPU usage on the server that lasts until I hit stop on the browser. I get no content from the server. I would debug further, but that would involve actually looking at the code, which I have so far managed to avoid ;-) I forced a core dump and put it in /raid1/httpd-cores/core.31834 on ajax. Here's the basics: #1 0x21007f30 in mbox_wrap_text ( str=0x2147c038 fotis 00/12/12 11:18:52\n\n Modified:.build.xml\n, ' ' repeats 15 times, docs/examples build.xml\n, ' ' repeats 15 times, docs/xml-docs fop.xml xml2xml.xsl\n, ' ' repeats 15 times, lib Fop.class Fop.java read...) at mod_mbox.c:155 #2 0x21012ff0 in mbox_static_message (r=0x60230ca0, f=0x60235f70) at mod_mbox_out.c:1158 #3 0x2100a900 in mbox_file_handler (r=0x60230ca0) at mod_mbox_file.c:320 #4 0x40035f90 in ap_run_handler (r=0x60230ca0) at config.c:153 #5 0x40036f70 in ap_invoke_handler (r=0x60230ca0) Joshua.
Re: mod_mbox spinning processes on ajax
Maxime Petazzoni wrote: Salut, * Joshua Slive [EMAIL PROTECTED] [2005-10-16 15:54:18]: There appear to be some httpd processes spinning on cpu on ajax. Attaching with gdb and asking for a backtrace gives me something like this: [snip] Could you send us the URL that raised the problem ? We don't see anything that could be wrong in the mbox_wrap_text code :( Sorry, I had to bounce the server to fix my mod_rewrite error (to get the raw format working again) and that killed all the spinning processes. None are currently spinning. We can keep an eye over the next few days to see if they reappear. Joshua.
