I'm looking at your error and seeing that it is merely reporting back that the server is busy but does not give a definitive cause. While it could be memory, I think I'd be looking for CPU usage issues. What does the CPU usage look like during this event?
I ran into a similar problem a couple of years back when the company I worked for at the time went to virtual machines (albeit we were using 32 bit Windows and not Linux). A process was running that had nothing to do with Jrun or IIS/WWW that was sucking up the CPU all the way to 100% and there was no CPU left to process requests. I was getting the same 503 error that you get when you try to request a page from the server. Restarts only postponed the problem and our fix was simply to spin up a new virtual machine (a 64bit instance). We never could completely isolate the exact process that was eating up the CPU. Just a thought, hope you're able to find the problem quickly! From: ad...@acfug.org [mailto:ad...@acfug.org] On Behalf Of Ajas Mohammed Sent: Thursday, August 08, 2013 11:56 PM To: discussion@acfug.org Subject: Re: [ACFUG Discuss] Out of Memory?!? My first question will be do you have 1. FusionReactor or are you using CF built in monitoring. See what requests were running or are running when this happens. <Ajas Mohammed /> iUseDropbox(http://db.tt/63Lvone9) http://ajashadi.blogspot.com We cannot become what we need to be, remaining what we are. No matter what, find a way. Because thats what winners do. You can't improve what you don't measure. Quality is never an accident; it is always the result of high intention, sincere effort, intelligent direction and skillful execution; it represents the wise choice of many alternatives. On Thu, Aug 8, 2013 at 8:23 PM, Frank Moorman <stretch...@franksdomain.net<mailto:stretch...@franksdomain.net>> wrote: FYI... This is what the user gets on their end: Server Error The server encountered an internal error and was unable to complete your request. Application server is busy. Either there are too many concurrent requests or the server is still starting up. Also, I have not received any CF template errors at the times the 503 errors occur, nor any java.outofmemory errors etc. The server is running and available except for the one or two requests. Looking in the apache access log, There is no pattern to the pages that were requested, (but I know that I need to dig deeper and see what pages were requested right before the errors occur.) Thanks in advance for any help, Frank On 08/08/2013 07:41 PM, Frank Moorman wrote: All, I'm trying to figure out and determine a Jrun Out of Memory error. I get the following in my logs: [Thu Aug 08 14:40:14 2013] [notice] jrApache[2937: 31182] returning error page for JRun too busy or out of memory [Thu Aug 08 15:50:09 2013] [notice] jrApache[1787: 63699] returning error page for JRun too busy or out of memory It doesn't happen often, (maybe once or occasionally twice a business day) but as everyone understands, users aren't happy when it happens to them. This is a linux box, 64bit Centos 6, CF9 Enterprise, 64bit jvm version 1.7. (The jvm was installed separately from CF for security and coldfusion uses it.) I doubt it is actually an out of memory condition (though I could be wrong) The server has 6GB of physical memory and another 6GB of swap. It rarely needs to use swap. (i.e. I have not observed it.) The jvm is given significant memory to use as well. It is using a 64bit jvm with the settings of 1GB min JVM heap, as well as a 3GB max. When I look through the server monitor, it is normal to see 1 to 1.5GB allocated and between 100-750MB used. (I see a normal sawtooth pattern with the memory usage, so it looks like what I would expect from the garbage collection routing. It does spike occasionally but I have never seen it close to the 3GB max. (I've never even seen it hit 2GB used.) The server is set for 40 template requests (I recently upped it from 10 to see if that was the problem and it still occurred with the same frequency.) Flash remoting is set to 2, webservice 1, CFC 1. (These remote settings are only set for the monitor, as the server does not provide any webservices outside the running application) Jrun is set to 50 requests, and 1000 queued. (Enough to cover the CF requests.) I looked at Charlie's blog... I have checked the logs, and other than the apache error log (above) I do not see anything. I've check the system /var/log/messages, I've checked all the CF logs (I also archived everything yesterday, and the cf logs are practically empty even after today's occurrence.) I did not find any jvm abort logs that Charlie mentioned in his blog. (I checked in the CF directory mentioned as well as the system logs and the actual JVM directory) I also checked the Jrun log (in /opt/jrun4/logs/cfusion-event.log ) and was surprised because the only entries were months ago. (Because of the age of the log, I'm curious if I am looking at the right place for it.) Does anyone have any ideas on what might be happening? or something else that I should check? I have searched the web and found different ideas (even the rare "add more memory") Another mentions the requests being overloaded, but I honestly do not believe that the 10 simultaneous template requests was low for the traffic for this site. After quadrupling it, with the problem still occurring, it is even less likely. I've seen some mentioning client variable storage, but the server is set to use cookies for that, not a database. While I do not use client storage, I know there are items like the last time visited etc, so I may just turn it off completely. Another one I found interested mentions a bug with MySql drivers with the "Maintain Connections" setting and suggested to uncheck this box. I search for this and found the bug mentioned, one site even speculated it was still a problem with CF9, but I could not find any details. Does anyone know of this issue, I've seen it mentioned, but a lack of any details other than its bad to have that checked. (The page that mentioned it did say it ate memory.) I'd love more ideas, I know these are not an easy or straight forward error. I may try removing the client storage next, but other ideas are welcome. (i.e. I'm not very convinced that the other things I found on the web will be effective.) Thanks, Frank ------------------------------------------------------------- To unsubscribe from this list, manage your profile @ http://www.acfug.org?fa=login.edituserform For more info, see http://www.acfug.org/mailinglists Archive @ http://www.mail-archive.com/discussion%40acfug.org/ List hosted by FusionLink<http://www.fusionlink.com> ------------------------------------------------------------- ------------------------------------------------------------- To unsubscribe from this list, manage your profile @ http://www.acfug.org?fa=login.edituserform For more info, see http://www.acfug.org/mailinglists Archive @ http://www.mail-archive.com/discussion%40acfug.org/ List hosted by FusionLink<http://www.fusionlink.com> ------------------------------------------------------------- ________________________________ No virus found in this message. Checked by AVG - www.avg.com<http://www.avg.com> Version: 2013.0.3392 / Virus Database: 3209/6559 - Release Date: 08/08/13 ------------------------------------------------------------- To unsubscribe from this list, manage your profile @ http://www.acfug.org?fa=login.edituserform For more info, see http://www.acfug.org/mailinglists Archive @ http://www.mail-archive.com/discussion%40acfug.org/ List hosted by http://www.fusionlink.com -------------------------------------------------------------
