FYI...
This is what the user gets on their end:
Server Error
The server encountered an internal error and was unable to complete your
request.
Application server is busy. Either there are too many concurrent
requests or the server is still starting up.
Also, I have not received any CF template errors at the times the 503
errors occur, nor any java.outofmemory errors etc. The server is running
and available except for the one or two requests. Looking in the apache
access log, There is no pattern to the pages that were requested, (but I
know that I need to dig deeper and see what pages were requested right
before the errors occur.)
Thanks in advance for any help,
Frank
On 08/08/2013 07:41 PM, Frank Moorman wrote:
All,
I'm trying to figure out and determine a Jrun Out of Memory error. I
get the following in my logs:
[Thu Aug 08 14:40:14 2013] [notice] jrApache[2937: 31182] returning error
page for JRun too busy or out of memory
[Thu Aug 08 15:50:09 2013] [notice] jrApache[1787: 63699] returning error
page for JRun too busy or out of memory
It doesn't happen often, (maybe once or occasionally twice a business
day) but as everyone understands, users aren't happy when it happens
to them.
This is a linux box, 64bit Centos 6, CF9 Enterprise, 64bit jvm version
1.7. (The jvm was installed separately from CF for security and
coldfusion uses it.)
I doubt it is actually an out of memory condition (though I could be
wrong) The server has 6GB of physical memory and another 6GB of swap.
It rarely needs to use swap. (i.e. I have not observed it.)
The jvm is given significant memory to use as well. It is using a
64bit jvm with the settings of 1GB min JVM heap, as well as a 3GB max.
When I look through the server monitor, it is normal to see 1 to 1.5GB
allocated and between 100-750MB used. (I see a normal sawtooth pattern
with the memory usage, so it looks like what I would expect from the
garbage collection routing. It does spike occasionally but I have
never seen it close to the 3GB max. (I've never even seen it hit 2GB
used.)
The server is set for 40 template requests (I recently upped it from
10 to see if that was the problem and it still occurred with the same
frequency.)
Flash remoting is set to 2, webservice 1, CFC 1. (These remote
settings are only set for the monitor, as the server does not provide
any webservices outside the running application) Jrun is set to 50
requests, and 1000 queued. (Enough to cover the CF requests.)
I looked at Charlie's blog... I have checked the logs, and other than
the apache error log (above) I do not see anything. I've check the
system /var/log/messages, I've checked all the CF logs (I also
archived everything yesterday, and the cf logs are practically empty
even after today's occurrence.) I did not find any jvm abort logs that
Charlie mentioned in his blog. (I checked in the CF directory
mentioned as well as the system logs and the actual JVM directory) I
also checked the Jrun log (in /opt/jrun4/logs/cfusion-event.log ) and
was surprised because the only entries were months ago. (Because of
the age of the log, I'm curious if I am looking at the right place for
it.)
Does anyone have any ideas on what might be happening? or something
else that I should check?
I have searched the web and found different ideas (even the rare "add
more memory")
Another mentions the requests being overloaded, but I honestly do not
believe that the 10 simultaneous template requests was low for the
traffic for this site. After quadrupling it, with the problem still
occurring, it is even less likely.
I've seen some mentioning client variable storage, but the server is
set to use cookies for that, not a database. While I do not use client
storage, I know there are items like the last time visited etc, so I
may just turn it off completely.
Another one I found interested mentions a bug with MySql drivers with
the "Maintain Connections" setting and suggested to uncheck this box.
I search for this and found the bug mentioned, one site even
speculated it was still a problem with CF9, but I could not find any
details. Does anyone know of this issue, I've seen it mentioned, but a
lack of any details other than its bad to have that checked. (The page
that mentioned it did say it ate memory.)
I'd love more ideas, I know these are not an easy or straight forward
error. I may try removing the client storage next, but other ideas are
welcome. (i.e. I'm not very convinced that the other things I found on
the web will be effective.)
Thanks,
Frank
-------------------------------------------------------------
To unsubscribe from this list, manage your profile @
http://www.acfug.org?fa=login.edituserform
For more info, see http://www.acfug.org/mailinglists
Archive @ http://www.mail-archive.com/discussion%40acfug.org/
List hosted by FusionLink <http://www.fusionlink.com>
-------------------------------------------------------------
-------------------------------------------------------------
To unsubscribe from this list, manage your profile @
http://www.acfug.org?fa=login.edituserform
For more info, see http://www.acfug.org/mailinglists
Archive @ http://www.mail-archive.com/discussion%40acfug.org/
List hosted by http://www.fusionlink.com
-------------------------------------------------------------
