But what do you mean by the stack trace showing that fine so where is it
caught up and not resolving?
A stack trace in and of itself isn't valuable. You need to see it over time.
You also said, I also monitored several of the long running threads (around
2 minutes) and capture several stack traces. They were all the same.
So do you really mean they all (what? All request running? And over several
thread dumps) showed being stuck with at
java.util.HashMap.get(HashMap.java:325) being the first line in the stack
trace? If not, then you've not really got anything to conclude from the
stack traces.
Sorry if it sounds like I'm being contentious. I just don't know what you
do or don't understand about all this, so am going only on your words.
As for the sudden burden of processing at startup, that could indeed be from
a bunch of requests all coming in at once. Perhaps you could queue them up
in such a case. There's no built-in singleton processing in CF, but you
could use CFLOCK surrounding some troublesome code that you permit to only
run one at a time, but your problem is that you only want to do it at
startup, or when things are going bad, so you need to use something other
than CFLOCK. You could code something in CFML, though, that acts like a
singleton (only let one request at a time run this during a period you
detect as a trouble time). Just thinking off the top of my head here.
But I'll share one other thought: others who experience slow startup times
often find that it's because that flush of new requests is triggering a lot
of java class loading, and there's a known issue in the JVM 1.6 version
implemented with CF. Some moved back to 1.5 to solve it, while others
updated to 1.6.10 (which fixed the problem) to make it go away. Have you
tried either?
There are various blogs about this. The first here explains the problem and
how to do it:
http://www.ghidinelli.com/2008/08/18/java16-u10-gives-big-boost-to-modelglue
-transfer-coldspring-performance
Others with more other detail are:
http://www.compoundtheory.com/?action=displayPostID=270
http://corfield.org/blog/index.cfm/do/blog.entry/entry/Java_6_and_ColdFusion
_8
Hope that's helpful.
/charlie
-Original Message-
From: cfaussie@googlegroups.com [mailto:cfaus...@googlegroups.com] On Behalf
Of Matthew
Sent: Wednesday, January 14, 2009 1:30 AM
To: cfaussie
Subject: [cfaussie] Re: JRUN hanging
Hi guys
We've just a server hang. It was a little different to usual
(perhaps because Fusion Reactor is installed). This time the website
was showing the JRun error message. When I looked at the JRun process
the RAM was up at 615MB!!! I restarted JRun.exe. As the website came
back up it started to chock again. There is obviously extra load as
all the objects, web services etc run for the first time. I know that
with the 3rd party web service which I've spoken of in previous posts
that always the first call takes a good couple of minutes.
Unfortunately when it happens on the live server you usually get at
least half a dozen users trying to hit the page which consumes this
web service so all requests take a long time and chew up a lot of
resources. I was watching things through FR and the Task Manager: the
CPU was up near 100% plus the RAM was quite high (it normally sits
around 200Mb but was 350+). Meanwhile I couldn't even hit a very light
weight page on the site. I used the KILL feature in FR to terminate
about 20 threads and eventually the CPU usage and RAM dropped down to
stable levels.
I also monitored several of the long running threads (around 2
minutes) and capture several stack traces. They were all the same.
Perhaps someone will be able to explain it to me so I can understand
what I'm reading. For example I read the below that the CFINVOKE was
fine so where is it caught up and not resolving?
at java.util.HashMap.get(HashMap.java:325)
at org.apache.axis.utils.JavaUtils.isEnumClass(JavaUtils.java:1040)
at org.apache.axis.encoding.ser.BeanSerializerFactory.init
(BeanSerializerFactory.java:49)
at org.apache.axis.encoding.ser.BeanSerializerFactory.
(BeanSerializerFactory.java:42)
at org.apache.axis.encoding.ser.BaseSerializerFactory.createFactory
(BaseSerializerFactory.java:235)
at org.apache.axis.client.Call.registerTypeMapping(Call.java:2296)
at
com.raileurope.web.ws.soap.RailEuropeWebServiceSoapBindingStub.createCall
(RailEuropeWebServiceSoapBindingStub.java:2454)
- locked 0x1534d630 (a
com.raileurope.web.ws.soap.RailEuropeWebServiceSoapBindingStub)
at
com.raileurope.web.ws.soap.RailEuropeWebServiceSoapBindingStub.doBuildPackag
eForCityPair
(RailEuropeWebServiceSoapBindingStub.java:2663)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke
(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke
(DelegatingMethodAccessorImpl.java:25)
at
