*We are having an issue with the tomcat service crashing version 4.1.31,
sometimes with these memory errors and sometimes not. We have a backup but
once the load moves to that server the backup crashes also almost
immediately after the load balancer switches. This has happened multiple
times to us over the past few days with no changes made to the server, just
a slight increase in load. We have no option to upgrade right now since
this is included in an enterprise application. We have the heap space set
to the maximum that it will allow us to. The vendor recommended 1536 but
this setting caused tomcat to not start at all, ~1200 was the highest I was
able to get it to go. I watched it crash yesterday and it did a clean
shutdown with no errors which was differrent. The vendor doesn't have a fix
yet and I wanted to see what you all think. There are other large logs, if
there is something specific that we would need to look at then let me know
and I will try and get it out.
Thanks,
Sean
Dec 5, 2007 4:11:05 PM - Error creating event for domain ss-ls01p:
java.lang.OutOfMemoryError: Java heap space
at java.util.ArrayList.<init>(ArrayList.java:113)
at java.util.ArrayList.<init>(ArrayList.java:120)
at
com.aprisma.spectrum.app.event.web.model.BackEndEventDataModel.makeEventObject
(BackEndEventDataModel.java:566)
at
com.aprisma.spectrum.app.event.web.model.BackEndEventDataModel.applyEventUpdate
(BackEndEventDataModel.java:748)
at
com.aprisma.spectrum.app.event.web.model.BackEndEventDataModel$QueuedEventUpdate.run
(BackEndEventDataModel.java:116)
at com.aprisma.util.thread.JobQueue.runJobThread(JobQueue.java:232)
at com.aprisma.util.thread.JobQueue.access$000(JobQueue.java:38)
at com.aprisma.util.thread.JobQueue$JobRunnable.run(JobQueue.java:47)
at java.lang.Thread.run(Thread.java:595)
java.lang.OutOfMemoryError: Java heap space
at com.aprisma.spectrum.core.idl.CsCAttribute.CsCAttrValueHelper.read(
CsCAttrValueHelper.java:53)
at com.aprisma.spectrum.core.idl.CsCAttribute.CsCAttrValListHelper.read(
CsCAttrValListHelper.java:56)
at
com.aprisma.spectrum.core.idl.CsCEventDomainPackage.CsCEventHelper.read(
CsCEventHelper.java:64)
at
com.aprisma.spectrum.core.idl.CsCEventDomainPackage.CsCEventListHelper.read(
CsCEventListHelper.java:63)
at com.aprisma.spectrum.core.idl.CsCEventWatchCBPOA._invoke(
CsCEventWatchCBPOA.java:66)
at com.aprisma.spectrum.core.idl.CsCEventWatchCBPOA._invoke(
CsCEventWatchCBPOA.java:51)
at com.inprise.vbroker.poa.POAImpl.invoke(POAImpl.java:2822)
at com.inprise.vbroker.poa.ActivationRecord.invoke(ActivationRecord.java
:186)
at com.inprise.vbroker.GIOP.GiopProtocolAdapter.doRequest(
GiopProtocolAdapter.java:838)
at com.inprise.vbroker.GIOP.GiopProtocolAdapter.dispatchMessage(
GiopProtocolAdapter.java:1120)
at com.inprise.vbroker.orb.TPDispatcherImpl$TPDispatcher.run(
TPDispatcherImpl.java:100)
at com.inprise.vbroker.orb.ThreadPool$PoolWorker.run(ThreadPool.java:76)
java.lang.OutOfMemoryError: Java heap space
Dec 5, 2007 4:12:21 PM - EventFormatHelper: Error parsing event table enum
file nmiEventType: java.lang.OutOfMemoryError: Java heap space
java.lang.OutOfMemoryError: Java heap space
java.lang.OutOfMemoryError: Java heap space
Exception in thread "HostConfig[localhost]" java.lang.OutOfMemoryError: Java
heap space
java.lang.OutOfMemoryError: Java heap space
Exception in thread "CORBAMonitorPool-243: IDLE" java.lang.OutOfMemoryError:
Java heap space
java.lang.OutOfMemoryError: Java heap space
Exception in thread "VBJ ThreadPool Worker" java.lang.OutOfMemoryError: Java
heap space
java.lang.OutOfMemoryError: Java heap space
Exception in thread "VBJ ThreadPool Worker" java.lang.OutOfMemoryError: Java
heap space
java.lang.OutOfMemoryError: Java heap space
Exception in thread "StandardManager[/examples]" java.lang.OutOfMemoryError:
Java heap space
java.lang.OutOfMemoryError: Java heap space
Exception in thread "CORBAMonitorPool-247: IDLE" Exception in thread "VBJ
ThreadPool Worker" java.lang.NullPointerException
at org.apache.coyote.tomcat4.OutputBuffer.realWriteChars(
OutputBuffer.java:563)
at org.apache.tomcat.util.buf.CharChunk.flushBuffer(CharChunk.java:435)
at org.apache.coyote.tomcat4.OutputBuffer.close(OutputBuffer.java:268)
at org.apache.coyote.tomcat4.CoyoteResponse.finishResponse(
CoyoteResponse.java:438)
at org.apache.coyote.tomcat4.CoyoteAdapter.service(CoyoteAdapter.java
:153)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java
:799)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection
(Http11Protocol.java:705)
at org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java
:577)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(
ThreadPool.java:683)
at java.lang.Thread.run(Thread.java:595)
java.lang.OutOfMemoryError: Java heap space
java.lang.OutOfMemoryError: Java heap space
java.lang.OutOfMemoryError: Java heap space
java.lang.OutOfMemoryError: Java heap space
java.lang.NullPointerException
at org.apache.coyote.tomcat4.OutputBuffer.realWriteChars(
OutputBuffer.java:563)
at org.apache.tomcat.util.buf.CharChunk.flushBuffer(CharChunk.java:435)
at org.apache.coyote.tomcat4.OutputBuffer.close(OutputBuffer.java:268)
at org.apache.coyote.tomcat4.CoyoteResponse.finishResponse(
CoyoteResponse.java:438)
at org.apache.coyote.tomcat4.CoyoteAdapter.service(CoyoteAdapter.java
:153)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java
:799)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection
(Http11Protocol.java:705)
at org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java
:577)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(
ThreadPool.java:683)
at java.lang.Thread.run(Thread.java:595)
java.lang.OutOfMemoryError: Java heap space
java.lang.OutOfMemoryError: Java heap space
- Error unregistering mbean
java.lang.OutOfMemoryError: Java heap space
Exception in thread "http-80-Processor519" java.lang.OutOfMemoryError: Java
heap space
java.lang.OutOfMemoryError: Java heap space
java.lang.OutOfMemoryError: Java heap space
Dec 5, 2007 4:14:19 PM (Thread-8) (MDL) - Administrator: Callback
notifications were missed from landscape ss-ls02p - resynching cached
alarms.
Dec 5, 2007 4:14:24 PM (Thread-8) (MDL) - Administrator: Callback
notifications were missed from landscape ss-ls08p - resynching cached
alarms.
Exception in thread "StandardManager[/customimages]"
java.lang.OutOfMemoryError: Java heap space
java.lang.OutOfMemoryError: Java heap space
java.lang.OutOfMemoryError: Java heap space
java.lang.OutOfMemoryError: Java heap space
Dec 5, 2007 4:15:07 PM (Thread-8) (MDL) - Administrator: Callback
notifications were missed from landscape ss-ls06p - resynching cached
alarms.
Dec 5, 2007 4:15:18 PM - AlarmUpdates: Error invoking job (
[EMAIL PROTECTED])
: java.lang.OutOfMemoryError: Java heap space
Exception in thread "CORBAMonitorPool-218: IDLE" java.lang.OutOfMemoryError:
Java heap space
java.lang.OutOfMemoryError: Java heap space
java.lang.OutOfMemoryError: Java heap space
Exception in thread "http-80-Processor304" java.lang.OutOfMemoryError: Java
heap space
java.lang.OutOfMemoryError: Java heap space
- Error unregistering mbean
java.lang.OutOfMemoryError: Java heap space
Exception in thread "CORBAMonitorPool-253: IDLE" Exception in thread
"StandardManager[/admin]" Exception in thread "CORBAMonitorPool-252: IDLE"
java.lang.OutOfMemoryError: Java heap space
Dec 5, 2007 4:17:05 PM (Thread-8) (MDL) - Administrator: Callback
notifications were missed from landscape ss-ls01p - resynching cached
alarms.
java.lang.OutOfMemoryError: Java heap space
java.lang.OutOfMemoryError: Java heap space
java.lang.OutOfMemoryError: Java heap space
java.lang.OutOfMemoryError: Java heap space
java.lang.OutOfMemoryError: Java heap space
Exception in thread "DatagramThread" java.lang.OutOfMemoryError: Java heap
space
java.lang.OutOfMemoryError: Java heap space
java.lang.OutOfMemoryError: Java heap space
Dec 5, 2007 4:18:17 PM - Error creating event for domain ss-ls01p:
java.lang.OutOfMemoryError: Java heap space
Dec 5, 2007 4:18:29 PM - [EMAIL PROTECTED]: Error invoking job
(
[EMAIL PROTECTED])
: java.lang.OutOfMemoryError: Java heap space
java.lang.OutOfMemoryError: Java heap space
java.lang.OutOfMemoryError: Java heap space
Dec 5, 2007 4:18:37 PM - The Location Server on ss-ls01p is not available.
Dec 5, 2007 4:18:37 PM - Could not reach location server at ss-ls01p
Dec 5, 2007 4:18:37 PM (Thread-8) (MDL) - Administrator: Callback
notifications were missed from landscape ss-ls01p - resynching cached
alarms.
Dec 5, 2007 4:18:37 PM - Error restarting association watch on domain:
ss-ls03p for models:
{0x4,0x12d23,0xc487,0x2ee89,0x2b67b,0x3a90a,0x3655,0x3c677,0x2f866,0x338bd,0x11b2a,0x3c4ca,0x3a0a4,0x33742,0x5085,0x8f92,0x199bb,0x5385,0xb088,0x2dee5,0x199ba,0x117d5,0x3b48,0x3a07a,0x5496,0x13324,0x3a918,0x117c8,0x4a7c,0x33613,0x2d92b,0x3c675,0x2f9e9,0x3a0a2,0x199bc,0x12d72,0x33740,0x33739,0x0,0x4d87,0x37f4,0xc33a,0xc486,0x12f8f,0x3a909,0x3c673,0x341a6,0xc377,0x1e66e,0x38ffb,0x2c4b6,0xc4c2,0x3a07e,0x3378a,0x3373e,0x39734,0x1da1f,0x2f48f,0x8f93,0x2fa6c,0x4f26,0x3c019,0x3915e,0x5735,0x3c679,0x34999,0xcfd1,0x3373c,0x3014b,0xcfd2,0x3a0a6}
with error: {0}java.lang.OutOfMemoryError: Java heap space
Dec 5, 2007 4:18:57 PM (Thread-8) (MDL) - Administrator: Callback
notifications were missed from landscape ss-ls02p - resynching cached
alarms.
Dec 5, 2007 4:20:08 PM (Thread-8) (MDL) - Administrator: Callback
notifications were missed from landscape ss-ls01p - resynching cached
alarms.
Dec 5, 2007 4:20:49 PM (Thread-8) (MDL) - Administrator: Callback
notifications were missed from landscape ss-ls02p - resynching cached
alarms.
Stopping service Tomcat-Standalone
- Stopping Coyote HTTP/1.1 on http-80
Dec 5, 2007 4:21:58 PM (Thread-8) (MDL) - Administrator: Callback
notifications were missed from landscape ss-ls08p - resynching cached
alarms.
*
--
"Any sufficiently advanced technology is indistinguishable from magic."
- Arthur C. Clarke
