Hi all,
I'm having a strange issue on a WO installation on EC2 (oracle jvm).
Same strange application which had immortal sessions…
Sometimes (quite rarely) a bounced application (put in refuse new sessions)
never quits and it's not accessible from JavaMonitor.
If I look at the logs i see:
Mar 19 12:38:52 B2C[2002] (ERXNSLogLog4jBridge.java:44) WARN NSLog -
<com.tla.calendar.Application>: refusing new clients and below min active
session threshold
Mar 19 12:38:52 B2C[2002] (ERXNSLogLog4jBridge.java:44) WARN NSLog -
<com.tla.calendar.Application>: about to terminate...
The only thing left is to kill the instance… Which is not nice.
I'm not overriding the terminate() method in Application.
I am attaching a stack trace if it helps.
Full thread dump Java HotSpot(TM) 64-Bit Server VM (20.7-b02 mixed mode):
"Attach Listener" daemon prio=10 tid=0x0000000041543800 nid=0x17fd runnable
[0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"Thread-21" prio=10 tid=0x00007f16f8784000 nid=0x1151 waiting on condition
[0x00007f16f75f4000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at
com.webobjects.appserver._private.WOClassicAdaptor.unregisterForEvents(WOClassicAdaptor.java:290)
- locked <0x00000000d043a8e0> (a
com.webobjects.appserver._private.WODefaultAdaptor)
at com.webobjects.appserver.WOApplication$1.run(WOApplication.java:1258)
at java.lang.Thread.run(Thread.java:662)
"Timer-0" daemon prio=10 tid=0x0000000041e93800 nid=0x1150 in Object.wait()
[0x00007f16f77f6000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000c5c898b0> (a java.util.TaskQueue)
at java.lang.Object.wait(Object.java:485)
at java.util.TimerThread.mainLoop(Timer.java:483)
- locked <0x00000000c5c898b0> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:462)
"ERMailSender" prio=10 tid=0x0000000041b41000 nid=0x1032 in Object.wait()
[0x00007f16f76f5000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000d2ea49d0> (a er.javamail.ERQueue)
at er.javamail.ERMailSender.run(ERMailSender.java:356)
- locked <0x00000000d2ea49d0> (a er.javamail.ERQueue)
at java.lang.Thread.run(Thread.java:662)
"WorkerThread11" prio=10 tid=0x0000000041848800 nid=0x1010 in Object.wait()
[0x00007f16f7cfa000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000d120b328> (a
com.webobjects.appserver.WOSessionStore$TimeoutEntry)
at java.lang.Object.wait(Object.java:485)
at
com.webobjects.appserver.WOSessionStore.checkOutSessionWithID(WOSessionStore.java:191)
- locked <0x00000000d120b328> (a
com.webobjects.appserver.WOSessionStore$TimeoutEntry)
at
com.webobjects.appserver.WOApplication.restoreSessionWithID(WOApplication.java:1913)
at
er.extensions.appserver.ERXApplication.restoreSessionWithID(ERXApplication.java:2403)
at
er.extensions.appserver.ERXWOContext.existingSession(ERXWOContext.java:57)
at er.extensions.appserver.ERXWOContext.hasSession(ERXWOContext.java:69)
at com.webobjects.appserver.WOAction.existingSession(WOAction.java:190)
at com.tla.calendar.DirectAction.goToAction(DirectAction.java:454)
at sun.reflect.GeneratedMethodAccessor160.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
com.webobjects.appserver.WODirectAction.performActionNamed(WODirectAction.java:144)
at
er.extensions.appserver.ERXDirectAction.performActionNamed(ERXDirectAction.java:418)
at
com.webobjects.appserver._private.WOActionRequestHandler._handleRequest(WOActionRequestHandler.java:259)
at
com.webobjects.appserver._private.WOActionRequestHandler.handleRequest(WOActionRequestHandler.java:161)
at
er.extensions.appserver.ERXDirectActionRequestHandler.handleRequest(ERXDirectActionRequestHandler.java:128)
at
com.webobjects.appserver.WOApplication.dispatchRequest(WOApplication.java:1687)
at
er.extensions.appserver.ERXApplication.dispatchRequestImmediately(ERXApplication.java:2109)
at
er.extensions.appserver.ERXApplication.dispatchRequest(ERXApplication.java:2074)
at
com.webobjects.appserver._private.WOWorkerThread.runOnce(WOWorkerThread.java:144)
at
com.webobjects.appserver._private.WOWorkerThread.run(WOWorkerThread.java:226)
at java.lang.Thread.run(Thread.java:662)
"WorkerThread9" prio=10 tid=0x0000000041c22000 nid=0x100e in Object.wait()
[0x00007f16f7efc000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000d120b328> (a
com.webobjects.appserver.WOSessionStore$TimeoutEntry)
at java.lang.Object.wait(Object.java:485)
at
com.webobjects.appserver.WOSessionStore.checkOutSessionWithID(WOSessionStore.java:191)
- locked <0x00000000d120b328> (a
com.webobjects.appserver.WOSessionStore$TimeoutEntry)
at
com.webobjects.appserver.WOApplication.restoreSessionWithID(WOApplication.java:1913)
at
er.extensions.appserver.ERXApplication.restoreSessionWithID(ERXApplication.java:2403)
at com.webobjects.appserver.WOContext.session(WOContext.java:364)
at com.webobjects.appserver.WOAction.session(WOAction.java:165)
at com.tla.calendar.DirectAction.sess(DirectAction.java:1720)
at
com.tla.calendar.DirectAction.commonBookAction(DirectAction.java:1593)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
com.webobjects.appserver.WODirectAction.performActionNamed(WODirectAction.java:144)
at
er.extensions.appserver.ERXDirectAction.performActionNamed(ERXDirectAction.java:418)
at
com.webobjects.appserver._private.WOActionRequestHandler._handleRequest(WOActionRequestHandler.java:259)
at
com.webobjects.appserver._private.WOActionRequestHandler.handleRequest(WOActionRequestHandler.java:161)
at
er.extensions.appserver.ERXDirectActionRequestHandler.handleRequest(ERXDirectActionRequestHandler.java:128)
at
com.webobjects.appserver.WOApplication.dispatchRequest(WOApplication.java:1687)
at
er.extensions.appserver.ERXApplication.dispatchRequestImmediately(ERXApplication.java:2109)
at
er.extensions.appserver.ERXApplication.dispatchRequest(ERXApplication.java:2074)
at
com.webobjects.appserver._private.WOWorkerThread.runOnce(WOWorkerThread.java:144)
at
com.webobjects.appserver._private.WOWorkerThread.run(WOWorkerThread.java:226)
at java.lang.Thread.run(Thread.java:662)
"MultiThreadedHttpConnectionManager cleanup" daemon prio=10
tid=0x0000000041449800 nid=0xff3 in Object.wait() [0x00007f16fc56c000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000d11948d0> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
- locked <0x00000000d11948d0> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
at
org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$ReferenceQueueThread.run(MultiThreadedHttpConnectionManager.java:1122)
"WorkerThread5" prio=10 tid=0x00007f16f8438800 nid=0xff0 in Object.wait()
[0x00007f16fc86e000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000d120b328> (a
com.webobjects.appserver.WOSessionStore$TimeoutEntry)
at java.lang.Object.wait(Object.java:485)
at
com.webobjects.appserver.WOSessionStore.checkOutSessionWithID(WOSessionStore.java:191)
- locked <0x00000000d120b328> (a
com.webobjects.appserver.WOSessionStore$TimeoutEntry)
at
com.webobjects.appserver.WOApplication.restoreSessionWithID(WOApplication.java:1913)
at
er.extensions.appserver.ERXApplication.restoreSessionWithID(ERXApplication.java:2403)
at com.webobjects.appserver.WOContext.session(WOContext.java:364)
at com.webobjects.appserver.WOAction.session(WOAction.java:165)
at com.tla.calendar.DirectAction.sess(DirectAction.java:1720)
at
com.tla.calendar.DirectAction.commonBookAction(DirectAction.java:1593)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
com.webobjects.appserver.WODirectAction.performActionNamed(WODirectAction.java:144)
at
er.extensions.appserver.ERXDirectAction.performActionNamed(ERXDirectAction.java:418)
at
com.webobjects.appserver._private.WOActionRequestHandler._handleRequest(WOActionRequestHandler.java:259)
at
com.webobjects.appserver._private.WOActionRequestHandler.handleRequest(WOActionRequestHandler.java:161)
at
er.extensions.appserver.ERXDirectActionRequestHandler.handleRequest(ERXDirectActionRequestHandler.java:128)
at
com.webobjects.appserver.WOApplication.dispatchRequest(WOApplication.java:1687)
at
er.extensions.appserver.ERXApplication.dispatchRequestImmediately(ERXApplication.java:2109)
at
er.extensions.appserver.ERXApplication.dispatchRequest(ERXApplication.java:2074)
at
com.webobjects.appserver._private.WOWorkerThread.runOnce(WOWorkerThread.java:144)
at
com.webobjects.appserver._private.WOWorkerThread.run(WOWorkerThread.java:226)
at java.lang.Thread.run(Thread.java:662)
"WorkerThread3" prio=10 tid=0x00007f16f842e800 nid=0xfee in Object.wait()
[0x00007f16fca70000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000d120b328> (a
com.webobjects.appserver.WOSessionStore$TimeoutEntry)
at java.lang.Object.wait(Object.java:485)
at
com.webobjects.appserver.WOSessionStore.checkOutSessionWithID(WOSessionStore.java:191)
- locked <0x00000000d120b328> (a
com.webobjects.appserver.WOSessionStore$TimeoutEntry)
at
com.webobjects.appserver.WOApplication.restoreSessionWithID(WOApplication.java:1913)
at
er.extensions.appserver.ERXApplication.restoreSessionWithID(ERXApplication.java:2403)
at com.webobjects.appserver.WOContext.session(WOContext.java:364)
at com.webobjects.appserver.WOAction.session(WOAction.java:165)
at com.tla.calendar.DirectAction.sess(DirectAction.java:1720)
at
com.tla.calendar.DirectAction.commonBookAction(DirectAction.java:1593)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
com.webobjects.appserver.WODirectAction.performActionNamed(WODirectAction.java:144)
at
er.extensions.appserver.ERXDirectAction.performActionNamed(ERXDirectAction.java:418)
at
com.webobjects.appserver._private.WOActionRequestHandler._handleRequest(WOActionRequestHandler.java:259)
at
com.webobjects.appserver._private.WOActionRequestHandler.handleRequest(WOActionRequestHandler.java:161)
at
er.extensions.appserver.ERXDirectActionRequestHandler.handleRequest(ERXDirectActionRequestHandler.java:128)
at
com.webobjects.appserver.WOApplication.dispatchRequest(WOApplication.java:1687)
at
er.extensions.appserver.ERXApplication.dispatchRequestImmediately(ERXApplication.java:2109)
at
er.extensions.appserver.ERXApplication.dispatchRequest(ERXApplication.java:2074)
at
com.webobjects.appserver._private.WOWorkerThread.runOnce(WOWorkerThread.java:144)
at
com.webobjects.appserver._private.WOWorkerThread.run(WOWorkerThread.java:226)
at java.lang.Thread.run(Thread.java:662)
"WorkerThread2" prio=10 tid=0x00007f16f8424800 nid=0xfed in Object.wait()
[0x00007f16fcb71000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000d120b328> (a
com.webobjects.appserver.WOSessionStore$TimeoutEntry)
at java.lang.Object.wait(Object.java:485)
at
com.webobjects.appserver.WOSessionStore.checkOutSessionWithID(WOSessionStore.java:191)
- locked <0x00000000d120b328> (a
com.webobjects.appserver.WOSessionStore$TimeoutEntry)
at
com.webobjects.appserver.WOApplication.restoreSessionWithID(WOApplication.java:1913)
at
er.extensions.appserver.ERXApplication.restoreSessionWithID(ERXApplication.java:2403)
at com.webobjects.appserver.WOContext.session(WOContext.java:364)
at com.webobjects.appserver.WOAction.session(WOAction.java:165)
at com.tla.calendar.DirectAction.sess(DirectAction.java:1720)
at
com.tla.calendar.DirectAction.commonBookAction(DirectAction.java:1593)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
com.webobjects.appserver.WODirectAction.performActionNamed(WODirectAction.java:144)
at
er.extensions.appserver.ERXDirectAction.performActionNamed(ERXDirectAction.java:418)
at
com.webobjects.appserver._private.WOActionRequestHandler._handleRequest(WOActionRequestHandler.java:259)
at
com.webobjects.appserver._private.WOActionRequestHandler.handleRequest(WOActionRequestHandler.java:161)
at
er.extensions.appserver.ERXDirectActionRequestHandler.handleRequest(ERXDirectActionRequestHandler.java:128)
at
com.webobjects.appserver.WOApplication.dispatchRequest(WOApplication.java:1687)
at
er.extensions.appserver.ERXApplication.dispatchRequestImmediately(ERXApplication.java:2109)
at
er.extensions.appserver.ERXApplication.dispatchRequest(ERXApplication.java:2074)
at
com.webobjects.appserver._private.WOWorkerThread.runOnce(WOWorkerThread.java:144)
at
com.webobjects.appserver._private.WOWorkerThread.run(WOWorkerThread.java:226)
at java.lang.Thread.run(Thread.java:662)
"Thread-1" daemon prio=10 tid=0x00007f16f8cc4800 nid=0xfea waiting on condition
[0x00007f16fce86000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at
com.webobjects.appserver.WOApplication$_LifebeatThread.run(WOApplication.java:1619)
"Session Timeout Thread" daemon prio=10 tid=0x00007f16f8cad000 nid=0xfe9
waiting on condition [0x00007f16fcf87000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at
com.webobjects.appserver.WOSessionStore$_SessionTimeoutManager.run(WOSessionStore.java:98)
at java.lang.Thread.run(Thread.java:662)
"Low Memory Detector" daemon prio=10 tid=0x0000000040a91000 nid=0xfe7 runnable
[0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C2 CompilerThread1" daemon prio=10 tid=0x0000000040a8c000 nid=0xfe6 waiting on
condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C2 CompilerThread0" daemon prio=10 tid=0x0000000040a89800 nid=0xfe5 waiting on
condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"Signal Dispatcher" daemon prio=10 tid=0x0000000040a87800 nid=0xfe4 runnable
[0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"Finalizer" daemon prio=10 tid=0x0000000040a65800 nid=0xfe3 in Object.wait()
[0x00007f16fdd20000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000d03fd4f0> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
- locked <0x00000000d03fd4f0> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
"Reference Handler" daemon prio=10 tid=0x0000000040a63800 nid=0xfe2 in
Object.wait() [0x00007f16fde21000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000d03fcdd8> (a java.lang.ref.Reference$Lock)
at java.lang.Object.wait(Object.java:485)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
- locked <0x00000000d03fcdd8> (a java.lang.ref.Reference$Lock)
"main" prio=10 tid=0x0000000040a07800 nid=0xfe0 in Object.wait()
[0x00007f1702351000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000c4763378> (a java.lang.Thread)
at java.lang.Thread.join(Thread.java:1186)
- locked <0x00000000c4763378> (a java.lang.Thread)
at java.lang.Thread.join(Thread.java:1239)
at com.webobjects.appserver.WOApplication.run(WOApplication.java:1267)
at er.extensions.appserver.ERXApplication.run(ERXApplication.java:1414)
at com.tla.common.TLAApplication.run(TLAApplication.java:225)
at com.webobjects.appserver.WOApplication.main(WOApplication.java:548)
at er.extensions.appserver.ERXApplication.main(ERXApplication.java:861)
at com.tla.calendar.Application.main(Application.java:33)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.webobjects._bootstrap.WOBootstrap.main(WOBootstrap.java:87)
"VM Thread" prio=10 tid=0x0000000040a5c800 nid=0xfe1 runnable
"VM Periodic Task Thread" prio=10 tid=0x0000000040a9c000 nid=0xfe8 waiting on
condition
JNI global references: 1831
Thanks,
Matteo Centro
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list (Webobjects-dev@lists.apple.com)
Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com
This email sent to arch...@mail-archive.com
