On Jun 18, 2009, at 7:46 PM, Rob Lockstone wrote:
The deadlock check is part of the <ping> functionality. There's an
<initial-sleep-time> that defaults to 15 minutes, because the ping
needs to wait for the server to start before pinging.
Interesting, so even though I haven't defined a jsp page for the
ping to check, it still operates? What is it checking?
It's a JDK capability. The JDK has an MBean
"java.lang:type=Threading" with the operation
"findMonitorDeadlockedThreads". So it doesn't need anything HTTP-
related.
-- Scott
In any event, good to know. I'm going to crank this value down since
our servers generally get started within about a minute or so. I'll
pad it out a little to account for recompiling jsp's and such.
Rob
I've added a bug report to at least clean up that log report.
-- Scott
Rob
Here's an example of the arguments we pass to Java. Most of our
servers are configured similarly with variations in the heap sizes.
<jvm-arg>-server</jvm-arg>
<jvm-arg>-verbose:gc</jvm-arg>
<jvm-arg>-Xmn500m</jvm-arg>
<jvm-arg>-Xms5000m</jvm-arg>
<jvm-arg>-Xmx5000m</jvm-arg>
<jvm-arg>-Xss128k</jvm-arg>
<jvm-arg>-Xrs</jvm-arg>
<jvm-arg>-Xloggc:log/gc.log</jvm-arg>
<jvm-arg>-Xdebug</jvm-arg>
<jvm-arg>-XX:+UseConcMarkSweepGC</jvm-arg>
<jvm-arg>-XX:+PrintGCTimeStamps</jvm-arg>
<jvm-arg>-XX:+PrintGCDetails</jvm-arg>
<jvm-arg>-Dhttp.maxConnections=400</jvm-arg>
<jvm-arg>-Dcom.sun.management.jmxremote</jvm-arg>
<jvm-arg>-Dcom.sun.management.jmxremote.port=9337</jvm-arg>
<jvm-arg>-Dcom.sun.management.jmxremote.ssl=false</jvm-arg>
<jvm-arg>-Dcom.sun.management.jmxremote.authenticate=false</
jvm-arg>
<jvm-arg>-agentlib:resin</jvm-arg>
And here's the portion of the log file from when a deadlock occurs.
RESIN [02:58:38.369] {resin-destroy} Server[id=,cluster=app-tier]
stopping [This is when the restart request is sent. --Rob]
RESIN [02:58:38.416] {http--80-2843$1916052035} WebApp[] active
RESIN [02:59:06.136] {main} Proxy Cache disk-size=1024M memory-
size=64M [Resin starts to restart here. --Rob]
RESIN [02:59:06.168] {main} PingThread[] starting, checking []
RESIN [02:59:06.340] {main}
RESIN [02:59:06.340] {main} Windows 2003 5.2 amd64
RESIN [02:59:06.340] {main} Java(TM) SE Runtime Environment
1.6.0_13-b03, Cp1252, en
RESIN [02:59:06.340] {main} Java HotSpot(TM) 64-Bit Server VM 11.3-
b02, 64, mixed mode, Sun Microsystems Inc.
RESIN [02:59:06.340] {main} user.name: SYSTEM
RESIN [02:59:06.340] {main} resin.home = d:\resin
RESIN [02:59:06.340] {main} resin.root = d:\resin
RESIN [02:59:06.340] {main} resin.conf = /d:/resin/conf/resin.conf
RESIN [02:59:06.340] {main}
RESIN [03:14:06.169] {resin-7} JDK detected deadlock. Restarting
Resin. [15 minutes later is when Resin has detected the deadlock
and restarts. --Rob]
RESIN [03:14:06.169] {resin-7}
javax
.management
.openmbean
.CompositeDataSupport
(compositeType
=
javax
.management
.openmbean
.CompositeType
(name
=
java
.lang
.management
.ThreadInfo
,items
=
((itemName
=
blockedCount
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.Long)),
(itemName
=
blockedTime
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.Long)),
(itemName
=
inNative
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.Boolean)),
(itemName
=
lockInfo
,itemType
=
javax
.management
.openmbean
.CompositeType
(name
=
java
.lang
.management
.LockInfo
,items
=
((itemName
=
className
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.String)),
(itemName
=
identityHashCode
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.Integer))))),
(itemName
=
lockName
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.String)),
(itemName
=
lockOwnerId
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.Long)),
(itemName
=
lockOwnerName
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.String)),
(itemName
=
lockedMonitors
,itemType
=
javax
.management
.openmbean
.ArrayType
(name
=
[Ljavax
.management
.openmbean
.CompositeData
;,dimension
=
1
,elementType
=
javax
.management
.openmbean
.CompositeType
(name
=
java
.lang
.management
.MonitorInfo
,items
=
((itemName
=
className
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.String)),
(itemName
=
identityHashCode
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.Integer)),
(itemName
=
lockedStackDepth
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.Integer)),
(itemName
=
lockedStackFrame
,itemType
=
javax
.management
.openmbean
.CompositeType
(name
=
java
.lang
.StackTraceElement
,items
=
((itemName
=
className
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.String)),
(itemName
=
fileName
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.String)),
(itemName
=
lineNumber
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.Integer)),
(itemName
=
methodName
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.String)),
(itemName
=
nativeMethod
,itemType
=
javax
.management
.openmbean
.SimpleType(name=java.lang.Boolean))))))),primitiveArray=false)),
(itemName
=
lockedSynchronizers
,itemType
=
javax
.management
.openmbean
.ArrayType
(name
=
[Ljavax
.management
.openmbean
.CompositeData
;,dimension
=
1
,elementType
=
javax
.management
.openmbean
.CompositeType
(name
=
java
.lang
.management
.LockInfo
,items
=
((itemName
=
className
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.String)),
(itemName
=
identityHashCode
,itemType
=
javax
.management
.openmbean
.SimpleType(name=java.lang.Integer)))),primitiveArray=false)),
(itemName
=
stackTrace
,itemType
=
javax
.management
.openmbean
.ArrayType
(name
=
[Ljavax
.management
.openmbean
.CompositeData
;,dimension
=
1
,elementType
=
javax
.management
.openmbean
.CompositeType
(name
=
java
.lang
.StackTraceElement
,items
=
((itemName
=
className
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.String)),
(itemName
=
fileName
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.String)),
(itemName
=
lineNumber
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.Integer)),
(itemName
=
methodName
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.String)),
(itemName
=
nativeMethod
,itemType
=
javax
.management
.openmbean
.SimpleType(name=java.lang.Boolean)))),primitiveArray=false)),
(itemName
=
suspended
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.Boolean)),
(itemName
=
threadId
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.Long)),
(itemName
=
threadName
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.String)),
(itemName
=
threadState
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.String)),
(itemName
=
waitedCount
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.Long)),
(itemName
=
waitedTime
,itemType
=
javax
.management
.openmbean
.SimpleType(name=java.lang.Long)))),contents={blockedCount=12,
blockedTime=-1, inNative=false,
lockInfo
=
javax
.management
.openmbean
.CompositeDataSupport
(compositeType
=
javax
.management
.openmbean
.CompositeType
(name
=
java
.lang
.management
.LockInfo
,items
=
((itemName
=
className
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.String)),
(itemName
=
identityHashCode
,itemType
=
javax
.management
.openmbean
.SimpleType
(name
=
java
.lang
.Integer
)))),contents={className=com.caucho.loader.EnvironmentClassLoader,
identityHashCode=324813035}),
lockname=com.caucho.loader.environmentclassloa...@135c40eb,
lockOwnerId=25, lockOwnerName=resin-8,
lockedMonitors
=[Ljavax.management.openmbean.CompositeData;@62135133,
lockedSynchronizers
=[Ljavax.management.openmbean.CompositeData;@62381d92,
stackTrace=[Ljavax.management.openmbean.CompositeData;@521d8e9e,
suspended=false, threadId=27, threadName=resin-10,
threadState=BLOCKED, waitedCount=7, waitedTime=-1})
RESIN [03:14:06.184] {resin-7}
javax
.management
.openmbean
.CompositeDataSupport
(compositeType
=
javax
.management
.openmbean
.CompositeType
(name
=
java
.lang
.management
.ThreadInfo
,items
=
((itemName
=
blockedCount
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.Long)),
(itemName
=
blockedTime
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.Long)),
(itemName
=
inNative
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.Boolean)),
(itemName
=
lockInfo
,itemType
=
javax
.management
.openmbean
.CompositeType
(name
=
java
.lang
.management
.LockInfo
,items
=
((itemName
=
className
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.String)),
(itemName
=
identityHashCode
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.Integer))))),
(itemName
=
lockName
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.String)),
(itemName
=
lockOwnerId
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.Long)),
(itemName
=
lockOwnerName
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.String)),
(itemName
=
lockedMonitors
,itemType
=
javax
.management
.openmbean
.ArrayType
(name
=
[Ljavax
.management
.openmbean
.CompositeData
;,dimension
=
1
,elementType
=
javax
.management
.openmbean
.CompositeType
(name
=
java
.lang
.management
.MonitorInfo
,items
=
((itemName
=
className
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.String)),
(itemName
=
identityHashCode
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.Integer)),
(itemName
=
lockedStackDepth
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.Integer)),
(itemName
=
lockedStackFrame
,itemType
=
javax
.management
.openmbean
.CompositeType
(name
=
java
.lang
.StackTraceElement
,items
=
((itemName
=
className
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.String)),
(itemName
=
fileName
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.String)),
(itemName
=
lineNumber
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.Integer)),
(itemName
=
methodName
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.String)),
(itemName
=
nativeMethod
,itemType
=
javax
.management
.openmbean
.SimpleType(name=java.lang.Boolean))))))),primitiveArray=false)),
(itemName
=
lockedSynchronizers
,itemType
=
javax
.management
.openmbean
.ArrayType
(name
=
[Ljavax
.management
.openmbean
.CompositeData
;,dimension
=
1
,elementType
=
javax
.management
.openmbean
.CompositeType
(name
=
java
.lang
.management
.LockInfo
,items
=
((itemName
=
className
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.String)),
(itemName
=
identityHashCode
,itemType
=
javax
.management
.openmbean
.SimpleType(name=java.lang.Integer)))),primitiveArray=false)),
(itemName
=
stackTrace
,itemType
=
javax
.management
.openmbean
.ArrayType
(name
=
[Ljavax
.management
.openmbean
.CompositeData
;,dimension
=
1
,elementType
=
javax
.management
.openmbean
.CompositeType
(name
=
java
.lang
.StackTraceElement
,items
=
((itemName
=
className
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.String)),
(itemName
=
fileName
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.String)),
(itemName
=
lineNumber
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.Integer)),
(itemName
=
methodName
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.String)),
(itemName
=
nativeMethod
,itemType
=
javax
.management
.openmbean
.SimpleType(name=java.lang.Boolean)))),primitiveArray=false)),
(itemName
=
suspended
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.Boolean)),
(itemName
=
threadId
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.Long)),
(itemName
=
threadName
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.String)),
(itemName
=
threadState
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.String)),
(itemName
=
waitedCount
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.Long)),
(itemName
=
waitedTime
,itemType
=
javax
.management
.openmbean
.SimpleType(name=java.lang.Long)))),contents={blockedCount=7,
blockedTime=-1, inNative=false,
lockInfo
=
javax
.management
.openmbean
.CompositeDataSupport
(compositeType
=
javax
.management
.openmbean
.CompositeType
(name
=
java
.lang
.management
.LockInfo
,items
=
((itemName
=
className
,itemType
=javax.management.openmbean.SimpleType(name=java.lang.String)),
(itemName
=
identityHashCode
,itemType
=
javax
.management
.openmbean
.SimpleType
(name
=
java
.lang
.Integer)))),contents={className=com.caucho.loader.ClassEntry,
identityHashCode=587476940}),
lockname=com.caucho.loader.classen...@23042fcc, lockOwnerId=27,
lockOwnerName=resin-10,
lockedMonitors
=[Ljavax.management.openmbean.CompositeData;@67de0c09,
lockedSynchronizers
=[Ljavax.management.openmbean.CompositeData;@22e85825,
stackTrace=[Ljavax.management.openmbean.CompositeData;@43f2f70a,
suspended=false, threadId=25, threadName=resin-8,
threadState=BLOCKED, waitedCount=1, waitedTime=-1})
RESIN [03:14:11.091] {main} Proxy Cache disk-size=1024M memory-
size=64M
RESIN [03:14:11.122] {main} PingThread[] starting, checking []
RESIN [03:14:11.263] {main}
RESIN [03:14:11.263] {main} Windows 2003 5.2 amd64
RESIN [03:14:11.278] {main} Java(TM) SE Runtime Environment
1.6.0_13-b03, Cp1252, en
RESIN [03:14:11.278] {main} Java HotSpot(TM) 64-Bit Server VM 11.3-
b02, 64, mixed mode, Sun Microsystems Inc.
RESIN [03:14:11.278] {main} user.name: SYSTEM
RESIN [03:14:11.278] {main} resin.home = d:\resin
RESIN [03:14:11.278] {main} resin.root = d:\resin
RESIN [03:14:11.278] {main} resin.conf = /d:/resin/conf/resin.conf
RESIN [03:14:11.278] {main}
_______________________________________________
resin-interest mailing list
resin-interest@caucho.com
http://maillist.caucho.com/mailman/listinfo/resin-interest
_______________________________________________
resin-interest mailing list
resin-interest@caucho.com
http://maillist.caucho.com/mailman/listinfo/resin-interest
_______________________________________________
resin-interest mailing list
resin-interest@caucho.com
http://maillist.caucho.com/mailman/listinfo/resin-interest
_______________________________________________
resin-interest mailing list
resin-interest@caucho.com
http://maillist.caucho.com/mailman/listinfo/resin-interest