Re: Tomcat 6 does not start in a cluster. It simply hangs... Any Idea..
Thanks Filip. I am going to ask the vendor. Filip Hanik - Dev Lists wrote: > > hi Jaspal, the problem seems to be in the com.valleyoak package. > > The main thread is stuck at > > at com.valleyoak.model.Payment.(Payment.java:136) > (at > com.valleyoak.util.IVosTableInfo.findColumnTables(IVosTableInfo.java:161)) > - last line > > the cluster thread is stuck at > at java.lang.Class.forName0(Native Method) > > because the main thread is stuck during a (this is class init, > happens directly after class loading, to do static initializers), the > cluster thread can't continue. > > what you need to investigate is why > at > com.valleyoak.util.IVosTableInfo.findColumnTables(IVosTableInfo.java:161)) > - last line > > is stuck, but it does look like its a problem with your code. > > Filip > > > > > > > jsandhu2204 wrote: >> HI Filip: >> >> I did three dumps. It is hanging on the same class. Please let me know >> any >> ideas... >> >> We are in production. I have same setup on development and it works. >> However, cluster setup on development is on the same box with apache web >> server in front. >> >> In production there are three servers with hardware load balancer in the >> the >> front. >> >> Thanks >> Jaspal >> >> jsandhu2204 wrote: >> >>> Please see the three stack dumps I did. >>> >>> http://www.nabble.com/file/p21367357/Thread-dump-three-times.txt >>> Thread-dump-three-times.txt >>> >>> >>> Filip Hanik - Dev Lists wrote: >>> >>>> Could you take more than one thread dump, take 3, wait 5 seconds >>>> between >>>> each. >>>> As you can see, the startup thread is still executing your application >>>> code com.valleyoak.util.IVosTableInfo, this is the thread that starts >>>> up >>>> tomcat >>>> >>>> Filip >>>> >>>> main thread - the one starting tomcat >>>> >>>> "main" prio=6 tid=0x01dc1c00 nid=0x15cc in Object.wait() >>>> [0x01f4d000..0x01f4fc70] >>>> >>>> java.lang.Thread.State: RUNNABLE >>>> at >>>> com.valleyoak.util.IVosTableInfo.findColumnTables(IVosTableInfo.java:161) >>>> at >>>> com.valleyoak.db.IVosStorable.findColumnTables(IVosStorable.java:1245) >>>> at com.valleyoak.model.Payment.(Payment.java:136) >>>> at >>>> com.valleyoak.model.ContextListener.contextInitialized(ContextListener.java:107) >>>> at >>>> org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:3843) >>>> at >>>> org.apache.catalina.core.StandardContext.start(StandardContext.java:4342) >>>> - locked <0x41f4f390> (a >>>> org.apache.catalina.core.StandardContext) >>>> at >>>> org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791) >>>> - locked <0x41eef8b8> (a java.util.HashMap) >>>> at >>>> org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771) >>>> at >>>> org.apache.catalina.core.StandardHost.addChild(StandardHost.java:525) >>>> at >>>> org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:926) >>>> at >>>> org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:889) >>>> at >>>> org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:492) >>>> >>>> >>>> One of the cluster threads have received the session state data from >>>> the >>>> other node and is deserializing it >>>> >>>> "pool-2-thread-1" prio=6 tid=0x01dc5c00 nid=0xf00 in >>>> Object.wait() [0x777c000..0x0777fae0] >>>> java.lang.Thread.State: RUNNABLE >>>> at java.lang.Class.forName0(Native Method) >>>> at java.lang.Class.forName(Class.java:169) >>>> at com.valleyoak.util.IVosTableInfo.class$(IVosTableInfo.java:504) >>>> at com.valleyoak.util.IVosTableInfo.loadTables(IVosTableInfo.java:522) >>>> at com.valleyoak.util.IVosTableInfo.(IVosTableInfo.java:53) >>>> at >>>> com.valleyoak.db.IVosStorable.findColumnTables(IVosStorable.java:1245) >>>> at com.valleyoak.model.User.(User.java:71) >>>> at java.io.ObjectStreamClass.hasStaticIniti
Re: Tomcat 6 does not start in a cluster. It simply hangs... Any Idea..
HI Filip: I did three dumps. It is hanging on the same class. Please let me know any ideas... We are in production. I have same setup on development and it works. However, cluster setup on development is on the same box with apache web server in front. In production there are three servers with hardware load balancer in the the front. Thanks Jaspal jsandhu2204 wrote: > > Please see the three stack dumps I did. > > http://www.nabble.com/file/p21367357/Thread-dump-three-times.txt > Thread-dump-three-times.txt > > > Filip Hanik - Dev Lists wrote: >> >> Could you take more than one thread dump, take 3, wait 5 seconds between >> each. >> As you can see, the startup thread is still executing your application >> code com.valleyoak.util.IVosTableInfo, this is the thread that starts up >> tomcat >> >> Filip >> >> main thread - the one starting tomcat >> >> "main" prio=6 tid=0x01dc1c00 nid=0x15cc in Object.wait() >> [0x01f4d000..0x01f4fc70] >> >> java.lang.Thread.State: RUNNABLE >> at >> com.valleyoak.util.IVosTableInfo.findColumnTables(IVosTableInfo.java:161) >> at com.valleyoak.db.IVosStorable.findColumnTables(IVosStorable.java:1245) >> at com.valleyoak.model.Payment.(Payment.java:136) >> at >> com.valleyoak.model.ContextListener.contextInitialized(ContextListener.java:107) >> at >> org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:3843) >> at >> org.apache.catalina.core.StandardContext.start(StandardContext.java:4342) >> - locked <0x41f4f390> (a >> org.apache.catalina.core.StandardContext) >> at >> org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791) >> - locked <0x41eef8b8> (a java.util.HashMap) >> at >> org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771) >> at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:525) >> at >> org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:926) >> at >> org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:889) >> at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:492) >> >> >> One of the cluster threads have received the session state data from the >> other node and is deserializing it >> >> "pool-2-thread-1" prio=6 tid=0x01dc5c00 nid=0xf00 in >> Object.wait() [0x777c000..0x0777fae0] >> java.lang.Thread.State: RUNNABLE >> at java.lang.Class.forName0(Native Method) >> at java.lang.Class.forName(Class.java:169) >> at com.valleyoak.util.IVosTableInfo.class$(IVosTableInfo.java:504) >> at com.valleyoak.util.IVosTableInfo.loadTables(IVosTableInfo.java:522) >> at com.valleyoak.util.IVosTableInfo.(IVosTableInfo.java:53) >> at com.valleyoak.db.IVosStorable.findColumnTables(IVosStorable.java:1245) >> at com.valleyoak.model.User.(User.java:71) >> at java.io.ObjectStreamClass.hasStaticInitializer(Native Method) >> at >> java.io.ObjectStreamClass.computeDefaultSUID(ObjectStreamClass.java:1696) >> at java.io.ObjectStreamClass.access$100(ObjectStreamClass.java:52) >> at java.io.ObjectStreamClass$1.run(ObjectStreamClass.java:205) >> at java.security.AccessController.doPrivileged(Native Method) >> at >> java.io.ObjectStreamClass.getSerialVersionUID(ObjectStreamClass.java:202) >> at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:558) >> at >> java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1583) >> at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1496) >> at >> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1732) >> >> >> >> jsandhu2204 wrote: >>> I have created stack trace when servers hangs. Please see attached. I >>> would >>> appreciate if you can help me with this. >>> If I put flag in the web.xml of the application. >>> Tomcat6 >>> does not start sometimes. It simply hangs. >>> >>> http://www.nabble.com/file/p21347289/CheatSheet-ltr1vs02-Tomcat6-hangs-if-put-distributeable-tag.rtf >>> CheatSheet-ltr1vs02-Tomcat6-hangs-if-put-distributeable-tag.rtf >>> >>> http://www.nabble.com/file/p21347289/CheatSheet-Tomcat6-hangs-if-put-distributeable-tag.rtf >>> CheatSheet-Tomcat6-hangs-if-put-distributeable-tag.rtf >>> >>> Filip Hanik - Dev Lists wrote: >>> >>>> When this happens, simply take a thread dump, and we can see why it is >>>> hanging &g
Re: Tomcat 6 does not start in a cluster. It simply hangs... Any Idea..
Please see the three stack dumps I did. http://www.nabble.com/file/p21367357/Thread-dump-three-times.txt Thread-dump-three-times.txt Filip Hanik - Dev Lists wrote: > > Could you take more than one thread dump, take 3, wait 5 seconds between > each. > As you can see, the startup thread is still executing your application > code com.valleyoak.util.IVosTableInfo, this is the thread that starts up > tomcat > > Filip > > main thread - the one starting tomcat > > "main" prio=6 tid=0x01dc1c00 nid=0x15cc in Object.wait() > [0x01f4d000..0x01f4fc70] > > java.lang.Thread.State: RUNNABLE > at > com.valleyoak.util.IVosTableInfo.findColumnTables(IVosTableInfo.java:161) > at com.valleyoak.db.IVosStorable.findColumnTables(IVosStorable.java:1245) > at com.valleyoak.model.Payment.(Payment.java:136) > at > com.valleyoak.model.ContextListener.contextInitialized(ContextListener.java:107) > at > org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:3843) > at > org.apache.catalina.core.StandardContext.start(StandardContext.java:4342) > - locked <0x41f4f390> (a org.apache.catalina.core.StandardContext) > at > org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791) > - locked <0x41eef8b8> (a java.util.HashMap) > at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771) > at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:525) > at > org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:926) > at > org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:889) > at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:492) > > > One of the cluster threads have received the session state data from the > other node and is deserializing it > > "pool-2-thread-1" prio=6 tid=0x01dc5c00 nid=0xf00 in > Object.wait() [0x777c000..0x0777fae0] > java.lang.Thread.State: RUNNABLE > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:169) > at com.valleyoak.util.IVosTableInfo.class$(IVosTableInfo.java:504) > at com.valleyoak.util.IVosTableInfo.loadTables(IVosTableInfo.java:522) > at com.valleyoak.util.IVosTableInfo.(IVosTableInfo.java:53) > at com.valleyoak.db.IVosStorable.findColumnTables(IVosStorable.java:1245) > at com.valleyoak.model.User.(User.java:71) > at java.io.ObjectStreamClass.hasStaticInitializer(Native Method) > at > java.io.ObjectStreamClass.computeDefaultSUID(ObjectStreamClass.java:1696) > at java.io.ObjectStreamClass.access$100(ObjectStreamClass.java:52) > at java.io.ObjectStreamClass$1.run(ObjectStreamClass.java:205) > at java.security.AccessController.doPrivileged(Native Method) > at > java.io.ObjectStreamClass.getSerialVersionUID(ObjectStreamClass.java:202) > at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:558) > at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1583) > at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1496) > at > java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1732) > > > > jsandhu2204 wrote: >> I have created stack trace when servers hangs. Please see attached. I >> would >> appreciate if you can help me with this. >> If I put flag in the web.xml of the application. Tomcat6 >> does not start sometimes. It simply hangs. >> >> http://www.nabble.com/file/p21347289/CheatSheet-ltr1vs02-Tomcat6-hangs-if-put-distributeable-tag.rtf >> CheatSheet-ltr1vs02-Tomcat6-hangs-if-put-distributeable-tag.rtf >> >> http://www.nabble.com/file/p21347289/CheatSheet-Tomcat6-hangs-if-put-distributeable-tag.rtf >> CheatSheet-Tomcat6-hangs-if-put-distributeable-tag.rtf >> >> Filip Hanik - Dev Lists wrote: >> >>> When this happens, simply take a thread dump, and we can see why it is >>> hanging >>> >>> Unix: >>> kill -3 >>> jstack -l >>> >>> Windows >>> Ctrl+Break in the console >>> jstack -l >>> >>> Filip >>> >>> jsandhu2204 wrote: >>> >>>> Problem1 : While starting tomcat6, sometimes it starts other times it >>>> simply >>>> hangs on the following message: >>>> INFO: Manager [localhost#/ivos]; session state send at 12/12/08 12:34 >>>> AM >>>> received in 110 ms. >>>> Tomcat Log: >>>> >>>> INFO: Register manager /ivos to cluster element Engine with name >>>> Catalina >>>> Dec 12, 2008 12:34:53 AM org.apache.catalina.ha.se
Re: Tomcat 6 does not start in a cluster. It simply hangs... Any Idea..
I have created stack trace when servers hangs. Please see attached. I would
appreciate if you can help me with this.
If I put flag in the web.xml of the application. Tomcat6
does not start sometimes. It simply hangs.
http://www.nabble.com/file/p21347289/CheatSheet-ltr1vs02-Tomcat6-hangs-if-put-distributeable-tag.rtf
CheatSheet-ltr1vs02-Tomcat6-hangs-if-put-distributeable-tag.rtf
http://www.nabble.com/file/p21347289/CheatSheet-Tomcat6-hangs-if-put-distributeable-tag.rtf
CheatSheet-Tomcat6-hangs-if-put-distributeable-tag.rtf
Filip Hanik - Dev Lists wrote:
>
> When this happens, simply take a thread dump, and we can see why it is
> hanging
>
> Unix:
> kill -3
> jstack -l
>
> Windows
> Ctrl+Break in the console
> jstack -l
>
> Filip
>
> jsandhu2204 wrote:
>> Problem1 : While starting tomcat6, sometimes it starts other times it
>> simply
>> hangs on the following message:
>> INFO: Manager [localhost#/ivos]; session state send at 12/12/08 12:34 AM
>> received in 110 ms.
>> Tomcat Log:
>>
>> INFO: Register manager /ivos to cluster element Engine with name Catalina
>> Dec 12, 2008 12:34:53 AM org.apache.catalina.ha.session.DeltaManager
>> start
>>
>> INFO: Starting clustering manager at /ivos Dec 12, 2008 12:34:53 AM
>> org.apache.catalina.ha.session.DeltaManager getAllClusterSessions
>>
>> WARNING: Manager [localhost#/ivos], requesting session state from
>> org.apache.catalina.tribes.membership.MemberImpl[tcp://{10, 88, 8,
>> 28}:4002,{10, 88, 8, 28},4002, alive=123501,id={27 -68 111 -64 -49 -117
>> 68
>> 38 -98 -27 -49 39 -22 6 -88 -38 }, payload={}, command={}, domain={}, ].
>> This operation will timeout if no session state has been received within
>> 60
>> seconds.
>>
>> Dec 12, 2008 12:34:53 AM org.apache.catalina.ha.session.DeltaManager
>> waitForSendAllSessions
>>
>> INFO: Manager [localhost#/ivos]; session state send at 12/12/08 12:34 AM
>> received in 110 ms.
>>
>>
>
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
>
>
--
View this message in context:
http://www.nabble.com/Tomcat-6-does-not-start-in-a-cluster.--It-simply-hangs...-Any-Idea..-tp21130862p21347289.html
Sent from the Tomcat - User mailing list archive at Nabble.com.
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
Tomcat 6 does not start in a cluster. It simply hangs... Any Idea..
Problem1 : While starting tomcat6, sometimes it starts other times it simply
hangs on the following message:
INFO: Manager [localhost#/ivos]; session state send at 12/12/08 12:34 AM
received in 110 ms.
Tomcat Log:
INFO: Register manager /ivos to cluster element Engine with name Catalina
Dec 12, 2008 12:34:53 AM org.apache.catalina.ha.session.DeltaManager start
INFO: Starting clustering manager at /ivos Dec 12, 2008 12:34:53 AM
org.apache.catalina.ha.session.DeltaManager getAllClusterSessions
WARNING: Manager [localhost#/ivos], requesting session state from
org.apache.catalina.tribes.membership.MemberImpl[tcp://{10, 88, 8,
28}:4002,{10, 88, 8, 28},4002, alive=123501,id={27 -68 111 -64 -49 -117 68
38 -98 -27 -49 39 -22 6 -88 -38 }, payload={}, command={}, domain={}, ].
This operation will timeout if no session state has been received within 60
seconds.
Dec 12, 2008 12:34:53 AM org.apache.catalina.ha.session.DeltaManager
waitForSendAllSessions
INFO: Manager [localhost#/ivos]; session state send at 12/12/08 12:34 AM
received in 110 ms.
--
View this message in context:
http://www.nabble.com/Tomcat-6-does-not-start-in-a-cluster.--It-simply-hangs...-Any-Idea..-tp21130862p21130862.html
Sent from the Tomcat - User mailing list archive at Nabble.com.
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
Tomcat 6 Cluster Setup- Server1 simply hangs sometimes - won't start...
We installed.tomcat6 in production on three high ended machines
Tomcat Version JVM Version JVM Vendor OS Name OS Version OS Architecture
Apache Tomcat/6.0.18 1.6.0_07-b06 Sun Microsystems Inc. Windows 2003 5.2
amd64
1) Cluster is setup correctly in the server.xml as follows
2) web.xml has flag
Problem1 : While starting tomcat6, sometimes it starts other times it
simply hangs on the following message:
INFO: Manager [localhost#/ivos]; session state send at 12/12/08 12:34 AM
received in 110 ms.
Tomcat Log:
INFO: Register manager /ivos to cluster element Engine with name Catalina
Dec 12, 2008 12:34:53 AM org.apache.catalina.ha.session.DeltaManager start
INFO: Starting clustering manager at /ivos Dec 12, 2008 12:34:53 AM
org.apache.catalina.ha.session.DeltaManager getAllClusterSessions
WARNING: Manager [localhost#/ivos], requesting session state from
org.apache.catalina.tribes.membership.MemberImpl[tcp://{10, 88, 8,
28}:4002,{10, 88, 8, 28},4002, alive=123501,id={27 -68 111 -64 -49 -117 68
38 -98 -27 -49 39 -22 6 -88 -38 }, payload={}, command={}, domain={}, ].
This operation will timeout if no session state has been received within 60
seconds.
Dec 12, 2008 12:34:53 AM org.apache.catalina.ha.session.DeltaManager
waitForSendAllSessions
INFO: Manager [localhost#/ivos]; session state send at 12/12/08 12:34 AM
received in 110 ms.
After this nothing happens simply hangs. Should I use
tcpHostListener="MYIPAddress" or "auto" May be auto is slowing it down?
Problem2: User does not transparently fail-over to node2 if the users fails
on node1.
However, the session count on all the nodes in cluster has correct number.
Means replication does work.
I have attached server.xml
Here is multicast information
Local Address Foreign Address
TCP10.88.8.27:40010.0.0.0:0 LISTENING
TCP10.88.8.27:400110.88.8.28:1993ESTABLISHED
TCP10.88.8.27:400110.88.8.28:2682ESTABLISHED
TCP10.88.8.27:400110.88.8.30:3532ESTABLISHED
TCP10.88.8.27:400110.88.8.30:4292ESTABLISHED
TCP10.88.8.27:424510.88.2.204:1433 ESTABLISHED
TCP10.88.8.27:425010.88.8.28:4002ESTABLISHED
TCP10.88.8.27:425210.88.8.28:4002ESTABLISHED
TCP10.88.8.27:425410.88.8.30:4003ESTABLISHED
TCP10.88.8.27:425510.88.8.30:4003ESTABLISHED
http://www.nabble.com/file/p20985367/server.xml server.xml
--
View this message in context:
http://www.nabble.com/Tomcat-6-Cluster-Setup--Server1-simply-hangs-sometimes---won%27t-start...-tp20985367p20985367.html
Sent from the Tomcat - User mailing list archive at Nabble.com.
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
