increase your send timeout, 3000 seconds is very aggressive, and you would have to tune away most long GC pauses to not timeout. increase your dropTimeout, it seems you're getting false positives, prolly GC again

Filip

On 12/11/2009 04:28 PM, Mitch Claborn wrote:
tomcat 6.0.20 cluster
See output below.
There are errors at the beginning.  Do the later messages (e.g. INFO:
Manager [localhost#]; session state send at 12/11/09 5:16 PM received in
7,894 ms.) indicate that it recovered?  Is there a way to somehow
determine that the cluster is operational after the tomcat has been
running for a while?

Mitch

Dec 11, 2009 5:16:43 PM org.apache.catalina.realm.JAASRealm setContainer
INFO: Set JAAS app name Catalina
Dec 11, 2009 5:16:43 PM org.apache.catalina.core.StandardService start
INFO: Starting service Catalina
Dec 11, 2009 5:16:43 PM org.apache.catalina.core.StandardEngine start
INFO: Starting Servlet Engine: Apache Tomcat/6.0.20
Dec 11, 2009 5:16:43 PM org.apache.catalina.ha.tcp.SimpleTcpCluster start
INFO: Cluster is about to start
Dec 11, 2009 5:16:44 PM
org.apache.catalina.tribes.transport.ReceiverBase bind
INFO: Receiver Server Socket bound to:/10.0.0.52:5000
Dec 11, 2009 5:16:44 PM
org.apache.catalina.tribes.membership.McastServiceImpl setupSocket
INFO: Setting cluster mcast soTimeout to 500
Dec 11, 2009 5:16:44 PM
org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers
INFO: Sleeping for 1000 milliseconds to establish cluster membership,
start level:4
Dec 11, 2009 5:16:44 PM org.apache.catalina.ha.tcp.SimpleTcpCluster
memberAdded
INFO: Replication member
added:org.apache.catalina.tribes.membership.MemberImpl[tcp://{10, 0, 0,
53}:5000,{10, 0, 0, 53},5000, alive=73495212,id={103 55 44 -101 55 19 65
-86 -80 19 79 49 -64 64 -48 10 }, payload={}, command={}, domain={}, ]
Dec 11, 2009 5:16:45 PM
org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers
INFO: Done sleeping, membership established, start level:4
Dec 11, 2009 5:16:45 PM
org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers
INFO: Sleeping for 1000 milliseconds to establish cluster membership,
start level:8
Dec 11, 2009 5:16:45 PM org.apache.catalina.tribes.io.BufferPool
getBufferPool
INFO: Created a buffer pool with max size:104857600 bytes of
type:org.apache.catalina.tribes.io.BufferPool15Impl
Dec 11, 2009 5:16:46 PM
org.apache.catalina.ha.session.ClusterSessionListener messageReceived
WARNING: Context manager doesn't exist:localhost#
Dec 11, 2009 5:16:46 PM
org.apache.catalina.ha.session.ClusterSessionListener messageReceived
WARNING: Context manager doesn't exist:localhost#
Dec 11, 2009 5:16:46 PM
org.apache.catalina.tribes.membership.McastServiceImpl waitForMembers
INFO: Done sleeping, membership established, start level:8
Dec 11, 2009 5:16:46 PM
org.apache.catalina.ha.session.ClusterSessionListener messageReceived
WARNING: Context manager doesn't exist:localhost#
Dec 11, 2009 5:16:47 PM
org.apache.catalina.ha.session.ClusterSessionListener messageReceived
WARNING: Context manager doesn't exist:localhost#
Dec 11, 2009 5:16:47 PM
org.apache.catalina.ha.session.ClusterSessionListener messageReceived
WARNING: Context manager doesn't exist:localhost#
Dec 11, 2009 5:16:48 PM
org.apache.catalina.ha.session.ClusterSessionListener messageReceived
WARNING: Context manager doesn't exist:localhost#
Dec 11, 2009 5:16:49 PM org.apache.catalina.ha.session.DeltaManager start
INFO: Register manager  to cluster element Engine with name Catalina
Dec 11, 2009 5:16:49 PM org.apache.catalina.ha.session.DeltaManager start
INFO: Starting clustering manager at
Dec 11, 2009 5:16:54 PM
org.apache.catalina.tribes.group.interceptors.TcpFailureDetector
memberDisappeared
INFO: Received
memberDisappeared[org.apache.catalina.tribes.membership.MemberImpl[tcp://{10,
0, 0, 53}:5000,{10, 0, 0, 53},5000, alive=73504284,id={103 55 44 -101 55
19 65 -86 -80 19 79 49 -64 64 -48 10 }, payload={}, command={},
domain={}, ]] message. Will verify.
Dec 11, 2009 5:16:54 PM
org.apache.catalina.tribes.group.interceptors.TcpFailureDetector
memberDisappeared
INFO: Verification complete. Member still
alive[org.apache.catalina.tribes.membership.MemberImpl[tcp://{10, 0, 0,
53}:5000,{10, 0, 0, 53},5000, alive=73504284,id={103 55 44 -101 55 19 65
-86 -80 19 79 49 -64 64 -48 10 }, payload={}, command={}, domain={}, ]]
Dec 11, 2009 5:16:54 PM org.apache.catalina.ha.tcp.SimpleTcpCluster send
SEVERE: Unable to send message through cluster sender.
org.apache.catalina.tribes.ChannelException: Operation has timed
out(3000 ms.).; Faulty members:tcp://{10, 0, 0, 53}:5000;
         at
org.apache.catalina.tribes.transport.nio.ParallelNioSender.sendMessage(ParallelNioSender.java:97)
         at
org.apache.catalina.tribes.transport.nio.PooledParallelSender.sendMessage(PooledParallelSender.java:53)
         at
org.apache.catalina.tribes.transport.ReplicationTransmitter.sendMessage(ReplicationTransmitter.java:80)
         at
org.apache.catalina.tribes.group.ChannelCoordinator.sendMessage(ChannelCoordinator.java:78)
         at
org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessage(ChannelInterceptorBase.java:75)
         at
org.apache.catalina.tribes.group.interceptors.ThroughputInterceptor.sendMessage(ThroughputInterceptor.java:61)
         at
org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessage(ChannelInterceptorBase.java:75)
         at
org.apache.catalina.tribes.group.interceptors.MessageDispatchInterceptor.sendMessage(MessageDispatchInterceptor.java:73)
         at
org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessage(ChannelInterceptorBase.java:75)
         at
org.apache.catalina.tribes.group.interceptors.TcpFailureDetector.sendMessage(TcpFailureDetector.java:87)
         at
org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessage(ChannelInterceptorBase.java:75)
         at
org.apache.catalina.tribes.group.GroupChannel.send(GroupChannel.java:216)
         at
org.apache.catalina.tribes.group.GroupChannel.send(GroupChannel.java:175)
         at
org.apache.catalina.ha.tcp.SimpleTcpCluster.send(SimpleTcpCluster.java:813)
         at
org.apache.catalina.ha.session.DeltaManager.getAllClusterSessions(DeltaManager.java:882)
         at
org.apache.catalina.ha.session.DeltaManager.start(DeltaManager.java:853)
         at
org.apache.catalina.core.ContainerBase.setManager(ContainerBase.java:438)
         at
org.apache.catalina.core.StandardContext.start(StandardContext.java:4358)
         at
org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
         at
org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
         at
org.apache.catalina.core.StandardHost.addChild(StandardHost.java:526)
         at
org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:630)
         at
org.apache.catalina.startup.HostConfig.deployDescriptors(HostConfig.java:556)
         at
org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:491)
         at
org.apache.catalina.startup.HostConfig.start(HostConfig.java:1206)
         at
org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:314)
         at
org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119)
         at
org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
         at
org.apache.catalina.core.StandardHost.start(StandardHost.java:722)
         at
org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
         at
org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
         at
org.apache.catalina.core.StandardService.start(StandardService.java:516)
         at
org.apache.catalina.core.StandardServer.start(StandardServer.java:710)
         at org.apache.catalina.startup.Catalina.start(Catalina.java:583)
         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
         at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
         at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
         at java.lang.reflect.Method.invoke(Unknown Source)
         at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:288)
         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
         at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
         at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
         at java.lang.reflect.Method.invoke(Unknown Source)
         at
org.apache.commons.daemon.support.DaemonLoader.start(DaemonLoader.java:177)
Dec 11, 2009 5:16:54 PM org.apache.catalina.ha.session.DeltaManager
getAllClusterSessions
WARNING: Manager [localhost#], requesting session state from
org.apache.catalina.tribes.membership.MemberImpl[tcp://{10, 0, 0,
53}:5000,{10, 0, 0, 53},5000, alive=73504284,id={103 55 44 -101 55 19 65
-86 -80 19 79 49 -64 64 -48 10 }, payload={}, command={}, domain={}, ].
This operation will timeout if no session state has been received within
60 seconds.
Dec 11, 2009 5:16:57 PM org.apache.catalina.ha.session.DeltaManager
waitForSendAllSessions
INFO: Manager [localhost#]; session state send at 12/11/09 5:16 PM
received in 7,894 ms.
Dec 11, 2009 5:16:57 PM org.apache.catalina.ha.session.DeltaManager
getAllClusterSessions
WARNING: Manager [localhost#]: Drop message SESSION-ACCESSED inside
GET_ALL_SESSIONS sync phase start date 12/11/09 5:16 PM message date
12/31/69 6:00 PM
Dec 11, 2009 5:16:57 PM org.apache.catalina.ha.session.DeltaManager
getAllClusterSessions
WARNING: Manager [localhost#]: Drop message SESSION-DELTA inside
GET_ALL_SESSIONS sync phase start date 12/11/09 5:16 PM message date
12/31/69 6:00 PM
Dec 11, 2009 5:16:57 PM org.apache.catalina.ha.session.DeltaManager
getAllClusterSessions
WARNING: Manager [localhost#]: Drop message SESSION-ACCESSED inside
GET_ALL_SESSIONS sync phase start date 12/11/09 5:16 PM message date
12/31/69 6:00 PM
Dec 11, 2009 5:16:57 PM org.apache.catalina.ha.session.DeltaManager
getAllClusterSessions
WARNING: Manager [localhost#]: Drop message SESSION-ACCESSED inside
GET_ALL_SESSIONS sync phase start date 12/11/09 5:16 PM message date
12/31/69 6:00 PM
Dec 11, 2009 5:16:57 PM org.apache.catalina.ha.session.DeltaManager
getAllClusterSessions
WARNING: Manager [localhost#]: Drop message SESSION-ACCESSED inside
GET_ALL_SESSIONS sync phase start date 12/11/09 5:16 PM message date
12/31/69 6:00 PM
Dec 11, 2009 5:16:57 PM org.apache.catalina.ha.session.DeltaManager
getAllClusterSessions
WARNING: Manager [localhost#]: Drop message SESSION-ACCESSED inside
GET_ALL_SESSIONS sync phase start date 12/11/09 5:16 PM message date
12/31/69 6:00 PM
Dec 11, 2009 5:16:57 PM org.apache.catalina.ha.session.DeltaManager
getAllClusterSessions
WARNING: Manager [localhost#]: Drop message SESSION-DELTA inside
GET_ALL_SESSIONS sync phase start date 12/11/09 5:16 PM message date
12/31/69 6:00 PM
Dec 11, 2009 5:16:57 PM org.apache.catalina.ha.session.DeltaManager
getAllClusterSessions
WARNING: Manager [localhost#]: Drop message SESSION-ACCESSED inside
GET_ALL_SESSIONS sync phase start date 12/11/09 5:16 PM message date
12/31/69 6:00 PM
Dec 11, 2009 5:16:57 PM org.apache.catalina.ha.session.DeltaManager
getAllClusterSessions
WARNING: Manager [localhost#]: Drop message SESSION-ACCESSED inside
GET_ALL_SESSIONS sync phase start date 12/11/09 5:16 PM message date
12/31/69 6:00 PM
Dec 11, 2009 5:16:57 PM org.apache.catalina.ha.session.DeltaManager
getAllClusterSessions
WARNING: Manager [localhost#]: Drop message SESSION-DELTA inside
GET_ALL_SESSIONS sync phase start date 12/11/09 5:16 PM message date
12/31/69 6:00 PM
Dec 11, 2009 5:16:57 PM org.apache.catalina.ha.session.DeltaManager
getAllClusterSessions
WARNING: Manager [localhost#]: Drop message SESSION-ACCESSED inside
GET_ALL_SESSIONS sync phase start date 12/11/09 5:16 PM message date
12/31/69 6:00 PM
Dec 11, 2009 5:16:57 PM org.apache.catalina.ha.session.DeltaManager
getAllClusterSessions
WARNING: Manager [localhost#]: Drop message SESSION-DELTA inside
GET_ALL_SESSIONS sync phase start date 12/11/09 5:16 PM message date
12/31/69 6:00 PM
Dec 11, 2009 5:16:57 PM org.apache.catalina.ha.session.DeltaManager
getAllClusterSessions
WARNING: Manager [localhost#]: Drop message SESSION-ACCESSED inside
GET_ALL_SESSIONS sync phase start date 12/11/09 5:16 PM message date
12/31/69 6:00 PM
Dec 11, 2009 5:16:57 PM
org.apache.catalina.tribes.transport.nio.NioReplicationTask sendAck
WARNING: Unable to send ACK back through channel, channel disconnected?:
null
com.csc.mm.MMCommonLogger logging initialized, email=err...@claborn.net,
guilogger=false, loaded from
file:/home/webhome/mm_web_server/webapps/ROOT/WEB-INF/classes/simplelog.properties
2009-12-11 17:16:57:654 [INFO] AbstractTilesApplicationContextFactory -
Initializing Tiles2 application context. . .
2009-12-11 17:16:57:664 [WARN] ChainedTilesApplicationContextFactory -
Cannot find TilesContextFactory class
org.apache.tiles.portlet.context.PortletTilesApplicationContextFactory
2009-12-11 17:16:57:664 [INFO] AbstractTilesApplicationContextFactory -
Finished initializing Tiles2 application context.
2009-12-11 17:16:57:664 [WARN] ChainedTilesApplicationContextFactory -
Cannot find TilesContextFactory class
org.apache.tiles.portlet.context.PortletTilesApplicationContextFactory
2009-12-11 17:16:57:690 [INFO] TilesContainerFactory - Initializing
Tiles2 container. . .
2009-12-11 17:16:57:692 [WARN] ChainedTilesApplicationContextFactory -
Cannot find TilesContextFactory class
org.apache.tiles.portlet.context.PortletTilesApplicationContextFactory
2009-12-11 17:16:57:694 [WARN] ChainedTilesRequestContextFactory -
Cannot find TilesRequestContextFactory class
org.apache.tiles.portlet.context.PortletTilesRequestContextFactory
2009-12-11 17:16:57:912 [INFO] TilesContainerFactory - Tiles2 container
initialized
2009-12-11 17:16:57:914 [INFO] TilesAccess - Publishing TilesContext for
context: org.apache.tiles.servlet.context.ServletTilesApplicationContext
2009-12-11 17:16:59:118 [INFO] XmlConfigurationProvider - Parsing
configuration file [struts-default.xml]
2009-12-11 17:16:59:495 [INFO] XmlConfigurationProvider - Parsing
configuration file [struts-plugin.xml]
2009-12-11 17:16:59:557 [INFO] XmlConfigurationProvider - Parsing
configuration file [struts.xml]
2009-12-11 17:17:01:077 [INFO] Initialize - Initializing server on host
[ccs052] build=[0000.00.00.00.00.00 unknown build]
2009-12-11 17:17:01:473 [INFO] Initialize - Loading cached objects from
local database
2009-12-11 17:17:24:372 [TRACE] WebUtil - loaded 1993 skus from db
2009-12-11 17:17:24:393 [INFO] Initialize - MM Web Server initialized
2009-12-11 17:17:24:393 [INFO] BackgroundObjectLoader - Beginning
background load of products, etc
Dec 11, 2009 5:17:24 PM org.apache.coyote.http11.Http11Protocol start
INFO: Starting Coyote HTTP/1.1 on http-8084
Dec 11, 2009 5:17:24 PM org.apache.catalina.startup.Catalina start
INFO: Server startup in 41147 ms
2009-12-11 17:19:01:373 [INFO] BackgroundObjectLoader - All data is
loaded, ready to rock and roll
2009-12-11 17:19:38:792 [TRACE] WebUtil - loading Products
Dec 11, 2009 5:20:53 PM
org.apache.catalina.tribes.group.interceptors.ThroughputInterceptor report
INFO: ThroughputInterceptor Report[
         Tx Msg:1 messages
         Sent:0.00 MB (total)
         Sent:0.00 MB (application)
         Time:0.00 seconds
         Tx Speed:0.26 MB/sec (total)
         TxSpeed:0.26 MB/sec (application)
         Error Msg:1
         Rx Msg:470 messages
         Rx Speed:0.02 MB/sec (since 1st msg)
         Received:5.53 MB]

2009-12-11 17:21:41:511 [TRACE] WebUtil - end loading Products
2009-12-11 17:21:41:512 [TRACE] WebUtil - Loading categories
ccs052:/home/webhome/mm_web_server/logs#





---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org




---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Reply via email to