Hi, folks.
On our environment occasionally were observed numerous
org.jboss.cache.lock.TimeoutException's that cause transactions to fail.
Moreover the problem seems to progress over time resulting in a system becoming
non-operational with restart required.
It was suggested to try to increase SynReplTimeOut up to 100 sec and set
loopback attribute to true (something that is recommended on windows machines).
As issue reproduces occasionally it is hard to verify that these changes will
help.
Any thoughts regarding this problem are appreciated.
We use:
JBossAS - 4.2.2.GA
The JBossCache - 1.4.1.SP5
JGroups - 2.4.1-SP4
The config
<mbean code="org.jboss.cache.TreeCache"
name="jboss.cache:service=HibernateTreeCache">
| <depends>jboss:service=Naming</depends>
| <depends>jboss:service=TransactionManager</depends>
| <attribute
name="TransactionManagerLookupClass">org.jboss.cache.JBossTransactionManagerLookup</attribute>
| <attribute name="IsolationLevel">REPEATABLE_READ</attribute>
| <attribute name="CacheMode">REPL_SYNC</attribute>
| <attribute name="ClusterName">DEFAULT</attribute>
|
| <attribute name="ClusterConfig">
| <config>
| <UDP mcast_addr="231.12.21.132" mcast_port="45047"
| ip_ttl="64" ip_mcast="true"
| mcast_send_buf_size="150000" mcast_recv_buf_size="80000"
| ucast_send_buf_size="150000" ucast_recv_buf_size="80000"
| loopback="false"/>
| <PING timeout="2000" num_initial_members="3"
| up_thread="false" down_thread="false"/>
| <MERGE2 min_interval="10000" max_interval="20000"/>
| <FD shun="true" up_thread="true" down_thread="true"/>
| <VERIFY_SUSPECT timeout="1500"
| up_thread="false" down_thread="false"/>
| <pbcast.NAKACK gc_lag="50"
retransmit_timeout="600,1200,2400,4800"
| up_thread="false" down_thread="false"/>
| <pbcast.STABLE desired_avg_gossip="20000"
| up_thread="false" down_thread="false"/>
| <UNICAST timeout="600,1200,2400" window_size="100"
min_threshold="10"
| down_thread="false"/>
| <FRAG frag_size="8192"
| down_thread="false" up_thread="false"/>
| <pbcast.GMS join_timeout="5000" join_retry_timeout="2000"
| shun="true" print_local_addr="true"/>
| <pbcast.STATE_TRANSFER up_thread="false"
down_thread="false"/>
| </config>
| </attribute>
| <attribute name="InitialStateRetrievalTimeout">20000</attribute>
| <attribute name="SyncReplTimeout">10000</attribute>
| <attribute name="LockAcquisitionTimeout">15000</attribute>
| <attribute name="EvictionPolicyClass"></attribute>
| </mbean>
the stack trace:
2008-09-15 04:44:32,548 WARN [http-0.0.0.0-80-2]
[com.arjuna.ats.arjuna.logging.arjLoggerI18N]
[com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator_2]
TwoPhaseCoordinator.beforeCompletion - failed for [EMAIL PROTECTED]
| java.lang.RuntimeException:
| at
org.jboss.cache.interceptors.TxInterceptor$LocalSynchronizationHandler.beforeCompletion(TxInterceptor.java:1182)
| at
org.jboss.cache.interceptors.OrderedSynchronizationHandler.beforeCompletion(OrderedSynchronizationHandler.java:75)
| at
com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:114)
| at
com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:247)
| at
com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:86)
| at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:177)
| at
com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1389)
| at
com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:135)
| at
com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:87)
| at
org.jboss.tm.usertx.client.ServerVMClientUserTransaction.commit(ServerVMClientUserTransaction.java:140)
| at
com.enkata.common.transaction.TransactionManager.commit(TransactionManager.java:75)
| at
com.enkata.common.transaction.TransactionManagerHelper.commit(TransactionManagerHelper.java:43)
| at com.enkata.common.transaction.ATransaction.run(ATransaction.java:91)
| at
com.enkata.common.transaction.NonCancellableTransaction.run(NonCancellableTransaction.java:33)
| at
com.enkata.common.transaction.ReturnObjectTransaction.executeTransaction(ReturnObjectTransaction.java:43)
| at
com.enkata.eac.to.analysis.AnalysisUIFacade.drill(AnalysisUIFacade.java:171)
| at
com.enkata.eac.ui.analyst.drillpath.DrillPathNavigationCmd.doRun(DrillPathNavigationCmd.java:72)
| at com.enkata.eac.ui.util.ACmd.run(ACmd.java:70)
| at com.enkata.eac.ui.framework.Page.runACmd(Page.java:746)
| at
com.enkata.eac.ui.framework.Page.runCmdAndAdjustScrollTop(Page.java:209)
| at com.enkata.eac.ui.framework.Page.validateAndUpdate(Page.java:192)
| at
com.enkata.eac.ui.framework.Controller.validateAndUpdate(Controller.java:373)
| at com.enkata.eac.ui.framework.Controller.process2(Controller.java:270)
| at com.enkata.eac.ui.framework.Controller.process(Controller.java:250)
| at com.enkata.eac.ui.framework.UIMgr.processRequest(UIMgr.java:133)
| at
com.enkata.eac.ui.servlet.EACServlet.processRequest(EACServlet.java:91)
| at com.enkata.eac.ui.servlet.EACServlet.doPost(EACServlet.java:67)
| at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
| at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
| at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
| at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
| at
com.enkata.eac.ui.servlet.DownloadExportFilter.doFilter(DownloadExportFilter.java:41)
| at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
| at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
| at
org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:83)
| at org.jboss.seam.web.MultipartFilter.doFilter(MultipartFilter.java:85)
| at
org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)
| at org.jboss.seam.web.ExceptionFilter.doFilter(ExceptionFilter.java:64)
| at
org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)
| at org.jboss.seam.web.RedirectFilter.doFilter(RedirectFilter.java:45)
| at
org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)
| at org.ajax4jsf.webapp.BaseXMLFilter.doXmlFilter(BaseXMLFilter.java:147)
| at org.ajax4jsf.webapp.BaseFilter.doFilter(BaseFilter.java:276)
| at org.ajax4jsf.Filter.doFilter(Filter.java:175)
| at org.jboss.seam.web.Ajax4jsfFilter.doFilter(Ajax4jsfFilter.java:60)
| at
org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)
| at org.jboss.seam.web.LoggingFilter.doFilter(LoggingFilter.java:58)
| at
org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)
| at org.jboss.seam.servlet.SeamFilter.doFilter(SeamFilter.java:158)
| at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
| at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
| at
org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
| at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
| at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
| at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230)
| at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
| at
org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:179)
| at
org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:84)
| at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
| at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
| at
org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:157)
| at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
| at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:262)
| at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844)
| at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
| at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:446)
| at java.lang.Thread.run(Unknown Source)
| Caused by: org.jboss.cache.ReplicationException:
rsp=sender=172.16.100.17:3216, retval=null, received=false, suspected=false
| at org.jboss.cache.TreeCache.callRemoteMethods(TreeCache.java:4436)
| at org.jboss.cache.TreeCache.callRemoteMethods(TreeCache.java:4358)
| at org.jboss.cache.TreeCache.callRemoteMethods(TreeCache.java:4469)
| at
org.jboss.cache.interceptors.BaseRpcInterceptor.replicateCall(BaseRpcInterceptor.java:110)
| at
org.jboss.cache.interceptors.BaseRpcInterceptor.replicateCall(BaseRpcInterceptor.java:88)
| at
org.jboss.cache.interceptors.ReplicationInterceptor.runPreparePhase(ReplicationInterceptor.java:152)
| at
org.jboss.cache.interceptors.ReplicationInterceptor.invoke(ReplicationInterceptor.java:69)
| at org.jboss.cache.interceptors.Interceptor.invoke(Interceptor.java:68)
| at
org.jboss.cache.interceptors.TxInterceptor.runPreparePhase(TxInterceptor.java:873)
| at
org.jboss.cache.interceptors.TxInterceptor$LocalSynchronizationHandler.beforeCompletion(TxInterceptor.java:1156)
| ... 66 more
| Caused by: org.jboss.cache.lock.TimeoutException: Response timed out:
sender=172.16.100.17:3216, retval=null, received=false, suspected=false
| at org.jboss.cache.TreeCache.callRemoteMethods(TreeCache.java:4434)
| ... 75 more
View the original post :
http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4179030#4179030
Reply to the post :
http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&p=4179030
_______________________________________________
jboss-user mailing list
jboss-user@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/jboss-user
