Hi Senthalan, Is this setup done in AWS? If so, shall we use AWS clustering mode? Refer to [1] for more information.
[1] https://docs.wso2.com/display/CLUSTER44x/Setting+up+a+Cluster+in+AWS+Mode On Mon, Jul 23, 2018 at 10:14 AM Senthalan Kanagalingam <sentha...@wso2.com> wrote: > Hi all, > > I have done a long-running test with IS 5.7.0-m2 to check the stability of > the adaptive authentication feature. I have implemented 2 node deployment > without the sticky session. And used the well-known address(WKA) based > clustering method. we faced a memory issue in one node for a particular > time. During that time that node was not responding and lost the database > connection and left from the cluster. But it recovered from it after some > minutes. But it has not joined again in the cluster again. > > The 2nd node has logged as the failed node has left. But, when the failed > node recovered, there are no logs in the 2nd node as the node has joined. > But the recovered node keeps sending the clustering message and throws > this exception all the time. > > TID: [-1234] [] [2018-07-20 11:41:11,948] ERROR > {org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent} - > Could not send cluster message > com.hazelcast.core.HazelcastInstanceNotActiveException: Hazelcast instance > is not active! > at > com.hazelcast.spi.AbstractDistributedObject.throwNotActiveException(AbstractDistributedObject.java:81) > at > com.hazelcast.spi.AbstractDistributedObject.lifecycleCheck(AbstractDistributedObject.java:76) > at > com.hazelcast.spi.AbstractDistributedObject.getNodeEngine(AbstractDistributedObject.java:70) > at com.hazelcast.topic.impl.TopicProxy.publish(TopicProxy.java:33) > at > org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent.sendMessage(HazelcastClusteringAgent.java:721) > at > org.wso2.carbon.caching.impl.clustering.ClusterCacheInvalidationRequestSender.send(ClusterCacheInvalidationRequestSender.java:98) > at > org.wso2.carbon.caching.impl.CacheImpl.remove(CacheImpl.java:525) > at > org.wso2.carbon.identity.application.common.cache.BaseCache.clearCacheEntry(BaseCache.java:200) > at > org.wso2.carbon.identity.oauth.cache.AuthorizationGrantCache.clearCacheEntryByCode(AuthorizationGrantCache.java:142) > at > org.wso2.carbon.identity.oauth2.token.AccessTokenIssuer.addUserAttributesToCache(AccessTokenIssuer.java:415) > at > org.wso2.carbon.identity.oauth2.token.AccessTokenIssuer.issue(AccessTokenIssuer.java:308) > at > org.wso2.carbon.identity.oauth2.OAuth2Service.issueAccessToken(OAuth2Service.java:225) > at > org.wso2.carbon.identity.oauth.endpoint.token.OAuth2TokenEndpoint.issueAccessToken(OAuth2TokenEndpoint.java:300) > at > org.wso2.carbon.identity.oauth.endpoint.token.OAuth2TokenEndpoint.issueAccessToken(OAuth2TokenEndpoint.java:90) > at sun.reflect.GeneratedMethodAccessor126.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(AbstractInvoker.java:188) > at > org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:104) > at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:204) > at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:101) > at > org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:58) > at > org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:94) > at > org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:272) > at > org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121) > at > org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:249) > at > org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:248) > at > org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:222) > at > org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:153) > at > org.apache.cxf.transport.servlet.CXFNonSpringServlet.invoke(CXFNonSpringServlet.java:171) > at > org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:289) > at > org.apache.cxf.transport.servlet.AbstractHTTPServlet.doPost(AbstractHTTPServlet.java:209) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:650) > at > org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:265) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > at > org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > at > org.wso2.carbon.webapp.mgt.filter.AuthorizationHeaderFilter.doFilter(AuthorizationHeaderFilter.java:85) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > at > org.wso2.carbon.ui.filters.cache.ContentTypeBasedCachePreventionFilter.doFilter(ContentTypeBasedCachePreventionFilter.java:53) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > at > org.apache.catalina.filters.HttpHeaderSecurityFilter.doFilter(HttpHeaderSecurityFilter.java:124) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > at > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219) > at > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:110) > at > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169) > at > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) > at > org.wso2.carbon.identity.context.rewrite.valve.TenantContextRewriteValve.invoke(TenantContextRewriteValve.java:80) > at > org.wso2.carbon.identity.authz.valve.AuthorizationValve.invoke(AuthorizationValve.java:91) > at > org.wso2.carbon.identity.auth.valve.AuthenticationValve.invoke(AuthenticationValve.java:60) > at > org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:99) > at > org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47) > at > org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:57) > at > org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47) > at > org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:62) > at > org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:159) > at > org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:962) > at > org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:57) > at > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) > at > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:445) > at > org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1115) > at > org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:637) > at > org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1775) > at > org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1734) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at > org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) > at java.lang.Thread.run(Thread.java:748) > > > thanks, > Senthalan > -- > > *Senthalan Kanagalingam* > *Software Engineer - WSO2 Inc.* > *Mobile : +94 (0) 77 18 77 466* > <http://wso2.com/signature> > _______________________________________________ > Dev mailing list > Dev@wso2.org > http://wso2.org/cgi-bin/mailman/listinfo/dev > -- Maduranga Siriwardena Associated Technical Lead WSO2 Inc; http://wso2.com/ Email: madura...@wso2.com Mobile: +94718990591 Blog: *https://madurangasiriwardena.wordpress.com/ <https://madurangasiriwardena.wordpress.com/>* <http://wso2.com/signature>
_______________________________________________ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
