I have setup the AWS clustering according to [1] but it seems the "AWSBasedMembershipScheme Member joined" message is not displayed once a new member is joined and no message when it left. This seems to be an issue with new kernel version.
<clustering class="org.wso2.carbon.core.clustering.hazelcast.HazelcastClusteringAgent" enable="true"> <parameter name="membershipScheme">aws</parameter> <parameter name="localMemberPort">5701</parameter> <parameter name="accessKey">xxxxxxx</parameter> <parameter name="secretKey">zzzzzzz</parameter> <parameter name="securityGroup">sg-16f4b571</parameter> <parameter name="region">us-east-1</parameter> <parameter name="tagKey">a_tag_key</parameter> <parameter name="tagValue">a_tag_value</parameter> [1] https://docs.wso2.com/display/CLUSTER420/Setting+up+a+Cluster+in+AWS+Mode Thanks. *Maninda Edirisooriya* Senior Software Engineer *WSO2, Inc.*lean.enterprise.middleware. *Blog* : http://maninda.blogspot.com/ *E-mail* : mani...@wso2.com *Skype* : @manindae *Twitter* : @maninda On Sun, Oct 25, 2015 at 6:59 PM, Afkham Azeez <az...@wso2.com> wrote: > In EC2 with AWS mode, you should use ports 5701 to 5800 > > On Sun, Oct 25, 2015 at 5:52 PM, Niranda Perera <nira...@wso2.com> wrote: > >> Hi Thusitha, >> >> 4000 is the local host port specified in the axis2.xml >> >> quote axis2.xml >> " <!-- The host name or IP address of this member --> >> >> <parameter name="localMemberHost">10.100.5.109</parameter> >> >> <!-- >> The bind adress of this member. The difference between >> localMemberHost & localMemberBindAddress >> is that localMemberHost is the one that is advertised by this >> member, while localMemberBindAddress >> is the address to which this member is bound to. >> --> >> <!-- >> <parameter name="localMemberBindAddress">10.100.5.109</parameter> >> --> >> >> <!-- >> The TCP port used by this member. This is the port through which >> other nodes will >> contact this member >> --> >> <parameter name="localMemberPort">4000</parameter>" >> >> in DAS we are using wka as the membership scheme. >> >> cheers >> >> On Sun, Oct 25, 2015 at 4:47 PM, Thusitha Thilina Dayaratne < >> thusit...@wso2.com> wrote: >> >>> Hi Anjana, >>> >>> I noticed following error line on the error stack that you posted. >>> >>> com.hazelcast.spi.exception.WrongTargetException: WrongTarget! >>>> this:Address[172.18.1.227]:4000 >>> >>> >>> Here it tries to connect using the port 4000. But AFAIK we usually use >>> ports between 5701 to 5800. Did you set the port to 4000 in any instance? >>> >>> Thanks >>> Thusitha >>> >>> On Sat, Oct 24, 2015 at 5:27 PM, Anjana Fernando <anj...@wso2.com> >>> wrote: >>> >>>> Hi, >>>> >>>> So I'd a chat with Azeez, and the first thing he mentioned was, we >>>> should be using the AWS mode in EC2 it seems, the docs can be found for >>>> this here [1]. And his comments are, these kind of issues can be seen >>>> sometimes there, but if it recovers eventually it would be okay, but if it >>>> continues to give same kind of errors always, that would be a problem. >>>> >>>> [1] >>>> https://docs.wso2.com/display/CLUSTER420/Setting+up+a+Cluster+in+AWS+Mode >>>> >>>> Cheers, >>>> Anjana. >>>> >>>> On Sat, Oct 24, 2015 at 5:11 PM, Anjana Fernando <anj...@wso2.com> >>>> wrote: >>>> >>>>> Hi everyone, >>>>> >>>>> When testing DAS in EC2, and also with the latest Kernel 4.4.2, QA has >>>>> reported some Hazlecast issues, specifically [1] and [2]. Some of the >>>>> errors as follows:- >>>>> >>>>> TID: [-1234] [] [2015-10-24 08:40:05,130] ERROR >>>>> {org.wso2.carbon.registry.core.jdbc.handlers.builtin.MountHandler} - >>>>> Unable to test existence of resource >>>>> {org.wso2.carbon.registry.core.jdbc.handlers.builtin.MountHandler} >>>>> com.hazelcast.spi.exception.WrongTargetException: WrongTarget! >>>>> this:Address[172.18.1.227]:4000, target:Address[172.18.1.228]:4000, >>>>> partitionId: 5, replicaIndex: 0, operation: >>>>> com.hazelcast.map.impl.operation.GetOperation, service: hz:impl:mapService >>>>> at >>>>> com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.ensureNoPartitionProblems(OperationRunnerImpl.java:246) >>>>> at >>>>> com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:127) >>>>> at >>>>> com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:315) >>>>> at >>>>> com.hazelcast.spi.impl.operationexecutor.classic.OperationThread.processPacket(OperationThread.java:142) >>>>> at >>>>> com.hazelcast.spi.impl.operationexecutor.classic.OperationThread.process(OperationThread.java:115) >>>>> at >>>>> com.hazelcast.spi.impl.operationexecutor.classic.OperationThread.doRun(OperationThread.java:101) >>>>> at >>>>> com.hazelcast.spi.impl.operationexecutor.classic.OperationThread.run(OperationThread.java:76) >>>>> at ------ End remote and begin local stack-trace ------.(Unknown >>>>> Source) >>>>> at >>>>> com.hazelcast.spi.impl.operationservice.impl.InvocationFuture.resolveApplicationResponse(InvocationFuture.java:384) >>>>> at >>>>> com.hazelcast.spi.impl.operationservice.impl.InvocationFuture.resolveApplicationResponseOrThrowException(InvocationFuture.java:334) >>>>> at >>>>> com.hazelcast.spi.impl.operationservice.impl.InvocationFuture.get(InvocationFuture.java:225) >>>>> at >>>>> com.hazelcast.spi.impl.operationservice.impl.InvocationFuture.get(InvocationFuture.java:204) >>>>> at >>>>> com.hazelcast.map.impl.proxy.MapProxySupport.invokeOperation(MapProxySupport.java:456) >>>>> at >>>>> com.hazelcast.map.impl.proxy.MapProxySupport.getInternal(MapProxySupport.java:275) >>>>> at >>>>> com.hazelcast.map.impl.proxy.MapProxyImpl.get(MapProxyImpl.java:82) >>>>> at >>>>> org.wso2.carbon.core.clustering.hazelcast.HazelcastDistributedMapProvider$DistMap.get(HazelcastDistributedMapProvider.java:141) >>>>> at org.wso2.carbon.caching.impl.CacheImpl.get(CacheImpl.java:203) >>>>> >>>>> ==================================================================== >>>>> >>>>> >>>>> TID: [-1234] [] [2015-10-24 08:44:18,800] ERROR >>>>> {org.wso2.carbon.registry.core.jdbc.handlers.builtin.MountHandler} - >>>>> Unable to test existence of resource >>>>> {org.wso2.carbon.registry.core.jdbc.handlers.builtin.MountHandler} >>>>> com.hazelcast.spi.exception.CallerNotMemberException: Not Member! >>>>> caller:Address[172.18.1.228]:4000, partitionId: 30, operation: >>>>> com.hazelcast.map.impl.operation.GetOperation, service: hz:impl:mapService >>>>> at >>>>> com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.ensureValidMember(OperationRunnerImpl.java:336) >>>>> at >>>>> com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:308) >>>>> at >>>>> com.hazelcast.spi.impl.operationexecutor.classic.OperationThread.processPacket(OperationThread.java:142) >>>>> at >>>>> com.hazelcast.spi.impl.operationexecutor.classic.OperationThread.process(OperationThread.java:115) >>>>> >>>>> ==================================================================== >>>>> >>>>> ERROR {com.hazelcast.cluster.impl.operations.JoinCheckOperation} - >>>>> [172.18.1.229]:4000 [wso2.qa.das.domain] [3.5.2] Cannot send response: >>>>> JoinRequest{packetVersion=4, buildNumber=20150826, >>>>> address=Address[172.18.1.229]:4000, >>>>> uuid='6848559f-141d-47cd-9e75-53bb2de10d52', credentials=null, >>>>> memberCount=4, tryCount=0} to Address[172.18.1.228]:4000 >>>>> {com.hazelcast.cluster.impl.operations.JoinCheckOperation} >>>>> com.hazelcast.core.HazelcastException: Cannot send response: >>>>> JoinRequest{packetVersion=4, buildNumber=20150826, >>>>> address=Address[172.18.1.229]:4000, >>>>> uuid='6848559f-141d-47cd-9e75-53bb2de10d52', credentials=null, >>>>> memberCount=4, tryCount=0} to Address[172.18.1.228]:4000 >>>>> at >>>>> com.hazelcast.spi.impl.ResponseHandlerFactory$RemoteInvocationResponseHandler.sendResponse(ResponseHandlerFactory.java:131) >>>>> at >>>>> com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.handleResponse(OperationRunnerImpl.java:204) >>>>> >>>>> ==================================================================== >>>>> >>>>> Have any one of you guys seen issues similar to the above in EC2? .. >>>>> and what are the possibilities this could arise, the most obvious thing >>>>> would be networking issues that can arise in EC2, which maybe related to a >>>>> higher latency that's there or something. Any of you guys know how to >>>>> maybe >>>>> tune HZ in this situations to have a higher timeout values or anything >>>>> like >>>>> that, or else, doing this in EC2 is altogether not recommended? .. >>>>> >>>>> [1] https://wso2.org/jira/browse/DAS-301 >>>>> [2] https://wso2.org/jira/browse/DAS-302 >>>>> >>>>> Cheers, >>>>> Anjana. >>>>> -- >>>>> *Anjana Fernando* >>>>> Senior Technical Lead >>>>> WSO2 Inc. | http://wso2.com >>>>> lean . enterprise . middleware >>>>> >>>> >>>> >>>> >>>> -- >>>> *Anjana Fernando* >>>> Senior Technical Lead >>>> WSO2 Inc. | http://wso2.com >>>> lean . enterprise . middleware >>>> >>>> _______________________________________________ >>>> Dev mailing list >>>> Dev@wso2.org >>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>> >>>> >>> >>> >>> -- >>> Thusitha Dayaratne >>> Software Engineer >>> WSO2 Inc. - lean . enterprise . middleware | wso2.com >>> >>> Mobile +94712756809 >>> Blog alokayasoya.blogspot.com >>> About http://about.me/thusithathilina >>> >>> >>> _______________________________________________ >>> Dev mailing list >>> Dev@wso2.org >>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>> >>> >> >> >> -- >> *Niranda Perera* >> Software Engineer, WSO2 Inc. >> Mobile: +94-71-554-8430 >> Twitter: @n1r44 <https://twitter.com/N1R44> >> https://pythagoreanscript.wordpress.com/ >> >> _______________________________________________ >> Dev mailing list >> Dev@wso2.org >> http://wso2.org/cgi-bin/mailman/listinfo/dev >> >> > > > -- > *Afkham Azeez* > Director of Architecture; WSO2, Inc.; http://wso2.com > Member; Apache Software Foundation; http://www.apache.org/ > * <http://www.apache.org/>* > *email: **az...@wso2.com* <az...@wso2.com> > * cell: +94 77 3320919 <%2B94%2077%203320919>blog: * > *http://blog.afkham.org* <http://blog.afkham.org> > *twitter: **http://twitter.com/afkham_azeez* > <http://twitter.com/afkham_azeez> > *linked-in: **http://lk.linkedin.com/in/afkhamazeez > <http://lk.linkedin.com/in/afkhamazeez>* > > *Lean . Enterprise . Middleware* > > _______________________________________________ > Dev mailing list > Dev@wso2.org > http://wso2.org/cgi-bin/mailman/listinfo/dev > >
_______________________________________________ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
