Hi,
It seams we are observing the same issue
https://github.com/hazelcast/hazelcast/issues/1019
On Tue, Mar 17, 2015 at 10:59 AM, Asanka Vithanage <asan...@wso2.com> wrote:
> Hi All,
>
> Observed following error on AS 520 manager Node when restarting the AS
> worker Nodes. Issue seams Intermittent.
> Have we observed this issue before ? Any solutions/explanations?
>
> *Cluster Details:*
> niginx fronted 1 Manager/ 2 worker node AS 520 (with Service
> Pack1-hazelcast_3.0.1.wso2v1.jar) cluster, each node runs on separate
> machines.
> Local registry mounted to H2
> Config and Governance registry mounted on a Mysql Database
>
> Manager Node: 192.168.48.90 Wrk Node 1: 192.168.48.91 Wrk Node 2:
> 192.168.48.92
>
>
> *Steps followed:*
> 1. Cluster was running nearly 2 weeks continuously
> 2. AS cluster used to host back-end services required for ESB
> 3. AS Services invoked thorough ESB continously as a long running/load test
> 4. Due to a OOM issue on worker Node2 , restarted the cluster
>
> *Issue:*
>
> TID: [0] [AS] [2015-03-17 02:06:34,207] INFO
> {org.wso2.carbon.core.clustering.hazelcast.wka.WKABasedMembershipScheme} -
> Member left [a3ee426a-153e-4ebc-95d5-95292370ec23]: /192.168.48.92:4000
> {org.wso2.carbon.core.clustering.hazelcast.wka.WKABasedMembershipScheme}
> TID: [0] [AS] [2015-03-17 04:49:47,567] INFO
> {org.wso2.carbon.core.clustering.hazelcast.wka.WKABasedMembershipScheme} -
> Member joined [a5d7cd86-fc94-409b-9720-e68210e6906c]: /192.168.48.92:4000
> {org.wso2.carbon.core.clustering.hazelcast.wka.WKABasedMembershipScheme}
> TID: [0] [AS] [2015-03-17 04:50:18,641] ERROR
> {com.hazelcast.spi.impl.PartitionIteratingOperation} -
> [192.168.48.90]:4000 [wso2.as.domain] [unknown]
> java.lang.NullPointerException
> {com.hazelcast.spi.impl.PartitionIteratingOperation}
> com.hazelcast.nio.serialization.HazelcastSerializationException:
> java.lang.NullPointerException
> at
> com.hazelcast.nio.serialization.SerializationServiceImpl.handleException(SerializationServiceImpl.java:298)
> at
> com.hazelcast.nio.serialization.SerializationServiceImpl.writeObject(SerializationServiceImpl.java:254)
> at
> com.hazelcast.nio.serialization.ByteArrayObjectDataOutput.writeObject(ByteArrayObjectDataOutput.java:262)
> at
> com.hazelcast.spi.impl.PartitionIteratingOperation$PartitionResponse.writeData(PartitionIteratingOperation.java:130)
> at
> com.hazelcast.nio.serialization.DataSerializer.write(DataSerializer.java:129)
> at
> com.hazelcast.nio.serialization.DataSerializer.write(DataSerializer.java:36)
> at
> com.hazelcast.nio.serialization.StreamSerializerAdapter.write(StreamSerializerAdapter.java:36)
> at
> com.hazelcast.nio.serialization.SerializationServiceImpl.writeObject(SerializationServiceImpl.java:252)
> at
> com.hazelcast.nio.serialization.ByteArrayObjectDataOutput.writeObject(ByteArrayObjectDataOutput.java:262)
> at
> com.hazelcast.spi.impl.NormalResponse.writeData(NormalResponse.java:79)
> at
> com.hazelcast.nio.serialization.DataSerializer.write(DataSerializer.java:129)
> at
> com.hazelcast.nio.serialization.DataSerializer.write(DataSerializer.java:36)
> at
> com.hazelcast.nio.serialization.StreamSerializerAdapter.write(StreamSerializerAdapter.java:48)
> at
> com.hazelcast.nio.serialization.SerializationServiceImpl.toData(SerializationServiceImpl.java:176)
> at
> com.hazelcast.nio.serialization.SerializationServiceImpl.toData(SerializationServiceImpl.java:157)
> at
> com.hazelcast.spi.impl.NodeEngineImpl.toData(NodeEngineImpl.java:151)
> at
> com.hazelcast.spi.impl.BasicOperationService.send(BasicOperationService.java:639)
> at
> com.hazelcast.spi.impl.ResponseHandlerFactory$RemoteInvocationResponseHandler.sendResponse(ResponseHandlerFactory.java:120)
> at
> com.hazelcast.spi.impl.BasicOperationService.processOperation(BasicOperationService.java:385)
> at
> com.hazelcast.spi.impl.BasicOperationService.processPacket(BasicOperationService.java:309)
> at
> com.hazelcast.spi.impl.BasicOperationService.access$400(BasicOperationService.java:102)
> at
> com.hazelcast.spi.impl.BasicOperationService$BasicOperationProcessorImpl.process(BasicOperationService.java:756)
> at
> com.hazelcast.spi.impl.BasicOperationScheduler$ProcessTask.run(BasicOperationScheduler.java:186)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> at
> com.hazelcast.util.executor.PoolExecutorThreadFactory$ManagedThread.run(PoolExecutorThreadFactory.java:59)
> Caused by: java.lang.NullPointerException
> at com.hazelcast.map.MapEntrySet.writeData(MapEntrySet.java:60)
> at
> com.hazelcast.nio.serialization.DataSerializer.write(DataSerializer.java:129)
> at
> com.hazelcast.nio.serialization.DataSerializer.write(DataSerializer.java:36)
> at
> com.hazelcast.nio.serialization.StreamSerializerAdapter.write(StreamSerializerAdapter.java:36)
> at
> com.hazelcast.nio.serialization.SerializationServiceImpl.writeObject(SerializationServiceImpl.java:252)
> ... 25 more
> TID: [0] [AS] [2015-03-17 04:52:41,699] INFO
> {org.wso2.carbon.core.clustering.hazelcast.wka.WKABasedMembershipScheme} -
> Member joined [83db16fc-65be-41de-9750-36fb5dd6f6d7]: /192.168.48.91:4000
> {org.wso2.carbon.core.clustering.hazelcast.wka.WKABasedMembershipScheme}
>
>
>
> --
> Asanka Vithanage
> Senior Software Engineer -QA
> Mobile: +94 0716286708
> Email: asan...@wso2.com
> WSO2 Inc. www.wso2.com
>
>
--
Asanka Vithanage
Senior Software Engineer -QA
Mobile: +94 0716286708
Email: asan...@wso2.com
WSO2 Inc. www.wso2.com
_______________________________________________
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev
