Hi Hemika, Does this happen at a specific moment when new nodes are joining etc. ? or purely random ? This could as well be a thrift connection breakup issue too.
Thanks On Thu, Mar 19, 2015 at 5:16 PM, Hemika Kodikara <[email protected]> wrote: > Hi All, > > I have setup a 3 MB node cluster with 3 datastax cassandra message stores > each on EC2 instances. > > When starting up the 2nd node of MB, I am getting the following error (The > error is coming from time to time, I believe its coming from a timer.) : > > TID: [] [] [2015-03-19 11:26:09,746] ERROR > {org.wso2.andes.thrift.MBThriftClient} - Error while sending slot deletion > safe zone update {org.wso2.andes.thrift.MBThriftClient} > org.apache.thrift.transport.TTransportException > at > org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132) > at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84) > at > org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378) > at > org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297) > at > org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204) > at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69) > at > org.wso2.andes.thrift.slot.gen.SlotManagementService$Client.recv_updateCurrentMessageIdForSafeZone(SlotManagementService.java:183) > at > org.wso2.andes.thrift.slot.gen.SlotManagementService$Client.updateCurrentMessageIdForSafeZone(SlotManagementService.java:169) > at > org.wso2.andes.thrift.MBThriftClient.updateSlotDeletionSafeZone(MBThriftClient.java:323) > at > org.wso2.andes.kernel.slot.SlotCoordinatorCluster.updateSlotDeletionSafeZone(SlotCoordinatorCluster.java:61) > at > org.wso2.andes.kernel.slot.SlotMessageCounter.submitCurrentSafeZone(SlotMessageCounter.java:134) > at > org.wso2.andes.kernel.slot.SlotMessageCounter.access$500(SlotMessageCounter.java:36) > at > org.wso2.andes.kernel.slot.SlotMessageCounter$1.run(SlotMessageCounter.java:99) > at java.util.TimerThread.mainLoop(Timer.java:555) > at java.util.TimerThread.run(Timer.java:505) > TID: [] [] [2015-03-19 11:26:22,357] INFO > {org.wso2.carbon.core.clustering.hazelcast.wka.WKABasedMembershipScheme} - > Member joined [8c25be13-8649-4826-a54a-cd881aaf07d8]: /10.23.156.141:4000 > {org.wso2.carbon.core.clustering.hazelcast.wka.WKABasedMembershipScheme} > TID: [] [] [2015-03-19 11:26:22,357] INFO > {org.wso2.andes.server.cluster.coordination.hazelcast.AndesMembershipListener} > - Handling cluster gossip: New member joined to the cluster. Member Socket > Address:/10.23.156.141:4000 UUID:8c25be13-8649-4826-a54a-cd881aaf07d8 > {org.wso2.andes.server.cluster.coordination.hazelcast.AndesMembershipListener} > TID: [] [] [2015-03-19 11:26:22,357] INFO > {org.wso2.andes.server.cluster.ClusterManager} - This node is elected as > the Slot Coordinator. Registering 10.23.10.50:7612 > {org.wso2.andes.server.cluster.ClusterManager} > TID: [] [] [2015-03-19 11:26:39,745] ERROR > {org.wso2.andes.thrift.MBThriftClient} - Error while sending slot deletion > safe zone update {org.wso2.andes.thrift.MBThriftClient} > org.apache.thrift.transport.TTransportException: java.net.SocketException: > Broken pipe > at > org.apache.thrift.transport.TIOStreamTransport.write(TIOStreamTransport.java:147) > at > org.apache.thrift.protocol.TBinaryProtocol.writeI32(TBinaryProtocol.java:163) > at > org.apache.thrift.protocol.TBinaryProtocol.writeMessageBegin(TBinaryProtocol.java:91) > at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:62) > at > org.wso2.andes.thrift.slot.gen.SlotManagementService$Client.send_updateCurrentMessageIdForSafeZone(SlotManagementService.java:177) > at > org.wso2.andes.thrift.slot.gen.SlotManagementService$Client.updateCurrentMessageIdForSafeZone(SlotManagementService.java:168) > at > org.wso2.andes.thrift.MBThriftClient.updateSlotDeletionSafeZone(MBThriftClient.java:323) > at > org.wso2.andes.kernel.slot.SlotCoordinatorCluster.updateSlotDeletionSafeZone(SlotCoordinatorCluster.java:61) > at > org.wso2.andes.kernel.slot.SlotMessageCounter.submitCurrentSafeZone(SlotMessageCounter.java:134) > at > org.wso2.andes.kernel.slot.SlotMessageCounter.access$500(SlotMessageCounter.java:36) > at > org.wso2.andes.kernel.slot.SlotMessageCounter$1.run(SlotMessageCounter.java:99) > at java.util.TimerThread.mainLoop(Timer.java:555) > at java.util.TimerThread.run(Timer.java:505) > Caused by: java.net.SocketException: Broken pipe > at java.net.SocketOutputStream.socketWrite0(Native Method) > at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:113) > at java.net.SocketOutputStream.write(SocketOutputStream.java:159) > at > org.apache.thrift.transport.TIOStreamTransport.write(TIOStreamTransport.java:145) > ... 12 more > TID: [] [] [2015-03-19 11:27:09,745] ERROR > {org.wso2.andes.thrift.MBThriftClient} - Error while sending slot deletion > safe zone update {org.wso2.andes.thrift.MBThriftClient} > org.apache.thrift.transport.TTransportException: java.net.SocketException: > Broken pipe > at > org.apache.thrift.transport.TIOStreamTransport.write(TIOStreamTransport.java:147) > at > org.apache.thrift.protocol.TBinaryProtocol.writeI32(TBinaryProtocol.java:163) > at > org.apache.thrift.protocol.TBinaryProtocol.writeMessageBegin(TBinaryProtocol.java:91) > at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:62) > at > org.wso2.andes.thrift.slot.gen.SlotManagementService$Client.send_updateCurrentMessageIdForSafeZone(SlotManagementService.java:177) > at > org.wso2.andes.thrift.slot.gen.SlotManagementService$Client.updateCurrentMessageIdForSafeZone(SlotManagementService.java:168) > at > org.wso2.andes.thrift.MBThriftClient.updateSlotDeletionSafeZone(MBThriftClient.java:323) > at > org.wso2.andes.kernel.slot.SlotCoordinatorCluster.updateSlotDeletionSafeZone(SlotCoordinatorCluster.java:61) > at > org.wso2.andes.kernel.slot.SlotMessageCounter.submitCurrentSafeZone(SlotMessageCounter.java:134) > at > org.wso2.andes.kernel.slot.SlotMessageCounter.access$500(SlotMessageCounter.java:36) > at > org.wso2.andes.kernel.slot.SlotMessageCounter$1.run(SlotMessageCounter.java:99) > at java.util.TimerThread.mainLoop(Timer.java:555) > at java.util.TimerThread.run(Timer.java:505) > Caused by: java.net.SocketException: Broken pipe > at java.net.SocketOutputStream.socketWrite0(Native Method) > at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:113) > at java.net.SocketOutputStream.write(SocketOutputStream.java:159) > at > org.apache.thrift.transport.TIOStreamTransport.write(TIOStreamTransport.java:145) > ... 12 more > > Any idea what can be done ? The pack was taken from jenkins. > > Regards, > Hemika > > > Hemika Kodikara > Software Engineer > WSO2 Inc. > lean . enterprise . middleware > http://wso2.com > > Mobile : +94777688882 > -- Cheers, Hasitha Amal De Silva Software Engineer Mobile : 0772037426 Blog : http://devnutshell.tumblr.com/ WSO2 Inc.: http://wso2.com ( lean.enterprise.middleware. )
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
