Re: what to try next for replica that will not stay up.

2016-09-01 Thread Shalin Shekhar Mangar
My guess is that recovery is indeed successful but the leader is repeatedly marking this replicas as 'down' using what we call Leader-Initiated-Recovery or LIR. We need to understand why that is happening. Are there GC issues on this new node? Can we see the logs on the leader from a bit before

RE: what to try next for replica that will not stay up.

2016-09-01 Thread Jon Hawkesworth
Well, thanks for your suggestion, it may well come to that. My impression the problem is with the shard in some way in that we have created the replica on other nodes and it has done the same thing. We went from 2 nodes to 4 to add processing capacity to the cluster a couple of weeks back, but

Re: what to try next for replica that will not stay up.

2016-09-01 Thread John Bickerstaff
This may be too simplistic of course, but what if you totally wipe Solr off that machine, re-install it from scratch, bring it up and let it join the cluster, then add the replica? If it was me (since I have the luxury of VM's) I'd turn it off and build a new VM from the ground up, but I get that

what to try next for replica that will not stay up.

2016-09-01 Thread Jon Hawkesworth
Hi If anyone has any suggestions of things I could try to resolve my issue where one replica on one of my solcloud 6.0.1 shards refuses to stay up, I'd love to hear them. In fact, I'll get you something off your amazon wishlist, within reason, if you can solve this puzzle. Today we pruned