Re: Send kill -9 to a node and can not delete down replicas with onlyIfDown.

2016-07-29 Thread Chris Hostetter
> onlyIfDown=false to indicate that you know the replica you are deleting : > isn't 'down' intentionally, but you want do delete it anyway. : > : > : > : > : > : > On Wed, 20 Jul 2016, Erick Erickson wrote: : > : > : Date: Wed, 20 Jul 2016 08:26:32 -0700 : >

Re: Send kill -9 to a node and can not delete down replicas with onlyIfDown.

2016-07-20 Thread Jerome Yang
t; either: a) try to bring the replica back up; b) delete the replica using > onlyIfDown=false to indicate that you know the replica you are deleting > isn't 'down' intentionally, but you want do delete it anyway. > > > > > > On Wed, 20 Jul 2016, Erick Erick

Re: Send kill -9 to a node and can not delete down replicas with onlyIfDown.

2016-07-20 Thread Chris Hostetter
hat you know the replica you are deleting isn't 'down' intentionally, but you want do delete it anyway. On Wed, 20 Jul 2016, Erick Erickson wrote: : Date: Wed, 20 Jul 2016 08:26:32 -0700 : From: Erick Erickson : Reply-To: solr-user@lucene.apache.org : To: solr-user : Subject: R

Re: Send kill -9 to a node and can not delete down replicas with onlyIfDown.

2016-07-20 Thread Erick Erickson
Yes, it's the intended behavior. The whole point of the onlyIfDown flag was as a safety valve for those who wanted to be cautious and guard against typos and the like. If you specify onlyIfDown=false and the node still isn't removed from ZK, it's not right. Best, Erick On Tue, Jul 19, 2016 at 10

Re: Send kill -9 to a node and can not delete down replicas with onlyIfDown.

2016-07-19 Thread Jerome Yang
What I'm doing is to simulate host crashed situation. Consider this, a host is not connected to the cluster. So, if a host crashed, I can not delete the down replicas by using onlyIfDown='true'. But in solr admin ui, it shows down for these replicas. And whiteout "onlyIfDown", it still show a fai

Re: Send kill -9 to a node and can not delete down replicas with onlyIfDown.

2016-07-19 Thread Justin Lee
Thanks for taking the time for the detailed response. I completely get what you are saying. Makes sense. On Tue, Jul 19, 2016 at 10:56 AM Erick Erickson wrote: > Justin: > > Well, "kill -9" just makes it harder. The original question > was whether a replica being "active" was a bug, and it's > no

Re: Send kill -9 to a node and can not delete down replicas with onlyIfDown.

2016-07-19 Thread Erick Erickson
Justin: Well, "kill -9" just makes it harder. The original question was whether a replica being "active" was a bug, and it's not when you kill -9; the Solr node has no chance to tell Zookeeper it's going away. ZK does modify the live_nodes by itself, thus there are checks as necessary when a repli

Re: Send kill -9 to a node and can not delete down replicas with onlyIfDown.

2016-07-19 Thread Justin Lee
Pardon me for hijacking the thread, but I'm curious about something you said, Erick. I always thought that the point (in part) of going through the pain of using zookeeper and creating replicas was so that the system could seamlessly recover from catastrophic failures. Wouldn't an OOM condition h

Re: Send kill -9 to a node and can not delete down replicas with onlyIfDown.

2016-07-19 Thread Erick Erickson
First of all, killing with -9 is A Very Bad Idea. You can leave write lock files laying around. You can leave the state in an "interesting" place. You haven't given Solr a chance to tell Zookeeper that it's going away. (which would set the state to "down"). In short when you do this you have to dea

Send kill -9 to a node and can not delete down replicas with onlyIfDown.

2016-07-18 Thread Jerome Yang
Hi all, Here's the situation. I'm using solr5.3 in cloud mode. I have 4 nodes. After use "kill -9 pid-solr-node" to kill 2 nodes. These replicas in the two nodes still are "ACTIVE" in zookeeper's state.json. The problem is, when I try to delete these down replicas with parameter onlyIfDown='tru