Re: Solrcloud Index corruption

2015-03-12 Thread Martin de Vries
Ahhh, ok. When you reloaded the cores, did you do it core-by-core? Yes, but maybe we reloaded the wrong core or something like that. We also noticed that the startTime doesn't update in the admin-ui while switching between cores (you have to reload the page). We still use 4.8.1, so maybe it

Re: Solrcloud Index corruption

2015-03-10 Thread Martin de Vries
Hi, this _sounds_ like you somehow don't have indexed=true set for the field in question. We investigated a lot more. The CheckIndex tool didn't find any error. We now think the following happened: - We changed the schema two months ago: we changed a field to indexed=true. We reloaded the

Re: Solrcloud Index corruption

2015-03-10 Thread Erick Erickson
Ahhh, ok. When you reloaded the cores, did you do it core-by-core? I can see how something could get dropped in that case. However, if you used the Collections API and two cores mysteriously failed to reload that would be a bug. Assuming the replicas in question were up and running at the time

Re: Solrcloud Index corruption

2015-03-06 Thread Erick Erickson
bq: You say in our case some docs didn't made it to the node, but that's not really true: the docs can be found on the corrupted nodes when I search on ID. The docs are also complete. The problem is that the docs do not appear when I filter on certain fields this _sounds_ like you somehow don't

Re: Solrcloud Index corruption

2015-03-05 Thread Martin de Vries
Hi Erick, Thank you for your detailed reply. You say in our case some docs didn't made it to the node, but that's not really true: the docs can be found on the corrupted nodes when I search on ID. The docs are also complete. The problem is that the docs do not appear when I filter on certain

RE: Solrcloud Index corruption

2015-03-05 Thread Garth Grimm
Subject: Re: Solrcloud Index corruption Hi Erick, Thank you for your detailed reply. You say in our case some docs didn't made it to the node, but that's not really true: the docs can be found on the corrupted nodes when I search on ID. The docs are also complete. The problem is that the docs do

Re: Solrcloud Index corruption

2015-03-05 Thread Mark Miller
to the leader of the appropriate shard, no matter what server first receives the request. -Original Message- From: Martin de Vries [mailto:mar...@downnotifier.com] Sent: Thursday, March 05, 2015 4:14 PM To: solr-user@lucene.apache.org Subject: Re: Solrcloud Index corruption Hi Erick

Re: Solrcloud Index corruption

2015-03-05 Thread Shawn Heisey
On 3/5/2015 3:13 PM, Martin de Vries wrote: I understand there is not a master in SolrCloud. In our case we use haproxy as a load balancer for every request. So when indexing every document will be sent to a different solr server, immediately after each other. Maybe SolrCloud is not able to

Re: Solrcloud Index corruption

2015-03-05 Thread Erick Erickson
Wait up. There's no master index in SolrCloud. Raw documents are forwarded to each replica, indexed and put in the local tlog. If a replica falls too far out of synch (say you take it offline), then the entire index _can_ be replicated from the leader and, if the leader's index was incomplete then

Solrcloud Index corruption

2015-03-05 Thread Martin de Vries
Hi, We have index corruption on some cores on our Solrcloud running version 4.8.1. The index is corrupt on several servers. (for example: when we do an fq search we get results on some servers, on other servers we don't, while the stored document contains the field on all servers). A full

Re: Solrcloud Index corruption

2015-03-05 Thread Andrew Butkus
We had a similar issue, when this happened we did a fetch index on each core out of sync to put them back right again Sent from my iPhone On 5 Mar 2015, at 14:40, Martin de Vries mar...@downnotifier.com wrote: Hi, We have index corruption on some cores on our Solrcloud running version

RE: Solrcloud Index corruption

2015-03-05 Thread Andrew Butkus
Subject: Re: Solrcloud Index corruption We had a similar issue, when this happened we did a fetch index on each core out of sync to put them back right again Sent from my iPhone On 5 Mar 2015, at 14:40, Martin de Vries mar...@downnotifier.com wrote: Hi, We have index corruption on some

RE: Solrcloud Index corruption

2015-03-05 Thread Martin de Vries
Hi Andrew, Even our master index is corrupt, so I'm afraid this won't help in our case. Martin Andrew Butkus schreef op 05.03.2015 16:45: Force a fetchindex on slave from master command: http://slave_host:port/solr/replication?command=fetchindex - from