Re: Data from deleted from Solr (Solr cloud)
Thanks Mark. Unfortunately I only took log file backup of the Solr instance which failed. Even if we stopped and started all instances in a shard at same time, isn't there a possibility that the Solr Instance with less data (or no data) might declare itself leader and the one with the data will start copying the empty index folder from the leader? We tried using the latest branch_4x few weeks back and hit the SOLR-4144 <https://issues.apache.org/jira/browse/SOLR-4144> issue. Although it was fixed in a day, we decided to use the released version only (4.0) since we cannot have such extended periods of outage. We are looking forward to 4.1 and will start using it as soon as its released. --Shreejay -- View this message in context: http://lucene.472066.n3.nabble.com/Data-from-deleted-from-Solr-Solr-cloud-tp4028055p4029212.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Data from deleted from Solr (Solr cloud)
Hmm…have not seen a report of this before. For the future, FWIW, if you are in a bad state, the best thing to do is stop all the replicas in the shard and then start it again. That way everyone will work together to trade data and determine the best leader. When you restarted instance 2, instance 1 was already the leader - so 2 just replicates from him. If you stopped the replicas and started them, that is when the best leader is chosen. I'd also encourage you to try out a nightly build if you can - would be good to be sure whatever this problem is is fixed. It would be interesting if you could post the logs of the node that became leader without all of the docs. - Mark On Dec 19, 2012, at 11:08 AM, shreejay wrote: > Hi All, > > I have a solrlcoud instance with 3 shards. Each shard has 2 instance (2 > servers each running a instance of solr) > > Lets say I had Instance1 and instance2 in shard1 … At some point, instance2 > went down due to OOM (out of memory) . instance1 for some reason was not > replicating the data properly and when it became the leader, it had only > around 1% of the data that instance2 had. I restarted instance2, and hoped > that instance1 will replicate from 2, but instead instanace2 replicated from > instance1 . and ended up deleting the original index folder it had. There > were around 2 million documents in that instance. > > Can any one of solrlcoud users give any hints if I can recover this data? > > > > > --Shreejay > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Data-from-deleted-from-Solr-Solr-cloud-tp4028055.html > Sent from the Solr - User mailing list archive at Nabble.com.
Re: Data from deleted from Solr (Solr cloud)
Note that you don't have to re-compile, the latest successful build is always available, you can find the from here: http://wiki.apache.org/solr/NightlyBuilds FWIW, Erick On Thu, Dec 20, 2012 at 3:36 AM, shreejay wrote: > Thanks John. > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Data-from-deleted-from-Solr-Solr-cloud-tp4028055p4028252.html > Sent from the Solr - User mailing list archive at Nabble.com. >
Re: Data from deleted from Solr (Solr cloud)
Yeah, I ran into this issue myself with solr-4.0.0. To fix it, I had to compile my own version from the solr-4x branch. That is, I assume it's fixed as I have been unable to replicate it after the switch. I'm afraid you will have to reindex your data. -- Med venlig hilsen / Best regards *John Nielsen* Programmer *MCB A/S* Enghaven 15 DK-7500 Holstebro Kundeservice: +45 9610 2824 p...@mcb.dk www.mcb.dk On Wed, Dec 19, 2012 at 5:08 PM, shreejay wrote: > Hi All, > > I have a solrlcoud instance with 3 shards. Each shard has 2 instance (2 > servers each running a instance of solr) > > Lets say I had Instance1 and instance2 in shard1 … At some point, instance2 > went down due to OOM (out of memory) . instance1 for some reason was not > replicating the data properly and when it became the leader, it had only > around 1% of the data that instance2 had. I restarted instance2, and hoped > that instance1 will replicate from 2, but instead instanace2 replicated > from > instance1 . and ended up deleting the original index folder it had. There > were around 2 million documents in that instance. > > Can any one of solrlcoud users give any hints if I can recover this data? > > > > > --Shreejay > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Data-from-deleted-from-Solr-Solr-cloud-tp4028055.html > Sent from the Solr - User mailing list archive at Nabble.com. >
