Right, this is expected behavior. The real problem isn't data loss, but how do you know which doc should "win"? Merging indexes is for a rather narrowly-defined use-case, it was never intended to remove duplicates.
Best, Erick On Tue, Apr 8, 2014 at 12:36 AM, Cihad Guzel <cguz...@gmail.com> wrote: > Hi. > > I have encountered a similar situation when I tested solr merge index . ( > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201403.mbox/%3CCAMrn6cOVWohxooRzZ8NmwYQUda2GW+gYD+edvC_b_kGT=f4...@mail.gmail.com%3E > ) > > I have had duplicates. But the duplicates are gone when I post same data > for indexing. I think this was done in order to prevent data loss while > merging index. > > > > > 2014-04-07 23:04 GMT+03:00 Erick Erickson <erickerick...@gmail.com>: > >> Oh my yes! I feel a great sense of relief every time an intermittent >> problem becomes reproducible... The problem is not solved, but at >> least I have a good feeling that once I don't see it any more it's >> _really_ gone! >> >> One possibility is index merging, see: >> https://wiki.apache.org/solr/MergingSolrIndexes. When you merge >> indexes, there is no duplicate id checking performed, so you can well >> have duplicates. That's a wild shot in the dark though. >> >> Best, >> Erick >> >> On Mon, Apr 7, 2014 at 12:26 PM, Simon <sim...@gallerysystems.com> wrote: >> > Erick, >> > >> > It's indeed quite odd. And after I trigger re-indexing all documents >> (via >> > the normal process of existing program). The duplication is gone. It can >> > not be reproduced easily. But it did occur occasionally and that makes >> it a >> > frustrating task to troubleshoot. >> > >> > Thanks, >> > Simon >> > >> > >> > >> > -- >> > View this message in context: >> http://lucene.472066.n3.nabble.com/Duplicate-Unique-Key-tp4129651p4129701.html >> > Sent from the Solr - User mailing list archive at Nabble.com. >>
