how to use RemoveDuplicatesTokenFilterFactory?

2012-12-25 Thread vrpar...@gmail.com
? -- View this message in context: http://lucene.472066.n3.nabble.com/how-to-use-RemoveDuplicatesTokenFilterFactory-tp4029004.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: how to use RemoveDuplicatesTokenFilterFactory?

2012-12-25 Thread Ahmet Arslan
I want to avoid duplicate values in one multivalued field. i am using dataimport handler to import data,  the particular multivalued field are being filled from xml source. now that xml has duplicate values, but i want to have unique valued in this multivalued field. e.g. xml data    

Re: how to use RemoveDuplicatesTokenFilterFactory?

2012-12-25 Thread Ahmet Arslan
The values are at same logical position. You mean positionIncrementGap set to 0? can you see that duplicates are removed in analysis page? By the way returned values are original (stored) values. Analysis (tokenfilter tokenizer etc) are about indexed values. UpdateProcessorFactory can change