Bravo Grant! Rajesh, I believe the following will work: - delete your small index - optimize your big index (needed? Not 100% sure, but I think it is) - loop through the docs in your "big" index - for each document in the big index, add a document to the small index
When you are done you have big+small with docIDs in sync. Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch ----- Original Message ---- > From: Grant Ingersoll <[EMAIL PROTECTED]> > To: java-user@lucene.apache.org > Sent: Wednesday, April 30, 2008 5:48:33 PM > Subject: Re: ParalleReader and synchronization between indexes > > Rajesh, > > You are asking a fairly complicated question on a seldom used piece of > functionality. Constantly pinging the list is just making it less > likely that someone will respond with an answer. The likelihood that > the 1 person who understand that code (and trust me, it really is > likely very few people who know how to practically employ it) enough > to give practical advice have read it in the time period you have > alloted us to respond is next to nil. We are all volunteers with day > jobs. > > Have you bothered to search the dev and user mailing list for > information on the class in question? I would look for threads from > Doug or Chuck Williams. > > -Grant > > > On Apr 30, 2008, at 5:00 PM, Rajesh parab wrote: > > > Hi Guys, > > > > Any comments on this? > > > > I was looking into Lucene archive and came across this > > thread what asks the same question. > > > > > http://www.gossamer-threads.com/lists/lucene/java-user/50477?search_string=parallelreader;#50477 > > > > Any pointers will be helpful. > > > > Regards, > > Rajesh > > > > --- Rajesh parab wrote: > > > >> Hi All, > >> > >> Any suggestions/comments on my questions in this > >> thread will be really helpful. > >> > >> We are planning to use Lucene indexes throughout the > >> application and exploring possibilites of > >> partitioning > >> data between multiple indexes. > >> > >> Regards, > >> Rajesh > >> > >> --- Rajesh parab wrote: > >> > >>> Hi, > >>> > >>> This is from javadoc of ParallelReader: > >>> > >>> > >> > > ====================================================== > >>> > >>> An IndexReader which reads multiple, parallel > >>> indexes. > >>> Each index added must have the same number of > >>> documents, but typically each contains different > >>> fields. Each document contains the union of the > >>> fields > >>> of all documents with the same document number. > >> When > >>> searching, matches for a query term are from the > >>> first > >>> index added that has the field. > >>> > >>> This is useful, e.g., with collections that have > >>> large > >>> fields which change rarely and small fields that > >>> change more frequently. The smaller fields may be > >>> re-indexed in a new index and both indexes may be > >>> searched together. > >>> > >>> > >> > > ====================================================== > >>> > >>> I have a similar use case as mentioned above and > >>> hence > >>> would like to use ParallelReader to search across > >>> multiple indexes. > >>> > >>> I have an object that has 50 fields. Out of these > >> 50 > >>> fields, 45 are relatively static and other 5 are > >>> modified very often. So, I am planning to > >> partition > >>> this objects data into 2 indexes such that 45 > >> static > >>> fields will be part of one index and remaining 5 > >>> dynamic fields will constitute second index. While > >>> generating the index for the first time, I can > >> make > >>> sure that the document order for documents inside > >>> both > >>> these indexes is same and hence ParallelReader > >> will > >>> work properly with it. > >>> > >>> The question is - > >>> What if the data inside second (smaller) index > >>> changes? In order to update index document, I will > >>> have to delete it and re-insert it again as Lucene > >>> does not support document update. This action (of > >>> delete and re-insert) will change internal > >> document > >>> id > >>> for updated document inside second index and in > >>> order > >>> to sync it with first index, I will have to also > >>> modify first (relatively big and static) index. If > >>> we > >>> will have to update both the indexes, how it is > >>> different from having a single index with all the > >>> fields? What is the use case in which > >> ParallelReader > >>> will get used? As per documentation, I was > >> thinking > >>> that it will apply for my use case, but > >>> synchronizing > >>> the indexes seems to be a problem. > >>> > >>> Please help. > >>> > >>> Regards, > >>> Rajesh > >>> > >>> > >>> > >>> > >>> > >> > > > ____________________________________________________________________________________ > >>> Be a better friend, newshound, and > >>> know-it-all with Yahoo! Mobile. Try it now. > >>> > >> > > http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ > >>> > >>> > >> > > --------------------------------------------------------------------- > >>> To unsubscribe, e-mail: > >>> [EMAIL PROTECTED] > >>> For additional commands, e-mail: > >>> [EMAIL PROTECTED] > >>> > >>> > >> > >> > >> > >> > >> > > > ____________________________________________________________________________________ > >> Be a better friend, newshound, and > >> know-it-all with Yahoo! Mobile. Try it now. > >> > > http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ > >> > >> > > --------------------------------------------------------------------- > >> To unsubscribe, e-mail: > >> [EMAIL PROTECTED] > >> For additional commands, e-mail: > >> [EMAIL PROTECTED] > >> > >> > > > > > > > > > > > ____________________________________________________________________________________ > > Be a better friend, newshound, and > > know-it-all with Yahoo! Mobile. Try it now. > http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > -------------------------- > Grant Ingersoll > > Lucene Helpful Hints: > http://wiki.apache.org/lucene-java/BasicsOfPerformance > http://wiki.apache.org/lucene-java/LuceneFAQ > > > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
