I'm still a bit confused about exactly what you're indexing, when, but if you have a unique id and don't want to add or update a doc that's already present, add the unique id to the index and search (TermQuery probably) for each one and skip if already present.
Can't you change the log rotation/copying/indexing so that you only index new data? To start a fresh index, use IndexWriterConfig.OpenMode.CREATE. -- Ian. On Tue, Oct 1, 2013 at 4:51 PM, gudiseashok <gudise.as...@gmail.com> wrote: > Hi > > Basically my log folder consists of four log files like > abc.log,abc1.log,abc2.log,abc3.log, as my log appender is doing. Every 30 > minutes content will be changed of all these file , for example after 30 > minutes refresh my conent of abc1.log will be replaced with existing abc.log > content and abc.log will have new content (Timestamp is DD-MM-YYYY MM-ss:S). > Since I am goingthrough the re-indexing for every 30 minutes, I dont want to > re-index the same record which was already present with same timstamp. > > Also if I want to do clean-up for every week, (clean up in the sense I want > to delete all indexes , and I want to do a fresh indexing for these 4 > files), how to do this efficiently. > > I really appreciate your time reading this, and kindly suggest a better way. > > > Regards > Ashok Gudise > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Rendexing-problem-Indexing-folder-size-is-keep-on-growing-for-same-remote-folder-tp4092835p4092963.html > Sent from the Lucene - Java Users mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
