On Wed, Nov 30, 2016 at 9:37 AM, Rob Audenaerde <rob.audenae...@gmail.com> wrote: > Thanks for the quick reply! > >>What do you mean by "Lucene complain about too-many uncommitted docs"? > > --> good question, I was thoughtlessly echoing words from my colleague. I > asked him and he said that it was about taking very long to commit and > memory issues. So maybe this wasn't the best opening statement :)
Ahhh. Well, fsync (which Lucene invokes on multiple files when you call commit) is inherently a costly (in wall clock time) operation since the OS and IO system may have to move many bytes to the underlying durable storage. But you should not hit memory issues. > For the other part of the question: we need users to see the changed > documents immediately, but I think we have this covered by using NRT Readers > and the SearcherManager. > > Am I correct to conclude calling commit() is not necessary for finding > recently changed documents? That's correct: NRT readers give you fast visibility of changes, while commit gives you durability. The two are entirely orthogonal in Lucene, as long as you don't use a single thread to call commit and call refresh. > I think we can then switch to a time based commit() where we just call > commit every 5 minutes, in effect losing a maximum of 5 minutes of work > (which we can mitigate in another way) > when the server somehow stops working. Exactly. > Thank you, You're welcome! Mike McCandless http://blog.mikemccandless.com --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
