Re: Soft commits and segments

2015-03-30 Thread Robert Muir
Yes, pending docs buffered in indexwriter are flushed when you reopen.

one way to avoid revising your model is to use NRTCachingDirectory.
When properly configured this can defer the writes to the filesystem
until they truly really need to be there, e.g. indexwriter.commit.

On Mon, Mar 30, 2015 at 7:55 PM, Erick Erickson erickerick...@gmail.com wrote:
 Is it the expected case that when a soft commit happens, a new segment
 is opened? 'Cause that's what I'm seeing. Thinking about it I don't
 see how Lucene could successfully MMap the underlying disk files if
 new segments weren't opened, and if they were all held in Java's
 memory BOOM (that as a Big OOM).

 So I'm guessing at this point that I need to revise my model from
 soft commits do not write new segments to soft commits do not write
 new segments _durably_.

 Thanks,
 Erick

 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



Re: Soft commits and segments

2015-03-30 Thread Erick Erickson
Robert, thanks a bunch, your comment got me looking in the right place
I'm seeing what I expect now. I can at least try to explain it without
stumbling too much..

Thanks again!
Erick

On Mon, Mar 30, 2015 at 5:24 PM, Robert Muir rcm...@gmail.com wrote:
 Yes, pending docs buffered in indexwriter are flushed when you reopen.

 one way to avoid revising your model is to use NRTCachingDirectory.
 When properly configured this can defer the writes to the filesystem
 until they truly really need to be there, e.g. indexwriter.commit.

 On Mon, Mar 30, 2015 at 7:55 PM, Erick Erickson erickerick...@gmail.com 
 wrote:
 Is it the expected case that when a soft commit happens, a new segment
 is opened? 'Cause that's what I'm seeing. Thinking about it I don't
 see how Lucene could successfully MMap the underlying disk files if
 new segments weren't opened, and if they were all held in Java's
 memory BOOM (that as a Big OOM).

 So I'm guessing at this point that I need to revise my model from
 soft commits do not write new segments to soft commits do not write
 new segments _durably_.

 Thanks,
 Erick

 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org


 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org