[jira] [Commented] (LUCENE-2955) Add utitily class to manage NRT reopening
[ https://issues.apache.org/jira/browse/LUCENE-2955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13046644#comment-13046644 ] Jason Rutherglen commented on LUCENE-2955: -- Perhaps we can merge this functionality with SOLR-2565 and/or SOLR-2566, such that Solr utilizes it for reader opening. However why would this issue use a background thread and Solr performs a max time reopen? > Add utitily class to manage NRT reopening > - > > Key: LUCENE-2955 > URL: https://issues.apache.org/jira/browse/LUCENE-2955 > Project: Lucene - Java > Issue Type: Improvement > Components: core/index >Reporter: Michael McCandless >Assignee: Michael McCandless > Fix For: 3.3 > > Attachments: LUCENE-2955.patch, LUCENE-2955.patch > > > I created a simple class, NRTManager, that tries to abstract away some > of the reopen logic when using NRT readers. > You give it your IW, tell it min and max nanoseconds staleness you can > tolerate, and it privately runs a reopen thread to periodically reopen > the searcher. > It subsumes the SearcherManager from LIA2. Besides running the reopen > thread, it also adds the notion of a "generation" containing changes > you've made. So eg it has addDocument, returning a long. You can > then take that long value and pass it back to the getSearcher method > and getSearcher will return a searcher that reflects the changes made > in that generation. > This gives your app the freedom to force "immediate" consistency (ie > wait for the reopen) only for those searches that require it, like a > verifier that adds a doc and then immediately searches for it, but > also use "eventual consistency" for other searches. > I want to also add support for the new "applyDeletions" option when > pulling an NRT reader. > Also, this is very new and I'm sure buggy -- the concurrency is either > wrong over overly-locking. But it's a start... -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-2955) Add utitily class to manage NRT reopening
[ https://issues.apache.org/jira/browse/LUCENE-2955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13046518#comment-13046518 ] Chris Male commented on LUCENE-2955: I agree with Simon. I think providing a ReopenStrategy abstraction will be helpful. > Add utitily class to manage NRT reopening > - > > Key: LUCENE-2955 > URL: https://issues.apache.org/jira/browse/LUCENE-2955 > Project: Lucene - Java > Issue Type: Improvement > Components: core/index >Reporter: Michael McCandless >Assignee: Michael McCandless > Fix For: 3.3 > > Attachments: LUCENE-2955.patch, LUCENE-2955.patch > > > I created a simple class, NRTManager, that tries to abstract away some > of the reopen logic when using NRT readers. > You give it your IW, tell it min and max nanoseconds staleness you can > tolerate, and it privately runs a reopen thread to periodically reopen > the searcher. > It subsumes the SearcherManager from LIA2. Besides running the reopen > thread, it also adds the notion of a "generation" containing changes > you've made. So eg it has addDocument, returning a long. You can > then take that long value and pass it back to the getSearcher method > and getSearcher will return a searcher that reflects the changes made > in that generation. > This gives your app the freedom to force "immediate" consistency (ie > wait for the reopen) only for those searches that require it, like a > verifier that adds a doc and then immediately searches for it, but > also use "eventual consistency" for other searches. > I want to also add support for the new "applyDeletions" option when > pulling an NRT reader. > Also, this is very new and I'm sure buggy -- the concurrency is either > wrong over overly-locking. But it's a start... -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-2955) Add utitily class to manage NRT reopening
[ https://issues.apache.org/jira/browse/LUCENE-2955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13046496#comment-13046496 ] Simon Willnauer commented on LUCENE-2955: - Mike, nice work so far :) I have to admit that I really don't like the reopen thread. I think reopen in the background should be abstracted and the Reopen thread should not be part of the core manager. By default I think we should consult a ReopenStrategy on change and hijack indexing threads to reopen the reader. we can still sychronized the reopeing with a lock.tryLock() and by default go with a timed reopen policy. Thoughts? simon > Add utitily class to manage NRT reopening > - > > Key: LUCENE-2955 > URL: https://issues.apache.org/jira/browse/LUCENE-2955 > Project: Lucene - Java > Issue Type: Improvement > Components: core/index >Reporter: Michael McCandless >Assignee: Michael McCandless > Fix For: 3.3 > > Attachments: LUCENE-2955.patch, LUCENE-2955.patch > > > I created a simple class, NRTManager, that tries to abstract away some > of the reopen logic when using NRT readers. > You give it your IW, tell it min and max nanoseconds staleness you can > tolerate, and it privately runs a reopen thread to periodically reopen > the searcher. > It subsumes the SearcherManager from LIA2. Besides running the reopen > thread, it also adds the notion of a "generation" containing changes > you've made. So eg it has addDocument, returning a long. You can > then take that long value and pass it back to the getSearcher method > and getSearcher will return a searcher that reflects the changes made > in that generation. > This gives your app the freedom to force "immediate" consistency (ie > wait for the reopen) only for those searches that require it, like a > verifier that adds a doc and then immediately searches for it, but > also use "eventual consistency" for other searches. > I want to also add support for the new "applyDeletions" option when > pulling an NRT reader. > Also, this is very new and I'm sure buggy -- the concurrency is either > wrong over overly-locking. But it's a start... -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
