[
https://issues.apache.org/jira/browse/LUCENE-3092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13032947#comment-13032947
]
Chris Male commented on LUCENE-3092:
------------------------------------
bq. I don't like events either. They look fragile and binding them to threads
is a WTF. With all our pausing/unpausing magic there's no guarantee merge will
end on the same thread it started on.
You're absolutely right that tying them to threads is not a good idea, but I
couldn't disagree more that this is an issue with an Event model. Yes we
should be wary of what information we make available in Events (I only added
Threads to the Events in my patch because I was showing a simple alternative to
Mike's initial impl). But to say Events are fragile and therefore we shouldn't
pursue a listener pattern seems a big leap. Many large applications use event
listeners and I don't think they're wrong.
bq. It goes downhill from there. What if FP and MP fire events simultaneously?
What should other listeners do?
Good point. Again, those are issues we need to resolve (one way applications
tend to address this is event queues).
bq. Try looking at a bigger picture. Merges are not your problem. Neither are
flushes.
You're right, neither merges nor flushes are the problem. We have a series of
components in Lucene; Directories, IndexWriter, MergeScheduler etc, and we have
some crosscutting concerns such as merges themselves. We should strive to
decouple these components where possible and think big picture.
I agree with your suggestions for how we can solve the problem here now for
NRTCachingDirectory, but I think we can do better for the longer term.
> NRTCachingDirectory, to buffer small segments in a RAMDir
> ---------------------------------------------------------
>
> Key: LUCENE-3092
> URL: https://issues.apache.org/jira/browse/LUCENE-3092
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Store
> Reporter: Michael McCandless
> Priority: Minor
> Fix For: 3.2, 4.0
>
> Attachments: LUCENE-3092-listener.patch, LUCENE-3092.patch
>
>
> I created this simply Directory impl, whose goal is reduce IO
> contention in a frequent reopen NRT use case.
> The idea is, when reopening quickly, but not indexing that much
> content, you wind up with many small files created with time, that can
> possibly stress the IO system eg if merges, searching are also
> fighting for IO.
> So, NRTCachingDirectory puts these newly created files into a RAMDir,
> and only when they are merged into a too-large segment, does it then
> write-through to the real (delegate) directory.
> This lets you spend some RAM to reduce I0.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
