[ https://issues.apache.org/jira/browse/LUCENE-2793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13041617#comment-13041617 ]
Michael McCandless commented on LUCENE-2793: -------------------------------------------- bq. If bean or copy-on-write builder I need to see what properties should be on it. so lets keep that for later. OK let's defer this decision for now. I just see "builder pattern" being too much of a hammer in search of nails... {quote} I really don't like the idea of exploiting OneMerge to the Directory. We might introduce some other internface that holds enough metadata to get the info and let OneMerge implement this but we should not pass it down to the dir. Same is true for SI I don't want directory mess around with those classes. {quote} I agree, it's spooky letting such a low level class (Dir) have access to high level stuff (OneMerge, SegmentInfo)... but it's really the only way to ensure we don't "miss" useful context about this file? Ie these high level classes already encompass the full context for a given file. So, if we really don't want to include these classes, and instead pick & choose what "digested" properties to include... then we risk some Dir impl not having access to something "interesting" because we forgot to include it, which means they'll have to hack something up, like NRTCachingDir does now in coupling threads to OneMerge instances. But, maybe the restrictions won't be so bad in practice... if we can try to brainstorm what Dirs may want to "know" about the full context and include that in IOCtx. For example, just by adding estimatedFullSegmentSizeBytes, for both Merge and Write (hmm: maybe Flush?), NRTCachingDir can cutover to IOCtx. > Directory createOutput and openInput should take an IOContext > ------------------------------------------------------------- > > Key: LUCENE-2793 > URL: https://issues.apache.org/jira/browse/LUCENE-2793 > Project: Lucene - Java > Issue Type: Improvement > Components: core/store > Reporter: Michael McCandless > Assignee: Simon Willnauer > Labels: gsoc2011, lucene-gsoc-11, mentor > Attachments: LUCENE-2793.patch, LUCENE-2793.patch, LUCENE-2793.patch, > LUCENE-2793.patch > > > Today for merging we pass down a larger readBufferSize than for searching > because we get better performance. > I think we should generalize this to a class (IOContext), which would hold > the buffer size, but then could hold other flags like DIRECT (bypass OS's > buffer cache), SEQUENTIAL, etc. > Then, we can make the DirectIOLinuxDirectory fully usable because we would > only use DIRECT/SEQUENTIAL during merging. > This will require fixing how IW pools readers, so that a reader opened for > merging is not then used for searching, and vice/versa. Really, it's only > all the open file handles that need to be different -- we could in theory > share del docs, norms, etc, if that were somehow possible. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org