[ https://issues.apache.org/jira/browse/LUCENE-2793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12980551#action_12980551 ]
Shai Erera commented on LUCENE-2793: ------------------------------------ If we drop bufferSize, how will I be able to tell Lucene I'm willing to spare, say, 1 MB buffer for this IndexInput/Output? Am I supposed to create my own Directory and open II/IO, depending on the IOContext.Mode and decide on the buffer size then? If so, then I don't understand how this would work (think that's what Jason is asking) - if I impl MyDirectory (extending Directory? FSDirectory?) which is probably going to be a wrapper Directory, what II/IO impl should I invoke? I'll need to extend BufferedIndexInput/Output, impl its abstract methods, just for delegating to the wrapped Directory's II/IO? If I misunderstood the intentions, I'd appreciate if you can clarify them. But if not, I think IOContext should include a bufferSize hint. If the Directory does not intend to do anything special for 'merging', then it can take bufferSize into account. If however it's a Linux Directory that wants to set the O_DIRECT, then it can ignore bufferSize. But I think it's a useful hint. > Directory createOutput and openInput should take an IOContext > ------------------------------------------------------------- > > Key: LUCENE-2793 > URL: https://issues.apache.org/jira/browse/LUCENE-2793 > Project: Lucene - Java > Issue Type: Improvement > Components: Store > Reporter: Michael McCandless > Attachments: LUCENE-2793.patch > > > Today for merging we pass down a larger readBufferSize than for searching > because we get better performance. > I think we should generalize this to a class (IOContext), which would hold > the buffer size, but then could hold other flags like DIRECT (bypass OS's > buffer cache), SEQUENTIAL, etc. > Then, we can make the DirectIOLinuxDirectory fully usable because we would > only use DIRECT/SEQUENTIAL during merging. > This will require fixing how IW pools readers, so that a reader opened for > merging is not then used for searching, and vice/versa. Really, it's only > all the open file handles that need to be different -- we could in theory > share del docs, norms, etc, if that were somehow possible. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org