[
https://issues.apache.org/jira/browse/LUCENE-2793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12980625#action_12980625
]
Shai Erera commented on LUCENE-2793:
------------------------------------
bq. No you dont... you can also call the .setBufferSize like the skiplist
reader does.
I'd still need to impl my Directory though right? That's the overkill I'm
trying to avoid.
But, I've been thinking about how would a merge/search code, which can only
tell the Directory it's in SEARCH / MERGE context, get the buffer size the
application wanted to use in that context. I don't think it has a way to do so
without either using a static class, something we try to avoid, or propagating
those settings down everywhere, which does not make sense either.
So, Robert, I think you're right -- bufferSize should not exist on IOContext. A
custom Directory impl seems unavoided. So I think it'd be good if we can create
a BufferedDirectoryWrapper which wraps a Directory and offers a
BufferedIndexInput/OutputWrapper which delegate the necessary calls to the
wrapped Directory. We've found ourselves needing to implement that kind of
Directory several times already, and it'd be good if Lucene can offer one.
That Directory would then take IOContext -> bufferSize map/properties/whatever
and can take that into account in openInput/createOutput.
If users will need to impl Directory wrapping, if they want to control buffer
sizes, I suggest we make that as painless as possible.
> Directory createOutput and openInput should take an IOContext
> -------------------------------------------------------------
>
> Key: LUCENE-2793
> URL: https://issues.apache.org/jira/browse/LUCENE-2793
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Store
> Reporter: Michael McCandless
> Attachments: LUCENE-2793.patch
>
>
> Today for merging we pass down a larger readBufferSize than for searching
> because we get better performance.
> I think we should generalize this to a class (IOContext), which would hold
> the buffer size, but then could hold other flags like DIRECT (bypass OS's
> buffer cache), SEQUENTIAL, etc.
> Then, we can make the DirectIOLinuxDirectory fully usable because we would
> only use DIRECT/SEQUENTIAL during merging.
> This will require fixing how IW pools readers, so that a reader opened for
> merging is not then used for searching, and vice/versa. Really, it's only
> all the open file handles that need to be different -- we could in theory
> share del docs, norms, etc, if that were somehow possible.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
