[
https://issues.apache.org/jira/browse/LUCENE-5038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13679778#comment-13679778
]
Simon Willnauer commented on LUCENE-5038:
-----------------------------------------
I think we should make this two different settings to be honest! Frist of all I
think we should only call MP if we merge otherwise it's not a MP anymore?
Further I think it is important to not imply any place where we check this in
the name ie. if you have _flush_ in the name we should check on flush and
should not make the decision ahead of time which we might want to do in the
future. For instance you have set it to true and then set it to false
afterwards but we already checked the boolean and wrapped the Directory in a
CFSDirectory which might be confusing.
I actually think it's a good thing to have this in two different places since
those are two different usecases ie. for merges you might not want CFS at all
but for new writes you want it because NRT can flush / write lots of segments.
The defaults on MP are for merging and the defaults on IWC are for indexing.
Btw. I think _useCompoundFileOnWrite_ is a good fit?!
bq. How complicated is it to keep on MP only?
I'd really appreciate to move it out of MP. it's not a merge setting and I
think it's rather confusing today since we take the merge setting and use it
for indexing. Aside of this, I think this is pretty expert too. Adding a
boolean for this to MP would feel odd and I think it's the wrong thing as well.
> Don't call MergePolicy / IndexWriter during DWPT Flush
> ------------------------------------------------------
>
> Key: LUCENE-5038
> URL: https://issues.apache.org/jira/browse/LUCENE-5038
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/index
> Affects Versions: 4.3
> Reporter: Simon Willnauer
> Assignee: Simon Willnauer
> Fix For: 4.4
>
> Attachments: LUCENE-5038.patch, LUCENE-5038.patch, LUCENE-5038.patch
>
>
> We currently consult the indexwriter -> merge policy to decide if we need to
> write CFS or not which is bad in many ways.
> - we should call mergepolicy only during merges
> - we should never sync on IW during DWPT flush
> - we should be able to make the decision if we need to write CFS or not
> before flush, ie. we could write parts of the flush directly to CFS or even
> start writing stored fields directly.
> - in the NRT case it might make sense to write all flushes to CFS to minimize
> filedescriptors independent of the index size.
> I wonder if we can use a simple boolean for this in the IWC and get away with
> not consulting merge policy. This would simplify concurrency a lot here
> already.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
