[
https://issues.apache.org/jira/browse/LUCENE-5123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13773028#comment-13773028
]
Robert Muir commented on LUCENE-5123:
-------------------------------------
The only reason merge() exists there is so they can implement some "bulk
merging" optimizations?
Can we remove these optimizations? Has there ever been a benchmark showing they
help at all?
We shouldnt have such scary code in lucene because it "looks faster". Every
time I look at infostreams from merge, its completely dominated by postings and
other things.
> invert the codec postings API
> -----------------------------
>
> Key: LUCENE-5123
> URL: https://issues.apache.org/jira/browse/LUCENE-5123
> Project: Lucene - Core
> Issue Type: Wish
> Reporter: Robert Muir
> Assignee: Michael McCandless
> Fix For: 5.0
>
> Attachments: LUCENE-5123.patch, LUCENE-5123.patch, LUCENE-5123.patch,
> LUCENE-5123.patch, LUCENE-5123.patch
>
>
> Currently FieldsConsumer/PostingsConsumer/etc is a "push" oriented api, e.g.
> FreqProxTermsWriter streams the postings at flush, and the default merge()
> takes the incoming codec api and filters out deleted docs and "pushes" via
> same api (but that can be overridden).
> It could be cleaner if we allowed for a "pull" model instead (like
> DocValues). For example, maybe FreqProxTermsWriter could expose a Terms of
> itself and just passed this to the codec consumer.
> This would give the codec more flexibility to e.g. do multiple passes if it
> wanted to do things like encode high-frequency terms more efficiently with a
> bitset-like encoding or other things...
> A codec can try to do things like this to some extent today, but its very
> difficult (look at buffering in Pulsing). We made this change with DV and it
> made a lot of interesting optimizations easy to implement...
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
