[
https://issues.apache.org/jira/browse/LUCENE-6115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14248718#comment-14248718
]
Robert Muir commented on LUCENE-6115:
-------------------------------------
We need to make it clear how bad the problem is in trunk: if you use a
leafreader or are upgrading/changing codec today, with my test data merging is
7x slower merging with LZ4, and 64x slower merging with deflate.
This means indexing data that ordinarily takes a minute takes an hour instead.
Unfortunately, this patch won't solve the performance issues with FilterReaders
at all.
The problem is: nobody will ever call getMergeInstance in that case. See
LUCENE-6065 for some details (I am not happy with any solution yet, i need
help).
However, the patch will work in the case of a user upgrading or changing codec,
just as long as IW is passed a segmentreader. So its a great step. It is
separate from LeafReader's brokenness.
> Add getMergeInstance to CompressingStoredFieldsReader
> -----------------------------------------------------
>
> Key: LUCENE-6115
> URL: https://issues.apache.org/jira/browse/LUCENE-6115
> Project: Lucene - Core
> Issue Type: Improvement
> Reporter: Adrien Grand
> Assignee: Adrien Grand
> Priority: Minor
> Attachments: LUCENE-6115.patch, LUCENE-6115.patch
>
>
> CompressingStoredFieldsReader is currently terrible at merging with different
> codecs or wrapped readers since it does not keep state. So if you want to get
> 5 documents that come from the same block, it means that you will have to
> decode the block header and decompress 5 times. It has some optimizations so
> that if you want to get the 2nd doc of the block then it will stop
> decompressing soon after the 2nd document, but it doesn't help much with
> merging since we want all documents.
> We should implement getMergeInstance and have a different behaviour when
> merging by decompressing everything up-front and then reusing for all
> documents of the block.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
