[
https://issues.apache.org/jira/browse/LUCENE-1483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12664883#action_12664883
]
Mark Miller commented on LUCENE-1483:
-------------------------------------
Re: The FieldCache loading problem with MultiSegmentReader
I think this is just the overhead cost associated with how we currently handle
this.
To load a FieldCache for a single segment we get a SegmentTermDocs and just
keep calling next, which basically just calls readbyte and readint over and
over on the index. Pretty efficient. Do it for each term to be loaded.
To load a FieldCache for multiple segments we get a MultiSegmentReader to get
MultiTermDocs. As we call next on MultiTermDocs it will get a TermDocs for each
Reader and call seek to get to the Term. The seek appears pretty slow, and we
do it for the number of Readers x the number of Terms to be loaded.
Under seek, TermInfosReader.get looks slow with SegmentTermeEnum.scanTo looking
to be the worst offender under it.
All in all though, there is just a big difference in whats going on, that
compounds with more segments.
- Mark
> Change IndexSearcher multisegment searches to search each individual segment
> using a single HitCollector
> --------------------------------------------------------------------------------------------------------
>
> Key: LUCENE-1483
> URL: https://issues.apache.org/jira/browse/LUCENE-1483
> Project: Lucene - Java
> Issue Type: Improvement
> Affects Versions: 2.9
> Reporter: Mark Miller
> Priority: Minor
> Attachments: LUCENE-1483-partial.patch, LUCENE-1483.patch,
> LUCENE-1483.patch, LUCENE-1483.patch, LUCENE-1483.patch, LUCENE-1483.patch,
> LUCENE-1483.patch, LUCENE-1483.patch, LUCENE-1483.patch, LUCENE-1483.patch,
> LUCENE-1483.patch, LUCENE-1483.patch, LUCENE-1483.patch, LUCENE-1483.patch,
> LUCENE-1483.patch, LUCENE-1483.patch, LUCENE-1483.patch, LUCENE-1483.patch,
> LUCENE-1483.patch, LUCENE-1483.patch, LUCENE-1483.patch, LUCENE-1483.patch,
> LUCENE-1483.patch, LUCENE-1483.patch, LUCENE-1483.patch, LUCENE-1483.patch,
> LUCENE-1483.patch, LUCENE-1483.patch, LUCENE-1483.patch, LUCENE-1483.patch,
> sortBench.py, sortCollate.py
>
>
> FieldCache and Filters are forced down to a single segment reader, allowing
> for individual segment reloading on reopen.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-dev-h...@lucene.apache.org
