[
https://issues.apache.org/jira/browse/SOLR-17775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17956005#comment-17956005
]
David Smiley commented on SOLR-17775:
-------------------------------------
While I maintain my opinion that DocStreamer casts a wider net with greater
value, I'm on board with continuing with your existing work here – "progress
not perfection". I think doing only the first X (1000) is an easy way to cap
the memory risk, leaving a later time for improving to a batch model.
> Optimize ValueSourceAugmenter
> -----------------------------
>
> Key: SOLR-17775
> URL: https://issues.apache.org/jira/browse/SOLR-17775
> Project: Solr
> Issue Type: Improvement
> Components: search
> Reporter: Yura
> Priority: Minor
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> h3. Problem
> ValueSourceAugmenter currently calculates function values on-demand during
> transform(), performing expensive binary searches and reader lookups for each
> document individually.
> h3. Solution
> Pre-calculate function values for all result set documents during
> setContext() by:
> * Collecting and sorting document IDs from DocList
> * Sequential iteration through sorted documents to calculate values once per
> reader segment
> * Storing results in hash map for O(1) lookup during transform()
> * Fallback to on-demand calculation for documents outside the pre-calculated
> set (RTG cases)
> h3. Performance Benefit
> Replaces repeated "find document at position N" operations (binary search per
> document) with efficient "get next document" iteration (sequential processing
> within reader segments), significantly reducing lookup overhead.
> h3. Compatibility
> Maintains full backward compatibility through fallback mechanism for edge
> cases.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
