[ https://issues.apache.org/jira/browse/LUCENE-5780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14038634#comment-14038634 ]
Robert Muir commented on LUCENE-5780: ------------------------------------- I agree conceptually its a bug, but I think this should be 4.10. Its not anything introduced in 4.9, its been this way since 4.2 I already am pushing back on other issues such as SOLR-6178, because i ran 125 rounds of tests last night and want to keep things stable. > OrdinalMap's mapping from global ords to segment ords is sometimes wasteful > --------------------------------------------------------------------------- > > Key: LUCENE-5780 > URL: https://issues.apache.org/jira/browse/LUCENE-5780 > Project: Lucene - Core > Issue Type: Bug > Reporter: Adrien Grand > Assignee: Adrien Grand > Fix For: 4.9, 5.0 > > > Robert found a case when the ordinal map can be quite wasteful in terms of > memory usage: in order to be able to resolve values given a global ordinals, > it stores two things: > - an identifier of the segment where the value is > - the difference between the ordinal on the segment and the global ordinal > The issue is that OrdinalMap currently picks any of the segments that contain > the value but we can do better: we can pick the first segment that has the > value. This will help for two reasons: > - it will potentially require fewer bits per value to store the segment ids > if NRT segments don't introduce new values > - if all values happen to appear in the first segment, then the map from > global ords to deltas only stores zeros. > I just tested on an index where all values are in the first segment and this > helped reduce memory usage of the ordinal map by 4x (from 3.5MB to 800KB). -- This message was sent by Atlassian JIRA (v6.2#6252) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org