[ https://issues.apache.org/jira/browse/LUCENE-6759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14717707#comment-14717707 ]
Karl Wright commented on LUCENE-6759: ------------------------------------- [~mikemccand]: So, your code: {code} long y = (long) (x * (Integer.MAX_VALUE / max)); {code} ... can be rewritten: {code} long y = (long) (x / max * Integer.MAX_VALUE); {code} So it is basically scaling so that Integer.MAX_VALUE is returned for a value of max. So far so good. But in order to do that, it's rounding down for positive numbers, and rounding up for negative ones. So: 1.5 -> 1 1.1 -> 1 1.9 -> 1 2.1 -> 2 -1.5 -> -1 -1.9 -> -1 But: 0.9 -> 0 -0.9 -> 0 So, the cell at 0 is in fact representing a range that is twice as great as the cell at 1 or the cell at -1. I'm not sure you really want that. ;-) > Integrate lat/long BKD and spatial 3d, part 2 > --------------------------------------------- > > Key: LUCENE-6759 > URL: https://issues.apache.org/jira/browse/LUCENE-6759 > Project: Lucene - Core > Issue Type: New Feature > Reporter: Michael McCandless > Attachments: LUCENE-6699.patch, LUCENE-6699.patch, LUCENE-6699.patch, > LUCENE-6699.patch, LUCENE-6699.patch, LUCENE-6699.patch, LUCENE-6699.patch > > > This is just a continuation of LUCENE-6699, which became too big. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org