dajiaohuang commented on PR #20059: URL: https://github.com/apache/pinot/pull/20059#issuecomment-7033037706
Pushed `fa8379795e`, which addresses both substantive findings from the review at `0c5d4dfa`. Replies are in the two existing threads; this is the summary. **The overshoot.** The tolerance was the bug. It grew with the magnitude of the bounds, so once a bound was spaced far wider than the increment it could reach half a step and promote a genuine fractional count. Your repro, `generateFloatArray(91321.76f, 91322.45f, 0.09f)`, has a step count of 7.7257 and a tolerance of about 0.347, so `rint` took it to 8 and appended 91322.48, three float ulps past the end. I agree with your diagnosis that no tolerance value can fix this, and I took the first option you suggested rather than the "cap it below 0.5" one. Capping does not separate the cases: the decimal repros need a tolerance above roughly 2e-9 and your float repro breaks at about 0.274, so a cap only moves which ranges are wrong. Instead the element that would actually be generated is checked, in the element type, and only a quotient that check cannot settle goes to the exact decimal quotient. All three repros now give the intended answer: 8 elements ending 91322.39f, `[1000000]`, and `[1e15, 1e15 + 1]`. **The float arithmetic.** The float path now promotes an operand before the subtraction, so both the subtraction and the division are in double precision, and the stale Javadoc wording is gone with the rewrite rather than reworded around. **Regression cases** are added for both types, in `GenerateArrayFunctionsTest`: the overshoot repro, ordinary float fractional steps, double at 1e15, and an end that is exactly on a step. Test run reported exactly as requested: ``` ./mvnw -pl pinot-common -am -Dtest=GenerateArrayFunctionsTest -Dsurefire.failIfNoSpecifiedTests=false -Ppinot-fastdev test Tests run: 22, Failures: 0, Errors: 0, Skipped: 0 BUILD SUCCESS ``` (JDK 26.0.2; `checkstyle:check -pl pinot-common` also passes.) **Risk and mixed-version rollout.** You are right that this is HIGH risk, and the scope is wider than the length function: calls on literal arguments are folded into the array literal at planning time, so this changes results as well as runtime evaluation. During a rolling upgrade two brokers or segments on different versions can return different-length arrays for the same query, and a plan folded by a newer broker can differ from what an older one would have produced. In a gap-filling grid that is a different row count, not an error. There is no wire-format concern and nothing to gate, but it is worth a release-note line so a rolling upgrade is not mistaken for nondeterminism. I have added that to the PR description. Two limits I would rather state than leave implied. A step that is not representable at the magnitude of the bounds (0.0141 at 3.4e5, where one ulp is 0.0151) has no unique count; the base, this revision and the previous one all differ there, and the count now follows the generated elements. And a range whose only in-bounds element repeats its predecessor still ends on the earlier one, as the base did. Beyond the suite I fuzzed the length computation standalone: 1.5M float and 1.1M double ranges, checking that the last element never passes the end and that the end is never dropped, and 800k decimal-step ranges per type against the exact decimal count. No violations in any of those. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
