On Tue, 18 Aug 2026 23:36:30 GMT, Michael Strauß <[email protected]> wrote:
>>> the terms of the calculation were already snapped to begin with, so the >>> idealized result is guaranteed to be pixel-aligned >> >> Can we emphasize this please? > >> Can we emphasize this please? > > I've changed this paragraph to emphasize it. There's two things here: 1. Using `ScaledMath.ceil` where it subtracts the `ulp` works incorrectly when inputs are so large that they don't have a significant fractional part anymore (all digits are allocated to the integer part) -- this causes `Double.MAX_VALUE` to get mangled which shows up in unit tests that can't just use `== Double.MAX_VALUE` -- the rounding functions have no such problems 2. The current `ceil` function forces absolute correct use of snapping, so that even with pre-snapped values `left + size + right` must be snapped again. With a more lenient `ceil` this final snapping becomes unnecessary as the total error introduced with pre-snapped values will never become significant enough that we can't `ceil` to the **intended pixel** if we were using a constant instead of the `ulp`. ## Example for situation 1: ### Using ULP Initial value | ULP | ceil(initial − ULP) | Difference ---|---|---|--- 1.797693134862**3157**E308 | 1.9958403095347198E292 | 1.797693134862**3155**E308 | 2E292 2.0E20 | 3.2768E4 | 1.9999999999999997E20 | 30000 2.0E10 | 3.814697265625E-6 | 2.0E10 | 0 2.0E5 | 2.9103830456733704E-11 | 2.0E5 | 0 2.0E0 | 4.440892098500626E-16 | 2.0E0 | 0 ### Using tiny fractional constant Initial value | Constant | ceil(initial − Constant) | Difference ---|---|---|--- 1.7976931348623157E308 | 1.0E-6 | 1.7976931348623157E308 | 0 2.0E20 | 1.0E-6 | 2.0E20 | 0 2.0E10 | 1.0E-6 | 2.0E10 | 0 2.0E5 | 1.0E-6 | 2.0E5 | 0 2.0E0 | 1.0E-6 | 2.0E0 | 0 ------------- PR Review Comment: https://git.openjdk.org/jfx/pull/2260#discussion_r3810120178
