Re: [RFC PATCH v4 03/42] drm: Correctly round for fixp2int_round

2024-03-11 Thread Pekka Paalanen
On Mon, 26 Feb 2024 16:10:17 -0500 Harry Wentland wrote: > A value of 0x8000 and higher should round up, and > below should round down. VKMS Kunit tests for lerp_u16 > showed that this is not the case. Fix it. > > 1 << (DRM_FIXED_POINT_HALF - 1) = > 1 << 15 = 0x8000 > > This is not 0.5,

[RFC PATCH v4 03/42] drm: Correctly round for fixp2int_round

2024-02-26 Thread Harry Wentland
A value of 0x8000 and higher should round up, and below should round down. VKMS Kunit tests for lerp_u16 showed that this is not the case. Fix it. 1 << (DRM_FIXED_POINT_HALF - 1) = 1 << 15 = 0x8000 This is not 0.5, but 0.0762939453125. Instead of some smart math use a simple if/else to