[PATCH] drm/amd/display: Fix 64 bit wraparound resulting in illegal drm mode

2022-06-17 Thread Leslie Shi 
[Why]
For m = drm_display_mode{.clock = 533250, .htotal = 4000, .vtotal = },
common_rates[i] = 6, the result of target_vtotal is 2221. This cause
wraparound of variable target_vtotal_diff.

[How]
Skip the loop if target_vtotal less than m->vtotal

Signed-off-by: Leslie Shi 
---
 drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 4 
 1 file changed, 4 insertions(+)

diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c 
b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
index c2bc7db85d7e..ed7ab089b224 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
@@ -8537,6 +8537,10 @@ static uint add_fs_modes(struct amdgpu_dm_connector 
*aconnector)
num = (unsigned long long)m->clock * 1000 * 1000;
den = common_rates[i] * (unsigned long long)m->htotal;
target_vtotal = div_u64(num, den);
+
+   if (target_vtotal < m->vtotal)
+   continue;
+
target_vtotal_diff = target_vtotal - m->vtotal;
 
/* Check for illegal modes */
-- 
2.25.1

RE: [PATCH] drm/amd/display: Fix 64 bit wraparound resulting in illegal drm mode

2022-06-17 Thread Shi, Leslie 
[AMD Official Use Only - General]

Please skip this patch, the description is incorrect. I will provide a new 
patch soon.

-Original Message-
From: Shi, Leslie  
Sent: Friday, June 17, 2022 4:56 PM
To: amd-gfx@lists.freedesktop.org; Deucher, Alexander 

Cc: Chen, Guchun ; Shi, Leslie 
Subject: [PATCH] drm/amd/display: Fix 64 bit wraparound resulting in illegal 
drm mode

[Why]
For m = drm_display_mode{.clock = 533250, .htotal = 4000, .vtotal = }, 
common_rates[i] = 6, the result of target_vtotal is 2221. This cause 
wraparound of variable target_vtotal_diff.

On RHEL7.9 kernel, the type of vsync_start is int and the value of new 
calculated vsync_start equals 67698. This lead to the following call
trace:

WARNING: CPU: 9 PID: 424 at drivers/gpu/drm/drm_modes.c:1642 
drm_mode_convert_to_umode+0x42/0x1c0 [drm] timing values too large for mode info
CPU: 9 PID: 424 Comm: plymouthd Tainted: G   OE     
3.10.0-1160.el7.x86_64 #1
Hardware name: HP HP Z8 G4 Workstation/81C7, BIOS P60 v02.73 06/16/2021 Call 
Trace:
[] dump_stack+0x19/0x1b [] 
__warn+0xd8/0x100 [] warn_slowpath_fmt+0x5f/0x80 
[] ? __drm_mode_object_find+0x72/0x100 [drm] 
[] drm_mode_convert_to_umode+0x42/0x1c0 [drm] 
[] drm_mode_getconnector+0x3c6/0x560 [drm] 
[] ? drm_connector_property_set_ioctl+0x60/0x60 [drm] 
[] drm_ioctl_kernel+0xbc/0x100 [drm] [] 
drm_ioctl+0x244/0x450 [drm] [] ? 
drm_connector_property_set_ioctl+0x60/0x60 [drm] [] 
amdgpu_drm_ioctl+0x4e/0x80 [amdgpu] [] 
do_vfs_ioctl+0x3a0/0x5b0 [] ? __do_page_fault+0x238/0x500 
[] SyS_ioctl+0xa1/0xc0 [] 
system_call_fastpath+0x25/0x2a

[How]
Skip the loop if target_vtotal less than m->vtotal

Signed-off-by: Leslie Shi 
---
 drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 4 
 1 file changed, 4 insertions(+)

diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c 
b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
index c2bc7db85d7e..ed7ab089b224 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
@@ -8537,6 +8537,10 @@ static uint add_fs_modes(struct amdgpu_dm_connector 
*aconnector)
num = (unsigned long long)m->clock * 1000 * 1000;
den = common_rates[i] * (unsigned long long)m->htotal;
target_vtotal = div_u64(num, den);
+
+   if (target_vtotal < m->vtotal)
+   continue;
+
target_vtotal_diff = target_vtotal - m->vtotal;
 
/* Check for illegal modes */
--
2.25.1

[PATCH] drm/amd/display: Fix 64 bit wraparound resulting in illegal drm mode

2022-06-17 Thread Leslie Shi 
[Why]
For m = drm_display_mode{.clock = 533250, .htotal = 4000, .vtotal = },
common_rates[i] = 6, the result of target_vtotal is 2221. This cause
wraparound of variable target_vtotal_diff.

On RHEL7.9 kernel, the type of vsync_start is int and the value of new
calculated vsync_start equals 67698. This lead to the following call
trace:

WARNING: CPU: 9 PID: 424 at drivers/gpu/drm/drm_modes.c:1642 
drm_mode_convert_to_umode+0x42/0x1c0 [drm]
timing values too large for mode info
CPU: 9 PID: 424 Comm: plymouthd Tainted: G   OE     
3.10.0-1160.el7.x86_64 #1
Hardware name: HP HP Z8 G4 Workstation/81C7, BIOS P60 v02.73 06/16/2021
Call Trace:
[] dump_stack+0x19/0x1b
[] __warn+0xd8/0x100
[] warn_slowpath_fmt+0x5f/0x80
[] ? __drm_mode_object_find+0x72/0x100 [drm]
[] drm_mode_convert_to_umode+0x42/0x1c0 [drm]
[] drm_mode_getconnector+0x3c6/0x560 [drm]
[] ? drm_connector_property_set_ioctl+0x60/0x60 [drm]
[] drm_ioctl_kernel+0xbc/0x100 [drm]
[] drm_ioctl+0x244/0x450 [drm]
[] ? drm_connector_property_set_ioctl+0x60/0x60 [drm]
[] amdgpu_drm_ioctl+0x4e/0x80 [amdgpu]
[] do_vfs_ioctl+0x3a0/0x5b0
[] ? __do_page_fault+0x238/0x500
[] SyS_ioctl+0xa1/0xc0
[] system_call_fastpath+0x25/0x2a

[How]
Skip the loop if target_vtotal less than m->vtotal

Signed-off-by: Leslie Shi 
---
 drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 4 
 1 file changed, 4 insertions(+)

diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c 
b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
index c2bc7db85d7e..ed7ab089b224 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
@@ -8537,6 +8537,10 @@ static uint add_fs_modes(struct amdgpu_dm_connector 
*aconnector)
num = (unsigned long long)m->clock * 1000 * 1000;
den = common_rates[i] * (unsigned long long)m->htotal;
target_vtotal = div_u64(num, den);
+
+   if (target_vtotal < m->vtotal)
+   continue;
+
target_vtotal_diff = target_vtotal - m->vtotal;
 
/* Check for illegal modes */
-- 
2.25.1