Both amdgpu_gem_align_pitch() and radeon_align_pitch() use signed int for the pitch calculation. When alignment rounding pushes the width to a boundary value, 'aligned * cpp' overflows signed 32-bit int to 0.
This defeats the overflow guards in drm_mode_create_dumb() because these drivers bypass drm_mode_size_dumb() and perform their own alignment rounding, which can push the pitch past the pre-validated range. A zero pitch propagates to a zero-size GEM object allocation reachable from unprivileged userspace via DRM_IOCTL_MODE_CREATE_DUMB on the render node. Both drivers need the same fix: add an overflow check in the alignment function and reject zero pitch/size in the dumb_create callback. The proper long-term fix is to convert both drivers to use drm_mode_size_dumb() as Thomas Zimmermann's series is doing for other drivers. Werner Kasselman (2): drm/amdgpu: fix integer overflow in amdgpu_gem_align_pitch() drm/radeon: fix integer overflow in radeon_align_pitch() drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 13 +++++++++++++ drivers/gpu/drm/radeon/radeon_gem.c | 9 +++++++++ 2 files changed, 22 insertions(+) -- 2.43.0
