[Qemu-devel] [PATCH] vmsvga: correct bitmap and pixmap size checks

2016-09-08 Thread P J P 
From: Prasad J Pandit 

When processing svga command DEFINE_CURSOR in vmsvga_fifo_run,
the computed BITMAP and PIXMAP size are checked against the
'cursor.mask[]' and 'cursor.image[]' array sizes in bytes.
Correct these checks to avoid OOB memory access.

Reported-by: Qinghao Tang 
Reported-by: Li Qiang 
Signed-off-by: Prasad J Pandit 
---
 hw/display/vmware_vga.c | 12 +++-
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/hw/display/vmware_vga.c b/hw/display/vmware_vga.c
index e51a05e..6599cf0 100644
--- a/hw/display/vmware_vga.c
+++ b/hw/display/vmware_vga.c
@@ -676,11 +676,13 @@ static void vmsvga_fifo_run(struct vmsvga_state_s *s)
 cursor.bpp = vmsvga_fifo_read(s);
 
 args = SVGA_BITMAP_SIZE(x, y) + SVGA_PIXMAP_SIZE(x, y, cursor.bpp);
-if (cursor.width > 256 ||
-cursor.height > 256 ||
-cursor.bpp > 32 ||
-SVGA_BITMAP_SIZE(x, y) > sizeof cursor.mask ||
-SVGA_PIXMAP_SIZE(x, y, cursor.bpp) > sizeof cursor.image) {
+if (cursor.width > 256
+|| cursor.height > 256
+|| cursor.bpp > 32
+|| SVGA_BITMAP_SIZE(x, y)
+> sizeof(cursor.mask) / sizeof(cursor.mask[0])
+|| SVGA_PIXMAP_SIZE(x, y, cursor.bpp)
+> sizeof(cursor.image) / sizeof(cursor.image[0])) {
 goto badcmd;
 }
 
-- 
2.5.5

Re: [Qemu-devel] [PATCH] vmsvga: correct bitmap and pixmap size checks

2016-09-13 Thread P J P 
+-- On Thu, 8 Sep 2016, P J P wrote --+
| From: Prasad J Pandit 
| 
| When processing svga command DEFINE_CURSOR in vmsvga_fifo_run,
| the computed BITMAP and PIXMAP size are checked against the
| 'cursor.mask[]' and 'cursor.image[]' array sizes in bytes.
| Correct these checks to avoid OOB memory access.
| 
| Reported-by: Qinghao Tang 
| Reported-by: Li Qiang 
| Signed-off-by: Prasad J Pandit 
| ---
|  hw/display/vmware_vga.c | 12 +++-
|  1 file changed, 7 insertions(+), 5 deletions(-)
| 
| diff --git a/hw/display/vmware_vga.c b/hw/display/vmware_vga.c
| index e51a05e..6599cf0 100644
| --- a/hw/display/vmware_vga.c
| +++ b/hw/display/vmware_vga.c
| @@ -676,11 +676,13 @@ static void vmsvga_fifo_run(struct vmsvga_state_s *s)
|  cursor.bpp = vmsvga_fifo_read(s);
|  
|  args = SVGA_BITMAP_SIZE(x, y) + SVGA_PIXMAP_SIZE(x, y, 
cursor.bpp);
| -if (cursor.width > 256 ||
| -cursor.height > 256 ||
| -cursor.bpp > 32 ||
| -SVGA_BITMAP_SIZE(x, y) > sizeof cursor.mask ||
| -SVGA_PIXMAP_SIZE(x, y, cursor.bpp) > sizeof cursor.image) {
| +if (cursor.width > 256
| +|| cursor.height > 256
| +|| cursor.bpp > 32
| +|| SVGA_BITMAP_SIZE(x, y)
| +> sizeof(cursor.mask) / sizeof(cursor.mask[0])
| +|| SVGA_PIXMAP_SIZE(x, y, cursor.bpp)
| +> sizeof(cursor.image) / sizeof(cursor.image[0])) {
|  goto badcmd;
|  }

Ping...!
--
Prasad J Pandit / Red Hat Product Security Team
47AF CE69 3A90 54AA 9045 1053 DD13 3D32 FE5B 041F

Re: [Qemu-devel] [PATCH] vmsvga: correct bitmap and pixmap size checks

2016-09-13 Thread Gerd Hoffmann 
On Do, 2016-09-08 at 18:15 +0530, P J P wrote:
> From: Prasad J Pandit 
> 
> When processing svga command DEFINE_CURSOR in vmsvga_fifo_run,
> the computed BITMAP and PIXMAP size are checked against the
> 'cursor.mask[]' and 'cursor.image[]' array sizes in bytes.
> Correct these checks to avoid OOB memory access.

Added to vga patch queue.

thanks,
  Gerd