Changes: v3: check for INT_MAX / 4 as well, and use ROUND_UP (Thanks Max)
v2 now also checks that Max Table Entries is < SIZE_MAX / 4 (Thanks Stefan) This series fixes a bug found by Richard Jones. When we allocate the pagetable based on max_table_entries, we multiply the max table entry value by 4 to accomodate a table of 32-bit integers. However, max_table_entries is a uint32_t, and the VPC driver accepts ranges for that entry over 0x40000000. So during this allocation: s->pagetable = qemu_try_blockalign(bs->file, s->max_table_entries * 4); The size arg overflows, allocating significantly less memory than expected. Since qemu_try_blockalign() size argument is size_t, cast the multiplication correctly to prevent overflow. The value of "max_table_entries * 4" is used elsewhere in the code as well, so store the correct value for use in all those cases. Jeff Cody (2): block: vpc - prevent overflow if max_table_entries >= 0x40000000 block: qemu-iotests - add check for multiplication overflow in vpc block/vpc.c | 18 +++++++-- tests/qemu-iotests/135 | 54 ++++++++++++++++++++++++++ tests/qemu-iotests/135.out | 5 +++ tests/qemu-iotests/group | 1 + tests/qemu-iotests/sample_images/afl5.img.bz2 | Bin 0 -> 175 bytes 5 files changed, 74 insertions(+), 4 deletions(-) create mode 100755 tests/qemu-iotests/135 create mode 100644 tests/qemu-iotests/135.out create mode 100644 tests/qemu-iotests/sample_images/afl5.img.bz2 -- 1.9.3
