Hi,

Please review this fix for dumping of long arrays, and general cleanup of types in heapDumper.cpp.

Problem:
At several places in heapDumper.cpp overflows could happen when dumping long arrays. Also the hprof format uses an u4 as a record length field, but arrays can be longer than that (counted in bytes).

Fix:
Many types that were previously signed are changed to equivalent unsigned types and/or to a larger type to prevent overflow. The bulk of the change though is the addition of calculate_array_max_length, which for a given array returns the number of elements we can dump. That length is then used to truncate arrays that are too long.
Whenever an array is truncated a warning is printed:
Java HotSpot(TM) 64-Bit Server VM warning: cannot dump array of type object[] with length 1,073,741,823; truncating to length 536,870,908

Much of the rest of the change is moving functions needed by calculate_array_max_length to the DumpWriter or DumperSupport class so that they can be accessed.

Added a test that relies on the hprof parser, which also had a couple of overflow problems (top repo changes). I've also run this change against a couple of other tests, but they are problematic in JPRT because they are using large heaps and lots of disk.

Bug:
8129419: heapDumper.cpp: assert(length_in_bytes > 0) failed: nothing to copy
https://bugs.openjdk.java.net/browse/JDK-8129419

Also fixed in this change is the problems seen in these two bugs:
8133317: Integer overflow in heapDumper.cpp leads to corrupt HPROF dumps
https://bugs.openjdk.java.net/browse/JDK-8133317

8144732: VM_HeapDumper hits assert with bad dump_len
https://bugs.openjdk.java.net/browse/JDK-8144732

Webrev:
Top repo: http://cr.openjdk.java.net/~aeriksso/8129419/webrev.00/jdk9-hs-rt/
Hotspot: http://cr.openjdk.java.net/~aeriksso/8129419/webrev.00/hotspot/

Regards,
Andreas

Reply via email to