Unexepcted OutOfMemoryError from Arrays.deepToString
Hello. I came across unexpected behaviour of Arrays.deepToString method. It throws OOM even on non-huge arrays. For example this code: int size = Integer.MAX_VALUE / 19; Integer[] integers = new Integer[size]; Arrays.fill(integers, 0); System.out.println(Arrays.deepToString(integers)); Fails with following stack trace: Exception in thread "main" java.lang.OutOfMemoryError: Requested array size exceeds VM limit at java.base/java.lang.AbstractStringBuilder.(AbstractStringBuilder.java:86) at java.base/java.lang.StringBuilder.(StringBuilder.java:112) at java.base/java.util.Arrays.deepToString(Arrays.java:5160) I believe it should be able to handle such arrays and not throw OOM Andrey Turbanov
Re: Unexepcted OutOfMemoryError from Arrays.deepToString
This error has two causes. The first cause is that the VM cannot allocate arrays whose length exceeds Integer.MAX_VALUE - 8 (MAX_ARRAY_SIZE). The second cause is that Arrays.deepToString tries to pre-allocate 20 chars per string representation for each array element and maxes out at Integer.MAX_VALUE, which is above MAX_ARRAY_SIZE. One solution would be to change Arrays.deepToString to max out at MAX_ARRAY_SIZE. Separately, java.lang.AbstractStringBuilder#MAX_ARRAY_SIZE seems unused; I wonder how that happened. -Pavel > On 9 Oct 2021, at 11:38, Andrey Turbanov wrote: > > Hello. > I came across unexpected behaviour of Arrays.deepToString method. > It throws OOM even on non-huge arrays. > For example this code: > >int size = Integer.MAX_VALUE / 19; >Integer[] integers = new Integer[size]; >Arrays.fill(integers, 0); > System.out.println(Arrays.deepToString(integers)); > > Fails with following stack trace: > > Exception in thread "main" java.lang.OutOfMemoryError: Requested array > size exceeds VM limit >at > java.base/java.lang.AbstractStringBuilder.(AbstractStringBuilder.java:86) >at java.base/java.lang.StringBuilder.(StringBuilder.java:112) >at java.base/java.util.Arrays.deepToString(Arrays.java:5160) > > > I believe it should be able to handle such arrays and not throw OOM > > > Andrey Turbanov
Re: Unexepcted OutOfMemoryError from Arrays.deepToString
> On 9 Oct 2021, at 13:07, Pavel Rappo wrote: > > > > Separately, java.lang.AbstractStringBuilder#MAX_ARRAY_SIZE seems unused; I > wonder how that happened. I found what happened: $ git log -S "MAX_ARRAY_SIZE" -- src/java.base/share/classes/java/lang/AbstractStringBuilder.java commit 03642a01af7123298d6524a98c99a3934d35c11b Author: Jim Laskey Date: Thu Jun 11 10:08:23 2020 -0300 8230744: Several classes throw OutOfMemoryError without message Reviewed-by: psandoz, martin, bchristi, rriggs, smarks Looking at the corresponding review thread, https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-June/066874.html, the question about "the local orphan" MAX_ARRAY_SIZE was raised by Martin and subsequently addressed by Jim. The problem is, there were two of these fields. The one in PriorityBlockingQueue was spotted and deleted, the one in AbstractStringBuilder was not. Andrey, regardless of the outcome of your main question (OOM from Arrays.deepToString), would be willing to publish a PR to delete that field in AbstractStringBuilder? -Pavel > > -Pavel > >> On 9 Oct 2021, at 11:38, Andrey Turbanov wrote: >> >> Hello. >> I came across unexpected behaviour of Arrays.deepToString method. >> It throws OOM even on non-huge arrays. >> For example this code: >> >> int size = Integer.MAX_VALUE / 19; >> Integer[] integers = new Integer[size]; >> Arrays.fill(integers, 0); >> System.out.println(Arrays.deepToString(integers)); >> >> Fails with following stack trace: >> >> Exception in thread "main" java.lang.OutOfMemoryError: Requested array >> size exceeds VM limit >> at >> java.base/java.lang.AbstractStringBuilder.(AbstractStringBuilder.java:86) >> at java.base/java.lang.StringBuilder.(StringBuilder.java:112) >> at java.base/java.util.Arrays.deepToString(Arrays.java:5160) >> >> >> I believe it should be able to handle such arrays and not throw OOM >> >> >> Andrey Turbanov >
Re: Unexepcted OutOfMemoryError from Arrays.deepToString
Created PR to remove AbstractStringBuilder.MAX_ARRAY_SIZE - https://github.com/openjdk/jdk/pull/5878 BTW, I found one more place where Integer.MAX_VALUE is used as maximum array length - java.util.concurrent.ScheduledThreadPoolExecutor.DelayedWorkQueue#grow I think this method could be reworked to take advantage of ArraysSupport.newLength method too. Andrey Turbanov сб, 9 окт. 2021 г. в 20:09, Pavel Rappo : > > > > > On 9 Oct 2021, at 13:07, Pavel Rappo wrote: > > > > > > > > Separately, java.lang.AbstractStringBuilder#MAX_ARRAY_SIZE seems unused; I > > wonder how that happened. > > I found what happened: > > $ git log -S "MAX_ARRAY_SIZE" -- > src/java.base/share/classes/java/lang/AbstractStringBuilder.java > commit 03642a01af7123298d6524a98c99a3934d35c11b > Author: Jim Laskey > Date: Thu Jun 11 10:08:23 2020 -0300 > > 8230744: Several classes throw OutOfMemoryError without message > > Reviewed-by: psandoz, martin, bchristi, rriggs, smarks > > Looking at the corresponding review thread, > https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-June/066874.html, > the question about "the local orphan" MAX_ARRAY_SIZE was raised by Martin and > subsequently addressed by Jim. The problem is, there were two of these > fields. The one in PriorityBlockingQueue was spotted and deleted, the one in > AbstractStringBuilder was not. > > Andrey, regardless of the outcome of your main question (OOM from > Arrays.deepToString), would be willing to publish a PR to delete that field > in AbstractStringBuilder? > > -Pavel > > > > > -Pavel > > > >> On 9 Oct 2021, at 11:38, Andrey Turbanov wrote: > >> > >> Hello. > >> I came across unexpected behaviour of Arrays.deepToString method. > >> It throws OOM even on non-huge arrays. > >> For example this code: > >> > >> int size = Integer.MAX_VALUE / 19; > >> Integer[] integers = new Integer[size]; > >> Arrays.fill(integers, 0); > >> System.out.println(Arrays.deepToString(integers)); > >> > >> Fails with following stack trace: > >> > >> Exception in thread "main" java.lang.OutOfMemoryError: Requested array > >> size exceeds VM limit > >> at > >> java.base/java.lang.AbstractStringBuilder.(AbstractStringBuilder.java:86) > >> at java.base/java.lang.StringBuilder.(StringBuilder.java:112) > >> at java.base/java.util.Arrays.deepToString(Arrays.java:5160) > >> > >> > >> I believe it should be able to handle such arrays and not throw OOM > >> > >> > >> Andrey Turbanov > > >
