[ https://issues.apache.org/jira/browse/BEAM-2334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Luke Cwik reassigned BEAM-2334: ------------------------------- Assignee: Romain Yon (was: Kenneth Knowles) > OutOfMemoryError in RandomAccessData.java:350 > --------------------------------------------- > > Key: BEAM-2334 > URL: https://issues.apache.org/jira/browse/BEAM-2334 > Project: Beam > Issue Type: Bug > Components: runner-core > Affects Versions: 2.0.0 > Reporter: Romain Yon > Assignee: Romain Yon > > Got the following trace: > ``` > Caused by: java.lang.OutOfMemoryError: Requested array size exceeds VM limit > at java.util.Arrays.copyOf(Arrays.java:3236) > at > org.apache.beam.runners.dataflow.util.RandomAccessData.ensureCapacity(RandomAccessData.java:350) > at > org.apache.beam.runners.dataflow.util.RandomAccessData.access$300(RandomAccessData.java:51) > at > org.apache.beam.runners.dataflow.util.RandomAccessData$1.write(RandomAccessData.java:258) > at com.esotericsoftware.kryo.io.Output.flush(Output.java:185) > ``` > However the machine was not out of heap space. Seems like the problem is in > RandomAccessData.java:357: > Arrays.copyOf(buffer, Integer.MAX_VALUE) > Proof: > ``` > scala> Arrays.copyOf(v, Int.MaxValue) > java.lang.OutOfMemoryError: Requested array size exceeds VM limit > at java.util.Arrays.copyOf(Arrays.java:3236) > ... 31 elided > scala> Arrays.copyOf(v, Int.MaxValue-2) > java.lang.OutOfMemoryError: Java heap space > at java.util.Arrays.copyOf(Arrays.java:3236) > ... 31 elided > scala> Arrays.copyOf(v, Int.MaxValue-1) > java.lang.OutOfMemoryError: Requested array size exceeds VM limit > at java.util.Arrays.copyOf(Arrays.java:3236) > ... 31 elided > ``` > I'll submit a corresponding PR. -- This message was sent by Atlassian JIRA (v6.3.15#6346)