[
https://issues.apache.org/jira/browse/SPARK-36900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean R. Owen resolved SPARK-36900.
----------------------------------
Fix Version/s: 3.3.0
Resolution: Fixed
Issue resolved by pull request 34284
[https://github.com/apache/spark/pull/34284]
> "SPARK-36464: size returns correct positive number even with over 2GB data"
> will oom with JDK17
> ------------------------------------------------------------------------------------------------
>
> Key: SPARK-36900
> URL: https://issues.apache.org/jira/browse/SPARK-36900
> Project: Spark
> Issue Type: Sub-task
> Components: Spark Core
> Affects Versions: 3.3.0
> Reporter: Yang Jie
> Assignee: Yang Jie
> Priority: Minor
> Fix For: 3.3.0
>
>
> Execute
>
> {code:java}
> build/mvn clean install -pl core -am -Dtest=none
> -DwildcardSuites=org.apache.spark.util.io.ChunkedByteBufferOutputStreamSuite
> {code}
> with JDK 17,
> {code:java}
> ChunkedByteBufferOutputStreamSuite:
> - empty output
> - write a single byte
> - write a single near boundary
> - write a single at boundary
> - single chunk output
> - single chunk output at boundary size
> - multiple chunk output
> - multiple chunk output at boundary size
> *** RUN ABORTED ***
> java.lang.OutOfMemoryError: Java heap space
> at java.base/java.lang.Integer.valueOf(Integer.java:1081)
> at scala.runtime.BoxesRunTime.boxToInteger(BoxesRunTime.java:67)
> at
> org.apache.spark.util.io.ChunkedByteBufferOutputStream.allocateNewChunkIfNeeded(ChunkedByteBufferOutputStream.scala:87)
> at
> org.apache.spark.util.io.ChunkedByteBufferOutputStream.write(ChunkedByteBufferOutputStream.scala:75)
> at java.base/java.io.OutputStream.write(OutputStream.java:127)
> at
> org.apache.spark.util.io.ChunkedByteBufferOutputStreamSuite.$anonfun$new$22(ChunkedByteBufferOutputStreamSuite.scala:127)
> at
> org.apache.spark.util.io.ChunkedByteBufferOutputStreamSuite$$Lambda$179/0x00000008011a75d8.apply(Unknown
> Source)
> at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
> at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
> at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
