[jira] [Commented] (SPARK-17082) Replace ByteBuffer with ChunkedByteBuffer
[ https://issues.apache.org/jira/browse/SPARK-17082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15432284#comment-15432284 ] Guoqiang Li commented on SPARK-17082: - OK > Replace ByteBuffer with ChunkedByteBuffer > - > > Key: SPARK-17082 > URL: https://issues.apache.org/jira/browse/SPARK-17082 > Project: Spark > Issue Type: Sub-task > Components: Shuffle, Spark Core >Reporter: Guoqiang Li > > The size of ByteBuffers can not be greater than 2G, should be replaced by > ChunkedByteBuffer -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17082) Replace ByteBuffer with ChunkedByteBuffer
[ https://issues.apache.org/jira/browse/SPARK-17082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15432274#comment-15432274 ] Sean Owen commented on SPARK-17082: --- I think we need to back up and discuss the design before submitting PRs. I'd prefer to close the PR and even this JIRA. It's not clear how it related to the previous ones, or even https://issues.apache.org/jira/browse/SPARK-17184 > Replace ByteBuffer with ChunkedByteBuffer > - > > Key: SPARK-17082 > URL: https://issues.apache.org/jira/browse/SPARK-17082 > Project: Spark > Issue Type: Sub-task > Components: Shuffle, Spark Core >Reporter: Guoqiang Li > > The size of ByteBuffers can not be greater than 2G, should be replaced by > ChunkedByteBuffer -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17082) Replace ByteBuffer with ChunkedByteBuffer
[ https://issues.apache.org/jira/browse/SPARK-17082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15422409#comment-15422409 ] Apache Spark commented on SPARK-17082: -- User 'witgo' has created a pull request for this issue: https://github.com/apache/spark/pull/14662 > Replace ByteBuffer with ChunkedByteBuffer > - > > Key: SPARK-17082 > URL: https://issues.apache.org/jira/browse/SPARK-17082 > Project: Spark > Issue Type: Sub-task > Components: Shuffle, Spark Core >Reporter: Guoqiang Li > > the various 2G limit we have in Spark, due to the use of ByteBuffers. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
