[
https://issues.apache.org/jira/browse/SPARK-9858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15047222#comment-15047222
]
Yin Huai commented on SPARK-9858:
---------------------------------
[~aroberts] Thanks for your comments.
For 1, can you provide more details? What is the rowBuffer you referred to?
For 2 and 3, I feel the size differences are caused by the differences of
platforms. In our tests, I got those numbers in assertion from my machine.
Those numbers work well with jenkins. Do you have any suggestion on how we can
make these tests robust to different platforms?
btw, have you changed {{spark.sql.shuffle.partitions}}?
> Introduce an ExchangeCoordinator to estimate the number of post-shuffle
> partitions.
> -----------------------------------------------------------------------------------
>
> Key: SPARK-9858
> URL: https://issues.apache.org/jira/browse/SPARK-9858
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Reporter: Yin Huai
> Assignee: Yin Huai
> Fix For: 1.6.0
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
