[
https://issues.apache.org/jira/browse/SPARK-13531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15267611#comment-15267611
]
Yin Huai commented on SPARK-13531:
--
I think this one has been fixed by SPARK-14578
[
https://issues.apache.org/jira/browse/SPARK-13531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15267569#comment-15267569
]
koert kuipers commented on SPARK-13531:
---
i do not see this exact error anymore. it has now become a
[
https://issues.apache.org/jira/browse/SPARK-13531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15267390#comment-15267390
]
Yin Huai commented on SPARK-13531:
--
I tried it out and it seems work (fixed at some point). [~koert] Can
[
https://issues.apache.org/jira/browse/SPARK-13531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15213612#comment-15213612
]
koert kuipers commented on SPARK-13531:
---
this seems to be like the simplest code that reproduces
[
https://issues.apache.org/jira/browse/SPARK-13531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15213363#comment-15213363
]
koert kuipers commented on SPARK-13531:
---
oh its because statistics for UnaryNode is based on
[
https://issues.apache.org/jira/browse/SPARK-13531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15213361#comment-15213361
]
koert kuipers commented on SPARK-13531:
---
before the optimization both MapPartitions have output
[
https://issues.apache.org/jira/browse/SPARK-13531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15213358#comment-15213358
]
koert kuipers commented on SPARK-13531:
---
i did a little more digging and i think this is what
[
https://issues.apache.org/jira/browse/SPARK-13531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15179201#comment-15179201
]
Apache Spark commented on SPARK-13531:
--
User 'zuowang' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-13531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177441#comment-15177441
]
Zuo Wang commented on SPARK-13531:
--
Caused by the commit in
[
https://issues.apache.org/jira/browse/SPARK-13531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15171420#comment-15171420
]
koert kuipers commented on SPARK-13531:
---
can you try this?
{noformat}
val df1 = sc.makeRDD(1 to
[
https://issues.apache.org/jira/browse/SPARK-13531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15171354#comment-15171354
]
Zuo Wang commented on SPARK-13531:
--
Unable to reproduce, here is my steps in spark-shell:
val rowRDD1 =
11 matches
Mail list logo