[jira] [Commented] (SPARK-13531) Some DataFrame joins stopped working with UnsupportedOperationException: No size estimation available for objects

2016-05-02 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15267611#comment-15267611 ] Yin Huai commented on SPARK-13531: -- I think this one has been fixed by SPARK-14578

[jira] [Commented] (SPARK-13531) Some DataFrame joins stopped working with UnsupportedOperationException: No size estimation available for objects

2016-05-02 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15267569#comment-15267569 ] koert kuipers commented on SPARK-13531: --- i do not see this exact error anymore. it has now become a

[jira] [Commented] (SPARK-13531) Some DataFrame joins stopped working with UnsupportedOperationException: No size estimation available for objects

2016-05-02 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15267390#comment-15267390 ] Yin Huai commented on SPARK-13531: -- I tried it out and it seems work (fixed at some point). [~koert] Can

[jira] [Commented] (SPARK-13531) Some DataFrame joins stopped working with UnsupportedOperationException: No size estimation available for objects

2016-03-27 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15213612#comment-15213612 ] koert kuipers commented on SPARK-13531: --- this seems to be like the simplest code that reproduces

[jira] [Commented] (SPARK-13531) Some DataFrame joins stopped working with UnsupportedOperationException: No size estimation available for objects

2016-03-27 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15213363#comment-15213363 ] koert kuipers commented on SPARK-13531: --- oh its because statistics for UnaryNode is based on

[jira] [Commented] (SPARK-13531) Some DataFrame joins stopped working with UnsupportedOperationException: No size estimation available for objects

2016-03-27 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15213361#comment-15213361 ] koert kuipers commented on SPARK-13531: --- before the optimization both MapPartitions have output

[jira] [Commented] (SPARK-13531) Some DataFrame joins stopped working with UnsupportedOperationException: No size estimation available for objects

2016-03-27 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15213358#comment-15213358 ] koert kuipers commented on SPARK-13531: --- i did a little more digging and i think this is what

[jira] [Commented] (SPARK-13531) Some DataFrame joins stopped working with UnsupportedOperationException: No size estimation available for objects

2016-03-03 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15179201#comment-15179201 ] Apache Spark commented on SPARK-13531: -- User 'zuowang' has created a pull request for this issue:

[jira] [Commented] (SPARK-13531) Some DataFrame joins stopped working with UnsupportedOperationException: No size estimation available for objects

2016-03-02 Thread Zuo Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177441#comment-15177441 ] Zuo Wang commented on SPARK-13531: -- Caused by the commit in

[jira] [Commented] (SPARK-13531) Some DataFrame joins stopped working with UnsupportedOperationException: No size estimation available for objects

2016-02-28 Thread koert kuipers (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15171420#comment-15171420 ] koert kuipers commented on SPARK-13531: --- can you try this? {noformat} val df1 = sc.makeRDD(1 to

[jira] [Commented] (SPARK-13531) Some DataFrame joins stopped working with UnsupportedOperationException: No size estimation available for objects

2016-02-28 Thread Zuo Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15171354#comment-15171354 ] Zuo Wang commented on SPARK-13531: -- Unable to reproduce, here is my steps in spark-shell: val rowRDD1 =