[jira] [Commented] (SPARK-22622) OutOfMemory thrown by Closure Serializer without proper failure propagation

2017-11-28 Thread Raghavendra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269974#comment-16269974 ] Raghavendra commented on SPARK-22622: - [~srowen] I am broadcasting all my large data strucures and im

[jira] [Commented] (SPARK-22622) OutOfMemory thrown by Closure Serializer without proper failure propagation

2017-11-28 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269935#comment-16269935 ] Sean Owen commented on SPARK-22622: --- The error isn't out of memory. It's that an array is too large,

[jira] [Commented] (SPARK-22622) OutOfMemory thrown by Closure Serializer without proper failure propagation

2017-11-28 Thread Raghavendra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269917#comment-16269917 ] Raghavendra commented on SPARK-22622: - The output of previous Stage is around 70GB and my driver