[
https://issues.apache.org/jira/browse/SPARK-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reynold Xin updated SPARK-2521:
-------------------------------
Description:
This can substantially reduce task size, as well as being able to support very
large closures (e.g. closures that reference large variables).
Once this is in, we can also remove broadcasting the Hadoop JobConf.
was:
This can substantially reduce task size, as well as being able to support very
large closures (e.g. closures that reference large variables).
> Broadcast RDD object once per TaskSet (instead of sending it for every task)
> ----------------------------------------------------------------------------
>
> Key: SPARK-2521
> URL: https://issues.apache.org/jira/browse/SPARK-2521
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Reporter: Reynold Xin
> Assignee: Reynold Xin
>
> This can substantially reduce task size, as well as being able to support
> very large closures (e.g. closures that reference large variables).
> Once this is in, we can also remove broadcasting the Hadoop JobConf.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
