[ https://issues.apache.org/jira/browse/SPARK-6584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Apache Spark reassigned SPARK-6584: ----------------------------------- Assignee: Apache Spark > Provide ExecutorPrefixTaskLocation to support the rdd which can be aware of > partition's executor location. > ----------------------------------------------------------------------------------------------------------- > > Key: SPARK-6584 > URL: https://issues.apache.org/jira/browse/SPARK-6584 > Project: Spark > Issue Type: Sub-task > Components: Spark Core > Affects Versions: 1.4.0 > Reporter: SaintBacchus > Assignee: Apache Spark > > The function *RDD.getPreferredLocations* can only be set the host awareness > prefer locations. > If some *RDD* wants to be scheduled by executor(such as BlockRDD), spark can > do nothing for this. > So I want to provide *ExecutorPrefixTaskLocation* to support the rdd which > can be aware of partition's executor location. This mechanism can avoid data > transfor in the case of many executor in the same host. > I think it's very useful especially for *SparkStreaming* since the > *Receriver* save data into the *BlockManger* and then become a BlockRDD -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org