]]) = {
this(config)
this.preferredNodeLocationData = preferredNodeLocationData
}
--
bit1...@163.com
*From:* Shushant Arora shushantaror...@gmail.com
*Date:* 2015-05-31 22:54
*To:* user user@spark.apache.org
*Subject:* data localisation in spark
I want
@spark.apache.org
*Subject:* data localisation in spark
I want to understand how spark takes care of data localisation in
cluster mode when run on YARN.
1.Driver program asks ResourceManager for executors. Does it tell yarn's
RM to check HDFS blocks of input data and then allocate executors
}
--
bit1...@163.com
*From:* Shushant Arora shushantaror...@gmail.com
*Date:* 2015-05-31 22:54
*To:* user user@spark.apache.org
*Subject:* data localisation in spark
I want to understand how spark takes care of data localisation in cluster
mode when run on YARN
...@gmail.com
*Date:* 2015-05-31 22:54
*To:* user user@spark.apache.org
*Subject:* data localisation in spark
I want to understand how spark takes care of data localisation in
cluster mode when run on YARN.
1.Driver program asks ResourceManager for executors. Does it tell yarn's
RM
Hi Shushant,
Spark currently makes no effort to request executors based on data locality
(although it does try to schedule tasks within executors based on data
locality). We're working on adding this capability at SPARK-4352
https://issues.apache.org/jira/browse/SPARK-4352.
-Sandy
On Sun, May
I want to understand how spark takes care of data localisation in cluster
mode when run on YARN.
1.Driver program asks ResourceManager for executors. Does it tell yarn's RM
to check HDFS blocks of input data and then allocate executors to it.
And executors remain fixed throughout application or
