[ https://issues.apache.org/jira/browse/SPARK-10149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Apache Spark reassigned SPARK-10149: ------------------------------------ Assignee: Apache Spark > Locality Level is ANY on "Details for Stage" WebUI page > ------------------------------------------------------- > > Key: SPARK-10149 > URL: https://issues.apache.org/jira/browse/SPARK-10149 > Project: Spark > Issue Type: Bug > Components: Spark Core, Web UI > Affects Versions: 1.4.1 > Reporter: Yun Zhao > Assignee: Apache Spark > > Locality Level is ANY on "Details for Stage" WebUI page > When a sc.textFile(XX) program is running, Locality Level is ANY which should > be NODE_LOCAL on stage 0. > org.apache.spark.scheduler.TaskSetManager > {quote} > // Check for node-local tasks > if (TaskLocality.isAllowed(locality, TaskLocality.NODE_LOCAL)) { > for (index <- speculatableTasks if canRunOnHost(index)) { > val locations = tasks(index).preferredLocations.map(_.host) > if (locations.contains(host)) { > speculatableTasks -= index > return Some((index, TaskLocality.NODE_LOCAL)) > } > } > } > {quote} > The variable "locations" is hostname of HDFS split, which is from > InetAddress.getHostName. > The variable "host" is ip of Executor, which is from > InetAddress.getLocalHost.getHostAddress. > org.apache.spark.deploy.worker.WorkerArguments > {quote} > var host = Utils.localHostName() > {quote} -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org