[ https://issues.apache.org/jira/browse/SPARK-10149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14718272#comment-14718272 ]
Xu Chen edited comment on SPARK-10149 at 8/28/15 9:35 AM: ---------------------------------------------------------- I encounter this issue too at Spark 1.4.1 , so I look around code 1. If you use SparkStandalone you can config {code}SPARK_LOCAL_HOSTNAME{code} at {code}spark-env.sh{code} on each Worker 2 If you deploy Spark on Yarn , you can config {code}yarn.nodemanager.bind-host{code} at {code}yarn-site.xml{code} on Nodemanager BTW, I have confuse for this point , why spark do not automatically get hostname and hold a configuration which is hosename or ip address by manually? was (Author: xuchencn): I encounter this issue too at Spark 1.4.1 , so I look around code 1. If you use SparkStandalone you can config {code}SPARK_LOCAL_HOSTNAME{code} at {code}spark-env.sh{code} 2 If you deploy Spark on Yarn , you can config {code}yarn.nodemanager.bind-host{code} at {code}yarn-site.xml{code} BTW, I have confuse for this point , why spark do not automatically get hostname and hold a configuration which is hosename or ip address by manually? > Locality Level is ANY on "Details for Stage" WebUI page > ------------------------------------------------------- > > Key: SPARK-10149 > URL: https://issues.apache.org/jira/browse/SPARK-10149 > Project: Spark > Issue Type: Bug > Components: Spark Core, Web UI > Affects Versions: 1.4.1 > Reporter: Yun Zhao > > Locality Level is ANY on "Details for Stage" WebUI page > When a sc.textFile(XX) program is running, Locality Level is ANY which should > be NODE_LOCAL on stage 0. > org.apache.spark.scheduler.TaskSetManager > {quote} > // Check for node-local tasks > if (TaskLocality.isAllowed(locality, TaskLocality.NODE_LOCAL)) { > for (index <- speculatableTasks if canRunOnHost(index)) { > val locations = tasks(index).preferredLocations.map(_.host) > if (locations.contains(host)) { > speculatableTasks -= index > return Some((index, TaskLocality.NODE_LOCAL)) > } > } > } > {quote} > The variable "locations" is hostname of HDFS split, which is from > InetAddress.getHostName. > The variable "host" is ip of Executor, which is from > InetAddress.getLocalHost.getHostAddress. > org.apache.spark.deploy.worker.WorkerArguments > {quote} > var host = Utils.localHostName() > {quote} -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org