[jira] [Commented] (SPARK-18483) spark on yarn always connect to yarn resourcemanager at 0.0.0.0:8032
[ https://issues.apache.org/jira/browse/SPARK-18483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15676265#comment-15676265 ] Sean Owen commented on SPARK-18483: --- I don't think that path is valid, even on Windows. Try a Linux-style path. To be honest I'm not sure all of this works on Windows to begin with. > spark on yarn always connect to yarn resourcemanager at 0.0.0.0:8032 > -- > > Key: SPARK-18483 > URL: https://issues.apache.org/jira/browse/SPARK-18483 > Project: Spark > Issue Type: Bug > Components: Spark Core >Affects Versions: 2.0.1 > Environment: java8 > SBT0.13 > scala2.11.8 > spark-2.0.1-bin-hadoop2.6 >Reporter: inred > > I have installed the yarn resource manager at 192.168.13.159:8032 and have > set YARN_CONF_DIR environment var and have the yarn-site.xml > configured as the following, but it always connects to 0.0.0.0:8032 instead > of > 192.168.13.159:8032 > set environment > E:\app>set yarn > YARN_CONF_DIR=D:\Documents\download\hadoop > E:\app>set had > HADOOP_HOME=D:\Documents\download\hadoop > E:\app>cat D:\Documents\download\hadoop\yarn-site.xml > > yarn.resourcemanager.address > 192.168.13.159:8032 > > "C:\Program Files\Java\jdk1.8.0_92\bin\java" -Didea.launcher.port=7532 > "-Didea.launcher.bin.path=C:\Program Files (x86)\JetBrains\IntelliJ IDEA > Community Edition 2016.2.5\bin" -Dfile.encoding=UTF-8 -classpath "C:\Program > Files\Java\jdk1.8.0_92\jre\lib\charsets.jar;C:\Program > Files\Java\jdk1.8.0_92\jre\lib\deploy.jar;C:\Program > Files\Java\jdk1.8.0_92\jre\lib\ext\access-bridge-64.jar;C:\Program > Files\Java\jdk1.8.0_92\jre\lib\ext\cldrdata.jar;C:\Program > Files\Java\jdk1.8.0_92\jre\lib\ext\dnsns.jar;C:\Program > Files\Java\jdk1.8.0_92\jre\lib\ext\jaccess.jar;C:\Program > Files\Java\jdk1.8.0_92\jre\lib\ext\jfxrt.jar;C:\Program > Files\Java\jdk1.8.0_92\jre\lib\ext\localedata.jar;C:\Program > Files\Java\jdk1.8.0_92\jre\lib\ext\nashorn.jar;C:\Program > Files\Java\jdk1.8.0_92\jre\lib\ext\sunec.jar;C:\Program > Files\Java\jdk1.8.0_92\jre\lib\ext\sunjce_provider.jar;C:\Program > Files\Java\jdk1.8.0_92\jre\lib\ext\sunmscapi.jar;C:\Program > Files\Java\jdk1.8.0_92\jre\lib\ext\sunpkcs11.jar;C:\Program > Files\Java\jdk1.8.0_92\jre\lib\ext\zipfs.jar;C:\Program > Files\Java\jdk1.8.0_92\jre\lib\javaws.jar;C:\Program > Files\Java\jdk1.8.0_92\jre\lib\jce.jar;C:\Program > Files\Java\jdk1.8.0_92\jre\lib\jfr.jar;C:\Program > Files\Java\jdk1.8.0_92\jre\lib\jfxswt.jar;C:\Program > Files\Java\jdk1.8.0_92\jre\lib\jsse.jar;C:\Program > Files\Java\jdk1.8.0_92\jre\lib\management-agent.jar;C:\Program > Files\Java\jdk1.8.0_92\jre\lib\plugin.jar;C:\Program > Files\Java\jdk1.8.0_92\jre\lib\resources.jar;C:\Program >
[jira] [Commented] (SPARK-18483) spark on yarn always connect to yarn resourcemanager at 0.0.0.0:8032
[ https://issues.apache.org/jira/browse/SPARK-18483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15675388#comment-15675388 ] inred commented on SPARK-18483: --- it failed even when i set HADOOP_CONF_DIR=%HADOOP_HOME%\etc\hadoop, i submit it from windows development node to remote Linux yarn cluster > spark on yarn always connect to yarn resourcemanager at 0.0.0.0:8032 > -- > > Key: SPARK-18483 > URL: https://issues.apache.org/jira/browse/SPARK-18483 > Project: Spark > Issue Type: Bug > Components: Spark Core >Affects Versions: 2.0.1 > Environment: java8 > SBT0.13 > scala2.11.8 > spark-2.0.1-bin-hadoop2.6 >Reporter: inred > > I have installed the yarn resource manager at 192.168.13.159:8032 and have > set YARN_CONF_DIR environment var and have the yarn-site.xml > configured as the following, but it always connects to 0.0.0.0:8032 instead > of > 192.168.13.159:8032 > set environment > E:\app>set yarn > YARN_CONF_DIR=D:\Documents\download\hadoop > E:\app>set had > HADOOP_HOME=D:\Documents\download\hadoop > E:\app>cat D:\Documents\download\hadoop\yarn-site.xml > > yarn.resourcemanager.address > 192.168.13.159:8032 > > "C:\Program Files\Java\jdk1.8.0_92\bin\java" -Didea.launcher.port=7532 > "-Didea.launcher.bin.path=C:\Program Files (x86)\JetBrains\IntelliJ IDEA > Community Edition 2016.2.5\bin" -Dfile.encoding=UTF-8 -classpath "C:\Program > Files\Java\jdk1.8.0_92\jre\lib\charsets.jar;C:\Program > Files\Java\jdk1.8.0_92\jre\lib\deploy.jar;C:\Program > Files\Java\jdk1.8.0_92\jre\lib\ext\access-bridge-64.jar;C:\Program > Files\Java\jdk1.8.0_92\jre\lib\ext\cldrdata.jar;C:\Program > Files\Java\jdk1.8.0_92\jre\lib\ext\dnsns.jar;C:\Program > Files\Java\jdk1.8.0_92\jre\lib\ext\jaccess.jar;C:\Program > Files\Java\jdk1.8.0_92\jre\lib\ext\jfxrt.jar;C:\Program > Files\Java\jdk1.8.0_92\jre\lib\ext\localedata.jar;C:\Program > Files\Java\jdk1.8.0_92\jre\lib\ext\nashorn.jar;C:\Program > Files\Java\jdk1.8.0_92\jre\lib\ext\sunec.jar;C:\Program > Files\Java\jdk1.8.0_92\jre\lib\ext\sunjce_provider.jar;C:\Program > Files\Java\jdk1.8.0_92\jre\lib\ext\sunmscapi.jar;C:\Program > Files\Java\jdk1.8.0_92\jre\lib\ext\sunpkcs11.jar;C:\Program > Files\Java\jdk1.8.0_92\jre\lib\ext\zipfs.jar;C:\Program > Files\Java\jdk1.8.0_92\jre\lib\javaws.jar;C:\Program > Files\Java\jdk1.8.0_92\jre\lib\jce.jar;C:\Program > Files\Java\jdk1.8.0_92\jre\lib\jfr.jar;C:\Program > Files\Java\jdk1.8.0_92\jre\lib\jfxswt.jar;C:\Program > Files\Java\jdk1.8.0_92\jre\lib\jsse.jar;C:\Program > Files\Java\jdk1.8.0_92\jre\lib\management-agent.jar;C:\Program > Files\Java\jdk1.8.0_92\jre\lib\plugin.jar;C:\Program > Files\Java\jdk1.8.0_92\jre\lib\resources.jar;C:\Program >
