[
https://issues.apache.org/jira/browse/SPARK-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Patrick Wendell resolved SPARK-1950.
------------------------------------
Resolution: Duplicate
> spark on yarn can't start
> --------------------------
>
> Key: SPARK-1950
> URL: https://issues.apache.org/jira/browse/SPARK-1950
> Project: Spark
> Issue Type: Bug
> Components: YARN
> Affects Versions: 1.0.0
> Reporter: Guoqiang Li
> Priority: Blocker
>
> {{HADOOP_CONF_DIR=/etc/hadoop/conf ./bin/spark-submit --archives
> /input/lbs/recommend/toona/spark/conf toona-assembly.jar 20140521}}throw an
> exception:
> {code}
> Exception in thread "main" java.io.FileNotFoundException: File
> file:/input/lbs/recommend/toona/spark/conf does not exist
> at
> org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:511)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:724)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:501)
> at
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:402)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:337)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:289)
> at
> org.apache.spark.deploy.yarn.ClientBase$class.org$apache$spark$deploy$yarn$ClientBase$$copyRemoteFile(ClientBase.scala:162)
> at
> org.apache.spark.deploy.yarn.ClientBase$$anonfun$prepareLocalResources$4$$anonfun$apply$2.apply(ClientBase.scala:237)
> at
> org.apache.spark.deploy.yarn.ClientBase$$anonfun$prepareLocalResources$4$$anonfun$apply$2.apply(ClientBase.scala:232)
> at
> scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
> at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:108)
> at
> org.apache.spark.deploy.yarn.ClientBase$$anonfun$prepareLocalResources$4.apply(ClientBase.scala:232)
> at
> org.apache.spark.deploy.yarn.ClientBase$$anonfun$prepareLocalResources$4.apply(ClientBase.scala:230)
> at scala.collection.immutable.List.foreach(List.scala:318)
> at
> org.apache.spark.deploy.yarn.ClientBase$class.prepareLocalResources(ClientBase.scala:230)
> at
> org.apache.spark.deploy.yarn.Client.prepareLocalResources(Client.scala:39)
> at org.apache.spark.deploy.yarn.Client.runApp(Client.scala:74)
> at org.apache.spark.deploy.yarn.Client.run(Client.scala:96)
> at org.apache.spark.deploy.yarn.Client$.main(Client.scala:186)
> at org.apache.spark.deploy.yarn.Client.main(Client.scala)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:292)
> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:55)
> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
> {code}
> {{HADOOP_CONF_DIR=/etc/hadoop/conf ./bin/spark-submit --archives
> hdfs://10dian72:8020/input/lbs/recommend/toona/spark/conf toona-assembly.jar
> 20140521}} work.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
