When I try to execute my task with Spark it starts to copy the jars it
needs to HDFS and it finally fails, I don't know exactly why. I have
checked HDFS and it copies the files, so, it seems to work that part.
I changed the log level to debug but there's nothing else to help.
What else does Spark need to copy that it could be an empty string?
Spark assembly has been built with Hive, including Datanucleus jars on classpath
15/01/08 14:06:32 WARN NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where
applicable
15/01/08 14:06:32 INFO RMProxy: Connecting to ResourceManager at
vmlbyarnl01.lvtc.gsnet.corp/180.133.240.174:8050
15/01/08 14:06:33 INFO Client: Got cluster metric info from
ResourceManager, number of NodeManagers: 3
15/01/08 14:06:33 INFO Client: Max mem capabililty of a single
resource in this cluster 97280
15/01/08 14:06:33 INFO Client: Preparing Local resources
15/01/08 14:06:34 WARN BlockReaderLocal: The short-circuit local reads
feature cannot be used because libhadoop cannot be loaded.
15/01/08 14:06:34 INFO Client: Uploading
file:/home/spark-1.1.1-bin-hadoop2.4/lib/spark-assembly-1.1.1-hadoop2.4.0.jar
to
hdfs://vmlbnanodl01.lvtc.gsnet.corp:8020/user/hdfs/.sparkStaging/application_1417607109980_0017/spark-assembly-1.1.1-hadoop2.4.0.jar
15/01/08 14:06:42 INFO Client: Uploading
file:/user/local/etc/lib/my-spark-streaming-scala.jar to
hdfs://vmlbnanodl01.lvtc.gsnet.corp:8020/user/hdfs/.sparkStaging/application_1417607109980_0017/my-spark-streaming-scala.jar
Exception in thread "main" java.lang.IllegalArgumentException: Can not
create a Path from an empty string
at org.apache.hadoop.fs.Path.checkPathArg(Path.java:127)
at org.apache.hadoop.fs.Path.<init>(Path.java:135)
at org.apache.hadoop.fs.Path.<init>(Path.java:94)
at
org.apache.spark.deploy.yarn.ClientBase$class.copyRemoteFile(ClientBase.scala:159)
at org.apache.spark.deploy.yarn.Client.copyRemoteFile(Client.scala:37)
at
org.apache.spark.deploy.yarn.ClientBase$$anonfun$prepareLocalResources$5$$anonfun$apply$2.apply(ClientBase.scala:236)
at
org.apache.spark.deploy.yarn.ClientBase$$anonfun$prepareLocalResources$5$$anonfun$apply$2.apply(ClientBase.scala:231)
at
scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:108)
at
org.apache.spark.deploy.yarn.ClientBase$$anonfun$prepareLocalResources$5.apply(ClientBase.scala:231)
at
org.apache.spark.deploy.yarn.ClientBase$$anonfun$prepareLocalResources$5.apply(ClientBase.scala:229)
at scala.collection.immutable.List.foreach(List.scala:318)
at
org.apache.spark.deploy.yarn.ClientBase$class.prepareLocalResources(ClientBase.scala:229)
at
org.apache.spark.deploy.yarn.Client.prepareLocalResources(Client.scala:37)
at org.apache.spark.deploy.yarn.Client.runApp(Client.scala:74)
at org.apache.spark.deploy.yarn.Client.run(Client.scala:96)
at org.apache.spark.deploy.yarn.Client$.main(Client.scala:176)
at org.apache.spark.deploy.yarn.Client.main(Client.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:329)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org
