DK created SPARK-13284:
--------------------------
Summary: Cannot submit app from Windows
java.io.FileNotFoundException: /C:
Key: SPARK-13284
URL: https://issues.apache.org/jira/browse/SPARK-13284
Project: Spark
Issue Type: Bug
Components: Spark Submit
Affects Versions: 1.6.0
Reporter: DK
I've tried multiple different ways to submit my spark java application from my
Windows machine to a remote Spark master running on Linux.
I'm consistently hitting {{java.io.FileNotFoundException: /C:}} error.
I've tried surrounding the jar path with and without quotes.
I've also tried including {{file:///}} and {{file:/}}
The file has full permissions
{code}
C:\>Icacls C:/spark-app-1.0-SNAPSHOT.jar
C:/spark-app-1.0-SNAPSHOT.jar BUILTIN\Administrators:(M,WDAC,WO)
GLOBAL\Domain Users:(RX,W)
Everyone:(RX,W)
NT AUTHORITY\Authenticated Users:(RX,W)
NT AUTHORITY\SYSTEM:(I)(RX,W)
BUILTIN\Users:(I)(RX)
NT AUTHORITY\Authenticated Users:(I)(RX,W)
Mandatory Label\High Mandatory Level:(I)(NW)
Submit failure:
{code}
C:\>"C:/spark-1.6.0-bin-hadoop2.6/bin/spark-submit.cmd" --master
spark://10.134.41.16:6066 --deploy-mode cluster --class
com.avaya.ingensg.devops.spark.es.JavaDirectKafkaWordCount
file:///C:/spark-app-1.0-SNAPSHOT.jar kafka-server:9092 dessie
Running Spark using the REST application submission protocol.
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
16/02/11 16:00:58 INFO RestSubmissionClient: Submitting a request to launch an
application in spark://10.134.41.16:6066.
16/02/11 16:00:58 INFO RestSubmissionClient: Submission successfully created as
driver-20160211155905-0013. Polling submission state...
16/02/11 16:00:58 INFO RestSubmissionClient: Submitting a request for the
status of submission driver-20160211155905-0013 in spark://10.134.41.16:6066.
16/02/11 16:00:58 INFO RestSubmissionClient: State of driver
driver-20160211155905-0013 is now ERROR.
16/02/11 16:00:58 INFO RestSubmissionClient: Driver is running on worker
worker-20160211152455-172.18.0.8-55808 at 172.18.0.8:55808.
16/02/11 16:00:58 ERROR RestSubmissionClient: Exception from the cluster:
java.io.FileNotFoundException: /C:/spark-app-1.0-SNAPSHOT.jar (No such file or
directory)
java.io.FileInputStream.open0(Native Method)
java.io.FileInputStream.open(FileInputStream.java:195)
java.io.FileInputStream.<init>(FileInputStream.java:138)
org.spark-project.guava.io.Files$FileByteSource.openStream(Files.java:124)
org.spark-project.guava.io.Files$FileByteSource.openStream(Files.java:114)
org.spark-project.guava.io.ByteSource.copyTo(ByteSource.java:202)
org.spark-project.guava.io.Files.copy(Files.java:436)
org.apache.spark.util.Utils$.org$apache$spark$util$Utils$$copyRecursive(Utils.scala:539)
org.apache.spark.util.Utils$.copyFile(Utils.scala:510)
org.apache.spark.util.Utils$.doFetchFile(Utils.scala:595)
org.apache.spark.util.Utils$.fetchFile(Utils.scala:394)
org.apache.spark.deploy.worker.DriverRunner.org$apache$spark$deploy$worker$DriverRunner$$downloadUserJar(DriverRunner.scala:150)
org.apache.spark.deploy.worker.DriverRunner$$anon$1.run(DriverRunner.scala:79)
16/02/11 16:00:58 INFO RestSubmissionClient: Server responded with
CreateSubmissionResponse:
{
"action" : "CreateSubmissionResponse",
"message" : "Driver successfully submitted as driver-20160211155905-0013",
"serverSparkVersion" : "1.6.0",
"submissionId" : "driver-20160211155905-0013",
"success" : true
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
