RE: Spark-submit and Windows / Linux mixed network
jar not found :( Seems if I create a directory sim link so that the share path in the same on the unix mount point as in windows, and submit from the drive where the mount point is, then it works. Granted, that's quite an ugly hack. Reverting to serving jar off http (i.e. using a relative path) for the time being. Date: Tue, 11 Nov 2014 20:15:17 +0530 Subject: Re: Spark-submit and Windows / Linux mixed network From: riteshoneinamill...@gmail.com To: as...@live.com CC: user@spark.apache.org Never tried this form but just guessing, What's the output when you submit this jar: \\shares\publish\Spark\app1\someJar.jarusing spark-submit.cmd
Spark-submit and Windows / Linux mixed network
Hi, I'm trying to submit a spark application fro network share to the spark master. Network shares are configured so that the master and all nodes have access to the target ja at (say): \\shares\publish\Spark\app1\someJar.jar And this is mounted on each linux box (i.e. master and workers) at: /mnt/spark/app1/someJar.jar I'm using the following to submit the app from a windows machine: spark-submit.cmd --class Main --master spark://mastername:7077 local:/mnt/spark/app1/someJar.jar However, I get an error saying: Warning: Local jar \mnt\spark\app1\someJar.jar does not exist, skipping. Followed by a ClassNotFoundException: Main. Notice the \'s instead of /s. Does anybody have experience getting something similar to work? Regards, Ashic.
Re: Spark-submit and Windows / Linux mixed network
Never tried this form but just guessing, What's the output when you submit this jar: \\shares\publish\Spark\app1\ someJar.jar using spark-submit.cmd