[ https://issues.apache.org/jira/browse/SPARK-12825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15102163#comment-15102163 ]
Alex Nederlof commented on SPARK-12825: --------------------------------------- It's a `307 temporary redirect` response code, returned by Nexus. The full command was {code} spark-submit --master spark://spark.ourserver.com:6066 --deploy-mode cluster --class me.our.SparkJob --executor-memory 4g --driver-memory 4g "http://our.nexus.repo/service/local/artifact/maven/redirect?r=snapshots&g=me.magnet&a=sparkjob&v=LATEST" {code} In curl you'd see {code} [alex in ~]$ curl -v "http://our.nexus.repo/service/local/artifact/maven/redirect?r=snapshots&g=me.magnet&a=sparkjob&v=LATEST" * Trying 10.8.0.2... * Connected to our.nexus.repo (10.8.0.2) port 80 (#0) > GET redirect?r=snapshots&g=me.magnet&a=sparkjob&v=LATEST HTTP/1.1 > Host: our.nexus.repo > User-Agent: curl/7.43.0 > Accept: */* > < HTTP/1.1 307 Temporary Redirect < Server: nginx/1.8.0 < Date: Fri, 15 Jan 2016 17:36:58 GMT < Content-Type: application/xml; charset=ISO-8859-1 < Content-Length: 198 < Connection: keep-alive < X-Frame-Options: SAMEORIGIN < X-Content-Type-Options: nosniff < Location: http://our.nexus.repo/service/local/repositories/snapshots/content/me/magnet/spark-job/1.1-SNAPSHOT/spark-job-1.1-20151231.112230-92.jar < Vary: Accept-Charset, Accept-Encoding, Accept-Language, Accept {code} > Spark-submit Jar URL loading fails on redirect > ---------------------------------------------- > > Key: SPARK-12825 > URL: https://issues.apache.org/jira/browse/SPARK-12825 > Project: Spark > Issue Type: Bug > Affects Versions: 1.6.0 > Reporter: Alex Nederlof > Priority: Minor > > When you use spark-submit, and pass the jar as a URL, it fails when the URL > redirects. > The log prints: > {code} > 16/01/14 14:26:43 INFO Utils: Fetching http://myUrl/my.jar to > /opt/spark/spark-1.6.0-bin-hadoop2.6/work/driver-20160114142642-0010/fetchFileTemp8495494631100918254.tmp > {code} > However, that file doesn't exist, but a file called "redirect" is created, > with the appropriate content. > After that, the driver fails with > {code} > 16/01/14 14:26:43 WARN Worker: Driver driver-20160114142642-0010 failed with > unrecoverable exception: java.lang.Exception: Did not see expected jar my.jar > in /opt/spark/spark-1.6.0-bin-hadoop2.6/work/driver-20160114142642-0010 > {code} > Here's the related code: > https://github.com/apache/spark/blob/56cdbd654d54bf07a063a03a5c34c4165818eeb2/core/src/main/scala/org/apache/spark/util/Utils.scala#L583-L603 > My Scala chops aren't up to this challenge, otherwise I would have made a > patch. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org