[ https://issues.apache.org/jira/browse/SPARK-12559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16080101#comment-16080101 ]
Apache Spark commented on SPARK-12559: -------------------------------------- User 'skonto' has created a pull request for this issue: https://github.com/apache/spark/pull/18587 > Cluster mode doesn't work with --packages > ----------------------------------------- > > Key: SPARK-12559 > URL: https://issues.apache.org/jira/browse/SPARK-12559 > Project: Spark > Issue Type: Bug > Components: Spark Submit > Affects Versions: 1.3.0 > Reporter: Andrew Or > > From the mailing list: > {quote} > Another problem I ran into that you also might is that --packages doesn't > work with --deploy-mode cluster. It downloads the packages to a temporary > location on the node running spark-submit, then passes those paths to the > node that is running the Driver, but since that isn't the same machine, it > can't find anything and fails. The driver process *should* be the one > doing the downloading, but it isn't. I ended up having to create a fat JAR > with all of the dependencies to get around that one. > {quote} > The problem is that we currently don't upload jars to the cluster. It seems > to fix this we either (1) do upload jars, or (2) just run the packages code > on the driver side. I slightly prefer (2) because it's simpler. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org