[
https://issues.apache.org/jira/browse/SPARK-9384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643248#comment-14643248
]
Mark Grover edited comment on SPARK-9384 at 7/27/15 7:25 PM:
-------------------------------------------------------------
Thanks Sean. {{--jars}} does something similar but not quite.
{{*.*.extraClassPath}} properties modifies the system classpath of both the
executors and drivers while {{--jars}} only modifies the class loader running
the driver/executor code.
This change would allow advanced users to override libraries shipped with
Spark, or add jars like some JDBC driver, that don't work unless they are
adding to the system classpath.
was (Author: mgrover):
Thanks Sean. {{--jars}} does something similar but not quite.
{{*.*.extraClassPath}} properties modifies the system classpath of both the
executors and drivers while {{--jars}} only modifies the class loader running
the driver/executor code.
This change would allow advanced users to override libraries shipped with
Spark, or add jars like some JDBC driver, that don't work unless they are
adding to the system classpath.
extraClassPath modifies the system classpath (not the class loader running the
executor code, for example), which allows you to override libraries shipped
with Spark, or add stuff that does not work when not added to the system
classpath (like some JDBC drivers).
> Easier setting of executor and driver classpath
> -----------------------------------------------
>
> Key: SPARK-9384
> URL: https://issues.apache.org/jira/browse/SPARK-9384
> Project: Spark
> Issue Type: Improvement
> Components: Spark Submit
> Affects Versions: 1.4.1
> Reporter: Mark Grover
> Priority: Minor
>
> Currently one can set extra jars in the driver classpath using
> {{spark.driver.extraClassPath}} property. Similarly, the executor's extra
> classpath can be set using {{spark.executor.extraClassPath}} property.
> However, many users (myself) included have often suffered from the dogma of
> setting one and missing the other. Causing a lot of wasted time. And, this
> JIRA is an effort to see if the status quo can be made better.
> More details in comments.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
