[jira] [Commented] (SPARK-21752) Config spark.jars.packages is ignored in SparkSession config

[ https://issues.apache.org/jira/browse/SPARK-21752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16138948#comment-16138948 ] Jakub Nowacki commented on SPARK-21752: --- OK I did one more extra test and, indeed, on the newest

[jira] [Commented] (SPARK-21752) Config spark.jars.packages is ignored in SparkSession config

[ https://issues.apache.org/jira/browse/SPARK-21752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16135669#comment-16135669 ] Stavros Kontopoulos commented on SPARK-21752: - In order to be correct before we update

[jira] [Commented] (SPARK-21752) Config spark.jars.packages is ignored in SparkSession config

[ https://issues.apache.org/jira/browse/SPARK-21752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16135082#comment-16135082 ] Sean Owen commented on SPARK-21752: --- Docs can't hurt, especially if they can be applied consistently. I

[jira] [Commented] (SPARK-21752) Config spark.jars.packages is ignored in SparkSession config

[ https://issues.apache.org/jira/browse/SPARK-21752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16135015#comment-16135015 ] Jakub Nowacki commented on SPARK-21752: --- [~srowen] Do you think we can create some sort of

[jira] [Commented] (SPARK-21752) Config spark.jars.packages is ignored in SparkSession config

[ https://issues.apache.org/jira/browse/SPARK-21752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16134327#comment-16134327 ] Sean Owen commented on SPARK-21752: --- This is an interesting discussion about what happens to work when,

[jira] [Commented] (SPARK-21752) Config spark.jars.packages is ignored in SparkSession config

[ https://issues.apache.org/jira/browse/SPARK-21752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16133565#comment-16133565 ] Stavros Kontopoulos commented on SPARK-21752: - [~jsnowacki] What I am doing is not manual its

[jira] [Commented] (SPARK-21752) Config spark.jars.packages is ignored in SparkSession config

[ https://issues.apache.org/jira/browse/SPARK-21752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16131945#comment-16131945 ] Jakub Nowacki commented on SPARK-21752: --- [~skonto] What you are doing is in fact starting manually

[jira] [Commented] (SPARK-21752) Config spark.jars.packages is ignored in SparkSession config

[ https://issues.apache.org/jira/browse/SPARK-21752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16130525#comment-16130525 ] Stavros Kontopoulos commented on SPARK-21752: - [~jsnowacki] I dont think I am doing anything

[jira] [Commented] (SPARK-21752) Config spark.jars.packages is ignored in SparkSession config

[ https://issues.apache.org/jira/browse/SPARK-21752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16130282#comment-16130282 ] Jakub Nowacki commented on SPARK-21752: --- [~skonto] Well, I'm not sure where you're failing here. If

[jira] [Commented] (SPARK-21752) Config spark.jars.packages is ignored in SparkSession config

[ https://issues.apache.org/jira/browse/SPARK-21752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16130274#comment-16130274 ] Jakub Nowacki commented on SPARK-21752: --- OK I get the point. I think we should only consider this

[jira] [Commented] (SPARK-21752) Config spark.jars.packages is ignored in SparkSession config

[ https://issues.apache.org/jira/browse/SPARK-21752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16130259#comment-16130259 ] Stavros Kontopoulos commented on SPARK-21752: - [~jsnowacki] Do you have a step by step script

[jira] [Commented] (SPARK-21752) Config spark.jars.packages is ignored in SparkSession config

[ https://issues.apache.org/jira/browse/SPARK-21752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16130161#comment-16130161 ] Sean Owen commented on SPARK-21752: --- I think this is, in any event, not something that's intended to

[jira] [Commented] (SPARK-21752) Config spark.jars.packages is ignored in SparkSession config

[ https://issues.apache.org/jira/browse/SPARK-21752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16130150#comment-16130150 ] Jakub Nowacki commented on SPARK-21752: --- [~skonto] Jupyter is not passing many environmental

[jira] [Commented] (SPARK-21752) Config spark.jars.packages is ignored in SparkSession config

[ https://issues.apache.org/jira/browse/SPARK-21752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16130132#comment-16130132 ] Stavros Kontopoulos commented on SPARK-21752: - [~jerryshao] That is true. I was curious which

[jira] [Commented] (SPARK-21752) Config spark.jars.packages is ignored in SparkSession config

[ https://issues.apache.org/jira/browse/SPARK-21752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16130078#comment-16130078 ] Saisai Shao commented on SPARK-21752: - For configurations like "spark.jars.packages" should be

[jira] [Commented] (SPARK-21752) Config spark.jars.packages is ignored in SparkSession config

[ https://issues.apache.org/jira/browse/SPARK-21752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16129589#comment-16129589 ] Stavros Kontopoulos commented on SPARK-21752: - [~jsnowacki] I tried the second example with

[jira] [Commented] (SPARK-21752) Config spark.jars.packages is ignored in SparkSession config

[ https://issues.apache.org/jira/browse/SPARK-21752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16129512#comment-16129512 ] Stavros Kontopoulos commented on SPARK-21752: - [~jsnowacki] Actually it didnt help I tried

[jira] [Commented] (SPARK-21752) Config spark.jars.packages is ignored in SparkSession config

[ https://issues.apache.org/jira/browse/SPARK-21752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16129502#comment-16129502 ] Jakub Nowacki commented on SPARK-21752: --- Not sure if [SPARK-11520] would help in this case. I'll

[jira] [Commented] (SPARK-21752) Config spark.jars.packages is ignored in SparkSession config

[ https://issues.apache.org/jira/browse/SPARK-21752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16129488#comment-16129488 ] Stavros Kontopoulos commented on SPARK-21752: - [~jsnowacki] I didnt say parenthesis is the

[jira] [Commented] (SPARK-21752) Config spark.jars.packages is ignored in SparkSession config

[ https://issues.apache.org/jira/browse/SPARK-21752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16129475#comment-16129475 ] Jakub Nowacki commented on SPARK-21752: --- OK so you don't need session creation with pyspark command

[jira] [Commented] (SPARK-21752) Config spark.jars.packages is ignored in SparkSession config

[ https://issues.apache.org/jira/browse/SPARK-21752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16129472#comment-16129472 ] Stavros Kontopoulos commented on SPARK-21752: - @Jakub Nowacki I figured out the import, no I

[jira] [Commented] (SPARK-21752) Config spark.jars.packages is ignored in SparkSession config

[ https://issues.apache.org/jira/browse/SPARK-21752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16129466#comment-16129466 ] Jakub Nowacki commented on SPARK-21752: --- Not really, maybe you're missing {{import pyspark}} at the

[jira] [Commented] (SPARK-21752) Config spark.jars.packages is ignored in SparkSession config

[ https://issues.apache.org/jira/browse/SPARK-21752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16129454#comment-16129454 ] Stavros Kontopoulos commented on SPARK-21752: - I couldnt reproduce this one:

[jira] [Commented] (SPARK-21752) Config spark.jars.packages is ignored in SparkSession config

[ https://issues.apache.org/jira/browse/SPARK-21752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16129432#comment-16129432 ] Jakub Nowacki commented on SPARK-21752: --- [~skonto] Not sure which one you couldn't reproduce. Using

[jira] [Commented] (SPARK-21752) Config spark.jars.packages is ignored in SparkSession config

[ https://issues.apache.org/jira/browse/SPARK-21752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16129406#comment-16129406 ] Stavros Kontopoulos commented on SPARK-21752: - "unclear why passing SparkConf via

[jira] [Commented] (SPARK-21752) Config spark.jars.packages is ignored in SparkSession config

[ https://issues.apache.org/jira/browse/SPARK-21752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16129322#comment-16129322 ] Jakub Nowacki commented on SPARK-21752: --- Well, it seems so, but it is, at least logically, unclear

[jira] [Commented] (SPARK-21752) Config spark.jars.packages is ignored in SparkSession config

[ https://issues.apache.org/jira/browse/SPARK-21752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16129296#comment-16129296 ] Sean Owen commented on SPARK-21752: --- At that point, your app has already launched. You can't change the