[GitHub] zeppelin issue #1255: Adding extra libraries to livy
Github user felixcheung commented on the issue: https://github.com/apache/zeppelin/pull/1255 how about "Add Spark packages support to Livy interpreter"? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1255: Adding extra libraries to livy
Github user mfelgamal commented on the issue: https://github.com/apache/zeppelin/pull/1255 @felixcheung I suggests this title "Adding packages to livy interpreter", Is it Ok ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1255: Adding extra libraries to livy
Github user felixcheung commented on the issue: https://github.com/apache/zeppelin/pull/1255 could you update the title of this PR and JIRA ZEPPELIN-1258, if we are changing how we would approach this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1255: Adding extra libraries to livy
Github user mfelgamal commented on the issue: https://github.com/apache/zeppelin/pull/1255 @zjffdu do you have any further comments ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1255: Adding extra libraries to livy
Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/1255 You can use --conf spark.jars= and --conf spark.jars.packages= instead --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1255: Adding extra libraries to livy
Github user mfelgamal commented on the issue: https://github.com/apache/zeppelin/pull/1255 I think that the livy itself doesn't support add (--jars/--packages property in spark-submit) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1255: Adding extra libraries to livy
Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/1255 No, I mean add interpreter property to allow user to specify jars/packages. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1255: Adding extra libraries to livy
Github user jimdowling commented on the issue: https://github.com/apache/zeppelin/pull/1255 Do you mean by modifying the UI on notebook creation or using a param to selecting the interpreter like '%livy --jars [path] --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1255: Adding extra libraries to livy
Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/1255 @jimdowling Right, there's no ZeppelinContext in livy right now. Would allowing user to specify jars/packages when creating livy session more general ? (like --jars/--packages in spark-submit) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1255: Adding extra libraries to livy
Github user jimdowling commented on the issue: https://github.com/apache/zeppelin/pull/1255 @zjffdu - I don't think there's a ZeppelinContext available in Livy. Would this be acceptable until the ZeppelinContext becomes available? As of now, Livy is not that much use if you cannot add libraries to your Spark app. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] zeppelin issue #1255: Adding extra libraries to livy
Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/1255 I would not recommend user to do that to add extra library. Because most of time zeppelin user don't know the cluster details like which extra library are located in the cluster machines. I would suggest livy to support dynamic library loading as native spark interpreter. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
