Are there clear instructions how to use spark.jars.packages properties? For instance, if I want to depend on bintray repo https://dl.bintray.com/comp-bio-aging/main with "group.research.aging:spark-extensions_2.11:0.0.7.2" as a dependency, what should I do with newintepreter?
On 2019/10/12 01:18:09, Jeff Zhang <zjf...@gmail.com> wrote: > Glad to hear that. > > Mark Bidewell <mbide...@gmail.com> 于2019年10月12日周六 上午1:30写道： > > > Just wanted to say "thanks"! Using spark.jars.packages, etc worked great! > > > > On Fri, Oct 11, 2019 at 9:45 AM Jeff Zhang <zjf...@gmail.com> wrote: > > > >> That's right, document should also be updated > >> > >> Mark Bidewell <mbide...@gmail.com> 于2019年10月11日周五 下午9:28写道： > >> > >>> Also the interpreter setting UI is still listed as the first way to > >>> handle dependencies in the documentation - Maybe it should be marked as > >>> deprecated? > >>> > >>> http://zeppelin.apache.org/docs/0.8.2/interpreter/spark.html > >>> > >>> > >>> On Thu, Oct 10, 2019 at 9:58 PM Jeff Zhang <zjf...@gmail.com> wrote: > >>> > >>>> It looks like many users still get used to specify spark dependencies > >>>> in interpreter setting UI, spark.jars and spark.jars.packages seems too > >>>> difficult to understand and not transparent, so I create ticket > >>>> https://issues.apache.org/jira/browse/ZEPPELIN-4374 that user can > >>>> still set dependencies in interpreter setting UI. > >>>> > >>>> Jeff Zhang <zjf...@gmail.com> 于2019年10月11日周五 上午9:54写道： > >>>> > >>>>> Like I said above, try to set them via spark.jars and > >>>>> spark.jars.packages. > >>>>> > >>>>> Don't set them here > >>>>> > >>>>> [image: image.png] > >>>>> > >>>>> > >>>>> Mark Bidewell <mbide...@gmail.com> 于2019年10月11日周五 上午9:35写道： > >>>>> > >>>>>> I was specifying them in the interpreter settings in the UI. > >>>>>> > >>>>>> On Thu, Oct 10, 2019 at 9:30 PM Jeff Zhang <zjf...@gmail.com> wrote: > >>>>>> > >>>>>>> How do you specify your spark interpreter dependencies ? You need to > >>>>>>> specify it via property spark.jars or spark.jars.packages for > >>>>>>> non-local > >>>>>>> model. > >>>>>>> > >>>>>>> Mark Bidewell <mbide...@gmail.com> 于2019年10月11日周五 上午3:45写道： > >>>>>>> > >>>>>>>> I am running some initial tests of Zeppelin 0.8.2 and I am seeing > >>>>>>>> some weird issues with dependencies. When I use the old interpreter, > >>>>>>>> everything works as expected. When I use the new interpreter, > >>>>>>>> classes in > >>>>>>>> my interpreter dependencies cannot be resolved when connecting to a > >>>>>>>> master > >>>>>>>> that is not local[*], I did not encounter issues with either > >>>>>>>> interpreter > >>>>>>>> on 0.8.1. > >>>>>>>> > >>>>>>>> Has anyone else seen this? > >>>>>>>> > >>>>>>>> Thanks! > >>>>>>>> > >>>>>>>> -- > >>>>>>>> Mark Bidewell > >>>>>>>> http://www.linkedin.com/in/markbidewell > >>>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> -- > >>>>>>> Best Regards > >>>>>>> > >>>>>>> Jeff Zhang > >>>>>>> > >>>>>> > >>>>>> > >>>>>> -- > >>>>>> Mark Bidewell > >>>>>> http://www.linkedin.com/in/markbidewell > >>>>>> > >>>>> > >>>>> > >>>>> -- > >>>>> Best Regards > >>>>> > >>>>> Jeff Zhang > >>>>> > >>>> > >>>> > >>>> -- > >>>> Best Regards > >>>> > >>>> Jeff Zhang > >>>> > >>> > >>> > >>> -- > >>> Mark Bidewell > >>> http://www.linkedin.com/in/markbidewell > >>> > >> > >> > >> -- > >> Best Regards > >> > >> Jeff Zhang > >> > > > > > > -- > > Mark Bidewell > > http://www.linkedin.com/in/markbidewell > > > > > -- > Best Regards > > Jeff Zhang >
