Re: Livy with Spark package

Thanks and agreed :) On Thu, Aug 24, 2017 at 12:01 PM, Saisai Shao wrote: > You could set "spark.jars.packages" in `conf` field of session post API ( > https://github.com/apache/incubator-livy/blob/master/ > docs/rest-api.md#post-sessions). This is equal to --package in

Re: Livy with Spark package

You could set "spark.jars.packages" in `conf` field of session post API ( https://github.com/apache/incubator-livy/blob/master/docs/rest-api.md#post-sessions). This is equal to --package in spark-submit. BTW you'd better ask livy question in u...@livy.incubator.apache.org. Thanks Jerry On Thu,

Livy with Spark package

Hi I have a python program which I invoke as spark-submit --packages com.databricks:spark-avro_2.11:3.2.0 somefile.py "2017-08-23 02:00:00" and it works Now I want to submit this file using Livy. I could work out most of the stuff (like putting files to HDFS etc) but not able to

Livy VS Spark Job Server

It will be helpful if someone can compare Livy and Spark Job Server. Thanks

Re: Livy with Spark

. This is basically no different compared to normal spark application. 2. Livy is just a Rest service, inside Livy you could create multiple sessions, each session is mapped to one spark application and resource managed under the cluster manager (like YARN). 3. One user could create one session, this session

Re: Livy with Spark

Livy at > http://henning.kropponline.de/2016/11/06/ > > On 5 Dec 2016, at 14:34, Mich Talebzadeh <mich.talebza...@gmail.com> > wrote: > > Hi, > > Has there been any experience using Livy with Spark to share multiple > Spark contexts? > > thanks > > > Dr

Re: Livy with Spark

You mean share a single spark context across multiple jobs? https://github.com/spark-jobserver/spark-jobserver does the same On Mon, Dec 5, 2016 at 9:33 AM, Mich Talebzadeh <mich.talebza...@gmail.com> wrote: > Hi, > > Has there been any experience using Livy with Spark to share m

Re: Livy with Spark

Dec 2016, at 14:34, Mich Talebzadeh <mich.talebza...@gmail.com> > wrote: > > Hi, > > Has there been any experience using Livy with Spark to share multiple > Spark contexts? > > thanks > > > Dr Mich Talebzadeh > > > > LinkedIn * > https://www

Re: Livy with Spark

There is a great write up on Livy at http://henning.kropponline.de/2016/11/06/ On 5 Dec 2016, at 14:34, Mich Talebzadeh <mich.talebza...@gmail.com<mailto:mich.talebza...@gmail.com>> wrote: Hi, Has there been any experience using Livy with Spark to share multiple Spark contexts?

Livy with Spark

Hi, Has there been any experience using Livy with Spark to share multiple Spark contexts? thanks Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw <https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV

Re: LIVY VS Spark Job Server

I have experience with both Livy & spark-jobserver. spark-jobserver gives you better API, particularly, if you want to work within a single spark context. Livy supports submitting python & R code while spark-jobserver doesn't support it. spark-jobserver code is more complex, it activ

LIVY VS Spark Job Server

Hi Team, I am evaluating different ways to submit & monitor spark Jobs using REST Interfaces. When to use Livy vs Spark Job Server? Regards, Sam -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/LIVY-VS-Spark-Job-Server-tp27722.html Sent from the Ap