Re: Building a REST Service with Spark back-end

I want to ask about something related to this. Does anyone know if there is or will be a command line equivalent of spark-shell client for Livy Spark Server or any other Spark Job Server? The reason that I am asking spark-shell does not handle multiple users on the same server well. Since a

Re: Building a REST Service with Spark back-end

Hi Yanlin, This is a fairly new effort and is not officially released/supported by Cloudera yet. I believe those numbers will be out once it is released. Guru Medasani gdm...@gmail.com > On Mar 2, 2016, at 10:40 AM, yanlin wang wrote: > > Did any one use Livy in real world

Re: Building a REST Service with Spark back-end

Did any one use Livy in real world high concurrency web app? I think it uses spark submit command line to create job... How about job server or notebook comparing with Livy? Thx, Yanlin Sent from my iPhone > On Mar 2, 2016, at 6:24 AM, Guru Medasani wrote: > > Hi Don, >

Re: Building a REST Service with Spark back-end

Hi Don, Here is another REST interface for interacting with Spark from anywhere. https://github.com/cloudera/livy Here is an example to estimate PI using Spark from Python using requests library. >>> data = { ... 'code': textwrap.dedent("""\ ... val

Re: Building a REST Service with Spark back-end

Have you looked at Apache Toree, http://toree.apache.org/. This was formerly the Spark-Kernel from IBM but contributed to apache. https://github.com/apache/incubator-toree You can find a good overview on the spark-kernel here:

Building a REST Service with Spark back-end

I'm interested in building a REST service that utilizes a Spark SQL Context to return records from a DataFrame (or IndexedRDD?) and even add/update records. This will be a simple REST API, with only a few end-points. I found this example: https://github.com/alexmasselot/spark-play-activator