The spark jobserver will do what you describe for you. I have built an app
where we have a bunch of queries being submitted via
http://something/query/
via POST (all parameters for the query are in JSON POST request). This is a
scalatra layer that talks to spark jobserver via HTTP.
On Sun, Nov 6,
Hi,
Spark jobserver seems to be more mature than Livy but both would work I
think. You will just get more functionalities with the jobserver except the
impersonation that is only in Livy.
If you need to publish business API I would recommend to use Akka http with
Spark actors sharing a preloaded
Hi
I have written multiple spark driver programs that load some data from HDFS
to data frames and accomplish spark sql queries on it and persist the
results in HDFS again. Now I need to provide a long running java program in
order to receive requests and their some parameters(such as the number
