Well if you are using Hortonworks distribution there is Livy2 which is compatible with Spark2 and scala 2.11.
https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.0/bk_command-line-installation/content/install_configure_livy2.html On Sun, Jun 4, 2017 at 1:55 PM, kant kodali <kanth...@gmail.com> wrote: > Hi, > > Thanks for this but here is what the documentation says: > > "To run the Livy server, you will also need an Apache Spark installation. > You can get Spark releases at https://spark.apache.org/downloads.html. > Livy requires at least Spark 1.4 and currently only supports Scala 2.10 > builds of Spark. To run Livy with local sessions, first export these > variables:" > > I am using spark 2.1.1 and scala 2.11.8 and I would like to use Dataframes > and Dataset API so it sounds like this is not an option for me? > > Thanks! > > On Sun, Jun 4, 2017 at 12:23 AM, Sandeep Nemuri <nhsande...@gmail.com> > wrote: > >> Check out http://livy.io/ >> >> >> On Sun, Jun 4, 2017 at 11:59 AM, kant kodali <kanth...@gmail.com> wrote: >> >>> Hi All, >>> >>> I am wondering what is the easiest way for a Micro service to query data >>> on HDFS? By easiest way I mean using minimal number of tools. >>> >>> Currently I use spark structured streaming to do some real time >>> aggregations and store it in HDFS. But now, I want my Micro service app to >>> be able to query and access data on HDFS. It looks like SparkSession can >>> only be accessed through CLI but not through a JDBC like API or whatever. >>> Any suggestions? >>> >>> Thanks! >>> >> >> >> >> -- >> * Regards* >> * Sandeep Nemuri* >> > > -- * Regards* * Sandeep Nemuri*
