Thanks Denny for your information! For #1, what I meant is the Spark SQL beta/official release date (as today it is still in alpha phase)… thought today I see it has most basic functionalities, I don’t know when will the next milestone happen? i.e. Beta? For #2, thanks for the information! I read it and it’s really useful! My take is that, Hive on Spark is still Hive (thus having all the metastore information and Hive interfaces such as the REST APIs), while Spark SQL is the expansion of Spark and use several interfaces (HiveContext for example) to support run Hive queries. Is this correct?
Then a following question would be, does Spark SQL has some REST APIs, just as what WebHCat exposes, to help users to submit queries remotely, other than logging into a cluster and execute the command in spark-sql command line? Xiaoyong From: Denny Lee [mailto:denny.g....@gmail.com] Sent: Saturday, December 13, 2014 10:59 PM To: Xiaoyong Zhu; user@spark.apache.org Subject: Re: Spark SQL Roadmap? Hi Xiaoyong, SparkSQL has already been released and has been part of the Spark code-base since Spark 1.0. The latest stable release is Spark 1.1 (here's the Spark SQL Programming Guide<http://spark.apache.org/docs/1.1.0/sql-programming-guide.html>) and we're currently voting on Spark 1.2. Hive on Spark is an initiative by Cloudera to help folks whom are already using Hive but instead of using traditional MR it will utilize Spark. For more information, check out http://blog.cloudera.com/blog/2014/07/apache-hive-on-apache-spark-motivations-and-design-principles/. For anyone who is building new projects in Spark, IMHO I would suggest jumping to SparkSQL first. HTH! Denny On Sat Dec 13 2014 at 5:00:56 AM Xiaoyong Zhu <xiaoy...@microsoft.com<mailto:xiaoy...@microsoft.com>> wrote: Dear spark experts, I am very interested in Spark SQL availability in the future – could someone share with me the information about the following questions? 1. Is there some ETAs for the Spark SQL release? 2. I heard there is a Hive on Spark program also – what’s the difference between Spark SQL and Hive on Spark? Thanks! Xiaoyong
