On Mon, Jul 30, 2018 at 10:57 AM bill.yunfu <guangcheng....@alibaba-inc.com> wrote: > > May I take this issue --hbase-spark > > Hi community > I am working in one HBase team which service hundreds customers. We find > that along increasing amount of data in the HBase, many customers have > analysis requirement for their data on Hbase. For example they want use > Spark to do some analysis which may query more data from Hbase and may also > join with other tables, the tables may be in Hbase or Spark. > But Hbase can not support this scenario very well. So we plan use spark > to support this. > We found the Apache Hbase already has one module called Hbase-spark, but > this module is not updated recently and not formally released. Besides we > found there are others project support Sql On Hbase. For example Hive on > Hbase which give good sql syntax support. > Even there are many projects for Spark on Hbase, but I think now no one > is the public knowing for users. Because our customer have more and more > requirement for Spark on Hbase, So we want take this issue. Initial goal is > make a standard and public knowing Spark on Hbase in apache Hbase > community. > Our initial idea is: > SQL support: Now the hbase-spark model can not spark-sql command to > create table, We want make it support sql command which may like the sql > syntax from Hive on HBase or the SQL syntax from SHC. > Performance improved: this part is not very clearly now, the goal is use > spark sql query HBase data has a good performance. > > We want to get some suggestions from community. Then I will raise a JIRA to > track it and put a design document. >
Thanks for showing up to help Bill. I second what our Sean said. I owe work on moving connectors out of hbase core. It was suggested we might move Spark out as part of the hbase-connector effort. Lets see. It be deserving of its own dedicated repository if its rate of change is far in excess of that of other connectors. Lets see. Spark SQL support is a long-time ask. Anything you could do to forward this project would be much appreciated. S > Best Regards > Bill > > > > > -- > Sent from: > http://apache-hbase.679495.n3.nabble.com/HBase-Developer-f679493.html
