bq. ROW FORMAT SERDE 'org.apache.hadoop.hive.hbase.HBaseSerDe'
The above implies dependency on some class from Hive.
Which Hive release would you use if you choose the above route ?
Looking forward to your demo.
On Tue, Jul 31, 2018 at 9:09 AM bill.yunfu <guangcheng....@alibaba-inc.com>
wrote:
> hi Ted
> Thank you for replying.
> The sql support means user can directly use spark sql to create table and
> query data from HBase. we found two sql support on HBase
> SHC use following command to create table in spark sql:
> CREATE TABLE spark_hbase USING
> org.apache.spark.sql.execution.datasources.hbase
> OPTIONS ('catalog'=
> '{"table":{"namespace":"default", "name":"test",
> "tableCoder":"PrimitiveType"},"rowkey":"key",
> "columns":{
> "col0":{"cf":"rowkey", "col":"key", "type":"string"},
> "col1":{"cf":"cf", "col":"a", "type":"string"}}}'
> )
> (SHC is a project can get details from:
> https://github.com/hortonworks-spark/shc)
> In spark sql also can use hive command to create table:
> create table spark_hbase (col0 string, col1 string)
> ROW FORMAT SERDE 'org.apache.hadoop.hive.hbase.HBaseSerDe' with
> SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf:a")
> STORED AS
> INPUTFORMAT
> 'org.apache.hadoop.hive.hbase.HiveHBaseTableInputFormat'
> OUTPUTFORMAT
> 'org.apache.hadoop.hive.hbase.HiveHBaseTableOutputFormat'
> tblproperties ("hbase.table.name" = "test");
>
> So we want make a similar DDL to create the table for hbase-spark model and
> query with the spark sql.
>
> And for the Spark release, we suggestion first target at spark 2.y, for
> example the spark 2.2.2 which is stability now.
>
> We will create a demo base on hbase-spark model with sql support in local,
> then share here to discuss.
>
> Regards
> Bill
>
>
> Ted Yu-3 wrote
> > For SQL support, can you be more specific on how the SQL support would be
> > added ?
> >
> > Maybe you can illustrate some examples showing the enhanced SQL syntax.
> >
> > Also, which Spark release(s) would be targeted?
> >
> > Thanks
> >
> > On Mon, Jul 30, 2018 at 10:57 AM bill.yunfu <
>
> > guangcheng.zgc@
>
> > >
> > wrote:
>
>
>
>
>
> --
> Sent from:
> http://apache-hbase.679495.n3.nabble.com/HBase-Developer-f679493.html
>
