[ https://issues.apache.org/jira/browse/SPARK-36860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17421936#comment-17421936 ]
Hyukjin Kwon edited comment on SPARK-36860 at 9/29/21, 5:26 AM: ---------------------------------------------------------------- Unless it's explicitly documented, nothing is supported officially. was (Author: hyukjin.kwon): Unless it's explicitly documented, nothing is supported. > Create the external hive table for HBase failed > ------------------------------------------------ > > Key: SPARK-36860 > URL: https://issues.apache.org/jira/browse/SPARK-36860 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 3.1.2 > Reporter: wineternity > Priority: Major > Attachments: image-2021-09-27-14-18-10-910.png, > image-2021-09-27-14-25-28-900.png > > > We use follow sql to create hive external table , which read from hbase > {code:java} > CREATE EXTERNAL TABLE if not exists dev.sanyu_spotlight_headline_material( > rowkey string COMMENT 'HBase主键', > content string COMMENT '图文正文') > USING HIVE > ROW FORMAT SERDE > 'org.apache.hadoop.hive.hbase.HBaseSerDe' > STORED BY > 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' > WITH SERDEPROPERTIES ( > 'hbase.columns.mapping'=':key, cf1:content' > ) > TBLPROPERTIES ( > 'hbase.table.name'='spotlight_headline_material' > ); > {code} > But the sql failed in Spark 3.1.2, which throw this exception > {code:java} > 21/09/27 11:44:24 INFO scheduler.DAGScheduler: Asked to cancel job group > 26d7459f-7b58-4c18-9939-5f2737525ff2 > 21/09/27 11:44:24 ERROR thriftserver.SparkExecuteStatementOperation: Error > executing query with 26d7459f-7b58-4c18-9939-5f2737525ff2, currentState > RUNNING, > org.apache.spark.sql.catalyst.parser.ParseException: > Operation not allowed: Unexpected combination of ROW FORMAT SERDE > 'org.apache.hadoop.hive.hbase.HBaseSerDe' and STORED BY > 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'WITHSERDEPROPERTIES('hbase.columns.mapping'=':key, > cf1:content')(line 5, pos 0) > {code} > this check was introduced from this change: > [https://github.com/apache/spark/pull/28026] > > Could anyone gave the introduction how to create the external table for hbase > in spark3 now ? -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org