[
https://issues.apache.org/jira/browse/HIVE-10073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14383152#comment-14383152
]
Chengxiang Li commented on HIVE-10073:
--------------------------------------
[~xuefuz], the root cause should be just like Jimmy mentioned, some hbase table
properties are set to JobConf during checkOutputSpecs, and this method is not
invoked in HoS. Actually Spark checkout output specs while user build RDD graph
with certain actions, like PairRDDFunctions::saveAsHadoopDataset,
PairRDDFunctions::saveAsNewAPIHadoopDataset, in HoS, we use foreach as action,
and write data to hadoop storage inside Hive, so it should be Hive's
reponsbility to check output specs.
> Runtime exception when querying HBase with Spark [Spark Branch]
> ---------------------------------------------------------------
>
> Key: HIVE-10073
> URL: https://issues.apache.org/jira/browse/HIVE-10073
> Project: Hive
> Issue Type: Bug
> Components: Spark
> Affects Versions: spark-branch
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Fix For: spark-branch
>
> Attachments: HIVE-10073.1-spark.patch, HIVE-10073.2-spark.patch,
> HIVE-10073.3-spark.patch
>
>
> When querying HBase with Spark, we got
> {noformat}
> Caused by: java.lang.IllegalArgumentException: Must specify table name
> at
> org.apache.hadoop.hbase.mapreduce.TableOutputFormat.setConf(TableOutputFormat.java:188)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73)
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
> at
> org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getHiveOutputFormat(HiveFileFormatUtils.java:276)
> at
> org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getHiveOutputFormat(HiveFileFormatUtils.java:266)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:331)
> {noformat}
> But it works fine for MapReduce.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
