[ https://issues.apache.org/jira/browse/HIVE-24706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422021#comment-17422021 ]
cadl commented on HIVE-24706: ----------------------------- I meet the issue too. After tracking the stacktrace, there are two problems that cause the issue. # The `org.apache.hadoop.hive.hbase.HiveHBaseTableInputFormat` not implements `org.apache.hadoop.mapreduce.InputFormat` completely. As [~Lysak] said, `HiveHBaseTableInputFormat` doesn't overwrite `getSplits(JobContext context)` and `createRecordReader`, and doesn't initialize the table. # Because of extending the `org.apache.hadoop.hbase.mapreduce.TableInputFormatBase extends InputFormat<ImmutableBytesWritable, Result>`, `HiveHBaseTableInputFormat` can't cast to `InputFormat<Writable, Writable>` at [spark createNewHadoopRDD|https://github.com/apache/spark/blob/35848385ae6518b4b72c2f5c1e9ca5a83a190723/sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala#L373] Is it safe to change `HiveHBaseTableInputFormat` from `InputFormat<ImmutableBytesWritable, Result>` to `InputFormat<ImmutableBytesWritable, ResultWritable>`? I'd like to submit a pull request about it. > Spark SQL access hive on HBase table access exception > ----------------------------------------------------- > > Key: HIVE-24706 > URL: https://issues.apache.org/jira/browse/HIVE-24706 > Project: Hive > Issue Type: Bug > Components: HBase Handler > Reporter: zhangzhanchang > Priority: Major > Attachments: image-2021-01-30-15-51-58-665.png > > > Hivehbasetableinputformat relies on two versions of inputformat,one is > org.apache.hadoop.mapred.InputFormat, the other is > org.apache.hadoop.mapreduce.InputFormat,Causes > spark 3.0(https://github.com/apache/spark/pull/31302) both conditions to be > true: > # classOf[oldInputClass[_, _]].isAssignableFrom(inputFormatClazz) is true > # classOf[newInputClass[_, _]].isAssignableFrom(inputFormatClazz) is true > !image-2021-01-30-15-51-58-665.png|width=430,height=137! > Hivehbasetableinputformat relies on inputformat to be changed to > org.apache.hadoop.mapreduce or org.apache.hadoop.mapred? > -- This message was sent by Atlassian Jira (v8.3.4#803005)