[
https://issues.apache.org/jira/browse/SPARK-34210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17626576#comment-17626576
]
Mehul Thakkar edited comment on SPARK-34210 at 10/31/22 12:55 PM:
------------------------------------------------------------------
Do you mean we have to download the spark source code from master branch and
update the code with the fix to make it working for Spark 3?
Could you please elaborate more on bug in Hadoop?
was (Author: JIRAUSER297345):
Do you mean we have to download the spark source code from master branch and
update the code with the fix to make it working for Spark 3?
> Cannot create a record reader because of a previous error when spark accesses
> the hive on HBase table
> ------------------------------------------------------------------------------------------------------
>
> Key: SPARK-34210
> URL: https://issues.apache.org/jira/browse/SPARK-34210
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.0.1
> Reporter: zhangzhanchang
> Priority: Major
>
> It is normal for version 2.4.6 to use spark SQL to access hive on HBase
> table,Upgrade to spark3.0.1 with the following exception：
> java.io.IOException: Cannot create a record reader because of a previous
> error. Please look at the previous logs lines from the task's full log for
> more details.
> at
> org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.getSplits(TableInputFormatBase.java:252)
> at org.apache.spark.rdd.NewHadoopRDD.getPartitions(NewHadoopRDD.scala:131)
> at org.apache.spark.rdd.RDD.$anonfun$partitions$2(RDD.scala:276)
> at scala.Option.getOrElse(Option.scala:189)
> at org.apache.spark.rdd.RDD.partitions(RDD.scala:272)
> at
> org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:49)
> at org.apache.spark.rdd.RDD.$anonfun$partitions$2(RDD.scala:276)
> at scala.Option.getOrElse(Option.scala:189)
> at org.apache.spark.rdd.RDD.partitions(RDD.scala:272)
> Caused by: java.lang.IllegalStateException: The input format instance has not
> been properly initialized. Ensure you call initializeTable either in your
> constructor or initialize method
> at
> org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.getTable(TableInputFormatBase.java:585)
> at
> org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.getSplits(TableInputFormatBase.java:247)
> ... 59 more
> java.io.IOException: Cannot create a record reader because of a previous
> error. Please look at the previous logs lines from the task's full log for
> more details.
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
