[jira] [Commented] (SPARK-44598) spark 3.2+ can not read hive table with hbase serde when hbase StorefileSize is 0
[ https://issues.apache.org/jira/browse/SPARK-44598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17749445#comment-17749445 ] ming95 commented on SPARK-44598: it seem is a hbase bug. and hbase fix this bug at 2.5+ https://issues.apache.org/jira/browse/HBASE-26340 > spark 3.2+ can not read hive table with hbase serde when hbase StorefileSize > is 0 > -- > > Key: SPARK-44598 > URL: https://issues.apache.org/jira/browse/SPARK-44598 > Project: Spark > Issue Type: Bug > Components: Spark Core >Affects Versions: 3.2.3 >Reporter: ming95 >Priority: Major > > We using spark to read a hive table with hbase serde . We found that when the > hbase table data is relatively small (hbase StorefileSize is 0), the data > read by spark 3.2 or 3.5 is empty, and there is no error message. > But when using spark2.4 or hive to read, the data can be read normally. Other > information shows that spark3.1 can also read data normally, can anyone > provide some ideas? -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-44598) spark 3.2+ can not read hive table with hbase serde when hbase StorefileSize is 0
[ https://issues.apache.org/jira/browse/SPARK-44598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17749444#comment-17749444 ] ming95 commented on SPARK-44598: [~ulysses] yes , it fix my case , thanks~ > spark 3.2+ can not read hive table with hbase serde when hbase StorefileSize > is 0 > -- > > Key: SPARK-44598 > URL: https://issues.apache.org/jira/browse/SPARK-44598 > Project: Spark > Issue Type: Bug > Components: Spark Core >Affects Versions: 3.2.3 >Reporter: ming95 >Priority: Major > > We using spark to read a hive table with hbase serde . We found that when the > hbase table data is relatively small (hbase StorefileSize is 0), the data > read by spark 3.2 or 3.5 is empty, and there is no error message. > But when using spark2.4 or hive to read, the data can be read normally. Other > information shows that spark3.1 can also read data normally, can anyone > provide some ideas? -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-44598) spark 3.2+ can not read hive table with hbase serde when hbase StorefileSize is 0
[ https://issues.apache.org/jira/browse/SPARK-44598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17749421#comment-17749421 ] XiDuo You commented on SPARK-44598: --- please try `--conf spark.hadoopRDD.ignoreEmptySplits=false` > spark 3.2+ can not read hive table with hbase serde when hbase StorefileSize > is 0 > -- > > Key: SPARK-44598 > URL: https://issues.apache.org/jira/browse/SPARK-44598 > Project: Spark > Issue Type: Bug > Components: Spark Core >Affects Versions: 3.2.3 >Reporter: ming95 >Priority: Major > > We using spark to read a hive table with hbase serde . We found that when the > hbase table data is relatively small (hbase StorefileSize is 0), the data > read by spark 3.2 or 3.5 is empty, and there is no error message. > But when using spark2.4 or hive to read, the data can be read normally. Other > information shows that spark3.1 can also read data normally, can anyone > provide some ideas? -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-44598) spark 3.2+ can not read hive table with hbase serde when hbase StorefileSize is 0
[ https://issues.apache.org/jira/browse/SPARK-44598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17749418#comment-17749418 ] Yuming Wang commented on SPARK-44598: - How to reproduce this issue? > spark 3.2+ can not read hive table with hbase serde when hbase StorefileSize > is 0 > -- > > Key: SPARK-44598 > URL: https://issues.apache.org/jira/browse/SPARK-44598 > Project: Spark > Issue Type: Bug > Components: Spark Core >Affects Versions: 3.2.3 >Reporter: ming95 >Priority: Major > > We using spark to read a hive table with hbase serde . We found that when the > hbase table data is relatively small (hbase StorefileSize is 0), the data > read by spark 3.2 or 3.5 is empty, and there is no error message. > But when using spark2.4 or hive to read, the data can be read normally. Other > information shows that spark3.1 can also read data normally, can anyone > provide some ideas? -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
