[
https://issues.apache.org/jira/browse/HIVE-29798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated HIVE-29798:
--------------------------------
Description:
This was discovered while testing HIVE-28822, repro attached:
{code}
Caused by: java.lang.NumberFormatException: For input string: "NION"
at
java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:67)
at java.base/java.lang.Long.parseLong(Long.java:709)
at java.base/java.lang.Long.parseLong(Long.java:832)
at
org.apache.hadoop.hive.ql.io.AcidUtils$ParsedDeltaLight.parse(AcidUtils.java:1081)
at
org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:355)
at
org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:175)
at
org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:170)
at
org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:2008)
at
org.apache.hadoop.hive.ql.io.RecordReaderWrapper.create(RecordReaderWrapper.java:76)
at
org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:469)
... 28 more
{code}
2 scenarios I can imagine:
1. this is a bug: alter table should take care of flattening the union subdirs
and converting the file structure to an ACID compliant layout
2. this is by design, and tables containing union subdirs must be flattened
BEFORE converting them to ACID
was:
This was discovered while testing HIVE-28822, repro attached:
{code}
Caused by: java.lang.NumberFormatException: For input string: "NION"
at
java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:67)
at java.base/java.lang.Long.parseLong(Long.java:709)
at java.base/java.lang.Long.parseLong(Long.java:832)
at
org.apache.hadoop.hive.ql.io.AcidUtils$ParsedDeltaLight.parse(AcidUtils.java:1081)
at
org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:355)
at
org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:175)
at
org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:170)
at
org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:2008)
at
org.apache.hadoop.hive.ql.io.RecordReaderWrapper.create(RecordReaderWrapper.java:76)
at
org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:469)
... 28 more
{code}
2 scenarios I can imagine:
1. this is a bug: alter table should take care of flattening the union subdirs
2. this is by design, and tables containing union subdirs must be flattened
BEFORE converting them to ACID
> NumberFormatException after converting table with UNION subdirs to ACID
> -----------------------------------------------------------------------
>
> Key: HIVE-29798
> URL: https://issues.apache.org/jira/browse/HIVE-29798
> Project: Hive
> Issue Type: Bug
> Reporter: László Bodor
> Assignee: László Bodor
> Priority: Major
> Attachments: TestUnionAllToAcidConversionReadFails.java
>
>
> This was discovered while testing HIVE-28822, repro attached:
> {code}
> Caused by: java.lang.NumberFormatException: For input string: "NION"
> at
> java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:67)
> at java.base/java.lang.Long.parseLong(Long.java:709)
> at java.base/java.lang.Long.parseLong(Long.java:832)
> at
> org.apache.hadoop.hive.ql.io.AcidUtils$ParsedDeltaLight.parse(AcidUtils.java:1081)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:355)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:175)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:170)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:2008)
> at
> org.apache.hadoop.hive.ql.io.RecordReaderWrapper.create(RecordReaderWrapper.java:76)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:469)
> ... 28 more
> {code}
> 2 scenarios I can imagine:
> 1. this is a bug: alter table should take care of flattening the union
> subdirs and converting the file structure to an ACID compliant layout
> 2. this is by design, and tables containing union subdirs must be flattened
> BEFORE converting them to ACID
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
