[
https://issues.apache.org/jira/browse/HIVE-29798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated HIVE-29798:
--------------------------------
Description:
This was discovered while testing HIVE-28822, repro attached:
{code}
Caused by: java.lang.NumberFormatException: For input string: "NION"
at
java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:67)
at java.base/java.lang.Long.parseLong(Long.java:709)
at java.base/java.lang.Long.parseLong(Long.java:832)
at
org.apache.hadoop.hive.ql.io.AcidUtils$ParsedDeltaLight.parse(AcidUtils.java:1081)
at
org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:355)
at
org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:175)
at
org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:170)
at
org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:2008)
at
org.apache.hadoop.hive.ql.io.RecordReaderWrapper.create(RecordReaderWrapper.java:76)
at
org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:469)
... 28 more
{code}
this happens after "alter table tbl convert to acid" as well as altering table
properties to transactional=true
2 scenarios I can imagine:
1. this is a bug: alter table should take care of flattening the union subdirs
and converting the file structure to an ACID compliant layout
2. this is by design, and tables containing union subdirs must be flattened
BEFORE converting them to ACID
I'm about to handle this as 1), because the alter command should at least throw
an exception if the conversion would not lead to a layout which can be used
afterwards
was:
This was discovered while testing HIVE-28822, repro attached:
{code}
Caused by: java.lang.NumberFormatException: For input string: "NION"
at
java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:67)
at java.base/java.lang.Long.parseLong(Long.java:709)
at java.base/java.lang.Long.parseLong(Long.java:832)
at
org.apache.hadoop.hive.ql.io.AcidUtils$ParsedDeltaLight.parse(AcidUtils.java:1081)
at
org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:355)
at
org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:175)
at
org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:170)
at
org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:2008)
at
org.apache.hadoop.hive.ql.io.RecordReaderWrapper.create(RecordReaderWrapper.java:76)
at
org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:469)
... 28 more
{code}
2 scenarios I can imagine:
1. this is a bug: alter table should take care of flattening the union subdirs
and converting the file structure to an ACID compliant layout
2. this is by design, and tables containing union subdirs must be flattened
BEFORE converting them to ACID
I'm about to handle this as 1), because the alter command should at least throw
an exception if the conversion would not lead to a layout which can be used
afterwards
> NumberFormatException after converting table with UNION subdirs to ACID
> -----------------------------------------------------------------------
>
> Key: HIVE-29798
> URL: https://issues.apache.org/jira/browse/HIVE-29798
> Project: Hive
> Issue Type: Bug
> Reporter: László Bodor
> Assignee: László Bodor
> Priority: Major
> Attachments: TestUnionAllToAcidConversionReadFails.java
>
>
> This was discovered while testing HIVE-28822, repro attached:
> {code}
> Caused by: java.lang.NumberFormatException: For input string: "NION"
> at
> java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:67)
> at java.base/java.lang.Long.parseLong(Long.java:709)
> at java.base/java.lang.Long.parseLong(Long.java:832)
> at
> org.apache.hadoop.hive.ql.io.AcidUtils$ParsedDeltaLight.parse(AcidUtils.java:1081)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:355)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:175)
> at
> org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:170)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:2008)
> at
> org.apache.hadoop.hive.ql.io.RecordReaderWrapper.create(RecordReaderWrapper.java:76)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:469)
> ... 28 more
> {code}
> this happens after "alter table tbl convert to acid" as well as altering
> table properties to transactional=true
> 2 scenarios I can imagine:
> 1. this is a bug: alter table should take care of flattening the union
> subdirs and converting the file structure to an ACID compliant layout
> 2. this is by design, and tables containing union subdirs must be flattened
> BEFORE converting them to ACID
> I'm about to handle this as 1), because the alter command should at least
> throw an exception if the conversion would not lead to a layout which can be
> used afterwards
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
