[
https://issues.apache.org/jira/browse/HIVE-30021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-30021:
----------------------------------
Labels: pull-request-available (was: )
> TIMESTAMP WITH LOCAL TIME ZONE columns cannot be created, written, or read
> correctly on Parquet tables
> ------------------------------------------------------------------------------------------------------
>
> Key: HIVE-30021
> URL: https://issues.apache.org/jira/browse/HIVE-30021
> Project: Hive
> Issue Type: Bug
> Reporter: Vikram Ahuja
> Assignee: Vikram Ahuja
> Priority: Major
> Labels: pull-request-available
>
> 1. CREATE TABLE fails outright.
> ArrayWritableObjectInspector.getObjectInspector(
> (ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/ArrayWritableObjectInspector.java)
> has no case for this type, so ParquetHiveSerDe.initialize() — called during
> Table.checkValidity() — throws immediately: UnsupportedOperationException:
> Unknown field type: timestamp with local time zone('US/Pacific'). So a table
> with this column type can never even be created with STORED AS PARQUET.
> 2. Schema generation has no case for it either.
> HiveSchemaConverter.convertType()
> (.../io/parquet/convert/HiveSchemaConverter.java) — which builds the physical
> Parquet MessageType at write time — falls through to throw new
> IllegalArgumentException("Unknown type: " + typeInfo).
> 3. Write path has no case either. DataWritableWriter.createWriter()
> (.../io/parquet/write/DataWritableWriter.java) explicitly rejects it:
> IllegalArgumentException: Unsupported primitive data type: TIMESTAMPLOCALTZ
> 4. Read path silently returns the wrong type. ETypeConverter
> (.../io/parquet/convert/ETypeConverter.java) has a case that matches
> TIMESTAMPLOCALTZ_TYPE_NAME for the INT64 encoding, but shares it with plain
> TIMESTAMP and always constructs a TimestampWritableV2 — never a
> TimestampLocalTZWritable. The INT96 converter (EINT96_TIMESTAMP_CONVERTER)
> has no type-specific handling for it at all, same
> default-to-TimestampWritableV2 problem.
>
> *Steps to reproduce:*
> {code:java}
> CREATE EXTERNAL TABLE t (
> tsval timestamp with local time zone
> )
> stored as parquet;{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
