[
https://issues.apache.org/jira/browse/HIVE-30170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Denys Kuzmenko updated HIVE-30170:
----------------------------------
Description:
HIVE-29184 models VARIANT inconsistently.
VariantTypeInfo has category VARIANT, but VariantObjectInspector extends
StandardStructObjectInspector, and serdes, vectorization and the inspector
utilities only know the struct. Plan types are derived from inspectors, so
result schemas, views, CTAS, DESCRIBE/SHOW CREATE TABLE and HS2 report
struct<metadata:binary,value:binary>, and variant-aware code identifies a
variant by the struct's name or shape.
Make VARIANT a category at the same level as STRUCT, MAP and LIST in both
layers:
- VariantObjectInspector becomes an interface (category VARIANT, type name
variant, Variant getVariant(Object)), with standard, constant, Iceberg and
struct-encoded (serde) flavours.
- Every serde stores a variant as the same two binaries as today, so on-disk
and shuffle bytes don't change. Vectors stay StructColumnVector(metadata,
value).
- Comparisons and hashing keep today's byte-wise results (semantic equality is
HIVE-30171).
- Calcite maps VARIANT to SqlTypeName.VARIANT. CAST(v AS STRING) returns the
JSON text.
- HS2 reports variant result columns as string until HIVE-30166.
User-visible changes: type names become variant. v.metadata/v.value and
inserting a struct into a variant column are rejected. Persistent non-Iceberg
tables reject VARIANT at any nesting depth.
was:
HIVE-29184 models VARIANT inconsistently. VariantTypeInfo has category VARIANT,
but VariantObjectInspector extends StandardStructObjectInspector, and serdes,
vectorization and the inspector utilities only know the struct. Plan types are
derived from inspectors, so result schemas, views, CTAS, DESCRIBE/SHOW CREATE
TABLE and HS2 report struct<metadata:binary,value:binary>, and variant-aware
code identifies a variant by the struct's name or shape.
Make VARIANT a category at the same level as STRUCT, MAP and LIST in both
layers:
- VariantObjectInspector becomes an interface (category VARIANT, type name
variant, Variant getVariant(Object)), with standard, constant, Iceberg and
struct-encoded (serde) flavours.
- Every serde stores a variant as the same two binaries as today, so on-disk
and shuffle bytes don't change. Vectors stay StructColumnVector(metadata,
value).
- Comparisons and hashing keep today's byte-wise results (semantic equality is
HIVE-30171).
- Calcite maps VARIANT to SqlTypeName.VARIANT. CAST(v AS STRING) returns the
JSON text.
- HS2 reports variant result columns as string until HIVE-30166.
User-visible changes: type names become variant. v.metadata/v.value and
inserting a struct into a variant column are rejected. Persistent non-Iceberg
tables reject VARIANT at any nesting depth.
> VARIANT as a first-class type: category VARIANT in object inspectors and at
> runtime
> -----------------------------------------------------------------------------------
>
> Key: HIVE-30170
> URL: https://issues.apache.org/jira/browse/HIVE-30170
> Project: Hive
> Issue Type: Sub-task
> Reporter: Denys Kuzmenko
> Priority: Major
>
> HIVE-29184 models VARIANT inconsistently.
> VariantTypeInfo has category VARIANT, but VariantObjectInspector extends
> StandardStructObjectInspector, and serdes, vectorization and the inspector
> utilities only know the struct. Plan types are derived from inspectors, so
> result schemas, views, CTAS, DESCRIBE/SHOW CREATE TABLE and HS2 report
> struct<metadata:binary,value:binary>, and variant-aware code identifies a
> variant by the struct's name or shape.
>
>
> Make VARIANT a category at the same level as STRUCT, MAP and LIST in both
> layers:
>
> - VariantObjectInspector becomes an interface (category VARIANT, type name
> variant, Variant getVariant(Object)), with standard, constant, Iceberg and
> struct-encoded (serde) flavours.
> - Every serde stores a variant as the same two binaries as today, so on-disk
> and shuffle bytes don't change. Vectors stay StructColumnVector(metadata,
> value).
> - Comparisons and hashing keep today's byte-wise results (semantic equality
> is HIVE-30171).
>
> - Calcite maps VARIANT to SqlTypeName.VARIANT. CAST(v AS STRING) returns the
> JSON text.
>
> - HS2 reports variant result columns as string until HIVE-30166.
>
>
>
>
>
> User-visible changes: type names become variant. v.metadata/v.value and
> inserting a struct into a variant column are rejected. Persistent non-Iceberg
> tables reject VARIANT at any nesting depth.
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
