[
https://issues.apache.org/jira/browse/FLINK-40839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-40839:
-----------------------------------
Labels: pull-request-available (was: )
> INSERT with a column list fails when an omitted column is of type VARIANT or
> BITMAP
> -----------------------------------------------------------------------------------
>
> Key: FLINK-40839
> URL: https://issues.apache.org/jira/browse/FLINK-40839
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
> Reporter: Ramin Gharib
> Assignee: Ramin Gharib
> Priority: Major
> Labels: pull-request-available
>
> An INSERT with an explicit column list fails during validation when the
> target table has a VARIANT or BITMAP column that is not listed.
> {code:sql}
> CREATE TABLE src (k INT) WITH ('connector' = 'datagen', 'number-of-rows' =
> '1');
> CREATE TABLE snk (k INT, v VARIANT, b BITMAP) WITH ('connector' =
> 'blackhole');
> INSERT INTO snk SELECT k, CAST(NULL AS VARIANT), CAST(NULL AS BITMAP) FROM
> src; -- works
> INSERT INTO snk (k) SELECT k FROM src;
> -- fails
> {code}
> {noformat}
> org.apache.flink.table.api.ValidationException: SQL validation failed.
> Unsupported type when convertTypeToSpec: VARIANT
> Caused by: java.lang.UnsupportedOperationException: Unsupported type when
> convertTypeToSpec: VARIANT
> at
> org.apache.calcite.sql.type.SqlTypeUtil.convertTypeToSpec(SqlTypeUtil.java:1326)
> at
> org.apache.calcite.sql.validate.SqlValidatorImpl.maybeCast(SqlValidatorImpl.java:819)
> at
> org.apache.flink.table.planner.calcite.FlinkCalciteSqlValidator.maybeCast(FlinkCalciteSqlValidator.java:491)
> at
> org.apache.flink.table.planner.calcite.PreValidateReWriter$.appendPartitionAndNullsProjects(PreValidateReWriter.scala:187)
> {noformat}
> BITMAP fails the same way with {{{}Unsupported type when convertTypeToSpec:
> OTHER{}}}. Nested types fail too, for example {{{}ARRAY<VARIANT>{}}},
> {{{}ROW<f VARIANT>{}}}, {{MAP<STRING, VARIANT>}} and {{{}ARRAY<BITMAP>{}}}.
> {{CAST(NULL AS VARIANT)}} will work once FLINK-40825 is resolved
> *Cause*
> {{PreValidateReWriter}} pads every omitted column with {{{}CAST(NULL AS
> <column type>){}}}. The cast target is built by
> {{{}SqlTypeUtil.convertTypeToSpec{}}}, which has no branch for either type:
> * VARIANT is not covered by {{{}SqlTypeUtil.isAtomic{}}}. Calcite fixed this
> in CALCITE-7293, which ships with Calcite 1.42.0. Flink is on 1.41.0.
> * BITMAP is a Flink type. {{BitmapRelDataType}} reports
> {{{}SqlTypeName.OTHER{}}}, so no branch matches. Calcite cannot fix this one.
> *Proposed fix*
> In Flink's copy of {{{}SqlTypeUtil{}}}:
> * Backport CALCITE-7293 by adding VARIANT to {{{}isAtomic{}}}. This part
> becomes a no-op once FLINK-40001 upgrades to Calcite 1.42.0.
> * Map {{BitmapRelDataType}} to {{SqlBitmapTypeNameSpec}} in
> {{{}convertTypeToSpec{}}}.
> Flink disables Calcite's implicit type coercion, so the {{isAtomic}} change
> has no effect outside {{{}convertTypeToSpec{}}}.
> The workaround is to list every column and write {{CAST(NULL AS VARIANT)}} or
> {{CAST(NULL AS BITMAP)}} for the omitted ones.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
