[
https://issues.apache.org/jira/browse/IMPALA-13296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17874560#comment-17874560
]
ASF subversion and git services commented on IMPALA-13296:
----------------------------------------------------------
Commit 82d8e382a3fa21e2930c0dd5e8198db647e36f17 in impala's branch
refs/heads/master from Gabor Kaszab
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=82d8e382a ]
IMPALA-13296: Check column compatibility earlier for table migration
When migrating a Hive table to Iceberg there can be columns in the
source table that are not supported by Iceberg. For these tables the
table migration process gives an error. However, this is a multi-step
process and the check for column compatibility is performed after the
source table had been renamed to a temporary name.
This patch brings the column type check to the analysis phase to avoid
renaming the table when there are incompatible columns.
Testing:
- Re-run the existing checks
- Adjusted the error message for this error case to reflect that we
return an AnalysisException.
Change-Id: I381e7359a55ebd84f9a1dcec3b665bdbea62a035
Reviewed-on: http://gerrit.cloudera.org:8080/21675
Reviewed-by: Peter Rozsa <pro...@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
> Hive to Iceberg table-migration: pre-check column compatibility
> ---------------------------------------------------------------
>
> Key: IMPALA-13296
> URL: https://issues.apache.org/jira/browse/IMPALA-13296
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Reporter: Gabor Kaszab
> Priority: Major
> Labels: impala-iceberg
>
> The table migration from a Hive table to an Iceberg table is a multi-step
> process that has a middle step to rename the original table to a temp name.
> If a later step fails the user gets an error but the table remains renamed
> (the user also gets a hint how to set the original name back).
> When the failure is column incompatibility (e.g. Iceberg doesn't support
> Hive's smallint, tinyint, varchar( n ) column types) we can do better because
> this imcompatibility could also be found during query analysis. That way the
> error could be sent before we rename the table. This results in a cleaner
> user experience.
> {code:java}
> Query: alter table hive_tbl convert to iceberg ERROR:
> IllegalArgumentException: Unsupported Hive type: VARCHAR, use string instead
> Your table might have been renamed. To reset the name try running: ALTER
> TABLE default.hive_tbl_tmp_8fb36dff RENAME TO default.hive_tbl;
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
