jackylee-ch opened a new pull request, #15814:
URL: https://github.com/apache/iceberg/pull/15814
When using a Hive catalog, schema evolution operations that change column
positions — such as ALTER TABLE ... DROP COLUMN (non-last column) and ALTER
TABLE ... ALTER COLUMN ... FIRST/AFTER (reorder) — fail with
InvalidOperationException from
MetaStoreUtils#throwExceptionIfIncompatibleColTypeChange. This is because the
Hive Metastore validates schema changes by comparing column types positionally
(by index, not by name), controlled by
hive.metastore.disallow.incompatible.col.type.changes (default true).
This limitation is not documented anywhere in the Iceberg docs, though the
Iceberg test suite itself works around it by setting
METASTORE_DISALLOW_INCOMPATIBLE_COL_TYPE_CHANGES=false
(TestHiveMetastore.java:269), and a test comment(HiveTableTest.java:283-287)
explicitly describes the issue.
This PR adds !!! warning admonition blocks to:
- spark-ddl.md — under the DROP COLUMN section (full explanation,
workaround, and trade-off) and the ALTER COLUMN reorder section (with
cross-reference)
- flink-ddl.md — under the Hive catalog configuration section
(engine-agnostic warning)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
