diqiu50 opened a new pull request, #14561: URL: https://github.com/apache/gravitino/pull/14561
### What changes were proposed in this pull request? - Alter Hive tables and schemas starting from the stored Glue object, so unmodeled fields are kept; pass `versionId` so concurrent changes fail with 409. Map `location`, `serde-name` and `serde.parameter.*` to the storage descriptor. - Store schema `location` as the Glue `LocationUri`. - Default the Hive storage format to text, and require explicit format classes to be complete and consistent. - Refuse partitioning, distribution, sort order and column definitions Glue cannot store. - Treat column names as case-insensitive and refuse duplicate names. - Map partition values by field name, escape partition names like Hive, and copy the table storage descriptor to new partitions. - Fix `double precision`, bare `decimal` and unsigned type conversion; validate struct field names. - Document that a Glue Data Catalog must be managed by only one Gravitino catalog. This PR is based on #14517 and includes its commits. ### Why are the changes needed? Altering Glue tables and schemas lost data written by other engines, and many invalid requests were accepted and then failed or were silently changed. Fixes #14560 ### Does this PR introduce _any_ user-facing change? Yes. - Hive tables created without `format` now default to text instead of Parquet. - Column names are case-insensitive; on Iceberg tables, columns added by alter are stored in lower case. - Invalid partitioning, distribution, sort order, column and format settings are rejected with 400. - Concurrent alters on the same Hive table return 409. ### How was this patch tested? - Added unit tests in `catalog-glue` (268 passed). - Added Moto ITs for native table alter, schema location and partition escaping; ran `MotoGlueCatalogIT` locally (35 passed). - Ran `AwsGlueCatalogIT` against real AWS Glue (35 passed). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
