niranjan-1408 opened a new pull request, #19761: URL: https://github.com/apache/hudi/pull/19761
### Describe the issue this Pull Request addresses When a column type changes on a partitioned table, `AWSGlueCatalogSyncClient` cascades the new columns to every partition. It rebuilt each partition's path from its values as `KEY=VALUE` via `getStringFromPartition`, then turned that string back into a `StorageDescriptor` location. With `hive_style_partitioning=false`, which is the default, the on-disk layout is bare values, so the cascade repointed every partition at a prefix holding no data. The partition values were still correct, because they go through `partitionValueExtractor`, so Glue matched the existing partition and performed an update rather than rejecting an unknown one. The catalog therefore silently disagreed with storage, and Hive metastore readers returned no rows for the affected partitions with no error raised. ### Summary and Changelog The cascade now reuses the location Glue already holds and the values it already returned, changing only the columns. That is correct for every key generator and encoding because it never derives a path. - `updateTableSchema` cascade now passes the `Partition` objects from `getAllPartitions` straight through instead of synthesizing path strings; `getStringFromPartition` is removed. - `updatePartitionsToTableInternal` and the cascade share one `updatePartitionsInternal`. Partitions are passed as a `Supplier` so a failure deriving them from a storage path is still wrapped, rather than escaping unwrapped and losing a level of the cause chain. - `TestAWSGlueSyncClient` covers the cascade for hive-style, bare and slash-encoded layouts, asserting both that the recorded location survives verbatim and that the table's columns reach the partition input. ### Impact Fixes silent data loss for Hive metastore readers (Athena, Redshift Spectrum, EMR Hive, Trino) on partitioned tables that do not use hive-style partitioning. Affected partitions resolved to an empty prefix and returned no rows. Spark's Hudi datasource is unaffected because it reads from the base path and timeline rather than Glue partition locations. Only partitions actively written to recovered, because `syncPartitions` rewrites their location on a later commit. On a date-partitioned table that is the current partition, so historical partitions stayed broken. ### Risk Level low The cascade writes strictly less derived data than before: it stops recomputing a field and reuses the value already stored. Verified on a live Glue catalog, where an `ALTER COLUMN TYPE` on a non-hive-style partitioned table left two of three partitions pointing at prefixes containing no objects, and Athena returned 22 of 31 rows with no error. ### Documentation Update none ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
