deepakpanda93 commented on issue #7503: URL: https://github.com/apache/hudi/issues/7503#issuecomment-4844697029
This is fixed in current Hudi. Root cause: during Hive sync, TableSchemaResolver read the table schema by opening a base file from the latest commit. When delete_partition removed exactly those latest files, the schema read failed with Failed to read schema from data file … File does not exist. Fix: the schema resolver no longer depends on those files — It reads the schema from commit metadata first (getTableSchemaFromLatestCommitMetadata), so the deleted data files aren't touched in the normal path; and when it does fall back to reading a data file, it now only inspects commits that actually wrote data (getLatestCommitMetadataWithInsertOrUpdate, filtering numInserts/numUpdateWrites > 0), so a delete_partition/replace commit is skipped — HUDI-9612 / [#13586](https://github.com/apache/hudi/pull/13586), in 1.1.0+. @ad1happy2go was also unable to reproduce this as far back as 0.13.0. Closing as fixed (hardened in 1.1.0). If you still hit Failed to read data schema on a delete_partition + Hive sync on 1.1.0+, please reopen with a minimal repro and the timeline listing. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
