deepakpanda93 commented on issue #13164: URL: https://github.com/apache/hudi/issues/13164#issuecomment-4850936095
@ckonehouse This is fixed. The root cause: when delete_partition runs iteratively, the Glue catalog sync computed a drop-partition list that still contained partitions already dropped in a previous iteration, so BatchDeletePartition failed with EntityNotFoundException: Partition [...] not found and the whole meta-sync errored. **Fix:** HUDI-9770 / [#13794](https://github.com/apache/hudi/pull/13794) ("During hive/glue sync, ensure drop partition events are generated when partition is present in the metastore") — the sync now only issues drops for partitions that actually exist in the catalog, so already-deleted partitions are skipped and this error no longer occurs. **Available in:** 1.1.0+, and backported to 0.15.1 ([#17857](https://github.com/apache/hudi/pull/17857)) and 0.14.2 ([#17924](https://github.com/apache/hudi/pull/17924)). Your report was on 0.14.0, which predates the fix. **Recommendation:** upgrade to 1.1.x (or a 0.14.2 / 0.15.1 bundle if you're pinned to the 0.x line on EMR-on-EKS). Closing as fixed — if you still hit EntityNotFoundException on drop-partition sync on those versions, please reopen with the sequence of delete_partition calls and the Glue table state. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
