suryaprasanna opened a new pull request, #17930: URL: https://github.com/apache/hudi/pull/17930
### Describe the issue this Pull Request addresses This PR adds graceful handling of metadata table service failures with a new configuration option and enhanced observability through metrics. ### Summary and Changelog Users can now configure whether metadata table service failures should fail the entire job or be handled gracefully. This provides more flexibility in production environments where metadata table operations might fail temporarily without affecting data writes. Changes: - Added new config `hoodie.metadata.fails.job.on.table.services.failures` (default: true) to control job failure behavior - Modified exception handling in `HoodieBackedTableMetadataWriter` to conditionally throw exceptions based on the config - Added try-catch blocks around compaction and log compaction operations - Added three new metrics: `compaction_failures`, `logcompaction_failures`, and `pending_compactions_failures` - Updated unit tests to pass the new metrics parameter ### Impact New configuration option: `hoodie.metadata.fails.job.on.table.services.failures` - When set to `true` (default): maintains existing behavior - job fails on metadata table service failures - When set to `false`: job continues even if metadata table services fail, but metrics are emitted New metrics added for better observability of metadata table service failures. ### Risk Level Low - The default behavior remains unchanged (jobs fail on metadata table service failures). The new behavior is opt-in. ### Documentation Update Config documentation added in `HoodieMetadataConfig.java` for the new configuration property `hoodie.metadata.fails.job.on.table.services.failures`. ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
