cshuo opened a new pull request, #19731: URL: https://github.com/apache/hudi/pull/19731
### Describe the issue this Pull Request addresses Closes #19025. Metadata table services currently run through `HoodieBackedTableMetadataWriter#performTableServices`, which requires a data-table write even when the only goal is to maintain the metadata table. This can lead users to generate empty commits to trigger MDT compaction, log compaction, cleaning, and archival. This change provides a standalone Spark-submit tool and reusable metadata-writer APIs so an external service can schedule and execute MDT table services without writing to the data table. Scheduling and timeline publication are serialized with data-table writers through the shared data-table lock, while expensive compaction execution uses the MDT OCC path added by #18295. ### Summary and Changelog - Add `HoodieMetadataTableServicesTool` under `hudi-utilities` with schedule, execute, and schedule-and-execute modes for MDT compaction, log compaction, cleaning, and archival. - Add reusable `scheduleTableServices` and `executeTableServices` APIs, backed by `MetadataTableServiceRequest`, to `HoodieTableMetadataWriter`. - Refactor inline MDT table services to use the shared request-driven implementation while preserving table-version-specific compaction semantics. - Add separate scheduling delegation through `hoodie.metadata.table.service.manager.schedule.actions` while retaining the existing execution delegation behavior. - Derive MDT OCC lock configuration from the data-table lock so external execution shares the same logical data-table lock during timeline transitions and completion. ### Impact This adds public metadata-writer APIs, a new Spark-submit utility, and a new advanced scheduling-delegation configuration. Existing inline table-service behavior remains the default. Users can opt into external MDT maintenance without producing empty data-table commits. Scheduling and short timeline publication/finalization sections contend on the existing data-table lock. Compaction and log-compaction execution do not hold that lock for the full execution duration. ### Risk Level Medium. The change affects MDT table-service orchestration and concurrent-writer locking. The implementation preserves the existing inline order, keeps table-version-six fallback behavior, reuses existing MDT write-client APIs, and includes focused coverage for request modes, delegation, lock derivation, failure propagation, and table-version-specific scheduling. ### Documentation Update Document the new `HoodieMetadataTableServicesTool` invocation and scheduling-delegation configuration before the feature is finalized. ### Contributor's checklist - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
