codope opened a new pull request, #20350: URL: https://github.com/apache/hudi/pull/20350
### Describe the issue this Pull Request addresses Data table writers run clean, compaction and archival on the metadata table inline: Spark and Java when a write starts (under the table lock), Flink when the coordinator starts and on each metadata table commit. Some jobs must not maintain the metadata table, for example a partition delete run by a table service platform. `hoodie.table.services.enabled=false` does not stop this. ### Summary and Changelog - New advanced config `hoodie.metadata.inline.table.services.enabled` (default true). When false, writers skip inline metadata table services. - The check is added in `SparkRDDWriteClient`, `HoodieJavaWriteClient`, `HoodieFlinkTableServiceClient.initMetadataTable` and `FlinkHoodieBackedTableMetadataWriter.commitInternal`. - This doesn't reuse `hoodie.table.services.enabled` because writers that keep that flag off rely on these paths to compact the metadata table, and data table archival cannot move past the latest metadata compaction. - Tests: new `TestSparkDeletePartitionTableServices` and `TestStreamWriteOperatorCoordinator#testSkipInlineMetadataTableServices`. Each checks that no metadata clean, compaction or archival happens with the setting off. Removing any of the checks fails a test. ### Impact None by default. With the setting off, the metadata table is maintained only by other writers or an external table service. ### Risk Level low ### Documentation Update Config doc string only. ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
