cshuo commented on issue #19025: URL: https://github.com/apache/hudi/issues/19025#issuecomment-5422127424
> we already encounter some issues for lock conrention between data table services and ingestion job: [#19587](https://github.com/apache/hudi/issues/19587), sharing the lock may make the issue more severe. @danny0405 This tool and the MDT writer API changes are not intended as a general table-service optimization. They target the specific use cases described in the issue motivation: https://github.com/apache/hudi/issues/17908#issuecomment-3923436575 : > For context, I added this since we want a way to have our table service platform perform all MDT table services for a dataset that is no longer receiving ingestion writes to the data table. Since in our org, we can do a backfill for clustering writes on a dataset, and our internal build of HUDI we don't perform MDT table services during clustering. For our use case it's fine if we hold the table lock the whole time (for table version 6) while calling performTableServices Such scenarios do not have a continuously running ingestion writer competing for the DT lock, so the severe Flink lock-contention pattern described in #19587 is not expected. For tables with regular ingestion, inline MDT table services—including compaction—continue to work as before, so this tool is unnecessary unless delegation is explicitly enabled. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
