majian1998 opened a new pull request, #10048: URL: https://github.com/apache/hudi/pull/10048
In the current implementation of run multi tables services, the clustering task and compaction task both build metaclient repeatedly for each table, causing additional overhead. To reduce this overhead, we extract the construction of metaclient and only construct it once for each table, passing it as a parameter to the corresponding task. At the same time, when running multi tables services, the write config lacks some information from the table config, such as the table name. This leads to empty strings when retrieving the table name in certain situations. For example, when configuring the prefix for metrics, if not specified, the table name is used as the prefix. However, in the current situation, without the table config, it's impossible to differentiate the metrics of different tables, resulting in an empty prefix. By adding the table config to the write config beforehand, we can obtain all the configuration information in the subsequent write config step. Additionally, we made a small modification by removing the redundant construction of metaclient in the clusteringJob's constructor. ### Change Logs We now construct metaclient only once per table and add table config to the write config to obtain all necessary information. Furthermore, unnecessary construction of metaclient in the clusteringJob's constructor has been removed. ### Impact None ### Risk level (write none, low medium or high below) None ### Documentation Update None ### Contributor's checklist - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [ ] Change Logs and Impact were stated clearly - [ ] Adequate tests were added if applicable - [ ] CI passed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
