[GitHub] [hudi] yihua commented on pull request #8238: [HUDI-5954] Infer cleaning policy based on clean configs
yihua commented on PR #8238: URL: https://github.com/apache/hudi/pull/8238#issuecomment-1492861245 CI is green. https://user-images.githubusercontent.com/2497195/229271805-6db678af-348c-48c9-8862-de80acbe56ae.png";> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] yihua commented on pull request #8238: [HUDI-5954] Infer cleaning policy based on clean configs
yihua commented on PR #8238: URL: https://github.com/apache/hudi/pull/8238#issuecomment-1492829259 > > > I'm so confused by these options, does the option hoodie.cleaner.policy make any sense here? If all the specific cleaning param: hoodie.cleaner.commits.retained, hoodie.cleaner.hours.retained, hoodie.cleaner.fileversions.retained all have detemistic policy, then this option should be eliminated. > > > For example, can we use a combination like HoodieCleaningPolicy.KEEP_LATEST_COMMITS policy and hoodie.cleaner.fileversions.retained, if not, introduce the redundant option key hoodie.cleaner.policy is totally unnecessary. > > > > > > @danny0405 what you mentioned totally makes sense. The reason I keep `hoodie.cleaner.policy` is that we should still make it backwards compatible, so that if `hoodie.cleaner.policy` is set, we should honor that. If none of `hoodie.cleaner.commits.retained`, `hoodie.cleaner.hours.retained`, or `hoodie.cleaner.fileversions.retained` is set, the specified policy should be used, and the default clean config value (commits, hours, or file versions retained) is used. > > Then let's mark this option `hoodie.cleaner.policy` as deprecated and add some release notes on the website. Makes sense. Fixed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] yihua commented on pull request #8238: [HUDI-5954] Infer cleaning policy based on clean configs
yihua commented on PR #8238: URL: https://github.com/apache/hudi/pull/8238#issuecomment-1491404273 > I'm so confused by these options, does the option hoodie.cleaner.policy make any sense here? If all the specific cleaning param: hoodie.cleaner.commits.retained, hoodie.cleaner.hours.retained, hoodie.cleaner.fileversions.retained all have detemistic policy, then this option should be eliminated. > > For example, can we use a combination like HoodieCleaningPolicy.KEEP_LATEST_COMMITS policy and hoodie.cleaner.fileversions.retained, if not, introduce the redundant option key hoodie.cleaner.policy is totally unnecessary. @danny0405 what you mentioned totally makes sense. The reason I keep `hoodie.cleaner.policy` is that we should still make it backwards compatible, so that if `hoodie.cleaner.policy` is set, we should honor that. If none of `hoodie.cleaner.commits.retained`, `hoodie.cleaner.hours.retained`, or `hoodie.cleaner.fileversions.retained` is set, the specified policy should be used, and the default clean config value (commits, hours, or file versions retained) is used. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
