nsivabalan commented on code in PR #17935:
URL: https://github.com/apache/hudi/pull/17935#discussion_r2720303288
##########
hudi-common/src/main/java/org/apache/hudi/common/config/HoodieMetadataConfig.java:
##########
@@ -151,6 +153,13 @@ public final class HoodieMetadataConfig extends
HoodieConfig {
.sinceVersion("0.14.0")
.withDocumentation("Controls the criteria to log compacted files groups
in metadata table.");
+ public static final ConfigProperty<String> CLEANER_POLICY = ConfigProperty
+ .key(METADATA_PREFIX + ".clean.policy")
+ .defaultValue(KEEP_LATEST_FILE_VERSIONS.name())
+ .markAdvanced()
+ .sinceVersion("1.2.0")
+ .withDocumentation("This config determines the cleaner policy for
metadata table.");
Review Comment:
can we also add a boolean Config property named
`hoodie.metadata.clean.follow.data.table.policy` and enable it by default.
out of the box, lets fall back to old behavior with 1.2X multiplier.
but advanced users will have a choice to override this and use
`hoodie.metadata.clean.policy` on a need basis.
I am worried that, some data table could have 7 day cleaner retention, but
switching to this FILE versions based and not deriving the value from data
table might have an impact
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
