koochiswathiTR commented on issue #3739: URL: https://github.com/apache/hudi/issues/3739#issuecomment-1423611752
@nsivabalan,@vinothchandar , @bhasudha , @bvaradar , @n3nash Cleanup tirggers after compaction? or Cleanup runs when an upsert on hudi dataset ? or cleanup triggers when configured criteria is met? We have configured our hudi with these configs HoodieCompactionConfig.CLEANER_POLICY.key() -> HoodieCleaningPolicy.KEEP_LATEST_COMMITS.name(), HoodieCompactionConfig.CLEANER_COMMITS_RETAINED.key() -> "936, HoodieCompactionConfig.MIN_COMMITS_TO_KEEP.key() -> "937", HoodieCompactionConfig.MAX_COMMITS_TO_KEEP.key() -> "960" Cleanup will not trigger until 936 commits on hudi timeline. Once the cleanup is triggered we see increase in processing time. We see processing time was at 4 mins 40 sec < 5 mints until cleanup started running. Processing time is at 6 mints once cleaned started running. Thanks in advance! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
