n3nash commented on issue #2771: URL: https://github.com/apache/hudi/issues/2771#issuecomment-814682374
@stackfun I see that you have some conflicting configs ``` "hoodie.datasource.compaction.async.enable": True, "hoodie.index.type": "SIMPLE", "hoodie.compact.inline": True, ``` You have enabled `inline` as well as `async` at the same time. Can you please enable one of them depending on if you want async or inline. Additionally, I see that you have set the num delta commits before compaction kicks in to 16 ``` hoodie.compact.inline.max.delta.commits : "16" ``` This means compaction will not kick in until you do 16 delta commits. Additionally, even after that, the default CompactionPolicy is `Bounded` https://github.com/apache/hudi/blob/master/hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieCompactionConfig.java#L97 which will compact at max 500GB https://github.com/apache/hudi/blob/master/hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieCompactionConfig.java#L94. Are you by any change writing more than 500GB during your 16 delta commits ? If yes, some log files may not get compacted. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org