n3nash commented on issue #2771:
URL: https://github.com/apache/hudi/issues/2771#issuecomment-814682374
@stackfun I see that you have some conflicting configs
```
"hoodie.datasource.compaction.async.enable": True,
"hoodie.index.type": "SIMPLE",
"hoodie.compact.inline": True,
```
You have enabled `inline` as well as `async` at the same time. Can you
please enable one of them depending on if you want async or inline.
Additionally, I see that you have set the num delta commits before
compaction kicks in to 16
```
hoodie.compact.inline.max.delta.commits : "16"
```
This means compaction will not kick in until you do 16 delta commits.
Additionally, even after that, the default CompactionPolicy is `Bounded`
https://github.com/apache/hudi/blob/master/hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieCompactionConfig.java#L97
which will compact at max 500GB
https://github.com/apache/hudi/blob/master/hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieCompactionConfig.java#L94.
Are you by any change writing more than 500GB during your 16 delta commits ?
If yes, some log files may not get compacted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
