Hello Xiaochi ,
I am not sure if I have understood the problem correctly but beware the
fact that only old log segments and not the current log segment are
taken into account for deletions. So if you want the data to be deleted
in a timely manner, you also need to configure a tighter interval for
rotating the log files.
See 'log.roll.hours' (broker config):
https://kafka.apache.org/documentation.html#brokerconfigs_log.roll.hours
Kind regards
Franz
Am 02.02.2022 um 20:37 schrieb Xiaochi Weng:
Hello,
I am currently using kafka 3.1.0 with java 1.8. I have set kafka log
retention policy in the server.properties like this:
log.retention.hours=6
log.retention.bytes=5368709120
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
log.cleanup.policy=delete
However, it seems that Kafka is still keeping the log opened and keeps
consuming the spaces. I have used the "df -h" and "du" commands and found
that the number of used spaces were different. When I checked with lsof |
grep "deleted", I found that kafka is still keeping deleted files.
Currently, it seems that the only way to resolve this issue is to kill the
kafka process and restart a new one.
Could you tell me whether there is an alternative way to resolve this issue?
Thanks,
Xiaochi Weng
