Hi Dustin,
thanks for the reply. to be honest, I am trying to work around
https://issues.apache.org/jira/browse/KAFKA-1194.
I implemented a small main() to call the LogCleaner for compaction and
it seemed to work, but left the zero-byte files. The idea is to stop the
broker, then run the compaction as a separate process, then restart the
broker in order to get around this problem where the whole process trips
over its own shoe laces due to Windows' file locking.
Since I haven't seen the compaction working on Windows yet, I was
wondering whether this is to be expected. Should the compaction remove
the zero-byte files are would the broker do this? I'm not yet enough
into the cleanup code to understand this.
Regards,
Harald
On 05.08.2016 16:23, Dustin Cote wrote:
Harald,
I note that your last modified times are all the same. Are you maybe using
Java 7? There's some details here that a JDK bug in Java 7 causes the last
modified time to get updated on broker restart:
https://issues.apache.org/jira/browse/KAFKA-3802
On Fri, Aug 5, 2016 at 6:12 AM, Harald Kirsch <harald.kir...@raytion.com>
wrote:
Hi,
experimenting with log compaction, I see Kafka go through all the steps,
in particular I see positive messages in log-cleaner.log and *.deleted
files. Yet once the *.deleted segment files have disappeared, the segment
and index files with size 0 are still kept.
I stopped and restarted Kafka and saw several rounds of the log cleaner go
by, but the empty files stay:
-rw-rw-r-- 1 0 Aug 5 11:58 00000000000000000000.index
-rw-rw-r-- 1 0 Aug 5 11:58 00000000000000000000.log
-rw-rw-r-- 1 0 Aug 5 11:58 00000000000000000051.index
-rw-rw-r-- 1 0 Aug 5 11:58 00000000000000000051.log
-rw-rw-r-- 1 0 Aug 5 11:58 00000000000000000096.index
-rw-rw-r-- 1 0 Aug 5 11:58 00000000000000000096.log
[... snip ...]
-rw-rw-r-- 1 92041 Aug 5 11:58 00000000000000000680.log
-rw-rw-r-- 1 0 Aug 5 11:58 00000000000000000727.index
-rw-rw-r-- 1 199822 Aug 5 11:58 00000000000000000727.log
-rw-rw-r-- 1 10485760 Aug 5 11:58 00000000000000000781.index
-rw-rw-r-- 1 95972 Aug 5 11:58 00000000000000000781.log
Is this expected behavior or is there yet another configuration option
that defines when these get purged?
Harald.
