Hi Fares, On Linux kernels, you can use the property "dirty_writeback_centisecs" [1] to configure the period between executions of kswapd, which does this "sync" job. The period is usually set to 30 seconds. There are few exceptions where Kafka explicitly forces a sync (via the force() method from the I/O API of the JDK), e.g. when a segment is rolled or Kafka shutting down.
The page writeback activity from your kernel is monitorable at different levels of granularity and depending on the instrumentation you are willing to use. Why would you want to monitor this activity in the first place? Do you want to know exactly *when* your data is on the disk? [1] https://www.kernel.org/doc/Documentation/sysctl/vm.txt Le lun. 9 mars 2020 à 15:58, Fares Oueslati <oueslati.fa...@gmail.com> a écrit : > > Hello, > > By default, both log.flush.interval.ms and log.flush.interval.messages are > set to Long.MAX_VALUE. > > As I understand, it makes Kafka flush log to disk (fsync) only depends on > file system. > > Is there any simple way to monitor that frequency ? > > Is there a rule of thumb to estimate that value depending on the os ? > > Thank you guys ! > Fares
