On Wed, Oct 12, 2022 at 3:02 PM Niket Goel <ng...@confluent.io.invalid> wrote: > 1. Do we need this value to be of the order of `ms`. Is it better off being > tunable to a minute granularity?
Hmm. The most common unit for time intervals in Kafka is milliseconds. Very rarely does Kafka express time intervals using another unit. There are two examples where Kafka expresses time intervals in seconds. I would argue that even those cases should have used milliseconds as the time unit. We could limit the range of possible values for this configuration but it is not clear to me that Kafka should strictly enforce a minimum in all cases. Except for enforcing that the interval should not be a negative number. > 2. Somewhat related to 1 - Do we need to make any tweaks to the way we > cleanup the metadata directory (time/size based cleanup)? My concern is if > the metadata-dir cleanup will trigger fast enough in all cases today. e.g. > If I were to configure the time-based snapshotting to something like every > 10 seconds, with enough load of course, will I end up flooding the disk.? This seems to be an implementation issue given how RaftMetadataLogCleanerManager deletes snapshots and log segments. I could imagine a different implementation that doesn't have this problem. I think we should avoid as much as possible having implementation details leak to the configuration and public APIs. -- -José
