Hey folks Before I file a KIP to change this in 4.0, I wanted to understand the historical context for the value of the following setting.
Currently, segment.ms minimum threshold is set to 1ms [1]. Segments are expensive. Every segment uses multiple file descriptors and it's easy to run out of OS limits when creating a large number of segments. Large number of segments also delays log loading on startup because of expensive operations such as iterating through all directories & conditionally loading all producer state. I am currently not aware of a reason as to why someone might want to work with a segment.ms of less than ~10s (number chosen arbitrary that looks sane) What was the historical context of setting the minimum threshold to 1ms for this setting? [1] https://kafka.apache.org/documentation.html#topicconfigs_segment.ms -- Divij Vaidya
