vamossagar12 commented on a change in pull request #9756:
URL: https://github.com/apache/kafka/pull/9756#discussion_r593032627
##########
File path: raft/src/main/java/org/apache/kafka/raft/RaftConfig.java
##########
@@ -76,6 +76,11 @@
"wait for writes to accumulate before flushing them to disk.";
public static final int DEFAULT_QUORUM_LINGER_MS = 25;
+ public static final String QUORUM_APPEND_MAX_UNFLUSHED_BYTES_CONFIG =
QUORUM_PREFIX + "append.max.unflushed.bytes";
Review comment:
I see what you are trying to say. Well, the premise of this ticket
originally was to trigger fsyncs happen the moment a configured amount of bytes
have been accumulated. Here is the original description for Jason in the ticket:
> In KAFKA-10601, we implemented linger semantics similar to the producer to
let the leader accumulate a batch of writes before fsyncing them to disk.
Currently the fsync is only based on the linger time, but it would be helpful
to make it size-based as well. In other words, if we accumulate a configurable
N bytes, then we should not wait for linger expiration and should just fsync
immediately.
But as you pointed out, it is also due to the fact that in the current
implementation batch append and fsync goes hand in hand.
With the future implementation on deferring fsync, this might just affect
the batch appends and considering that in mind, imo it makes sense to rename it
to `append.linger.bytes` . It also matches with `append.linger.ms` .
BTW, on the fsync deferral track. i had created a draft PR where i have
outlined my approach: https://github.com/apache/kafka/pull/10278
Request you or Jason to review this..
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
