nicktelford commented on code in PR #15264:
URL: https://github.com/apache/kafka/pull/15264#discussion_r1473167769
##########
streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java:
##########
@@ -1836,6 +1841,33 @@ public void
maybeThrowTaskExceptionsFromProcessingThreads() {
}
}
+ // track the size of the transaction buffer on each iteration to predict
when it will be exceeded in advance
+ private long lastUncommittedBytes = 0L;
+
+ boolean needsCommit(final boolean updateDelta) {
+ if (maxUncommittedStateBytes < 0) {
+ // if our transaction buffers are unbounded, we never need to
force an early commit
+ return false;
+ }
+
+ // force an early commit if the uncommitted bytes exceeds or is
*likely to exceed* the configured threshold
+ final long uncommittedBytes = tasks.approximateUncommittedStateBytes();
+
+ final long deltaBytes = Math.max(0, uncommittedBytes -
lastUncommittedBytes);
+
+ final boolean needsCommit = maxUncommittedStateBytes > -1 &&
uncommittedBytes + deltaBytes > maxUncommittedStateBytes;
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
