On 4/11/20 2:45 PM, Alex Craig wrote:
Yep, max poll interval is 2147483647 and session timeout is 12 (2
minutes). I don't have anything set for heartbeat.interval.ms, so it must
be using the default. (3 seconds I think?) Hmm, is it possible the
heartbeat might not happen if the client ap
Yep, max poll interval is 2147483647 and session timeout is 12 (2
minutes). I don't have anything set for heartbeat.interval.ms, so it must
be using the default. (3 seconds I think?) Hmm, is it possible the
heartbeat might not happen if the client app was so swamped with restoring
data (and m
Hey Alex,
Huh.
Unprefixed configs apply to all consumers, but in this case, it's
irrelevant because only the "main" consumer participates in group
management (so the config only applies to the main consumer).
So you actually have max.poll.interval.ms set to Integer.MAX_VALUE,
which amounts to 25
Thanks John, I double-checked my configs and I've actually got the
max.poll.interval.ms set to the max (not prefixed with anything so
presumably that’s the “main” consumer). So I think that means the problem
isn’t due to a single batch of messages not getting processed/committed
within the polling
Hi Alex,
It sounds like your theory is plausible. After a rebalance, Streams needs to
restore its stores from the changelog topics. Currently, Streams performs this
restore operation in the same loop that does processing and polls the consumer
for more records. If the restore batches (or the pr
Hi all, I’ve got a Kafka Streams application running in a Kubernetes
environment. The topology on this application has 2 aggregations (and
therefore 2 Ktables), both of which can get fairly large – the first is
around 200GB and the second around 500GB. As with any K8s platform, pods
can occasiona
