Re: Kafka Streams leave group behaviour

2021-08-18 Thread Sophie Blee-Goldman
As Boyang mentioned, Kafka Streams intentionally does not send a LeaveGroup request when shutting down. This is because often the shutdown is not due to a scaling down event but instead some transient closure, such as during a rolling bounce. In cases where the instance is expected to start up

Re: Kafka metrics to calculate number of messages in a topic

2021-08-18 Thread Eric Azama
That will get you a good approximation, but it's not guaranteed to be completely accurate. Offsets in Kafka are not guaranteed to be continuous. For topics with log compaction enabled, the removed records will leave (potentially very large) holes in the offsets. Even for topics without log