Hi, from the confluent docs, > Offset Management: The two main settings affecting offset management are whether auto-commit is enabled and the offset reset policy. First, if you set enable.auto.commit (which is the default), then the consumer will automatically commit offsets periodically at the interval set by auto.commit.interval.ms. The default is 5 seconds.
As you can see https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java#L161 the coordinator is the responsible of committing the offsets every interval. this is https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java#L358 as well another point when the offsets gets committed. >From your message: > If that is the case then the only way to guarantee that no message will get lost is to commit consumed offsets manually after processing the messages returned from poll(), right? messages in the consumers don't get lost, you mean you don't process them twice, right? Cheers Missatge de Peter Levart <peter.lev...@gmail.com> del dia dj., 31 de gen. 2019 a les 23:23: > Hello, > > I'm using Java KafkaConsumer and I'm wondering what is the guarantee of > consuming messages when enable.auto.commit is set to true. > > What is the earliest time the offsets of messages returned from last > poll() may be committed? Immediately after returning from poll() or upon > next call to poll()? > > If it is the former, am I right to assume that messages may get lost in > the event of consumer crash? > > If that is the case then the only way to guarantee that no message will > get lost is to commit consumed offsets manually after processing the > messages returned from poll(), right? > > Thanks, Peter > > -- Pere Urbon-Bayes Software Architect http://www.purbon.com https://twitter.com/purbon https://www.linkedin.com/in/purbon/
