Hi, Sorry for the late reply, let me clarify on this.
I am developing using Golang so I used a library based on librdkafka, and there's one function, ReadMesage(), which is a wrapper on top of the poll() function, except that it will only poll one message(record) at a time and return either one of the following, 1. (msg, nil) -> normal situation with no error; 2. (nil, err) -> err is a Kafka timeout error; 3. (nil, err) -> err is general error; 4. (msg, err) -> err is partition-specific error. When I was browsing the javadocs https://kafka.apache.org/26/javadoc/index.html?org/apache/kafka/clients/consumer/KafkaConsumer.html, I noticed the concept of `Offsets and Consumer Position`, so I am wondering that in the event of the situation 2, 3 and 4, will the position still increment by 1 even if what I get from ReadMessage() is an error? I tried to read the docs but didn't find anything useful I can relate to, I tried to ask for help from the contributors of the library I was using but didn't get an answer, not sure if this question is too obvious or too noobie lolol. Since that library I used is an implementation of the Kafka protocol, I decided to ask it here and I sincerely hope I can get some insights from a Kafka guru. Thanks, Zhen Zhang Software Engineer [image: Twilio] <https://www.twilio.com/?utm_source=email_signature> MOBILE (949) 771-6073 EMAIL zzh...@twilio.com On Wed, Sep 23, 2020 at 9:45 AM Matthias J. Sax <mj...@apache.org> wrote: > I guess it depends where the exception comes from? Can you clarify? > > -Matthias > > On 9/23/20 12:53 AM, Zhen Zhang wrote: > > Hi there, > > > > I am new to Kafka and I would like to get some clarifications for a > newbie > > question, > > > > Let's say if I have set up my consumer's "enable.auto.commit" to false, > and > > then poll the records one at a time. So when calling poll(), starting > from > > offset 0, if any exception is thrown, should I expect to get the record > at > > offset 0 or offset 1 when I call poll() again? The reason I'm asking for > > this is bc in the Kafka Doc, it says that, > > "The position of the consumer gives the offset of the next record that > will > > be given out. It will be one larger than the highest offset the consumer > > has seen in that partition. It automatically advances every time the > > consumer receives messages in a call to poll(Duration)." > > > > But in my described situation above, an exception is thrown, I'm not sure > > if this is counted as a successful poll (meaning that the next poll() > will > > give the next record) or a failed one (meaning that the next poll() will > > give the same record again). > > > > I would really appreciate it for your help. > > > > Thanks, > > Zhen Zhang > > Software Engineer > > [image: Twilio] <https://www.twilio.com/?utm_source=email_signature> > > MOBILE (949) 771-6073 > > EMAIL zzh...@twilio.com > > >
