Thank you for your reply. Oh is retention hours have affect on this? I didn't knew this. I have log.retention.hours set to 1, and during development we test this once a 15 mins or hour or 2. So do you think this is causing the issue?
Thanks, Pradeep Simha Technical Lead -----Original Message----- From: Steve Miller [mailto:st...@idrathernotsay.com] Sent: Tuesday, August 19, 2014 6:13 PM To: users@kafka.apache.org Subject: Re: Keep on getting kafka.common.OffsetOutOfRangeException: Random times Also, what do you have log.retention.hours set to? How often do you publish messages? I can envision a scenario in which you don't publish to a topic often, and in fact publish so infrequently that everything in the topic ages out from log.retention.hours first. I don't know exactly what happens should that occur, but I've seen some stuff that makes me think that the offsets might go back to zero -- or maybe they do if the broker restarts, so you might check to be sure that's not happening. From what I've seen in that regard, I've been wondering if part of the way most long-running Kafka consumers shouldn't be designed is to catch that exception and either set their offset to the first available message or the last available message, depending on whether their priority is to get every message or if it's to get the most recent messages. Though in that scenario maybe it's that the first and last messages are the same by definition since there aren't any messages left in the topic. (-: It's also possible that the specific topic weirdness that my specific installation has been running into is causing that and it only happens for the stuff I work on, so definitely take this with a grain of salt, I'm no expert, just relating the local folklore. -Steve On Tue, Aug 19, 2014 at 09:12:30AM +0000, pradeep.si...@wipro.com wrote: > Hi Team, > > Can someone please help me in this? This is really becoming road block to our > project we should decide whether to continue to use Kafka or some other > project as it is becoming too much of unstable. > > Thanks, > Pradeep Simha > Technical Lead > > -----Original Message----- > From: pradeep.si...@wipro.com [mailto:pradeep.si...@wipro.com] > Sent: Tuesday, August 19, 2014 9:30 AM > To: users@kafka.apache.org > Subject: RE: Keep on getting kafka.common.OffsetOutOfRangeException: > Random times > > Hi Neha, > > Yes, I am using the latest version ie (0.8.1.1). > > Hi Guozhang, > > These are the values: > > #log.retention.bytes= 1073741824 (Yes, this was commented by default) > > log.retention.check.interval.ms=60000 > > Am I doing anything wrong here? Since I haven't touched this properties file. > > Thanks, > Pradeep Simha > Technical Lead > > -----Original Message----- > From: Neha Narkhede [mailto:neha.narkh...@gmail.com] > Sent: Tuesday, August 19, 2014 2:27 AM > To: users@kafka.apache.org > Subject: Re: Keep on getting kafka.common.OffsetOutOfRangeException: > Random times > > Also, what version of Kafka are you using? 0.8.1.1 is the latest most stable > version. > > > On Mon, Aug 18, 2014 at 9:36 AM, Guozhang Wang <wangg...@gmail.com> wrote: > > > Hi Pradeep, > > > > It seems your logs gets deleted due to retention policy. Could you > > check the config values for log.retention.bytes and > > log.retention.check.interval.ms? > > > > http://kafka.apache.org/documentation.html#brokerconfigs > > > > Guozhang > > > > > > On Mon, Aug 18, 2014 at 5:49 AM, <pradeep.si...@wipro.com> wrote: > > > > > Hi Team, > > > > > > Of late I am facing strange issue w.r.t Kafka. Random times I keep > > > on getting these strange errors while consuming the topic: > > > > > > > > > kafka.common.OffsetOutOfRangeException: Request for offset 19 but > > > we only have log segments in the range 0 to 0. > > > Sometimes I get like this: > > > > > > > > > kafka.common.OffsetOutOfRangeException: Request for offset 19 but > > > we only have log segments in the range 19 to 22. > > > > > > That number keeps on changing (with random ranges). I don't know > > > what is the problem here. Both producer and consumer will work > > > perfectly, but I keep on getting these errors randomly. In that > > > situation if I clear the logs, remove the broker again it starts working > > > fine again. > > > > > > Can anyone please help me in this regard? This is affecting our > > > application stability, if any more information required I can > > > provide, > > also > > > we are using only the defaults provided by the kafka we didn't > > > changed > > any > > > settings. > > > > > > Thanks, > > > Pradeep Simha > > > Technical Lead > > > > > > The information contained in this electronic message and any > > > attachments to this message are intended for the exclusive use of > > > the addressee(s) > > and > > > may contain proprietary, confidential or privileged information. > > > If you > > are > > > not the intended recipient, you should not disseminate, distribute > > > or > > copy > > > this e-mail. Please notify the sender immediately and destroy all > > > copies > > of > > > this message and any attachments. > > > > > > WARNING: Computer viruses can be transmitted via email. The > > > recipient should check this email and any attachments for the presence of > > > viruses. > > > The company accepts no liability for any damage caused by any > > > virus transmitted by this email. > > > > > > www.wipro.com > > > > > > > > > > > -- > > -- Guozhang > > > > The information contained in this electronic message and any attachments to > this message are intended for the exclusive use of the addressee(s) and may > contain proprietary, confidential or privileged information. If you are not > the intended recipient, you should not disseminate, distribute or copy this > e-mail. Please notify the sender immediately and destroy all copies of this > message and any attachments. > > WARNING: Computer viruses can be transmitted via email. The recipient should > check this email and any attachments for the presence of viruses. The company > accepts no liability for any damage caused by any virus transmitted by this > email. > > www.wipro.com > > The information contained in this electronic message and any attachments to > this message are intended for the exclusive use of the addressee(s) and may > contain proprietary, confidential or privileged information. If you are not > the intended recipient, you should not disseminate, distribute or copy this > e-mail. Please notify the sender immediately and destroy all copies of this > message and any attachments. > > WARNING: Computer viruses can be transmitted via email. The recipient should > check this email and any attachments for the presence of viruses. The company > accepts no liability for any damage caused by any virus transmitted by this > email. > > www.wipro.com The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. www.wipro.com
