Re: commitOffsets by partition 0.8-beta

2014-04-23 Thread Jun Rao
or last consumed offset. -Original Message- From: Jun Rao [mailto:jun...@gmail.com] Sent: Tuesday, April 22, 2014 9:52 PM To: users@kafka.apache.org Subject: Re: commitOffsets by partition 0.8-beta Do you have auto commit disabled? Thanks, Jun On Tue, Apr 22, 2014 at 7:10 PM

RE: commitOffsets by partition 0.8-beta

2014-04-23 Thread Seshadri, Balaji
I'm not seeing that API in java MessageAndMeta,is this part of ConsumerIterator. -Original Message- From: Jun Rao [mailto:jun...@gmail.com] Sent: Wednesday, April 23, 2014 8:47 AM To: users@kafka.apache.org Subject: Re: commitOffsets by partition 0.8-beta The checkpointed offset should

Re: commitOffsets by partition 0.8-beta

2014-04-23 Thread Jun Rao
. -Original Message- From: Jun Rao [mailto:jun...@gmail.com] Sent: Wednesday, April 23, 2014 8:47 AM To: users@kafka.apache.org Subject: Re: commitOffsets by partition 0.8-beta The checkpointed offset should be the offset of the next message to be consumed. So, you should save

Re: commitOffsets by partition 0.8-beta

2014-04-23 Thread Ian Friedman
offset to be consumed or last consumed offset. -Original Message- From: Jun Rao [mailto:jun...@gmail.com] Sent: Tuesday, April 22, 2014 9:52 PM To: users@kafka.apache.org Subject: Re: commitOffsets by partition 0.8-beta Do you have auto commit disabled? Thanks, Jun

RE: commitOffsets by partition 0.8-beta

2014-04-23 Thread Seshadri, Balaji
] Sent: Wednesday, April 23, 2014 10:14 AM To: users@kafka.apache.org Subject: Re: commitOffsets by partition 0.8-beta Take a look at the example in https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+SimpleConsumer+Example Thanks, Jun On Wed, Apr 23, 2014 at 9:01 AM, Seshadri, Balaji

RE: commitOffsets by partition 0.8-beta

2014-04-23 Thread Seshadri, Balaji
Does check point avoid duplicate update to zookeeper ?. -Original Message- From: Jun Rao [mailto:jun...@gmail.com] Sent: Wednesday, April 23, 2014 10:14 AM To: users@kafka.apache.org Subject: Re: commitOffsets by partition 0.8-beta Take a look at the example in https://cwiki.apache.org

RE: commitOffsets by partition 0.8-beta

2014-04-23 Thread Seshadri, Balaji
: commitOffsets by partition 0.8-beta Does check point avoid duplicate update to zookeeper ?. -Original Message- From: Jun Rao [mailto:jun...@gmail.com] Sent: Wednesday, April 23, 2014 10:14 AM To: users@kafka.apache.org Subject: Re: commitOffsets by partition 0.8-beta Take a look at the example

Re: commitOffsets by partition 0.8-beta

2014-04-23 Thread Jun Rao
this ?. -Original Message- From: Seshadri, Balaji Sent: Wednesday, April 23, 2014 12:01 PM To: 'users@kafka.apache.org' Subject: RE: commitOffsets by partition 0.8-beta Does check point avoid duplicate update to zookeeper ?. -Original Message- From: Jun Rao [mailto:jun...@gmail.com

RE: commitOffsets by partition 0.8-beta

2014-04-22 Thread Seshadri, Balaji
: Seshadri, Balaji Sent: Friday, April 18, 2014 11:50 AM To: 'users@kafka.apache.org' Subject: RE: commitOffsets by partition 0.8-beta Thanks Jun. -Original Message- From: Jun Rao [mailto:jun...@gmail.com] Sent: Friday, April 18, 2014 11:37 AM To: users@kafka.apache.org Subject: Re

RE: commitOffsets by partition 0.8-beta

2014-04-22 Thread Seshadri, Balaji
, Balaji -Original Message- From: Seshadri, Balaji [mailto:balaji.sesha...@dish.com] Sent: Tuesday, April 22, 2014 8:10 PM To: 'users@kafka.apache.org' Subject: RE: commitOffsets by partition 0.8-beta I'm updating the latest offset consumed to the zookeeper directory. Say for eg if my last

Re: commitOffsets by partition 0.8-beta

2014-04-22 Thread Jun Rao
check zookeeper path it has 6 after sometimes. Does any other process updates it ?. From: Seshadri, Balaji Sent: Friday, April 18, 2014 11:50 AM To: 'users@kafka.apache.org' Subject: RE: commitOffsets by partition 0.8-beta Thanks Jun. -Original

RE: commitOffsets by partition 0.8-beta

2014-04-22 Thread Seshadri, Balaji
Yes I disabled it. My doubt is the path should have offset to be consumed or last consumed offset. -Original Message- From: Jun Rao [mailto:jun...@gmail.com] Sent: Tuesday, April 22, 2014 9:52 PM To: users@kafka.apache.org Subject: Re: commitOffsets by partition 0.8-beta Do you have

Re: commitOffsets by partition 0.8-beta

2014-04-18 Thread Jun Rao
We don't have the ability to commit offset at the partition level now. This feature probably won't be available until we are done with the consumer rewrite, which is 3-4 months away. If you want to do sth now and don't want to use SimpleConsumer, another hacky way is to turn off auto offset

RE: commitOffsets by partition 0.8-beta

2014-04-18 Thread Seshadri, Balaji
Thanks Jun. -Original Message- From: Jun Rao [mailto:jun...@gmail.com] Sent: Friday, April 18, 2014 11:37 AM To: users@kafka.apache.org Subject: Re: commitOffsets by partition 0.8-beta We don't have the ability to commit offset at the partition level now. This feature probably won't