Thanks for the help.
FWIW, I ended up writing a simple Util that I can use as my consumer is
starting up to move the offset back. It *seems* to work decently.
Thoughts? Would this be something that would be helpful for contribution
back to Kafka, or is the idea just poor?
/**
* Attempts to
I guess I missed a step between 4 and 5 -
4. Replace the exported offsets with these offsets
*Use ImportZkOffsets to import the offsets from the modified export file.*
5. Restart the consumer.
Thanks,
Neha
On Tue, Mar 19, 2013 at 11:00 AM, S Ahmed wrote:
> I thought since the offsets in .8 ar
I thought since the offsets in .8 are numeric and not byte offsets like in
0.7x, you can simply just take say the current offset - 1.
On Tue, Mar 19, 2013 at 12:16 PM, Neha Narkhede wrote:
> Jim,
>
> You can leverage the ExportZkOffsets/ImportZkOffsets tools to do this.
> ExportZkOffsets exp
Jim,
You can leverage the ExportZkOffsets/ImportZkOffsets tools to do this.
ExportZkOffsets exports the consumer offsets for your group to a file in a
certain format. You can then place the desired offset per partition you
want to reset your consumer to in the exported file.
1. Shutdown the consu
This API is exposed through the SimpleConsumer scala class. See
https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/consumer/SimpleConsumer.scala#L60
You will need to set earliestOrLatest to -1 for the latest offset.
There is also a command line tool
https://github.com/apache/
I'm still a bit lost. Where is the offsets API? I.e. which class?
On Tue, Mar 19, 2013 at 11:16 AM, David Arthur wrote:
> Using the Offsets API, you can get the latest offset by setting time to
> -1. Then you subtract 1
>
> There is no guarantee that 10k prior messages exist of course, so
Using the Offsets API, you can get the latest offset by setting time to
-1. Then you subtract 1
There is no guarantee that 10k prior messages exist of course, so you'd
need to handle that case.
-David
On 3/19/13 11:04 AM, James Englert wrote:
Hi,
I'm using Kafka 0.8. I would like to s
Hi,
I'm using Kafka 0.8. I would like to setup a consumer to fetch the last
10,000 messages and then start consuming messages.
I see the configuration autooffset.reset, but that isn't quite what I
want. I want only the last 10,000 messages.
Is there a good way to achieve this in 0.8, besides j
