To: users@kafka.apache.org
Subject: Re: committing particular offset with high level consumer?
Joris,
The high level consumer now has this API:
def commitOffsets(offsetsToCommit: immutable.Map[TopicAndPartition,
OffsetAndMetadata], retryOnFailure: Boolean)
I believe that was added in the 0.9 versions
Hi,
We would like to log the offset of a Kafka message if we fail to process it
(so we can try to re-process it later). Is it possible to get the offset
using the high level consumer?
I took a quick look at the code, and:
- It seems like the offset it private in the current Scala consumer
etsky" <eugene.miret...@gmail.com> wrote:
> Hi,
>
> We would like to log the offset of a Kafka message if we fail to process it
> (so we can try to re-process it later). Is it possible to get the offset
> using the high level consumer?
>
> I took a quick look at the code, a
Hi Neha,
Thanks for the quick reply. Could you tell me if there is some way of
determining the offset for a consumer from a high level Java consumer class
apart from ConsumerOffsetChecker tool? This tool can be run only from the
command line. Is it possible to use this in a Java class?
I write
The other option is jmx bean that exposes the lag. Also Kafka provides at
least once guarantees so even if your consumer lags occasionally, you will
eventually receive all mesaages. You need to provision enough consumers so
that they don't fall behind.
Thanks,
Neha
On May 23, 2013 5:30 AM, arathi
You can run the ConsumerOffsetChecker tool that ships with Kafka.
Thanks,
Neha
On Wed, May 22, 2013 at 2:02 PM, arathi maddula arathimadd...@gmail.comwrote:
Hi,
Could you tell me how to find the offset in a high level Java consumer ?
Thanks
Arathi
