ultiple threads (which is not
> recommended), similar principle applies - commit offsets only after
> processing them.
>
> Thanks,
>
> Jiangjie (Becket) Qin
>
> On 6/21/15, 10:50 PM, "Carl Heymann" wrote:
>
> >Thanks Jiangjie
> >
> >So you agree that w
nce, ideally we
> should wait until all the messages returned by iterator to be processed
> before commit offsets.
>
> In LinkedIn, we have wrapper around open source consumer iterator where we
> can implants those logics.
>
> Jiangjie (Becket) Qin
>
> On 6/19/15, 12:22 AM, "
ffset() for the
> last message processed. The MessageAndMetadata instance comes from the
> ConsumerIterator.
>
> On Fri, Jun 19, 2015 at 2:31 AM Carl Heymann wrote:
>
> > How are you tracking the offsets that you manually commit? I.e. where do
> > you get the metadata for
How are you tracking the offsets that you manually commit? I.e. where do
you get the metadata for the consumed messages?
On Thu, Jun 18, 2015 at 11:21 PM, noah wrote:
> We are in a situation where we need at least once delivery. We have a
> thread that pulls messages off the consumer, puts them
t; >>
> >> Content-Type: text/plain; charset=UTF-8
> >>
> >>
> >>
> >>
> >> With auto-commit one can only have at-most-once delivery guarantee -
> after
> >>
> >> commit but before message is delivered for processing, or
just so that you can commit offsets with
at-least-once semantics. The downside of this approach is more duplicate
deliveries after recovery from hard failure (but this is "at least once",
right, not "exactly once").
I don't propose that the code necessarily be changed like this in trunk, I
just want to know if the approach seems reasonable.
Regards
Carl Heymann
