The KIP has passed with three binding +1 votes (Guozhang, Ismael, Jason)
and no -1 or +0 votes.
Thanks to everyone for the feedback.
Apurva
On Mon, Sep 11, 2017 at 8:31 PM, Apurva Mehta wrote:
> Hi Becket,
>
> You are right: the calculations are per partition produced to by each
> idempotent p
Hi Becket,
You are right: the calculations are per partition produced to by each
idempotent producer. I actually think this makes the problem more acute
when we actually end up enabling the idempotent producer by default.
However, even the most optimized version will still result in an overhead
of
Hi Apurva,
Thanks for the explanation.
I think the STO will be per producer/partition, right? Am I missing
something? You are right that the proposal does not strengthen the
semantic. The goal is more about trying to bound the memory consumption to
some reasonable number and use the memory more e
Hi Becket,
Regarding the current implementation: we opted for a simpler server side
implementation where we _don't_ snapshot the metadata of the last 5 batches
to disk. So if a broker fails, comes back online, and is the leader again,
it will only have the last batch in memory. With max.in.flight
Hi Apurva,
Sorry for being late on this thread. I am trying to understand the
implementation of case that we will throw DuplicateSequenceException. My
understanding is the following:
1. On the broker side, we will cache 5 most recent
sequence/timestamp/offset (STO) for each of the producer ID.
2.
Thanks for the votes everyone.
One of the proposals here was to raise a 'DuplicateSequenceException' to
the user if the broker detected that one of the internal retries resulted
in the duplicate, and the metadata for the original batch was no longer
cached.
However, when implementing this change,
Thanks for the comments Ismael.
I have gone ahead and incorporated all your suggestions in the KIP
document. You convinced me on adding max.message.bytes :)
Apurva
On Thu, Sep 7, 2017 at 6:12 PM, Ismael Juma wrote:
> Thanks for the KIP. +1 (binding) from me. A few minor comments:
>
> 1. We sho
Thanks for the KIP. +1 (binding) from me. A few minor comments:
1. We should add a note to the backwards compatibility section explaining
the impact of throwing DuplicateSequenceException (a new exception) from
`send`. As I understand it, it's not an issue, but good to include it in
the KIP.
2. F
Thanks for the votes!
Jason: I updated the KIP so that the messageFormatVersion field is int8.
Guozhang: The type of the config field is a purely internal concept. The
public API of the KafkaProducer takes a Map type or a
Properties type (which is a map of String to String). So this change in the
+1.
A quick clarification question regarding the compatibility plan as for "The
legacy values for `enable.idempotence` will be interpreted as follows by
the new producer: true will mean required, false will mean off."
Right now "enable.idempotence" is defined as Type.BOOLEAN while we are
likely t
+1. Thanks for the KIP. One nit: we could use int8 to represent the message
format version. That is how it is represented in the messages themselves.
-Jason
On Thu, Sep 7, 2017 at 1:51 PM, Apurva Mehta wrote:
> Hi,
>
> I'd like to start a vote for KIP-192:
>
> https://cwiki.apache.org/confluenc
Hi,
I'd like to start a vote for KIP-192:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-192+%3A+Provide+cleaner+semantics+when+idempotence+is+enabled
Thanks,
Apurva
12 matches
Mail list logo