Hi Jason, Thanks for the comments:
1. I have also updated the KIP to indicate that the DuplicateSequenceException could be the new error code returned in the producer callback as a result of tightening up the semantics of the OutOfOrderSequenceException. 2. I think returning the message format version in the TopicMetadata of the MetadatResponse is a great idea. It is general enough, and could be put to other uses in the future. Also, it is more efficient to send the right message format to the broker to begin with, so that's another thing going for this solution. I have updated the KIP to reflect this change. Tom: I have update the KIP so that the 'safe' value of 'enable.idempotence' is converted to 'requested' everywhere. Thanks, Apurva On Tue, Sep 5, 2017 at 11:17 AM, Jason Gustafson <ja...@confluent.io> wrote: > The proposal looks good. Two minor comments: > > 1. Can we call out how we handle the duplicate case? This is a change in > behavior since we currently raise OutOfOrderSequence in this case. > > 2. Instead of passing through `idempotenceLevel` in the ProduceRequest, I > wonder if we should have a field for the minimum required message format. > When using enable.idempotence=required, we could set the minimum required > version to v2. For enable.idempotence=requested, we could use v0. The > advantage is that we may find other uses for a more general field in the > future. Alternatively, maybe we really should be returning the message > format version of each topic in the TopicMetadata response. A nice bonus of > doing so is that it gives the producer the ability to craft the right > format version ahead of time and avoid the need for conversion on the > broker. > > Thanks, > Jason > > On Tue, Aug 29, 2017 at 4:32 PM, Apurva Mehta <apu...@confluent.io> wrote: > > > Hi, > > > > In the discussion of KIP-185 (enable idempotence by default), we > discovered > > some shortcomings of the existing idempotent producer implementation. > > Fixing these issues requires changes to the ProduceRequest and > > ProduceResponse protocols as well as changes to the values of the > > 'enable.idempotence' producer config. > > > > Hence, I split off those changes into another KIP so as to decouple the > two > > issues. Please have a look at my follow up KIP below: > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > > 192+%3A+Provide+cleaner+semantics+when+idempotence+is+enabled > > > > KIP-185 depends on KIP-192, and I hope to make progress on the latter > > independently. > > > > Thanks, > > Apurva > > >
