If we add the message format version (a topic config) in the response of TopicMetadata, we should consider adding the max message bytes as well. That would allow us to later improve the implementation of KIP-126 to split the batch _before_ sending.
Ismael On Tue, Sep 5, 2017 at 7:17 PM, Jason Gustafson <ja...@confluent.io> wrote: > The proposal looks good. Two minor comments: > > 1. Can we call out how we handle the duplicate case? This is a change in > behavior since we currently raise OutOfOrderSequence in this case. > > 2. Instead of passing through `idempotenceLevel` in the ProduceRequest, I > wonder if we should have a field for the minimum required message format. > When using enable.idempotence=required, we could set the minimum required > version to v2. For enable.idempotence=requested, we could use v0. The > advantage is that we may find other uses for a more general field in the > future. Alternatively, maybe we really should be returning the message > format version of each topic in the TopicMetadata response. A nice bonus of > doing so is that it gives the producer the ability to craft the right > format version ahead of time and avoid the need for conversion on the > broker. > > Thanks, > Jason > > On Tue, Aug 29, 2017 at 4:32 PM, Apurva Mehta <apu...@confluent.io> wrote: > > > Hi, > > > > In the discussion of KIP-185 (enable idempotence by default), we > discovered > > some shortcomings of the existing idempotent producer implementation. > > Fixing these issues requires changes to the ProduceRequest and > > ProduceResponse protocols as well as changes to the values of the > > 'enable.idempotence' producer config. > > > > Hence, I split off those changes into another KIP so as to decouple the > two > > issues. Please have a look at my follow up KIP below: > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > > 192+%3A+Provide+cleaner+semantics+when+idempotence+is+enabled > > > > KIP-185 depends on KIP-192, and I hope to make progress on the latter > > independently. > > > > Thanks, > > Apurva > > >
