Hi Gwen/Team Can you please review the KIP. Hope we have clarified the question you have regarding proposal.
Thanks Renuka M On Mon, Aug 26, 2019 at 3:35 PM Renuka M <renumetuk...@gmail.com> wrote: > Hi Eric, > > We thought about that but we didn't find the strong enough reason for > having record itself in Acknowledgement. > Headers are supposed to carry metadata and that is the reason headers are > added to producer/consumer records. > Also we feel having headers information in record metadata is good enough > to bridge the gap and link the record to its metadata. > Its simple change since we are not adding any new method signatures. > Adding new method signatures requires adoption and deprecation of old ones > to reduce duplication. > If we get enough votes on adding new method signature, we are open to add > it. > > Thanks > Renuka M > > On Mon, Aug 26, 2019 at 10:54 AM Eric Azama <eazama...@gmail.com> wrote: > >> Have you considered adding a new onAcknowledgement method to the >> ProducerInterceptor with the signature onAcknowledgement(RecordMetadata >> metadata, Exception exception, ProducerRecord record)? I would also >> consider adding this to Producer Callbacks as well, since linking a >> Callback to a specific record currently requires creating a new Callback >> for every ProducerRecord sent. >> >> This seems like a more robust strategy compared to using Headers. Headers >> don't necessarily contain anything that connects them to the original >> ProducerRecord, and forcibly including information in the Headers seems >> like unnecessary bloat. If your goal is to link a RecordMetadata to a >> specific ProducerRecord, it seems simpler to make sure the original >> ProducerRecord is accessible at the same time as the RecordMetadata >> >> On Mon, Aug 26, 2019 at 10:26 AM Renuka M <renumetuk...@gmail.com> wrote: >> >> > Hi Gwen, >> > >> > 1.We are not doing any changes on the broker side. This change is only >> on >> > Kafka clients library. >> > 2. RecordMetaData is created by client library while appending record to >> > ProducerBatch where offset alone returned by broker. Here we are adding >> > headers to RecordMetaData while creating FutureRecordMetaData to create >> > context between record and its metadata. I have updated the snippet in >> KIP >> > proposed changes in step 3. >> > 3. As we mentioned in alternatives, client side we can link record and >> its >> > metadata using callback, but Interceptors having same RecordMetadata >> will >> > not have context on for which record this MetaData belongs to. To fill >> that >> > Gap, we are proposing these changes. >> > Please let us know if we are not clear. >> > >> > Thanks >> > Renuka M >> > >> > >> > >> > >> > On Fri, Aug 23, 2019 at 7:08 PM Gwen Shapira <g...@confluent.io> wrote: >> > >> > > I am afraid I don't understand the proposal. The RecordMetadata is >> > > information returned from the broker regarding the record. The >> > > producer already has the record (including the headers), so why would >> > > the broker need to send the headers back as part of the metadata? >> > > >> > > On Fri, Aug 23, 2019 at 4:22 PM Renuka M <renumetuk...@gmail.com> >> wrote: >> > > > >> > > > Hi All, >> > > > >> > > > I am starting this thread to discuss >> > > > >> > > >> > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-512%3AAdding+headers+to+RecordMetaData >> > > > . >> > > > >> > > > Please provide the feedback. >> > > > >> > > > Thanks >> > > > Renuka M >> > > >> > > >> > > >> > > -- >> > > Gwen Shapira >> > > Product Manager | Confluent >> > > 650.450.2760 | @gwenshap >> > > Follow us: Twitter | blog >> > > >> > >> >
