As Gwen commented earlier, the client already has the record that it sent,
including all the headers.
>
> Future<RecordMetadata> future = producer.send(myRecord, null);
> future.get();
> System.out.println("I sent myRecord with headers " + myRecord.headers());
>
best,
Colin
On Tue, Aug 27, 2019, at 17:06, Renuka M wrote:
> Hi Gwen/Team
>
> Can you please review the KIP. Hope we have clarified the question you have
> regarding proposal.
>
> Thanks
> Renuka M
>
> On Mon, Aug 26, 2019 at 3:35 PM Renuka M <renumetuk...@gmail.com> wrote:
>
> > Hi Eric,
> >
> > We thought about that but we didn't find the strong enough reason for
> > having record itself in Acknowledgement.
> > Headers are supposed to carry metadata and that is the reason headers are
> > added to producer/consumer records.
> > Also we feel having headers information in record metadata is good enough
> > to bridge the gap and link the record to its metadata.
> > Its simple change since we are not adding any new method signatures.
> > Adding new method signatures requires adoption and deprecation of old ones
> > to reduce duplication.
> > If we get enough votes on adding new method signature, we are open to add
> > it.
> >
> > Thanks
> > Renuka M
> >
> > On Mon, Aug 26, 2019 at 10:54 AM Eric Azama <eazama...@gmail.com> wrote:
> >
> >> Have you considered adding a new onAcknowledgement method to the
> >> ProducerInterceptor with the signature onAcknowledgement(RecordMetadata
> >> metadata, Exception exception, ProducerRecord record)? I would also
> >> consider adding this to Producer Callbacks as well, since linking a
> >> Callback to a specific record currently requires creating a new Callback
> >> for every ProducerRecord sent.
> >>
> >> This seems like a more robust strategy compared to using Headers. Headers
> >> don't necessarily contain anything that connects them to the original
> >> ProducerRecord, and forcibly including information in the Headers seems
> >> like unnecessary bloat. If your goal is to link a RecordMetadata to a
> >> specific ProducerRecord, it seems simpler to make sure the original
> >> ProducerRecord is accessible at the same time as the RecordMetadata
> >>
> >> On Mon, Aug 26, 2019 at 10:26 AM Renuka M <renumetuk...@gmail.com> wrote:
> >>
> >> > Hi Gwen,
> >> >
> >> > 1.We are not doing any changes on the broker side. This change is only
> >> on
> >> > Kafka clients library.
> >> > 2. RecordMetaData is created by client library while appending record to
> >> > ProducerBatch where offset alone returned by broker. Here we are adding
> >> > headers to RecordMetaData while creating FutureRecordMetaData to create
> >> > context between record and its metadata. I have updated the snippet in
> >> KIP
> >> > proposed changes in step 3.
> >> > 3. As we mentioned in alternatives, client side we can link record and
> >> its
> >> > metadata using callback, but Interceptors having same RecordMetadata
> >> will
> >> > not have context on for which record this MetaData belongs to. To fill
> >> that
> >> > Gap, we are proposing these changes.
> >> > Please let us know if we are not clear.
> >> >
> >> > Thanks
> >> > Renuka M
> >> >
> >> >
> >> >
> >> >
> >> > On Fri, Aug 23, 2019 at 7:08 PM Gwen Shapira <g...@confluent.io> wrote:
> >> >
> >> > > I am afraid I don't understand the proposal. The RecordMetadata is
> >> > > information returned from the broker regarding the record. The
> >> > > producer already has the record (including the headers), so why would
> >> > > the broker need to send the headers back as part of the metadata?
> >> > >
> >> > > On Fri, Aug 23, 2019 at 4:22 PM Renuka M <renumetuk...@gmail.com>
> >> wrote:
> >> > > >
> >> > > > Hi All,
> >> > > >
> >> > > > I am starting this thread to discuss
> >> > > >
> >> > >
> >> >
> >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-512%3AAdding+headers+to+RecordMetaData
> >> > > > .
> >> > > >
> >> > > > Please provide the feedback.
> >> > > >
> >> > > > Thanks
> >> > > > Renuka M
> >> > >
> >> > >
> >> > >
> >> > > --
> >> > > Gwen Shapira
> >> > > Product Manager | Confluent
> >> > > 650.450.2760 | @gwenshap
> >> > > Follow us: Twitter | blog
> >> > >
> >> >
> >>
> >
>
