Hi all,
Adding on a couple more points for motivation:
1. Users typically set MDC context from each record but this can break in
subtle ways for both sync/async threading models
for (var r: consumer.poll(…)) {
MDC.put(“kafka.offset”, r.offset());
// add rest of the fields
process(r); // throws an ex
MDC.clear();
}
In both sync and async models, record N+1 carries prev record’s context during
exceptions. The solution is a try finally for cleanup, yet more boilerplate for
teams to maintain manually. Across thread boundaries, we have to manually copy
threadlocals around from the poller loop (such as OTel,DD spans).
In KafkaShareConsumer, this behaviour is more common when worker threads can
perform individual redelivery, ack calls. Sometimes, a main poll loop for
try/catch does not neatly cleanup state between records.
2. I also see Confluent’s parallel-consumer implement a PollContext to achieve
a similar function to wrap each poll() action per-record.
Best,
Aditya
> On Apr 16, 2026, at 11:45, Aditya Kousik <[email protected]> wrote:
>
> Hi kafka dev,
>
> Gentle ping on this KIP. I’ve added a short [unit test
> reproducer](https://github.com/apache/kafka/pull/22077) that highlights the
> gap mentioned in the KIP.
>
> Would appreciate any eyes on both.
>
> Thanks,
> Aditya
>
>> On Apr 1, 2026, at 08:14, Aditya Kousik <[email protected]> wrote:
>>
>> Hi all,
>>
>> I'd like to start a discussion on KIP-1305: Per-Record Context Processing
>> for Kafka Consumers.
>>
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1305%3A+Per-Record+Context+Processing+for+Kafka+Consumers
>>
>> Spring Kafka, SmallRye, Micronaut, OTel, and Datadog have each independently
>> built per-record lifecycle hooks because the client doesn't provide one.
>> This KIP proposes ConsumerRecordContextProcessor, a configurable hook that
>> creates a per-record AutoCloseable context via record.newContext(), giving
>> frameworks and instrumentation libraries a single stable integration point.
>>
>> Looking forward to your feedback.
>>
>> Thanks,
>> Aditya Kousik
