lsc713 opened a new pull request, #23474:
URL: https://github.com/apache/kafka/pull/23474
A send invoked from a callback on the same producer's I/O thread can wait
for metadata that only that thread can fetch. Buffer allocation has the same
dependency: a completed batch's buffer is released after its callback returns,
so a nested send can exhaust the buffer and stall the Sender until max.block.ms
expires.
Give doSend() a zero metadata/buffer wait budget when the caller is that
producer's I/O thread. Sends with cached metadata and available buffer space
continue to work; otherwise they fail through the existing callback and
failed-Future path. Application-thread sends retain their configured wait
budget. Document synchronous failure callbacks and the risk of recursive
retries.
This draft proposes the behavior for discussion. Feedback is welcome on
whether the timing change is appropriate for KAFKA-10335 and whether it needs a
KIP. It does not prevent blocking in user serializers, partitioners, callbacks,
or locks, and does not solve recursive failure-callback retries.
Tests use the real Sender thread with MockClient broker responses and cover
missing metadata, exhausted buffer space, and a successful cached-metadata
send. Both zero-wait assertions failed before the production change; the
success comparison passed. No end-to-end broker test was run.
Validation on trunk 6866205623: 155 tests passed (105 KafkaProducerTest, 37
RecordAccumulatorTest, 13 BufferPoolTest), with Checkstyle and Spotless checks
passing.
```
./gradlew clients:test \
--tests org.apache.kafka.clients.producer.KafkaProducerTest \
--tests org.apache.kafka.clients.producer.internals.BufferPoolTest \
--tests org.apache.kafka.clients.producer.internals.RecordAccumulatorTest \
clients:checkstyleMain clients:checkstyleTest clients:spotlessJavaCheck
```
AI-assisted contribution using OpenAI Codex (GPT-6), attributed in the
commit trailer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
