Jason Gustafson created KAFKA-6352:
--------------------------------------
Summary: Delay message down-conversion until response is ready to
be sent
Key: KAFKA-6352
URL: https://issues.apache.org/jira/browse/KAFKA-6352
Project: Kafka
Issue Type: Improvement
Reporter: Jason Gustafson
We have observed some users beginning to use the new message format before
their clients have been upgraded. As we know, this can cause a lot of memory
pressure due to the fact that we lose zero-copy in order to down-convert the
messages. Currently we do down-conversion prior to enqueuing the response to be
sent by the Processor. This may lead to more heap utilization than needed with
a steady queue size.
A possible improvement is to delay the down-conversion until the response is
actually ready to be sent. Even better would be to do the down-conversion in a
streaming fashion, converting only as much as is needed at any time. One
potential drawback is that this moves the down-conversion into the network
threads.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
