Re: [DISCUSS] KIP-498: Add client-side configuration for maximum response size to protect against OOM

Hello, Thanks David to propose a new PR [1] for KIP-498 [2] to address KAFKA-4090 [3]. I find it interesting, I wonder how it stands w.r.t. avoiding inter-layer violation. [1] https://github.com/apache/kafka/pull/8066 [2] https://issues.apache.org/jira/browse/KAFKA-4090 [3]

Re: [DISCUSS] KIP-498: Add client-side configuration for maximum response size to protect against OOM

Hi Alexandre. Thanks for this analysis. IMHO, there are 4 ways to ago about this: 1. We don't fix the bug directly but instead update the Kafka documentation telling clients to configure themselves correctly - Silly but easy to achieve. 2. We adopt Stanislav's solution that fixes the problem -

Re: [DISCUSS] KIP-498: Add client-side configuration for maximum response size to protect against OOM

Hello, I wrote a small change [1] to make clients validate the size of messages received from a broker at the protocol-level. However I don't like the change. It does not really solve the problem which originally motivated the KIP - that is, protocol mismatch (plaintext to SSL endpoint). More

Re: [DISCUSS] KIP-498: Add client-side configuration for maximum response size to protect against OOM

Thanks Gokul and Stanislav for your answers. I went through the PR 5940 [1]. Indeed Gokul, your reasoning echoes the observations of Ismael about a potential inter-protocol layering violation. As you said Stanislav, the server-side SSL engine responds with an alert with code 80 (internal_error)

Re: [DISCUSS] KIP-498: Add client-side configuration for maximum response size to protect against OOM

Hey Alexandre, thanks for the KIP! I had personally hit the same problem and found it very annoying. Have you considered detecting such a scenario in the client and simply throwing an exception, rather than allocating any memory? I have an open PR that does just that -

Re: [DISCUSS] KIP-498: Add client-side configuration for maximum response size to protect against OOM

Hi Alex. A rejected alternative is to try to do SSL handshake from the plaintext transport layer. This couples various transport layers and is inflexible to adding new transport layers in the future. Further, if the plaintext transport layer does SSL handshake, it will always get an error,

[DISCUSS] KIP-498: Add client-side configuration for maximum response size to protect against OOM

Hello, I have created the KIP-498 to propose a minor addition to the producer configuration properties, in order to protect against OOM when the response