Hey Alexandre, thanks for the KIP! I had personally hit the same problem and found it very annoying. Have you considered detecting such a scenario in the client and simply throwing an exception, rather than allocating any memory? I have an open PR that does just that - https://github.com/apache/kafka/pull/5940 <https://github.com/apache/kafka/pull/5940/files>
Best, Stanislav On Wed, Jul 31, 2019 at 10:35 AM Gokul Ramanan Subramanian < gokul24...@gmail.com> wrote: > Hi Alex. > > A rejected alternative is to try to do SSL handshake from the plaintext > transport layer. This couples various transport layers and is inflexible to > adding new transport layers in the future. Further, if the plaintext > transport layer does SSL handshake, it will always get an error, > irrespective of whether or not the peer supports SSL. Therefore, the > plaintext transport layer would have to determine if the peer uses SSL or > not based on the type of error it gets back from the SSL handshake. This > fits right into the general anti-pattern of using exceptions as control > flow mechanisms. > > Another rejected alternative would be to have a single byte at the > transport layer represent the security protocol in use. This byte would > have to be present in every single message between clients and brokers and > between brokers and brokers. This change is backwards-incompatible and adds > overhead to Kafka. It is likely never going to get accepted by the > community. > > In the absence of a fool-proof way to do a handhskake across all security > protocols (plaintext, SSL, other future ones), I think that the proposed > KIP provides a good solution. Therefore, you have my +1. (Not sure if my +1 > counts, since I am a Kafka noob). > > Thanks. > Gokul Ramanan Subramanian > Senior SDE, Amazon AWS > > On 28/Jul/19 05:43:19PM +0100, Alexandre Dupriez wrote: > Hello, > > I have created the KIP-498 > < > https://cwiki.apache.org/confluence/display/KAFKA/KIP-498%3A+Add+client-side+configuration+for+maximum+response+size+to+protect+against+OOM > > > to > propose a minor addition to the producer configuration properties, in > order > to protect against OOM when the response message decoded by a client > indicates an "abnormally high" size to be allocated. > > This follows this discussion thread > <https://www.mail-archive.com/dev@kafka.apache.org/msg55658.html> and is > tracked by KAFKA-4090 <https://issues.apache.org/jira/browse/KAFKA-4090>. > > Please let me know what you think. I created this KIP even though the > change seems minimal because it affects client configuration, which is > mentioned as a type of change eligible for a KIP on this main wiki page > < > https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals > > > . > > Thanks, > Alexandre >
