Re: Default values for max.request.size and message.max.bytes seem wrong

2020-03-03 Thread Ismael Juma
This was fixed recently:

https://github.com/apache/kafka/commit/bd5a1c4d368b9e91398e48400965d30d3045062e

Ismael

On Tue, Mar 3, 2020 at 11:34 AM Michael Rubin 
wrote:

> (I am re-asking my SO question:
> https://stackoverflow.com/questions/60458132/default-values-for-max-request-size-and-message-max-bytes-seem-wrong
> <
> https://stackoverflow.com/questions/60458132/default-values-for-max-request-size-and-message-max-bytes-seem-wrong
> >)
>
> The default value for the Producer setting max.request.size is 1048576.
> This controls the maximum size of a request (to a broker) in bytes.
>
> The default value for the Broker setting message.max.bytes is 112.
> This controls the largest (record batch) size allowed (by the Kafka
> server/broker).
>
> 1048576 > 112
>
> It seems that the defaults are such that a Producer (with this default
> value) could happen to produce a message which is larger than a Broker
> (with this default value) will accept, resulting in MESSAGE_TOO_LARGE
> errors.
>
> Am I misunderstanding these configuration settings? Or are the Kafka
> defaults really such that it is possible to get MESSAGE_TOO_LARGE errors
> with the default values?
>
>
>


Default values for max.request.size and message.max.bytes seem wrong

2020-03-03 Thread Michael Rubin
(I am re-asking my SO question: 
https://stackoverflow.com/questions/60458132/default-values-for-max-request-size-and-message-max-bytes-seem-wrong
 
)

The default value for the Producer setting max.request.size is 1048576. This 
controls the maximum size of a request (to a broker) in bytes.

The default value for the Broker setting message.max.bytes is 112. This 
controls the largest (record batch) size allowed (by the Kafka server/broker).

1048576 > 112

It seems that the defaults are such that a Producer (with this default value) 
could happen to produce a message which is larger than a Broker (with this 
default value) will accept, resulting in MESSAGE_TOO_LARGE errors.

Am I misunderstanding these configuration settings? Or are the Kafka defaults 
really such that it is possible to get MESSAGE_TOO_LARGE errors with the 
default values?