The message size check is currently only done on the broker. If you enable trace level logging in RequestChannel, you will see the produce request, which includes the size of each partition.
Thanks, Jun On Wed, Aug 27, 2014 at 4:40 PM, Alexis Midon < alexis.mi...@airbedandbreakfast.com> wrote: > Hello, > > my brokers are reporting that some received messages exceed the > `message.max.bytes` value. > I'd like to know what producers are at fault but It is pretty much > impossible: > - the brokers don't log the content of the rejected messages > - the log messages do not contain the IP of the producers > - on the consumer side, no exception is thrown (afaik it is because Ack-0 > is used). The only kind of notification is to closed the connection. > > [1] Do you have any suggestions to track down the guilty producers or find > out the message content? > > Even though it makes total sense to have the limit defined and applied on > the brokers, I was thinking that this check could also be applied by the > producers. Some google results suggest that `message.max.bytes` might be > used by the producers but I can't find any trace of that behavior in the > code. > > The closest thing I have is > > https://github.com/apache/kafka/blob/0.8.1/core/src/main/scala/kafka/producer/SyncProducer.scala#L67 > but it simply logs the message size and content and the log level is trace. > > [2] could you please confirm if such a producer-side check exists? > > > thanks! > > Alexis >
