Big is a relative term. And the question you ask is quite difficult to answer because not other information is available - including the configuration of the Kafka cluster, hardware specification etcetera.
I suggest the following: (1) read a couple of benchmarks such as [1], (2) investigate onto this question of matter yourself by designing and running a microbenchmark - since you’re the only one who knows what throughput or for example latency you’d like to achieve. But in general, I do believe that those messages are larger then the expected. Hence, you may also want to take a look at compression in Kafka - although you’ll lose performance. [1] https://engineering.linkedin.com/kafka/benchmarking-apache-kafka-2-million-writes-second-three-cheap-machines > On 22 Nov 2016, at 13:00, Felipe Santos <felip...@gmail.com> wrote: > > I read on documentation that kafka is not optimized for big messages, what > is considered a big message? > > For us the messages will be on average from 20k ~ 40k? Is this a real > problem? > > Thanks > -- > Felipe Santos
