We upgraded our Kafka clusters from 0.8.2.1 to 0.10.0.1, but most of our consumers use older libraries that do not support the new message format. So we set the brokers' log.message.format.version to 0.8.2 while we work on upgrading our consumers.
In the meantime, I'm worried about a performance problem with consumers that have upgraded and are requesting messages using the new Kafka 10 versions of those API calls. I may be misunderstanding, but it seems logical that the performance problem isn't just about old consumers with a new broker. I would think the performance problem would also exist if we take new brokers, set the log format to an old version, then have our consumers make API calls using the Kafka 10 API calls. The broker would need to do on-the-fly conversion from the 0.8.2 log format up to the 0.10.0 format to send to the new consumers. This is the inverse problem of what's mentioned here: https://kafka.apache.org/documentation/#upgrade_10_performance_impact Is this a valid problem?
