The performance impact happens if the consumer doesn't support the message format defined by log.message.format.version. The old consumer in Kafka 0.10 supports the message format introduced in 0.10 (including timestamps and the ability to avoid recompression in the produce path) so what you propose is fine.
However, the old consumer does _not_ support the message format introduced in 0.11.0 (no matter the version) so you will eventually have to deal with down conversion if that application cannot be upgraded to use the Java consumer. Kafka 2.0.0 will contain down conversion efficiency improvements and you may consider testing if the down conversion costs are bearable for that application. If they are, the 0.11.0+ message format version is a good path forward as it is necessary for using headers, transactions and the idempotent producer. Ismael On Thu, Jun 21, 2018 at 11:30 PM Jeff Pollard <jeff.poll...@gmail.com> wrote: > Hello all, > > We are in the process of upgrading all our consumers to version 0.10+. One > of our pre-0.10 consumers still uses the deprecated ConsumerConnector API > (I believe colloquially called the "old consumer"). > > We're unfortunately not able to upgrade this consumer to the new consumer > API, as the consumer is baked into a larger piece of software we did not > write, and they have not offered a version of it with the new consumer. > They have, however, released a version using Kafka 0.10, which we have > upgraded to. > > We also have upgraded our *brokers* to 0.10+, but per the potential > performance impact > < > http://kafka.apache.org/0102/documentation.html#upgrade_10_performance_impact > > > noted > in documentation have set the log.message.format.version to 0.8.2. This has > worked well for us, as we've slowly been upgrading our consumers to 0.10+. > > My question is about using the old ConsumerConnector API with Kafka 0.10+ > client and a 0.10+ broker. From my reading of the documentation, it seems > the performance impact is translating the message format by the broker, and > is independent of the kind (old or new) consumer. I believe both the old > and new consumer both use the same fetch API, but I hadn't verified that > yet. > > We just wanted to make sure this was correct, as to avoid any unexpected > performance impacts when we finish our upgrade and change the on-broker > message format to the 0.10+ one. > > Thanks again. Happy to answer or clarify any points of this email as > needed. >
