Keith So created KAFKA-8724:
-------------------------------
Summary: log cleaner thread dies when attempting to clean a
__consumer_offsets partition after upgrade from 2.0->2.3
Key: KAFKA-8724
URL: https://issues.apache.org/jira/browse/KAFKA-8724
Project: Kafka
Issue Type: Bug
Components: log cleaner
Affects Versions: 2.3.0
Environment: Linux 3.10.0-862.2.3.el7.x86_64 #1 SMP Wed May 9 18:05:47
UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Reporter: Keith So
Attachments: KAFKA-308-stack-trace.txt
We are attempting an upgrade from Kafka 2.0 to 2.3 on a single cluster setup.
We have a mixture of Java/C++ and Python clients (Python clients are using
kafka-python libraries).
After the upgrade, the log cleaner occasionally dies with the attached stack
trace. Using timestamp correlation, we pinned it down to the cleaning of a
consumer_offsets topic. The config logged at initialization shows that
inter.broker.protocol.version = 2.3-IV1
log.message.format.version = 2.3-IV1
We initially thought this was to do with unclean upgrade from 2.0 to 2.3, but
after resetting the consumer offsets topic (via
https://medium.com/@nblaye/reset-consumer-offsets-topic-in-kafka-with-zookeeper-5910213284a2)
this still recurs on initially empty consumer offset partitions.
At the moment we are working around by toggling log.cleaner.threads option
using dynamic broker configuration to restore the log cleaner threads
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
