Martin Kamp Jensen created KAFKA-7563:
-----------------------------------------
Summary: Single broker sends incorrect metadata for topic
partitions
Key: KAFKA-7563
URL: https://issues.apache.org/jira/browse/KAFKA-7563
Project: Kafka
Issue Type: Bug
Affects Versions: 2.0.0
Reporter: Martin Kamp Jensen
Attachments: kafka.log, zookeeper.log
When starting our Kafka Streams application in a test setup with just one Kafka
broker we are seeing the following error roughly 1 out of 15 runs:
{{StreamsException: Existing internal topic
alarm-message-streams-alarm-from-unknown-asset-changelog has invalid
partitions: expected: 32; actual: 25. Use 'kafka.tools.StreamsResetter' tool to
clean up invalid topics before processing.}}
(Note: It is not always the same topic that causes the error.)
When we see the error above the actual number of partitions varies (expected is
32, actual is above 0 and below 32).
Before each test run the Kafka broker is started without data (using
[https://hub.docker.com/r/wurstmeister/kafka/]).
We have never seen this happen in non-test where we are running with 6 Kafka
brokers. However, we are running a significantly higher number of test runs
than deploys to non-test.
After some investigation (including using AdminClient to describe the topics
when the Kafka Streams application got the StreamsException and confirming that
AdminClient also reports that a topic has the wrong number of partitions!) we
implemented the following workaround: When the Kafka Streams application fails
with the exception, we stop the application, stop the Kafka broker, start the
Kafka broker, and finally start the application. Then the exception is not
thrown. Of course this does not explain or fix the real issue at hand but it is
still important because we all hate flaky tests.
Kafka and ZooKeeper log files from a run where the exception above occurred and
where applying the workaround described above enabled us to continue without
the exception are attached.
This issue was created by request of Matthias J. Sax at
https://stackoverflow.com/questions/52943653/existing-internal-topic-has-invalid-partitions.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
