David Arthur created KAFKA-15825:
------------------------------------
Summary: KRaft controller writes empty state to ZK after migration
Key: KAFKA-15825
URL: https://issues.apache.org/jira/browse/KAFKA-15825
Project: Kafka
Issue Type: Bug
Components: controller
Affects Versions: 3.6.0
Reporter: David Arthur
Assignee: David Arthur
Fix For: 3.7.0, 3.6.1
Immediately following the ZK migration, there is a race condition where the
KRaftMigrationDriver can use an empty MetadataImage when performing the full
"SYNC_KRAFT_TO_ZK" reconciliation.
After the next controller failover, or when the controller loads a metadata
snapshot, the correct state will be written to ZK.
The symptom of this bug is that we see the migration complete, and then all the
metadata removed from ZK. For example,
{code}
[KRaftMigrationDriver id=9990] Completed migration of metadata from ZooKeeper
to KRaft. 573 records were generated in 2204 ms across 51 batches. The record
types were {TOPIC_RECORD=41, PARTITION_RECORD=410, CONFIG_RECORD=121,
PRODUCER_IDS_RECORD=1}. The current metadata offset is now 503794 with an epoch
of 21. Saw 6 brokers in the migrated metadata [0, 1, 2, 3, 4, 5].
{code}
immediately followed by:
{code}
[KRaftMigrationDriver id=9990] Made the following ZK writes when reconciling
with KRaft state: {DeleteBrokerConfig=7, DeleteTopic=41, UpdateTopicConfig=41}
{code}
If affected by this, a quick workaround is to cause the controller to failover.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
