Hi Vaquar, VK4/VK8: We don't do PID mapping anymore. The KIP was updated some time ago with the new approach based on the new PID reset control record.
VK11: The transaction index is always built locally during log append, never copied. VK1: The 50,000 * 1MB = 50GB calculation misunderstands the fetch model. Fetcher threads don't allocate one buffer per partition. Actual peak memory is roughly num_fetcher_threads * response_max_bytes, not num_partitions * partition_max_bytes. With 1 fetcher thread and the default response max, the memory footprint is modest regardless of partition count. We are leveraging the same proven pattern used by the internal replication. VK3: The __mirror_state topic uses hash-based partitioning based on mirrorName, topicId and partition number. With the production default of 50 partitions, 50,000 partition transitions distribute across ~50 partition leaders on different brokers, not a single broker. This is the same proven pattern as __consumer_offsets, which handles millions of commits. VK12: The scenario described can only occur if offsets are force-written to an active group, which the design prevents. Cheers Fede
