Christo Lolov created KAFKA-15195: ------------------------------------- Summary: Regenerate segment-aligned producer snapshots when upgrading to a Kafka version supporting Tiered Storage Key: KAFKA-15195 URL: https://issues.apache.org/jira/browse/KAFKA-15195 Project: Kafka Issue Type: Sub-task Affects Versions: 3.6.0 Reporter: Christo Lolov Assignee: Christo Lolov
As mentioned in KIP-405: Kafka Tiered Storage#Upgrade a customer wishing to upgrade from a Kafka version < 2.8.0 to 3.6 and turn Tiered Storage on will have to wait for retention to clean up segments without an associated producer snapshot. However, in our experience, customers of Kafka expect to be able to immediately enable tiering on a topic once their cluster upgrade is complete. Once they do this, however, they start seeing NPEs and no data is uploaded to Tiered Storage (https://github.com/apache/kafka/blob/9e50f7cdd37f923cfef4711cf11c1c5271a0a6c7/storage/api/src/main/java/org/apache/kafka/server/log/remote/storage/LogSegmentData.java#L61). To achieve this, we propose changing Kafka to retroactively create producer snapshot files on upload whenever a segment is due to be archived and lacks one. -- This message was sent by Atlassian Jira (v8.20.10#820010)