lucasbru commented on PR #21738: URL: https://github.com/apache/kafka/pull/21738#issuecomment-4073832754
@nicktelford @bbejeck Can you please validate that kafkatest.tests.streams.streams_application_upgrade_test.StreamsUpgradeTest.test_app_upgrade works with these changes? It seems it is failing starting with this commit. Claude analysis: The previous analysis identified KAFKA-19434 (initializeStartupStores) as the cause. While KAFKA-19434 (Feb 20) introduced the startup initialization code path, the failure only started recently because KAFKA-19712 (commit 5740a26525a, March 16) fundamentally changed how offsets are read within that path: Before KAFKA-19712: initializeStoreOffsetsFromCheckpoint() read offsets from the .checkpoint file and used changelogOffsetFromCheckpointedOffset() to convert the OFFSET_UNKNOWN sentinel (-4) back to null. This properly handled state written by older Kafka versions. After KAFKA-19712: The method was rewritten to initializeStoreOffsets(), which calls store.stateStore.committedOffset(store.changelogPartition) instead of reading the checkpoint file. RocksDB stores now manage their own offsets (managesOffsets() == true) and are no longer wrapped in LegacyCheckpointingStateStore. The changelogOffsetFromCheckpointedOffset() method — which handled the -4 → null conversion — was removed entirely. When opening state directories written by an older Kafka Streams version during upgrade, the new RocksDB store's committedOffset() returns raw values that were never intended to be exposed directly. This produces the -3 value (from -4 + 1) that fails the sentinel validation in sumOfChangelogOffsets(). KAFKA-20257 (commit 823f0cfc8cb, same day) makes the same delegation change for GlobalStateManagerImpl and may have similar upgrade implications. Both commits landed on March 16 ~17:15 UTC, approximately 4 hours before the test workflow started (21:00 UTC), confirming they were in the tested build. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
