[
https://issues.apache.org/jira/browse/KAFKA-20543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bill Bejeck resolved KAFKA-20543.
---------------------------------
Resolution: Won't Do
> Drop RocksDB offsets column family on clean shutdown when downgrading from
> 4.3
> -------------------------------------------------------------------------------
>
> Key: KAFKA-20543
> URL: https://issues.apache.org/jira/browse/KAFKA-20543
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Affects Versions: 4.3.0
> Reporter: Bill Bejeck
> Assignee: Bill Bejeck
> Priority: Major
> Fix For: 4.3.1, 4.4.0
>
>
> KIP-1035 introduced a dedicated offsets column family in each RocksDB
> instance to persist changelog offsets per-store. KAFKA-19710 added a
> downgrade path that writes a legacy per-task .checkpoint file on clean
> shutdown when upgrade.from is set to a pre-4.3 version, but it does not
> remove the offsets column family from RocksDB. As a result, older Kafka
> Streams versions (which do not declare this column family at open time) fail
> with RocksDBException and the application crashes on startup, making
> in-flight downgrade impossible.
>
>
> This ticket covers extending the clean-shutdown downgrade path to also drop
> the offsets column family (via db.dropColumnFamily(handle)) in RocksDBStore,
> RocksDBTimestampedStore, RocksDBTimestampedStoreWithHeaders,
> RocksDBMigratingWindowStoreWithHeaders, and
> RocksDBMigratingSessionStoreWithHeaders before the underlying RocksDB
> instance is closed, so that older versions can open the store and resume from
> the legacy .checkpoint file written by KAFKA-19710. Once this is in place,
> the downgrade documentation in docs/streams/upgrade-guide.md should be
> updated to describe the supported in-flight upgrade.from procedure instead of
> the current wipe-and-restore workaround.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
