nicktelford opened a new pull request, #22625: URL: https://github.com/apache/kafka/pull/22625
Adds a transactional implementation of `RocksDBStore`, building on the transaction-buffer framework (KAFKA-20490) and the dual-column-family accessor work (KAFKA-20492). `RocksDBTransactionBuffer` implements `TransactionBuffer` for RocksDB-backed stores: writes are staged in the inherited `pendingWrites` map and accumulated in a RocksDB `WriteBatch` that is applied atomically on commit. Reads merge the staged writes over a base iterator opened against the underlying RocksDB instance, so transactions observe their own uncommitted writes. The buffer owns the full range-tombstone lifecycle — `stageDeleteRange`, the `rangeTombstones` map, and tombstone-aware overrides of `get`, `all`, `range`, `commit`, `rollback`, and `isEmpty` — since only RocksDB stores use `deleteRange`. `TransactionalDBAccessor`, a `DBAccessor` decorator, intercepts reads and writes to route them through the buffer. It is installed in `openDB()` only when transactional state stores are enabled, so the non-transactional path is unchanged. The store reports its pending write-buffer size via `approximateNumUncommittedBytes()`, feeding the uncommitted-bytes limit added in KAFKA-20491. For dual-column-family stores, all `put`/`delete`/`deleteRange` and `all`/`range`/`prefixScan` calls are routed unconditionally through CF-aware buffer overloads. This fixes two latent invariants under EOS: old-CF writes were previously flushed outside the shared `WriteBatch` (so they could commit independently of new-CF writes), and in-transaction iterators on the old CF saw the pre-delete committed value because the in-memory tombstone was never placed in `pendingWrites`. Because `DualColumnFamilyAccessor` always pairs writes symmetrically, a single shared `pendingWrites` map yields the correct final state for both CFs and a single `db.write` on commit applies all column families atomically. This is part of [KIP-892](https://cwiki.apache.org/confluence/display/KAFKA/KIP-892%3A+Transactional+Semantics+for+State+Stores). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
