Bill Bejeck created KAFKA-20471:
-----------------------------------
Summary: KIP-892: Implement Generic Transaction Buffer (Phase 0)
Key: KAFKA-20471
URL: https://issues.apache.org/jira/browse/KAFKA-20471
Project: Kafka
Issue Type: Sub-task
Reporter: Bill Bejeck
Deliver the TransactionBuffer abstraction as a standalone, independently
mergeable piece before KIP-892 core work begins. This must land and stabilize
before Phase 1 begins. Can proceed in parallel with the StateStore.java
interface design.
Files to create:
- state/internals/TransactionBuffer.java — interface
- state/internals/AbstractTransactionBuffer.java — shared ConcurrentSkipListMap
+ ReentrantReadWriteLock logic
- state/internals/RocksDBTransactionBuffer.java — RocksDB implementation (plain
WriteBatch)
- state/internals/InMemoryTransactionBuffer.java — in-memory implementation
- state/internals/StagedMergeIterator.java — merge iterator over staging
snapshot + base iterator
Files to modify:
- StreamsConfig.java — add transactional.state.stores boolean config
- state/internals/RocksDBStore.java — add TransactionalDBAccessor inner class;
wire in init()
- state/internals/InMemoryKeyValueStore.java — wire InMemoryTransactionBuffer
in init()
Unit tests:
- RocksDBTransactionBufferTest — stage/get/commit/rollback, cross-thread read
correctness
- InMemoryTransactionBufferTest — same for in-memory
- StagedMergeIteratorTest — overlapping keys, tombstones, ordering
- TransactionBufferConcurrencyTest — concurrent scan creation vs. commit, no
duplicates
Integration tests:
- RocksDBStoreWithGTBIntegrationTest — IQ sees staged writes before commit;
committed state correct after
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
