[ https://issues.apache.org/jira/browse/KAFKA-14976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
A. Sophie Blee-Goldman updated KAFKA-14976: ------------------------------------------- Labels: needs-kip (was: ) > Left/outer stream-stream joins create KV stores that aren't customizable > ------------------------------------------------------------------------ > > Key: KAFKA-14976 > URL: https://issues.apache.org/jira/browse/KAFKA-14976 > Project: Kafka > Issue Type: Bug > Components: streams > Reporter: A. Sophie Blee-Goldman > Priority: Major > Labels: needs-kip > > It appears that we only give the illusion of full customizability when it > comes to the state stores of a windowed join. This arose due to an > [optimization|https://github.com/apache/kafka/pull/11252] for the performance > of the spurious results fix, and means that these joins now come with one > additional, and possibly unexpected, state store: > > {code:java} > final StoreBuilder<KeyValueStore<TimestampedKeyAndJoinSide<K>, > LeftOrRightValue<V1, V2>>> builder = > new ListValueStoreBuilder<>( > |--[ persistent ? > this--> | Stores.persistentKeyValueStore(storeName) : > |--[ Stores.inMemoryKeyValueStore(storeName), > timestampedKeyAndJoinSideSerde, > leftOrRightValueSerde, > Time.SYSTEM > ); {code} > > where persistent is defined above that as > {code:java} > final boolean persistent = streamJoinedInternal.thisStoreSupplier() == null > || streamJoinedInternal.thisStoreSupplier().get().persistent(); {code} > > This means regardless of whether a custom state store implementation was > passed in to the join, we will still insert one of our RocksDB or InMemory > state stores. Which might be very surprising since the API makes it seem like > the underlying stores are fully configurable. > I'm adding a warning line for this in PR > [#13682|https://github.com/apache/kafka/pull/13682/files#diff-9ce43046fdef1233ab762e728abd1d3d44d7c270b28dcf6b63aa31a93a30af07R334-R336] > but we should really make this hidden state store fully configurable like > the window stores currently are (which will require a KIP) > -- This message was sent by Atlassian Jira (v8.20.10#820010)