Re: [PR] KAFKA-14412: Transactional StateStores [kafka]

2024-02-23 Thread via GitHub
nicktelford commented on PR #15137: URL: https://github.com/apache/kafka/pull/15137#issuecomment-1961419902 Closing this PR for now while I break changes out into separate PRs -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

Re: [PR] KAFKA-14412: Transactional StateStores [kafka]

2024-02-23 Thread via GitHub
nicktelford closed pull request #15137: KAFKA-14412: Transactional StateStores URL: https://github.com/apache/kafka/pull/15137 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

Re: [PR] KAFKA-14412: Transactional StateStores [kafka]

2024-01-25 Thread via GitHub
nicktelford commented on PR #15137: URL: https://github.com/apache/kafka/pull/15137#issuecomment-1910399080 > * Is it possible to have a separate PR for the config changes? @cadonna Yes, in fact I'm working on that as I speak. :grin: It'll be a little strange to essentially add config

Re: [PR] KAFKA-14412: Transactional StateStores [kafka]

2024-01-25 Thread via GitHub
cadonna commented on PR #15137: URL: https://github.com/apache/kafka/pull/15137#issuecomment-1909913075 Some suggestions/questions to split the PR in smaller parts (without having looked at the PR in much depth): - Is it possible to have a separate PR for the config changes? - Is it po

Re: [PR] KAFKA-14412: Transactional StateStores [kafka]

2024-01-25 Thread via GitHub
nicktelford commented on PR #15137: URL: https://github.com/apache/kafka/pull/15137#issuecomment-1909852487 > Hey @nicktelford , I'll try to take a look at this here and there and I can't speak for the Confluent folks, but I have to say it's pretty intimidating to see a PR with 3000 LOC and

Re: [PR] KAFKA-14412: Transactional StateStores [kafka]

2024-01-24 Thread via GitHub
ableegoldman commented on PR #15137: URL: https://github.com/apache/kafka/pull/15137#issuecomment-1909050890 Hey @nicktelford , I'll try to take a look at this here and there and I can't speak for the Confluent folks, but I have to say it's pretty intimidating to see a PR with 3000 LOC and

Re: [PR] KAFKA-14412: Transactional StateStores [kafka]

2024-01-12 Thread via GitHub
nicktelford commented on PR #15137: URL: https://github.com/apache/kafka/pull/15137#issuecomment-1888900920 The build says there are "19 new failed tests", but they all pass locally. Since all but 3 are in non-streams modules, this looks like poor CI performance induced timeouts. -- This

Re: [PR] KAFKA-14412: Transactional StateStores [kafka]

2024-01-05 Thread via GitHub
nicktelford commented on code in PR #15137: URL: https://github.com/apache/kafka/pull/15137#discussion_r1443144457 ## streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBStore.java: ## @@ -805,6 +959,194 @@ public void reset() { public void close() {

Re: [PR] KAFKA-14412: Transactional StateStores [kafka]

2024-01-05 Thread via GitHub
nicktelford commented on code in PR #15137: URL: https://github.com/apache/kafka/pull/15137#discussion_r1443141089 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/ProcessorStateManager.java: ## @@ -266,29 +269,20 @@ void initializeStoreOffsetsFromCheckpoint

Re: [PR] KAFKA-14412: Transactional StateStores [kafka]

2024-01-05 Thread via GitHub
nicktelford commented on code in PR #15137: URL: https://github.com/apache/kafka/pull/15137#discussion_r1443138486 ## checkstyle/checkstyle.xml: ## @@ -135,7 +135,7 @@ - + Review Comment: `TaskExecutor#commitOffsetsOrTransaction` -- This i

Re: [PR] KAFKA-14412: Transactional StateStores [kafka]

2024-01-05 Thread via GitHub
nicktelford commented on code in PR #15137: URL: https://github.com/apache/kafka/pull/15137#discussion_r1443136941 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java: ## @@ -262,8 +268,7 @@ private void closeDirtyAndRevive(final Collection ta

Re: [PR] KAFKA-14412: Transactional StateStores [kafka]

2024-01-05 Thread via GitHub
nicktelford commented on PR #15137: URL: https://github.com/apache/kafka/pull/15137#issuecomment-1878999651 @cadonna @mjsax @ableegoldman @lucasbru @wcarlson5 @bbejeck @vvcephei @guozhangwang My apologies to whoever draws the short straw of having to review this! This is the m

[PR] KAFKA-14412: Transactional StateStores [kafka]

2024-01-05 Thread via GitHub
nicktelford opened a new pull request, #15137: URL: https://github.com/apache/kafka/pull/15137 See [KIP-892: Transactional Semantics for StateStores][1] for a detailed explanation of the design. --- Writes to StateStores are now isolated from the underlying RocksDB database un