[jira] [Commented] (KAFKA-16073) Kafka Tiered Storage Bug: Consumer Fetch Error Due to Delayed localLogStartOffset Update During Segment Deletion

2024-01-05 Thread hzh0425 (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17803768#comment-17803768 ] hzh0425 commented on KAFKA-16073: - I think we can solve this issue by change the order the following

[jira] [Assigned] (KAFKA-13949) Connect /connectors endpoint should support querying the active topics and the task configs

2024-01-05 Thread Sudesh Wasnik (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sudesh Wasnik reassigned KAFKA-13949: - Assignee: Sudesh Wasnik (was: Viktor Somogyi-Vass) > Connect /connectors endpoint

[jira] [Commented] (KAFKA-13949) Connect /connectors endpoint should support querying the active topics and the task configs

2024-01-05 Thread Sudesh Wasnik (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17803765#comment-17803765 ] Sudesh Wasnik commented on KAFKA-13949: --- HI [~viktorsomogyi]! I would like to pick up this old

[jira] [Commented] (KAFKA-15682) Ensure internal remote log metadata topic does not expire its segments before deleting user-topic segments

2024-01-05 Thread Phuc Hong Tran (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17803749#comment-17803749 ] Phuc Hong Tran commented on KAFKA-15682: [~ckamal] is the retention rate of

Re: [PR] KAFKA-15134:Enrich the prompt reason in CommitFailedException [kafka]

2024-01-05 Thread via GitHub
github-actions[bot] commented on PR #13930: URL: https://github.com/apache/kafka/pull/13930#issuecomment-1879522834 This PR is being marked as stale since it has not had any activity in 90 days. If you would like to keep this PR alive, please ask a committer for review. If the PR has

Re: [PR] Remove consumer rejoin to the group's synchronized lock [kafka]

2024-01-05 Thread via GitHub
github-actions[bot] commented on PR #14241: URL: https://github.com/apache/kafka/pull/14241#issuecomment-1879522799 This PR is being marked as stale since it has not had any activity in 90 days. If you would like to keep this PR alive, please ask a committer for review. If the PR has

Re: [PR] KAFKA-15468 [1/2]: Prevent transaction coordinator reloads on already loaded leaders [kafka]

2024-01-05 Thread via GitHub
jolshan commented on code in PR #15139: URL: https://github.com/apache/kafka/pull/15139#discussion_r1443569935 ## metadata/src/main/java/org/apache/kafka/image/LocalReplicaChanges.java: ## @@ -27,21 +27,24 @@ public final class LocalReplicaChanges { private final Set

Re: [PR] KAFKA-15468 [1/2]: Prevent transaction coordinator reloads on already loaded leaders [kafka]

2024-01-05 Thread via GitHub
jolshan commented on code in PR #15139: URL: https://github.com/apache/kafka/pull/15139#discussion_r1443569042 ## metadata/src/main/java/org/apache/kafka/image/LocalReplicaChanges.java: ## @@ -27,21 +27,24 @@ public final class LocalReplicaChanges { private final Set

Re: [PR] KAFKA-15468 [1/2]: Prevent transaction coordinator reloads on already loaded leaders [kafka]

2024-01-05 Thread via GitHub
artemlivshits commented on code in PR #15139: URL: https://github.com/apache/kafka/pull/15139#discussion_r1443564382 ## metadata/src/main/java/org/apache/kafka/image/LocalReplicaChanges.java: ## @@ -27,21 +27,24 @@ public final class LocalReplicaChanges { private final

Re: [PR] KAFKA-16083: Exclude throttle time when expiring inflight requests on a connection [kafka]

2024-01-05 Thread via GitHub
jolshan commented on PR #15130: URL: https://github.com/apache/kafka/pull/15130#issuecomment-1879485562 Dont worry about `**/build/test-results/**/TEST-*.xml` (It has not worked as long as I can remember) I always just go to the tests tab:

Re: [PR] KAFKA-16059: close more kafkaApis instances [kafka]

2024-01-05 Thread via GitHub
jolshan commented on PR #15132: URL: https://github.com/apache/kafka/pull/15132#issuecomment-1879484649 > but there are more...  In all seriousness, thanks for taking care of this. -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] KAFKA-15468: Prevent transaction coordinator reloads on already loaded leaders [kafka]

2024-01-05 Thread via GitHub
jolshan commented on PR #14489: URL: https://github.com/apache/kafka/pull/14489#issuecomment-1879475298 Part 1 PR here: https://github.com/apache/kafka/pull/15139 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[PR] KAFKA-15468 [1/2]: Prevent transaction coordinator reloads on already loaded leaders [kafka]

2024-01-05 Thread via GitHub
jolshan opened a new pull request, #15139: URL: https://github.com/apache/kafka/pull/15139 This originally was https://github.com/apache/kafka/pull/14489 which covered 2 aspects -- reloading on partition epoch changes where leader epoch did not change and reloading when leader epoch

Re: [PR] KAFKA-15468: Prevent transaction coordinator reloads on already loaded leaders [kafka]

2024-01-05 Thread via GitHub
jolshan commented on PR #14489: URL: https://github.com/apache/kafka/pull/14489#issuecomment-1879456828 I've been asked to split these changes into the two parts I mentioned. Will follow up with that. -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] KAFKA-15987: Refactor ReplicaManager code for transaction verification [kafka]

2024-01-05 Thread via GitHub
jolshan commented on code in PR #15087: URL: https://github.com/apache/kafka/pull/15087#discussion_r1443537258 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -749,9 +749,11 @@ class ReplicaManager(val config: KafkaConfig, * @param responseCallback

[jira] [Updated] (KAFKA-16087) Tasks dropping incorrect records when errors.tolerance=all and errors reported asynchronously due to data race

2024-01-05 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Harris updated KAFKA-16087: Affects Version/s: 3.2.0 2.6.0 > Tasks dropping incorrect records when

[jira] [Updated] (KAFKA-16087) Tasks dropping incorrect records when errors.tolerance=all and errors reported asynchronously due to data race

2024-01-05 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Harris updated KAFKA-16087: Description: The ErrantRecordReporter introduced in KIP-610 (2.6.0) allows sink connectors to

[jira] [Updated] (KAFKA-16087) Tasks dropping incorrect records when errors.tolerance=all and errors reported asynchronously due to data race

2024-01-05 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Harris updated KAFKA-16087: Summary: Tasks dropping incorrect records when errors.tolerance=all and errors reported

[jira] [Updated] (KAFKA-16087) Sink connector dropping incorrect record when ErrantRecordReporter used asynchronously

2024-01-05 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Harris updated KAFKA-16087: Description: The ErrantRecordReporter introduced in KIP-610 allows sink connectors to push

Re: [PR] KAFKA-16083: Exclude throttle time when expiring inflight requests on a connection [kafka]

2024-01-05 Thread via GitHub
ditac commented on PR #15130: URL: https://github.com/apache/kafka/pull/15130#issuecomment-1879371419 > Some `ZkMigrationIntegrationTest` tests failed. Can we run a rebuild? The tests seem to have passed in the re-run. The step `**/build/test-results/**/TEST-*.xml` seems to have

[jira] [Created] (KAFKA-16087) Sink connector dropping incorrect record when ErrantRecordReporter used asynchronously

2024-01-05 Thread Greg Harris (Jira)
Greg Harris created KAFKA-16087: --- Summary: Sink connector dropping incorrect record when ErrantRecordReporter used asynchronously Key: KAFKA-16087 URL: https://issues.apache.org/jira/browse/KAFKA-16087

Re: [PR] KAFKA-16072: JUnit 5 extension to detect thread leak [kafka]

2024-01-05 Thread via GitHub
wernerdv commented on code in PR #15101: URL: https://github.com/apache/kafka/pull/15101#discussion_r1443373091 ## core/src/test/java/kafka/test/junit/LeakTestingExtension.java: ## @@ -0,0 +1,88 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + *

Re: [PR] KAFKA-15556: Remove NetworkClientDelegate methods isUnavailable, maybeThrowAuthFailure, and tryConnect [kafka]

2024-01-05 Thread via GitHub
Phuc-Hong-Tran commented on PR #15020: URL: https://github.com/apache/kafka/pull/15020#issuecomment-1879233506 @philipnee, is there anything else that I should change? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] KAFKA-16077: Streams with state updater fails to close task upon fencing [kafka]

2024-01-05 Thread via GitHub
lucasbru merged PR #15117: URL: https://github.com/apache/kafka/pull/15117 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-14133: Migrate consumer mock in TaskManagerTest to Mockito [kafka]

2024-01-05 Thread via GitHub
clolov commented on code in PR #15112: URL: https://github.com/apache/kafka/pull/15112#discussion_r1443172726 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/TaskManagerTest.java: ## @@ -4904,7 +4870,8 @@ public void suspend() { final Map>

Re: [PR] MINOR: Move Raft io thread implementation to Java [kafka]

2024-01-05 Thread via GitHub
hachikuji merged PR #15119: URL: https://github.com/apache/kafka/pull/15119 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-14412: Transactional StateStores [kafka]

2024-01-05 Thread via GitHub
nicktelford commented on code in PR #15137: URL: https://github.com/apache/kafka/pull/15137#discussion_r1443144457 ## streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBStore.java: ## @@ -805,6 +959,194 @@ public void reset() { public void close() {

Re: [PR] KAFKA-14412: Transactional StateStores [kafka]

2024-01-05 Thread via GitHub
nicktelford commented on code in PR #15137: URL: https://github.com/apache/kafka/pull/15137#discussion_r1443141089 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/ProcessorStateManager.java: ## @@ -266,29 +269,20 @@ void

Re: [PR] KAFKA-14412: Transactional StateStores [kafka]

2024-01-05 Thread via GitHub
nicktelford commented on code in PR #15137: URL: https://github.com/apache/kafka/pull/15137#discussion_r1443138486 ## checkstyle/checkstyle.xml: ## @@ -135,7 +135,7 @@ - + Review Comment: `TaskExecutor#commitOffsetsOrTransaction` -- This

Re: [PR] KAFKA-14412: Transactional StateStores [kafka]

2024-01-05 Thread via GitHub
nicktelford commented on code in PR #15137: URL: https://github.com/apache/kafka/pull/15137#discussion_r1443136941 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java: ## @@ -262,8 +268,7 @@ private void closeDirtyAndRevive(final Collection

Re: [PR] KAFKA-14412: Transactional StateStores [kafka]

2024-01-05 Thread via GitHub
nicktelford commented on PR #15137: URL: https://github.com/apache/kafka/pull/15137#issuecomment-1878999651 @cadonna @mjsax @ableegoldman @lucasbru @wcarlson5 @bbejeck @vvcephei @guozhangwang My apologies to whoever draws the short straw of having to review this! This is the

Re: [PR] MINOR: Add public documentation for metrics introduced in KIP-963 [kafka]

2024-01-05 Thread via GitHub
clolov closed pull request #15131: MINOR: Add public documentation for metrics introduced in KIP-963 URL: https://github.com/apache/kafka/pull/15131 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] MINOR: Add public documentation for metrics introduced in KIP-963 [kafka]

2024-01-05 Thread via GitHub
clolov commented on PR #15131: URL: https://github.com/apache/kafka/pull/15131#issuecomment-1878999520 Closing this PR and moving it to https://github.com/apache/kafka/pull/15138 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[PR] MINOR: Add public documentation for metrics introduced in KIP-963 [kafka]

2024-01-05 Thread via GitHub
clolov opened a new pull request, #15138: URL: https://github.com/apache/kafka/pull/15138 Adding the public documentation for metrics introduced in [KIP-963](https://cwiki.apache.org/confluence/display/KAFKA/KIP-963%3A+Additional+metrics+in+Tiered+Storage) -- This is an automated message

[PR] KAFKA-14412: Transactional StateStores [kafka]

2024-01-05 Thread via GitHub
nicktelford opened a new pull request, #15137: URL: https://github.com/apache/kafka/pull/15137 See [KIP-892: Transactional Semantics for StateStores][1] for a detailed explanation of the design. --- Writes to StateStores are now isolated from the underlying RocksDB database

Re: [PR] MINOR: Enable kraft test in kafka.api and kafka.network [kafka]

2024-01-05 Thread via GitHub
mimaison commented on PR #14595: URL: https://github.com/apache/kafka/pull/14595#issuecomment-1878961224 Thanks for the PR. It fails compiling: ``` > Task :core:compileTestScala [Error]

[jira] [Assigned] (KAFKA-14404) Fix & update docs on client configs controlled by Streams

2024-01-05 Thread Ayoub Omari (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayoub Omari reassigned KAFKA-14404: --- Assignee: Ayoub Omari (was: Sujay Hegde) > Fix & update docs on client configs controlled

[jira] [Updated] (KAFKA-16032) Review inconsistent error handling of OffsetFetch and OffsetCommit responses

2024-01-05 Thread Lianet Magrans (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianet Magrans updated KAFKA-16032: --- Description: OffsetFetch and OffsetCommit handle errors separately. There are 2 issues to

[jira] [Updated] (KAFKA-15922) Add MetadataVersion for JBOD

2024-01-05 Thread Proven Provenzano (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Proven Provenzano updated KAFKA-15922: -- Fix Version/s: 3.7.0 > Add MetadataVersion for JBOD > >

[jira] [Resolved] (KAFKA-15922) Add MetadataVersion for JBOD

2024-01-05 Thread Proven Provenzano (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Proven Provenzano resolved KAFKA-15922. --- Resolution: Fixed > Add MetadataVersion for JBOD > > >

Re: [PR] KAFKA-15704: Update clientId and clientHost in MemberMetadata when static member is replaced. [kafka]

2024-01-05 Thread via GitHub
vamossagar12 commented on PR #14666: URL: https://github.com/apache/kafka/pull/14666#issuecomment-1878869619 hey @divijvaidya , @showuon would you be able to review this PR? This seems like a useful change and things look fine once I reviewed. -- This is an automated message from the

Re: [PR] MINOR: Increase parallelism for Jenkins [kafka]

2024-01-05 Thread via GitHub
ijuma commented on PR #15099: URL: https://github.com/apache/kafka/pull/15099#issuecomment-1878866375 That is not surprising - it's the reason why we limited parallelism in the first place. :) -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] KAFKA-16086: Fix memory leak in RocksDBStore [kafka]

2024-01-05 Thread via GitHub
nicktelford commented on PR #15135: URL: https://github.com/apache/kafka/pull/15135#issuecomment-1878840727 IntelliJ will report a warning, provided that the object isn't passed to another method (as we do with `RocksDB.listColumnFamilies`). I can't get spotbugs to report this bug

Re: [PR] KAFKA-16016: Add docker wrapper in core and remove docker utility script [kafka]

2024-01-05 Thread via GitHub
viktorsomogyi commented on code in PR #15048: URL: https://github.com/apache/kafka/pull/15048#discussion_r1442969642 ## core/src/main/scala/kafka/docker/KafkaDockerWrapper.scala: ## @@ -0,0 +1,218 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + *

Re: [PR] KAFKA-16016: Add docker wrapper in core and remove docker utility script [kafka]

2024-01-05 Thread via GitHub
viktorsomogyi commented on code in PR #15048: URL: https://github.com/apache/kafka/pull/15048#discussion_r1442969642 ## core/src/main/scala/kafka/docker/KafkaDockerWrapper.scala: ## @@ -0,0 +1,218 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + *

Re: [PR] KAFKA-16086: Fix memory leak in RocksDBStore [kafka]

2024-01-05 Thread via GitHub
lucasbru merged PR #15135: URL: https://github.com/apache/kafka/pull/15135 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-15942: Implement ConsumerInterceptors in the async consumer [kafka]

2024-01-05 Thread via GitHub
lucasbru commented on PR #15000: URL: https://github.com/apache/kafka/pull/15000#issuecomment-1878769287 @philipnee some considerations: - Does the old consumer guarantee that the interceptor is called when `poll` or `close` are not called by the consumer anymore after an autocommit? I

Re: [PR] KAFKA-16086: Fix memory leak in RocksDBStore [kafka]

2024-01-05 Thread via GitHub
lucasbru commented on PR #15135: URL: https://github.com/apache/kafka/pull/15135#issuecomment-1878750810 My hunch is that spotbugs won't report it when it's leaked from the method via another method call (here, `RocksDB.listColumnFamilies`). But yeah, better to check -- This is an

Re: [PR] MINOR: Add 3.5.2 and 3.6.1 to system tests [kafka]

2024-01-05 Thread via GitHub
mimaison commented on PR #14932: URL: https://github.com/apache/kafka/pull/14932#issuecomment-1878710831 Thanks @mjsax! I've added 3.5.2 to this PR as well. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] KAFKA-16086: Fix memory leak in RocksDBStore [kafka]

2024-01-05 Thread via GitHub
divijvaidya commented on PR #15135: URL: https://github.com/apache/kafka/pull/15135#issuecomment-1878694769 I agree @nicktelford, there could be false positives. As a followup to this PR, can you please create a JIRA to investigate why spotbugs didn't catch it for us and make necessary

Re: [PR] KAFKA-14133: Migrate consumer mock in TaskManagerTest to Mockito [kafka]

2024-01-05 Thread via GitHub
divijvaidya commented on code in PR #15112: URL: https://github.com/apache/kafka/pull/15112#discussion_r1442911556 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/TaskManagerTest.java: ## @@ -4904,7 +4870,8 @@ public void suspend() { final Map>

[jira] [Commented] (KAFKA-14404) Fix & update docs on client configs controlled by Streams

2024-01-05 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17803543#comment-17803543 ] Matthias J. Sax commented on KAFKA-14404: - It seems [~sujayopensource] lost interest to work on

Re: [PR] Minor: Add KIP-923 to upgrade-guide.html and dsl-api.html [kafka]

2024-01-05 Thread via GitHub
mjsax commented on PR #14725: URL: https://github.com/apache/kafka/pull/14725#issuecomment-1878653401 Merged to `trunk` and cherry-picked to `3.7` and `3.6` branches. -- @wcarlson5 can you do a PR on `kafka-site` to get the change live for `3.6`? -- This is an automated message from the

Re: [PR] Minor: Add KIP-923 to upgrade-guide.html and dsl-api.html [kafka]

2024-01-05 Thread via GitHub
mjsax merged PR #14725: URL: https://github.com/apache/kafka/pull/14725 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] MINOR: Add 3.6.1 to system tests [kafka]

2024-01-05 Thread via GitHub
mjsax commented on PR #14932: URL: https://github.com/apache/kafka/pull/14932#issuecomment-1878645077 Uploaded 3.5.2 and 3.6.1 artifacts. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] KAFKA-14133: Migrate consumer mock in TaskManagerTest to Mockito [kafka]

2024-01-05 Thread via GitHub
clolov commented on code in PR #15112: URL: https://github.com/apache/kafka/pull/15112#discussion_r1442836205 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/TaskManagerTest.java: ## @@ -4809,11 +4768,14 @@ public void

Re: [PR] KAFKA-15721: KRaft support in DeleteTopicsRequestWithDeletionDisabledTest [kafka]

2024-01-05 Thread via GitHub
wernerdv commented on PR #15124: URL: https://github.com/apache/kafka/pull/15124#issuecomment-1878609362 Hello, @jolshan Please, take a look. The changes are similar to #15086. Tests passed

Re: [PR] KAFKA-16085: Add metric value consolidated for topics on a broker for tiered storage. [kafka]

2024-01-05 Thread via GitHub
showuon commented on PR #15133: URL: https://github.com/apache/kafka/pull/15133#issuecomment-1878605296 > Okay, correct me if I am wrong - your idea is that we will use the aggregate object to either store partitions->values or topics->values. For the case where we don't have a topic tag

Re: [PR] KAFKA-16086: Fix memory leak in RocksDBStore [kafka]

2024-01-05 Thread via GitHub
nicktelford commented on PR #15135: URL: https://github.com/apache/kafka/pull/15135#issuecomment-1878591135 > Is there a way we could have caught this? (perhaps spotbugs telling that an autocloseable object is not getting closed) > > see:

Re: [PR] KAFKA-16086: Fix memory leak in RocksDBStore [kafka]

2024-01-05 Thread via GitHub
divijvaidya commented on PR #15135: URL: https://github.com/apache/kafka/pull/15135#issuecomment-1878582350 Is there a way we could have caught this? (perhaps spotbugs telling that an autocloseable object is not getting closed) -- This is an automated message from the Apache Git Service.

Re: [PR] KAFKA-16085: add all topic metrics [kafka]

2024-01-05 Thread via GitHub
showuon commented on PR #15133: URL: https://github.com/apache/kafka/pull/15133#issuecomment-1878558194 > Should we write a test that could have caught this? Will do! Just want to make sure the solution makes sense. -- This is an automated message from the Apache Git Service. To

Re: [PR] KAFKA-14133: Migrate consumer mock in TaskManagerTest to Mockito [kafka]

2024-01-05 Thread via GitHub
divijvaidya commented on code in PR #15112: URL: https://github.com/apache/kafka/pull/15112#discussion_r1442788932 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/TaskManagerTest.java: ## @@ -1354,14 +1346,12 @@ public void

Re: [PR] KAFKA-16082: Avoid resuming future replica if current replica is in the same directory [kafka]

2024-01-05 Thread via GitHub
gaurav-narula commented on PR #15136: URL: https://github.com/apache/kafka/pull/15136#issuecomment-1878536740 CC: @OmniaGM @pprovenzano -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[jira] [Commented] (KAFKA-16073) Kafka Tiered Storage Bug: Consumer Fetch Error Due to Delayed localLogStartOffset Update During Segment Deletion

2024-01-05 Thread hzh0425 (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17803517#comment-17803517 ] hzh0425 commented on KAFKA-16073: - [~ckamal]  yes, Im working on it, and we can solve it together >

Re: [PR] MINOR: Add public documentation for metrics introduced in KIP-963 [kafka]

2024-01-05 Thread via GitHub
showuon commented on PR #15131: URL: https://github.com/apache/kafka/pull/15131#issuecomment-1878532279 I found the bug that we didn't update the all topic metric value: [KAFKA-16085](https://issues.apache.org/jira/browse/KAFKA-16085) and the drafted solution:

Re: [PR] MINOR: Address occasional UnnecessaryStubbingException in StreamThreadTest [kafka]

2024-01-05 Thread via GitHub
divijvaidya commented on PR #15134: URL: https://github.com/apache/kafka/pull/15134#issuecomment-1878531012 could you please help us understand in the description on what the cause of flakiness was and how adding lenient() fixes it. -- This is an automated message from the Apache Git

Re: [PR] KAFKA-16083: Exclude throttle time when expiring inflight requests on a connection [kafka]

2024-01-05 Thread via GitHub
stanislavkozlovski commented on PR #15130: URL: https://github.com/apache/kafka/pull/15130#issuecomment-1878479652 Some `ZkMigrationIntegrationTest` tests failed. Can we run a rebuild? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] KAFKA-16083: Exclude throttle time when expiring inflight requests on a connection [kafka]

2024-01-05 Thread via GitHub
stanislavkozlovski commented on code in PR #15130: URL: https://github.com/apache/kafka/pull/15130#discussion_r1442734095 ## clients/src/test/java/org/apache/kafka/clients/NetworkClientTest.java: ## @@ -560,6 +560,38 @@ public void testConnectionSetupTimeout() {

Re: [PR] KAFKA-16083: Exclude throttle time when expiring inflight requests on a connection [kafka]

2024-01-05 Thread via GitHub
stanislavkozlovski commented on code in PR #15130: URL: https://github.com/apache/kafka/pull/15130#discussion_r1442732922 ## clients/src/main/java/org/apache/kafka/clients/InFlightRequests.java: ## @@ -158,7 +158,9 @@ public Iterable clearAll(String node) { private

Re: [PR] KAFKA-16083: Exclude throttle time when expiring inflight requests on a connection [kafka]

2024-01-05 Thread via GitHub
stanislavkozlovski commented on code in PR #15130: URL: https://github.com/apache/kafka/pull/15130#discussion_r1442732279 ## clients/src/main/java/org/apache/kafka/clients/InFlightRequests.java: ## @@ -158,7 +158,9 @@ public Iterable clearAll(String node) { private

[jira] [Comment Edited] (KAFKA-16086) Kafka Streams has RocksDB native memory leak

2024-01-05 Thread Nicholas Telford (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17803506#comment-17803506 ] Nicholas Telford edited comment on KAFKA-16086 at 1/5/24 10:46 AM: --- As

[jira] [Comment Edited] (KAFKA-16086) Kafka Streams has RocksDB native memory leak

2024-01-05 Thread Nicholas Telford (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17803506#comment-17803506 ] Nicholas Telford edited comment on KAFKA-16086 at 1/5/24 10:45 AM: --- As

[jira] [Commented] (KAFKA-16086) Kafka Streams has RocksDB native memory leak

2024-01-05 Thread Nicholas Telford (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17803506#comment-17803506 ] Nicholas Telford commented on KAFKA-16086: -- As discussed on Slack:

[jira] [Updated] (KAFKA-16077) Streams fails to close task after restoration when input partitions are updated

2024-01-05 Thread Lucas Brutschy (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lucas Brutschy updated KAFKA-16077: --- Labels: streams (was: ) > Streams fails to close task after restoration when input

Re: [PR] MINOR: New year code cleanup - include final keyword [kafka]

2024-01-05 Thread via GitHub
mimaison commented on code in PR #15072: URL: https://github.com/apache/kafka/pull/15072#discussion_r1442700296 ## clients/src/main/java/org/apache/kafka/common/record/LazyDownConversionRecordsSend.java: ## @@ -36,9 +36,9 @@ public final class LazyDownConversionRecordsSend

[jira] [Commented] (KAFKA-16073) Kafka Tiered Storage Bug: Consumer Fetch Error Due to Delayed localLogStartOffset Update During Segment Deletion

2024-01-05 Thread Kamal Chandraprakash (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17803486#comment-17803486 ] Kamal Chandraprakash commented on KAFKA-16073: -- [~hzh0425@apache]  Are you working on this

Re: [PR] KAFKA-16016: Add docker wrapper in core and remove docker utility script [kafka]

2024-01-05 Thread via GitHub
mimaison commented on code in PR #15048: URL: https://github.com/apache/kafka/pull/15048#discussion_r1442692990 ## core/src/main/scala/kafka/docker/KafkaDockerWrapper.scala: ## @@ -0,0 +1,218 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + *

Re: [PR] MINOR: Small cleanups in Connect [kafka]

2024-01-05 Thread via GitHub
mimaison commented on code in PR #15128: URL: https://github.com/apache/kafka/pull/15128#discussion_r1442689284 ## connect/basic-auth-extension/src/main/java/org/apache/kafka/connect/rest/basic/auth/extension/JaasBasicAuthFilter.java: ## @@ -119,16 +117,16 @@ private boolean

Re: [PR] MINOR: Small cleanups in Connect [kafka]

2024-01-05 Thread via GitHub
mimaison merged PR #15128: URL: https://github.com/apache/kafka/pull/15128 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Assigned] (KAFKA-16086) Kafka Streams has RocksDB native memory leak

2024-01-05 Thread Lucas Brutschy (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lucas Brutschy reassigned KAFKA-16086: -- Assignee: Nicholas Telford > Kafka Streams has RocksDB native memory leak >

[jira] [Updated] (KAFKA-16086) Kafka Streams has RocksDB native memory leak

2024-01-05 Thread Lucas Brutschy (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lucas Brutschy updated KAFKA-16086: --- Affects Version/s: 3.7.0 > Kafka Streams has RocksDB native memory leak >

[jira] [Created] (KAFKA-16086) Kafka Streams has RocksDB native memory leak

2024-01-05 Thread Lucas Brutschy (Jira)
Lucas Brutschy created KAFKA-16086: -- Summary: Kafka Streams has RocksDB native memory leak Key: KAFKA-16086 URL: https://issues.apache.org/jira/browse/KAFKA-16086 Project: Kafka Issue Type:

Re: [PR] MINOR: Address occasional UnnecessaryStubbingException in StreamThreadTest [kafka]

2024-01-05 Thread via GitHub
clolov commented on PR #15134: URL: https://github.com/apache/kafka/pull/15134#issuecomment-1878383350 This pull request should hopefully address the problem which was discussed in https://github.com/apache/kafka/pull/15116#issuecomment-1877052454 -- This is an automated message from the

[PR] MINOR: Address occasional UnnecessaryStubbingException in StreamThreadTest [kafka]

2024-01-05 Thread via GitHub
clolov opened a new pull request, #15134: URL: https://github.com/apache/kafka/pull/15134 This pull request should address the occasional failures of the StreamThreadTest such as https://ci-builds.apache.org/blue/organizations/jenkins/Kafka%2Fkafka-pr/detail/PR-15116/1/tests A

[jira] [Updated] (KAFKA-14133) Remaining EasyMock to Mockito tests

2024-01-05 Thread Christo Lolov (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christo Lolov updated KAFKA-14133: -- Description: {color:#de350b}There are tests which use both PowerMock and EasyMock. I have

Re: [PR] KAFKA-16077: Streams with state updater fails to close task upon fencing [kafka]

2024-01-05 Thread via GitHub
lucasbru commented on PR #15117: URL: https://github.com/apache/kafka/pull/15117#issuecomment-1878354099 > @lucasbru What do you think about my previous question? > > > Should we also add a integration test that tests the scenario? We don't test the other cases of the pending

Re: [PR] KAFKA-16059: close more kafkaApis instances [kafka]

2024-01-05 Thread via GitHub
showuon commented on code in PR #15132: URL: https://github.com/apache/kafka/pull/15132#discussion_r1442645316 ## core/src/test/scala/unit/kafka/server/KafkaApisTest.scala: ## @@ -4075,6 +4122,7 @@ class KafkaApisTest extends Logging { assertEquals(authorizedTopic,

[jira] [Commented] (KAFKA-16060) Some questions about tiered storage capabilities

2024-01-05 Thread Satish Duggana (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17803470#comment-17803470 ] Satish Duggana commented on KAFKA-16060: [~jianbin] There are no short term plans to support

Re: [PR] KAFKA-16059: close more kafkaApis instances [kafka]

2024-01-05 Thread via GitHub
divijvaidya commented on code in PR #15132: URL: https://github.com/apache/kafka/pull/15132#discussion_r1442639976 ## core/src/test/scala/unit/kafka/server/KafkaApisTest.scala: ## @@ -4075,6 +4122,7 @@ class KafkaApisTest extends Logging { assertEquals(authorizedTopic,

Re: [PR] KAFKA-16085: add all topic metrics [kafka]

2024-01-05 Thread via GitHub
showuon commented on PR #15133: URL: https://github.com/apache/kafka/pull/15133#issuecomment-1878302828 @clolov @satishd @kamalcph , do you think this fix makes sense? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] KAFKA-16085: add all topic metrics [kafka]

2024-01-05 Thread via GitHub
showuon commented on code in PR #15133: URL: https://github.com/apache/kafka/pull/15133#discussion_r1442611638 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -529,18 +534,28 @@ class BrokerTopicMetrics(name: Option[String], configOpt:

[PR] KAFKA-16085: add all topic metrics [kafka]

2024-01-05 Thread via GitHub
showuon opened a new pull request, #15133: URL: https://github.com/apache/kafka/pull/15133 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of

[jira] [Commented] (KAFKA-16085) remote copy lag bytes/segments metrics don't update all topic value

2024-01-05 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17803452#comment-17803452 ] Luke Chen commented on KAFKA-16085: --- [~christo_lolov] [~satishd] [~ckamal], this is the part we

[jira] [Created] (KAFKA-16085) remote copy lag bytes/segments metrics don't update all topic value

2024-01-05 Thread Luke Chen (Jira)
Luke Chen created KAFKA-16085: - Summary: remote copy lag bytes/segments metrics don't update all topic value Key: KAFKA-16085 URL: https://issues.apache.org/jira/browse/KAFKA-16085 Project: Kafka

[jira] [Commented] (KAFKA-16060) Some questions about tiered storage capabilities

2024-01-05 Thread Jianbin Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17803449#comment-17803449 ] Jianbin Chen commented on KAFKA-16060: -- Thank you both for your replies, and allow me to ask an