[jira] [Commented] (KAFKA-15425) Compatibility break in Admin.listOffsets() (2)

2023-09-01 Thread Satish Duggana (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17761437#comment-17761437 ] Satish Duggana commented on KAFKA-15425: https://github.com/apache/kafka/pull/14

[GitHub] [kafka] ableegoldman commented on a diff in pull request #14326: KAFKA-15429: reset transactionInFlight on StreamsProducer close

2023-09-01 Thread via GitHub
ableegoldman commented on code in PR #14326: URL: https://github.com/apache/kafka/pull/14326#discussion_r1313721147 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamsProducer.java: ## @@ -371,6 +371,7 @@ void flush() { void close() { pr

[GitHub] [kafka] ableegoldman commented on a diff in pull request #14326: KAFKA-15429: reset transactionInFlight on StreamsProducer close

2023-09-01 Thread via GitHub
ableegoldman commented on code in PR #14326: URL: https://github.com/apache/kafka/pull/14326#discussion_r1313720481 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamsProducer.java: ## @@ -371,6 +371,7 @@ void flush() { void close() { pr

[GitHub] [kafka] vamossagar12 commented on a diff in pull request #14093: KAFKA-15248 Add BooleanConverter

2023-09-01 Thread via GitHub
vamossagar12 commented on code in PR #14093: URL: https://github.com/apache/kafka/pull/14093#discussion_r1313720397 ## connect/runtime/src/main/java/org/apache/kafka/connect/converters/BooleanConverter.java: ## @@ -0,0 +1,107 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] rodesai opened a new pull request, #14326: KAFKA-15429: reset transactionInFlight on StreamsProducer close

2023-09-01 Thread via GitHub
rodesai opened a new pull request, #14326: URL: https://github.com/apache/kafka/pull/14326 Resets the value of transactionInFlight to false when closing the StreamsProducer. This ensures we don't try to commit against a closed producer. Tested using a unit test of StreamsProducer. -- Thi

[GitHub] [kafka] rodesai opened a new pull request, #14325: KAFKA-15429: catch+log errors from unsubscribe in streamthread shutdown

2023-09-01 Thread via GitHub
rodesai opened a new pull request, #14325: URL: https://github.com/apache/kafka/pull/14325 Preliminary fix for KAFKA-15429 which updates StreamThread.completeShutdown to catch-and-log errors from consumer.unsubscribe. Though this does not prevent the exception, it does preserve the original

[GitHub] [kafka] jolshan opened a new pull request, #14324: KAFKA-15424: Make the transaction verification a dynamic configuration

2023-09-01 Thread via GitHub
jolshan opened a new pull request, #14324: URL: https://github.com/apache/kafka/pull/14324 This will allow enabling and disabling transaction verification (KIP-890 part 1) without having to roll the cluster. Tested that restarting the cluster persists the configuration. If a v

[GitHub] [kafka] C0urante commented on pull request #14320: KAFKA-15421: fix network thread leak in testThreadPoolResize

2023-09-01 Thread via GitHub
C0urante commented on PR #14320: URL: https://github.com/apache/kafka/pull/14320#issuecomment-1703531194 @showuon you're my hero, thanks for your diligent work on fixing these flaky tests! -- This is an automated message from the Apache Git Service. To respond to the message, please log o

[GitHub] [kafka] jolshan commented on a diff in pull request #14321: KAFKA-14499: [6/N] Add MemberId and MemberEpoch to OffsetFetchRequest

2023-09-01 Thread via GitHub
jolshan commented on code in PR #14321: URL: https://github.com/apache/kafka/pull/14321#discussion_r1313636482 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/consumer/ConsumerGroupTest.java: ## @@ -631,4 +631,33 @@ public void testValidateOffsetCommit() {

[GitHub] [kafka] jolshan commented on a diff in pull request #14321: KAFKA-14499: [6/N] Add MemberId and MemberEpoch to OffsetFetchRequest

2023-09-01 Thread via GitHub
jolshan commented on code in PR #14321: URL: https://github.com/apache/kafka/pull/14321#discussion_r1313635548 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/consumer/ConsumerGroupTest.java: ## @@ -631,4 +631,33 @@ public void testValidateOffsetCommit() {

[GitHub] [kafka] jolshan commented on a diff in pull request #14321: KAFKA-14499: [6/N] Add MemberId and MemberEpoch to OffsetFetchRequest

2023-09-01 Thread via GitHub
jolshan commented on code in PR #14321: URL: https://github.com/apache/kafka/pull/14321#discussion_r1313633100 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/OffsetMetadataManagerTest.java: ## @@ -1409,6 +1444,82 @@ public void testFetchAllOffsetsAtDiffer

[GitHub] [kafka] jolshan commented on a diff in pull request #14321: KAFKA-14499: [6/N] Add MemberId and MemberEpoch to OffsetFetchRequest

2023-09-01 Thread via GitHub
jolshan commented on code in PR #14321: URL: https://github.com/apache/kafka/pull/14321#discussion_r1313631916 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/ConsumerGroup.java: ## @@ -538,17 +537,46 @@ public void validateOffsetCommit( i

[GitHub] [kafka] jolshan commented on a diff in pull request #14321: KAFKA-14499: [6/N] Add MemberId and MemberEpoch to OffsetFetchRequest

2023-09-01 Thread via GitHub
jolshan commented on code in PR #14321: URL: https://github.com/apache/kafka/pull/14321#discussion_r1313618740 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/OffsetMetadataManagerTest.java: ## @@ -1409,6 +1444,82 @@ public void testFetchAllOffsetsAtDiffer

[jira] [Updated] (KAFKA-15428) Cluster-wide dynamic log adjustments for Connect

2023-09-01 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton updated KAFKA-15428: -- Labels: needs-kip (was: Needs-kip) > Cluster-wide dynamic log adjustments for Connect > -

[jira] [Updated] (KAFKA-15428) Cluster-wide dynamic log adjustments for Connect

2023-09-01 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton updated KAFKA-15428: -- Labels: Needs-kip (was: ) > Cluster-wide dynamic log adjustments for Connect > --

[jira] [Created] (KAFKA-15428) Cluster-wide dynamic log adjustments for Connect

2023-09-01 Thread Chris Egerton (Jira)
Chris Egerton created KAFKA-15428: - Summary: Cluster-wide dynamic log adjustments for Connect Key: KAFKA-15428 URL: https://issues.apache.org/jira/browse/KAFKA-15428 Project: Kafka Issue Type

[GitHub] [kafka] jolshan commented on a diff in pull request #14321: KAFKA-14499: [6/N] Add MemberId and MemberEpoch to OffsetFetchRequest

2023-09-01 Thread via GitHub
jolshan commented on code in PR #14321: URL: https://github.com/apache/kafka/pull/14321#discussion_r1313613392 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/OffsetMetadataManagerTest.java: ## @@ -181,10 +181,28 @@ public List fetchOffsets( S

[GitHub] [kafka] jolshan commented on a diff in pull request #14321: KAFKA-14499: [6/N] Add MemberId and MemberEpoch to OffsetFetchRequest

2023-09-01 Thread via GitHub
jolshan commented on code in PR #14321: URL: https://github.com/apache/kafka/pull/14321#discussion_r1313590994 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/generic/GenericGroup.java: ## @@ -823,9 +823,17 @@ public void validateOffsetCommit( /**

[GitHub] [kafka] jolshan commented on a diff in pull request #14321: KAFKA-14499: [6/N] Add MemberId and MemberEpoch to OffsetFetchRequest

2023-09-01 Thread via GitHub
jolshan commented on code in PR #14321: URL: https://github.com/apache/kafka/pull/14321#discussion_r1313587458 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/ConsumerGroup.java: ## @@ -538,17 +537,46 @@ public void validateOffsetCommit( i

[GitHub] [kafka] jolshan commented on a diff in pull request #14321: KAFKA-14499: [6/N] Add MemberId and MemberEpoch to OffsetFetchRequest

2023-09-01 Thread via GitHub
jolshan commented on code in PR #14321: URL: https://github.com/apache/kafka/pull/14321#discussion_r1313582619 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/OffsetMetadataManager.java: ## @@ -225,17 +225,21 @@ private Group validateOffsetCommit( }

[GitHub] [kafka] jolshan commented on a diff in pull request #14321: KAFKA-14499: [6/N] Add MemberId and MemberEpoch to OffsetFetchRequest

2023-09-01 Thread via GitHub
jolshan commented on code in PR #14321: URL: https://github.com/apache/kafka/pull/14321#discussion_r1313582042 ## clients/src/main/resources/common/message/OffsetFetchResponse.json: ## @@ -32,8 +32,21 @@ // Version 7 adds pending offset commit as new error response on partiti

[GitHub] [kafka] kirktrue commented on pull request #14305: KAFKA-14274: [1/7] basic refactoring

2023-09-01 Thread via GitHub
kirktrue commented on PR #14305: URL: https://github.com/apache/kafka/pull/14305#issuecomment-1703327044 @junrao let me know your thoughts on this PR. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [kafka] Cerchie commented on a diff in pull request #14322: KAFKA-15418: update statement on decompression

2023-09-01 Thread via GitHub
Cerchie commented on code in PR #14322: URL: https://github.com/apache/kafka/pull/14322#discussion_r1313519585 ## docs/design.html: ## @@ -136,8 +136,9 @@ -Kafka supports this with an efficient batching format. A batch of messages can be clumped together compressed and sen

[jira] [Resolved] (KAFKA-15081) Implement new consumer offsetsForTimes

2023-09-01 Thread Jun Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-15081. - Fix Version/s: 3.7.0 Resolution: Fixed merged the PR to trunk > Implement new consumer offsetsFo

[jira] [Resolved] (KAFKA-14965) Introduce OffsetsRequestManager to integrate ListOffsets requests into new consumer threading refactor

2023-09-01 Thread Jun Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-14965. - Fix Version/s: 3.7.0 Resolution: Fixed merged the PR to trunk > Introduce OffsetsRequestManager

[GitHub] [kafka] junrao merged pull request #14308: KAFKA-14965 - OffsetsRequestsManager implementation & API integration

2023-09-01 Thread via GitHub
junrao merged PR #14308: URL: https://github.com/apache/kafka/pull/14308 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.o

[GitHub] [kafka] junrao commented on pull request #14308: KAFKA-14965 - OffsetsRequestsManager implementation & API integration

2023-09-01 Thread via GitHub
junrao commented on PR #14308: URL: https://github.com/apache/kafka/pull/14308#issuecomment-1703310796 Implementation of the OffsetRequestsManager, responsible for building requests and processing responses for requests related to partition offsets. In this PR, the manager includes su

[GitHub] [kafka] lianetm opened a new pull request, #14323: [WIP] - KAFKA-15275 - Client state machine basic components, states and initial transitions

2023-09-01 Thread via GitHub
lianetm opened a new pull request, #14323: URL: https://github.com/apache/kafka/pull/14323 Core components for maintaining group membership based on heartbeats, following the new consumer group protocol. These includes the initial components, states and transitions, with a shape suit

[GitHub] [kafka] jolshan commented on pull request #14320: KAFKA-15421: fix network thread leak in testThreadPoolResize

2023-09-01 Thread via GitHub
jolshan commented on PR #14320: URL: https://github.com/apache/kafka/pull/14320#issuecomment-1703274740 For my understanding, the issue was from tests disabled previously (ie, not the 2 we just disabled?) -- This is an automated message from the Apache Git Service. To respond to the messa

[GitHub] [kafka] jeffkbkim commented on a diff in pull request #14182: KAFKA 14515: Optimized Uniform Rack Aware Assignor

2023-09-01 Thread via GitHub
jeffkbkim commented on code in PR #14182: URL: https://github.com/apache/kafka/pull/14182#discussion_r1313285352 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/OptimizedUniformAssignmentBuilder.java: ## @@ -0,0 +1,390 @@ +/* + * Licensed to the Ap

[GitHub] [kafka] jeffkbkim commented on a diff in pull request #14182: KAFKA 14515: Optimized Uniform Rack Aware Assignor

2023-09-01 Thread via GitHub
jeffkbkim commented on code in PR #14182: URL: https://github.com/apache/kafka/pull/14182#discussion_r1313213976 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/OptimizedUniformAssignmentBuilder.java: ## @@ -0,0 +1,399 @@ +/* + * Licensed to the Ap

[GitHub] [kafka] jeffkbkim commented on a diff in pull request #14182: KAFKA 14515: Optimized Uniform Rack Aware Assignor

2023-09-01 Thread via GitHub
jeffkbkim commented on code in PR #14182: URL: https://github.com/apache/kafka/pull/14182#discussion_r1313434102 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/assignor/OptimizedUniformAssignmentBuilderTest.java: ## @@ -0,0 +1,1070 @@ +/* + * Licensed to t

[jira] [Updated] (KAFKA-15372) MM2 rolling restart can drop configuration changes silently

2023-09-01 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Harris updated KAFKA-15372: Fix Version/s: (was: 3.6.0) > MM2 rolling restart can drop configuration changes silently > --

[jira] [Commented] (KAFKA-15372) MM2 rolling restart can drop configuration changes silently

2023-09-01 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17761381#comment-17761381 ] Greg Harris commented on KAFKA-15372: - I implemented the REST-less fix, but it's com

[GitHub] [kafka] jeffkbkim commented on a diff in pull request #14182: KAFKA 14515: Optimized Uniform Rack Aware Assignor

2023-09-01 Thread via GitHub
jeffkbkim commented on code in PR #14182: URL: https://github.com/apache/kafka/pull/14182#discussion_r1313408646 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/OptimizedUniformAssignmentBuilder.java: ## @@ -0,0 +1,390 @@ +/* + * Licensed to the Ap

[jira] [Commented] (KAFKA-15399) Enable OffloadAndConsumeFromLeader test

2023-09-01 Thread Lianet Magrans (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17761379#comment-17761379 ] Lianet Magrans commented on KAFKA-15399: The failure we are currently seeing is:

[GitHub] [kafka] lianetm commented on pull request #14308: KAFKA-14965 - OffsetsRequestsManager implementation & API integration

2023-09-01 Thread via GitHub
lianetm commented on PR #14308: URL: https://github.com/apache/kafka/pull/14308#issuecomment-1703196406 The failure they were after with that fix seems different to this though, here is about hanging threads on the base `QuorumTestHarness`, so I would say it is not specific to the `OffloadA

[jira] [Comment Edited] (KAFKA-15399) Enable OffloadAndConsumeFromLeader test

2023-09-01 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17761377#comment-17761377 ] Philip Nee edited comment on KAFKA-15399 at 9/1/23 6:49 PM:

[GitHub] [kafka] philipnee commented on pull request #14308: KAFKA-14965 - OffsetsRequestsManager implementation & API integration

2023-09-01 Thread via GitHub
philipnee commented on PR #14308: URL: https://github.com/apache/kafka/pull/14308#issuecomment-1703188741 [KAFKA-15399](https://issues.apache.org/jira/browse/KAFKA-15399) - Chris L. mentioned that he fixed the OffloadAndConsumeFromLeaderTest issue, but apparently we are still seeing it on t

[jira] [Commented] (KAFKA-15399) Enable OffloadAndConsumeFromLeader test

2023-09-01 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17761377#comment-17761377 ] Philip Nee commented on KAFKA-15399: [~christo_lolov] - We are still seeing this on

[GitHub] [kafka] divijvaidya commented on pull request #14320: KAFKA-15421: fix network thread leak in testThreadPoolResize

2023-09-01 Thread via GitHub
divijvaidya commented on PR #14320: URL: https://github.com/apache/kafka/pull/14320#issuecomment-1703186896 cc: @C0urante @jolshan since you folks were wondering about the mystery of threads leaks as well. -- This is an automated message from the Apache Git Service. To respond to the mess

[GitHub] [kafka] divijvaidya commented on a diff in pull request #14322: KAFKA-15418: update statement on decompression

2023-09-01 Thread via GitHub
divijvaidya commented on code in PR #14322: URL: https://github.com/apache/kafka/pull/14322#discussion_r1313393480 ## docs/design.html: ## @@ -136,8 +136,9 @@ -Kafka supports this with an efficient batching format. A batch of messages can be clumped together compressed and

[GitHub] [kafka] jeffkbkim commented on a diff in pull request #14182: KAFKA 14515: Optimized Uniform Rack Aware Assignor

2023-09-01 Thread via GitHub
jeffkbkim commented on code in PR #14182: URL: https://github.com/apache/kafka/pull/14182#discussion_r1313393423 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/OptimizedUniformAssignmentBuilder.java: ## @@ -0,0 +1,390 @@ +/* + * Licensed to the Ap

[GitHub] [kafka] lianetm commented on pull request #14308: KAFKA-14965 - OffsetsRequestsManager implementation & API integration

2023-09-01 Thread via GitHub
lianetm commented on PR #14308: URL: https://github.com/apache/kafka/pull/14308#issuecomment-1703178243 Build completed with 10 unrelated test failures (mainly due to unexpected threads found before running them & timeouts): Build / JDK 20 and Scala 2.13 / initializationError – org.a

[GitHub] [kafka] divijvaidya commented on pull request #14319: KAFKA-15427: Fix resource leak in integration tests for tiered storage

2023-09-01 Thread via GitHub
divijvaidya commented on PR #14319: URL: https://github.com/apache/kafka/pull/14319#issuecomment-1703178016 Nice find @clolov and @nikramakrishnan. Thank you! @satishd (quick administrative note) - the co-authored-by line goes at the very end (after reviewers) for GitHub to correct de

[GitHub] [kafka] divijvaidya commented on pull request #14320: KAFKA-15421: fix network thread leak in testThreadPoolResize

2023-09-01 Thread via GitHub
divijvaidya commented on PR #14320: URL: https://github.com/apache/kafka/pull/14320#issuecomment-1703171354 We should also enable `testThreadPoolResize` which was disabled in https://github.com/apache/kafka/commit/43fe13350f7a4c74cd101cbb69a01d062f5c9329 Do you want to do that in thi

[jira] [Comment Edited] (KAFKA-15411) DelegationTokenEndToEndAuthorizationWithOwnerTest is Flaky

2023-09-01 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17761369#comment-17761369 ] Divij Vaidya edited comment on KAFKA-15411 at 9/1/23 6:22 PM:

[jira] [Commented] (KAFKA-15411) DelegationTokenEndToEndAuthorizationWithOwnerTest is Flaky

2023-09-01 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17761369#comment-17761369 ] Divij Vaidya commented on KAFKA-15411: -- > Are we still seeing this issue? Yes, you

[GitHub] [kafka] cmccabe commented on a diff in pull request #14306: KAFKA-15369: Implement KIP-919: Allow AC to Talk Directly with Controllers

2023-09-01 Thread via GitHub
cmccabe commented on code in PR #14306: URL: https://github.com/apache/kafka/pull/14306#discussion_r1313371146 ## metadata/src/test/java/org/apache/kafka/image/publisher/ControllerRegistrationsPublisherTest.java: ## @@ -0,0 +1,149 @@ +/* + * Licensed to the Apache Software Found

[GitHub] [kafka] Cerchie opened a new pull request, #14322: KAFKA-15418: update statement on decompression

2023-09-01 Thread via GitHub
Cerchie opened a new pull request, #14322: URL: https://github.com/apache/kafka/pull/14322 The docs state:  -https://github.com/apache/kafka/blob/0912ca27e2a229d2ebe02f4d1dabc40ed5fab0bb/docs/design.html#L139-L140 However, brokers always perform some amount of batch decompressi

[GitHub] [kafka] cmccabe commented on a diff in pull request #14306: KAFKA-15369: Implement KIP-919: Allow AC to Talk Directly with Controllers

2023-09-01 Thread via GitHub
cmccabe commented on code in PR #14306: URL: https://github.com/apache/kafka/pull/14306#discussion_r1313363156 ## core/src/main/scala/kafka/server/AuthHelper.scala: ## @@ -130,4 +134,57 @@ class AuthHelper(authorizer: Option[Authorizer]) { } } + def computeDescribeClu

[jira] [Commented] (KAFKA-15418) Update statement on decompression location

2023-09-01 Thread Lucia Cerchie (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17761364#comment-17761364 ] Lucia Cerchie commented on KAFKA-15418: --- suggested replacement text: _Kafka suppo

[GitHub] [kafka] satishd merged pull request #14319: KAFKA-15427: Fix resource leak in integration tests for tiered storage

2023-09-01 Thread via GitHub
satishd merged PR #14319: URL: https://github.com/apache/kafka/pull/14319 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

[GitHub] [kafka] satishd commented on pull request #14319: KAFKA-15427: Fix resource leak in integration tests for tiered storage

2023-09-01 Thread via GitHub
satishd commented on PR #14319: URL: https://github.com/apache/kafka/pull/14319#issuecomment-1703109683 There are a few test failures unrelated to this change. Merging it to trunk and 3.6. -- This is an automated message from the Apache Git Service. To respond to the message, please log o

[GitHub] [kafka] CalvinConfluent commented on a diff in pull request #14321: KAFKA-14499: [6/N] Add MemberId and MemberEpoch to OffsetFetchRequest

2023-09-01 Thread via GitHub
CalvinConfluent commented on code in PR #14321: URL: https://github.com/apache/kafka/pull/14321#discussion_r1313307656 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/ConsumerGroup.java: ## @@ -538,17 +537,46 @@ public void validateOffsetCommit(

[GitHub] [kafka] jeffkbkim commented on a diff in pull request #14182: KAFKA 14515: Optimized Uniform Rack Aware Assignor

2023-09-01 Thread via GitHub
jeffkbkim commented on code in PR #14182: URL: https://github.com/apache/kafka/pull/14182#discussion_r1313294383 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/OptimizedUniformAssignmentBuilder.java: ## @@ -0,0 +1,390 @@ +/* + * Licensed to the Ap

[GitHub] [kafka] jeffkbkim commented on a diff in pull request #14182: KAFKA 14515: Optimized Uniform Rack Aware Assignor

2023-09-01 Thread via GitHub
jeffkbkim commented on code in PR #14182: URL: https://github.com/apache/kafka/pull/14182#discussion_r1313292815 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/assignor/OptimizedUniformAssignmentBuilderTest.java: ## @@ -0,0 +1,1070 @@ +/* + * Licensed to t

[GitHub] [kafka] gharris1727 commented on pull request #14316: KAFKA-14725: Interrupt source task poll thread when cancelled

2023-09-01 Thread via GitHub
gharris1727 commented on PR #14316: URL: https://github.com/apache/kafka/pull/14316#issuecomment-1703060492 @C0urante Thanks for mentioning `InterruptException`, that can certainly appear as a RuntimeException when plugins are using Kafka clients internally. You mention the converter

[GitHub] [kafka] jeffkbkim commented on a diff in pull request #14182: KAFKA 14515: Optimized Uniform Rack Aware Assignor

2023-09-01 Thread via GitHub
jeffkbkim commented on code in PR #14182: URL: https://github.com/apache/kafka/pull/14182#discussion_r1313285352 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/OptimizedUniformAssignmentBuilder.java: ## @@ -0,0 +1,390 @@ +/* + * Licensed to the Ap

[GitHub] [kafka] jeffkbkim commented on a diff in pull request #14182: KAFKA 14515: Optimized Uniform Rack Aware Assignor

2023-09-01 Thread via GitHub
jeffkbkim commented on code in PR #14182: URL: https://github.com/apache/kafka/pull/14182#discussion_r1313283639 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/UniformAssignor.java: ## @@ -0,0 +1,270 @@ +/* + * Licensed to the Apache Software Foun

[GitHub] [kafka] jeffkbkim commented on a diff in pull request #14182: KAFKA 14515: Optimized Uniform Rack Aware Assignor

2023-09-01 Thread via GitHub
jeffkbkim commented on code in PR #14182: URL: https://github.com/apache/kafka/pull/14182#discussion_r1313281824 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/OptimizedUniformAssignmentBuilder.java: ## @@ -0,0 +1,390 @@ +/* + * Licensed to the Ap

[GitHub] [kafka] rreddy-22 commented on a diff in pull request #14321: KAFKA-14499: [6/N] Add MemberId and MemberEpoch to OffsetFetchRequest

2023-09-01 Thread via GitHub
rreddy-22 commented on code in PR #14321: URL: https://github.com/apache/kafka/pull/14321#discussion_r1313249102 ## clients/src/main/resources/common/message/OffsetFetchRequest.json: ## @@ -47,8 +54,12 @@ ]}, { "name": "Groups", "type": "[]OffsetFetchRequestGroup", "ve

[GitHub] [kafka] rreddy-22 commented on a diff in pull request #14147: KAFKA-14502; Implement LeaveGroup protocol in new GroupCoordinator

2023-09-01 Thread via GitHub
rreddy-22 commented on code in PR #14147: URL: https://github.com/apache/kafka/pull/14147#discussion_r1313242009 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -2904,6 +2908,135 @@ private void validateGenericGroupHeartbeat

[GitHub] [kafka] rreddy-22 commented on a diff in pull request #14147: KAFKA-14502; Implement LeaveGroup protocol in new GroupCoordinator

2023-09-01 Thread via GitHub
rreddy-22 commented on code in PR #14147: URL: https://github.com/apache/kafka/pull/14147#discussion_r1313241768 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -2904,6 +2908,135 @@ private void validateGenericGroupHeartbeat

[GitHub] [kafka] rreddy-22 commented on a diff in pull request #14147: KAFKA-14502; Implement LeaveGroup protocol in new GroupCoordinator

2023-09-01 Thread via GitHub
rreddy-22 commented on code in PR #14147: URL: https://github.com/apache/kafka/pull/14147#discussion_r1313241475 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -2904,6 +2908,135 @@ private void validateGenericGroupHeartbeat

[GitHub] [kafka] rreddy-22 commented on a diff in pull request #14147: KAFKA-14502; Implement LeaveGroup protocol in new GroupCoordinator

2023-09-01 Thread via GitHub
rreddy-22 commented on code in PR #14147: URL: https://github.com/apache/kafka/pull/14147#discussion_r1313240470 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupCoordinatorShard.java: ## @@ -306,6 +308,22 @@ public CoordinatorResult commitOffset(

[GitHub] [kafka] jeffkbkim commented on a diff in pull request #14182: KAFKA 14515: Optimized Uniform Rack Aware Assignor

2023-09-01 Thread via GitHub
jeffkbkim commented on code in PR #14182: URL: https://github.com/apache/kafka/pull/14182#discussion_r1313213976 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/OptimizedUniformAssignmentBuilder.java: ## @@ -0,0 +1,399 @@ +/* + * Licensed to the Ap

[GitHub] [kafka] lianetm commented on pull request #14308: KAFKA-14965 - OffsetsRequestsManager implementation & API integration

2023-09-01 Thread via GitHub
lianetm commented on PR #14308: URL: https://github.com/apache/kafka/pull/14308#issuecomment-1702962811 Thanks for all the comment @junrao . All addressed. (I'm keeping an eye on the build and will update relevant results here) -- This is an automated message from the Apache Git Service.

[GitHub] [kafka] lianetm commented on a diff in pull request #14308: KAFKA-14965 - OffsetsRequestsManager implementation & API integration

2023-09-01 Thread via GitHub
lianetm commented on code in PR #14308: URL: https://github.com/apache/kafka/pull/14308#discussion_r1313203382 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/DefaultBackgroundThread.java: ## @@ -42,6 +42,7 @@ import static java.util.Objects.requireNonNull;

[GitHub] [kafka] lianetm commented on a diff in pull request #14308: KAFKA-14965 - OffsetsRequestsManager implementation & API integration

2023-09-01 Thread via GitHub
lianetm commented on code in PR #14308: URL: https://github.com/apache/kafka/pull/14308#discussion_r1313192482 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/OffsetsRequestManagerTest.java: ## @@ -0,0 +1,675 @@ +/* + * Licensed to the Apache Software Founda

[GitHub] [kafka] lianetm commented on a diff in pull request #14308: KAFKA-14965 - OffsetsRequestsManager implementation & API integration

2023-09-01 Thread via GitHub
lianetm commented on code in PR #14308: URL: https://github.com/apache/kafka/pull/14308#discussion_r1313167529 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/PrototypeAsyncConsumerTest.java: ## @@ -220,6 +225,95 @@ public void testAssignOnEmptyTopicInPartit

[GitHub] [kafka] omkreddy commented on a diff in pull request #14130: KAFKA-15273: Log common name of expired client certificates

2023-09-01 Thread via GitHub
omkreddy commented on code in PR #14130: URL: https://github.com/apache/kafka/pull/14130#discussion_r1313153829 ## clients/src/main/java/org/apache/kafka/common/security/ssl/CommonNameLoggingSslEngineFactory.java: ## @@ -0,0 +1,581 @@ +/* + * Licensed to the Apache Software Foun

[jira] [Updated] (KAFKA-15425) Compatibility break in Admin.listOffsets() (2)

2023-09-01 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton updated KAFKA-15425: -- Description: The behavioral change that warrants this ticket is identical to the change noted

[GitHub] [kafka] lianetm commented on a diff in pull request #14308: KAFKA-14965 - OffsetsRequestsManager implementation & API integration

2023-09-01 Thread via GitHub
lianetm commented on code in PR #14308: URL: https://github.com/apache/kafka/pull/14308#discussion_r1313134230 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/OffsetsRequestManagerTest.java: ## @@ -0,0 +1,675 @@ +/* + * Licensed to the Apache Software Founda

[GitHub] [kafka] dajac opened a new pull request, #14321: KAFKA-14499: [6/N] Add MemberId and MemberEpoch to OffsetFetchRequest

2023-09-01 Thread via GitHub
dajac opened a new pull request, #14321: URL: https://github.com/apache/kafka/pull/14321 This patch adds the MemberId and the MemberEpoch fields to the OffsetFetchRequest. Those fields will be populated when the new consumer group protocol is used to ensure that the member fetching the offs

[GitHub] [kafka] lianetm commented on a diff in pull request #14308: KAFKA-14965 - OffsetsRequestsManager implementation & API integration

2023-09-01 Thread via GitHub
lianetm commented on code in PR #14308: URL: https://github.com/apache/kafka/pull/14308#discussion_r1313096461 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/OffsetsRequestManagerTest.java: ## @@ -0,0 +1,675 @@ +/* + * Licensed to the Apache Software Founda

[GitHub] [kafka] lianetm commented on a diff in pull request #14308: KAFKA-14965 - OffsetsRequestsManager implementation & API integration

2023-09-01 Thread via GitHub
lianetm commented on code in PR #14308: URL: https://github.com/apache/kafka/pull/14308#discussion_r1313096461 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/OffsetsRequestManagerTest.java: ## @@ -0,0 +1,675 @@ +/* + * Licensed to the Apache Software Founda

[jira] [Commented] (KAFKA-15411) DelegationTokenEndToEndAuthorizationWithOwnerTest is Flaky

2023-09-01 Thread Proven Provenzano (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17761324#comment-17761324 ] Proven Provenzano commented on KAFKA-15411: --- A doc change which ran all the te

[GitHub] [kafka] lianetm commented on a diff in pull request #14308: KAFKA-14965 - OffsetsRequestsManager implementation & API integration

2023-09-01 Thread via GitHub
lianetm commented on code in PR #14308: URL: https://github.com/apache/kafka/pull/14308#discussion_r1313041137 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/OffsetsRequestManagerTest.java: ## @@ -0,0 +1,675 @@ +/* + * Licensed to the Apache Software Founda

[GitHub] [kafka] dajac merged pull request #14122: KAFKA-14500; [7/7] Refactor GroupMetadataManagerTest

2023-09-01 Thread via GitHub
dajac merged PR #14122: URL: https://github.com/apache/kafka/pull/14122 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

[GitHub] [kafka] ethaden commented on a diff in pull request #14130: KAFKA-15273: Log common name of expired client certificates

2023-09-01 Thread via GitHub
ethaden commented on code in PR #14130: URL: https://github.com/apache/kafka/pull/14130#discussion_r1313025236 ## clients/src/main/java/org/apache/kafka/common/security/ssl/CommonNameLoggingSslEngineFactory.java: ## @@ -0,0 +1,581 @@ +/* + * Licensed to the Apache Software Found

[GitHub] [kafka] abhijeetk88 commented on a diff in pull request #14113: KAFKA-15260: RLM Task should handle uninitialized RLMM for the associated topic-parititon

2023-09-01 Thread via GitHub
abhijeetk88 commented on code in PR #14113: URL: https://github.com/apache/kafka/pull/14113#discussion_r1313011389 ## core/src/test/java/kafka/log/remote/RemoteLogManagerTest.java: ## @@ -754,6 +755,62 @@ void testCopyLogSegmentsToRemoteShouldNotCopySegmentForFollower() throws

[GitHub] [kafka] yashmayya commented on a diff in pull request #14152: KAFKA-7438: Migrate WindowStoreBuilderTest from EasyMock to Mockito

2023-09-01 Thread via GitHub
yashmayya commented on code in PR #14152: URL: https://github.com/apache/kafka/pull/14152#discussion_r1313019711 ## streams/src/test/java/org/apache/kafka/streams/state/internals/WindowStoreBuilderTest.java: ## @@ -164,17 +160,6 @@ public void shouldThrowNullPointerIfInnerIsNull

[GitHub] [kafka] abhijeetk88 commented on a diff in pull request #14113: KAFKA-15260: RLM Task should handle uninitialized RLMM for the associated topic-parititon

2023-09-01 Thread via GitHub
abhijeetk88 commented on code in PR #14113: URL: https://github.com/apache/kafka/pull/14113#discussion_r1313011389 ## core/src/test/java/kafka/log/remote/RemoteLogManagerTest.java: ## @@ -754,6 +755,62 @@ void testCopyLogSegmentsToRemoteShouldNotCopySegmentForFollower() throws

[jira] [Resolved] (KAFKA-15399) Enable OffloadAndConsumeFromLeader test

2023-09-01 Thread Christo Lolov (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christo Lolov resolved KAFKA-15399. --- Resolution: Fixed > Enable OffloadAndConsumeFromLeader test > --

[jira] [Commented] (KAFKA-15399) Enable OffloadAndConsumeFromLeader test

2023-09-01 Thread Christo Lolov (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17761301#comment-17761301 ] Christo Lolov commented on KAFKA-15399: --- I have detailed the reason for the failin

[GitHub] [kafka] showuon commented on a diff in pull request #14113: KAFKA-15260: RLM Task should handle uninitialized RLMM for the associated topic-parititon

2023-09-01 Thread via GitHub
showuon commented on code in PR #14113: URL: https://github.com/apache/kafka/pull/14113#discussion_r1312968348 ## core/src/test/java/kafka/log/remote/RemoteLogManagerTest.java: ## @@ -754,6 +755,62 @@ void testCopyLogSegmentsToRemoteShouldNotCopySegmentForFollower() throws Exce

[GitHub] [kafka] clolov commented on a diff in pull request #14152: KAFKA-7438: Migrate WindowStoreBuilderTest from EasyMock to Mockito

2023-09-01 Thread via GitHub
clolov commented on code in PR #14152: URL: https://github.com/apache/kafka/pull/14152#discussion_r1312965699 ## streams/src/test/java/org/apache/kafka/streams/state/internals/WindowStoreBuilderTest.java: ## @@ -183,18 +168,7 @@ public void shouldThrowNullPointerIfTimeIsNull() {

[GitHub] [kafka] abhijeetk88 commented on a diff in pull request #14113: KAFKA-15260: RLM Task should handle uninitialized RLMM for the associated topic-parititon

2023-09-01 Thread via GitHub
abhijeetk88 commented on code in PR #14113: URL: https://github.com/apache/kafka/pull/14113#discussion_r1312963078 ## core/src/test/java/kafka/log/remote/RemoteLogManagerTest.java: ## @@ -754,6 +755,59 @@ void testCopyLogSegmentsToRemoteShouldNotCopySegmentForFollower() throws

[GitHub] [kafka] abhijeetk88 commented on a diff in pull request #14113: KAFKA-15260: RLM Task should handle uninitialized RLMM for the associated topic-parititon

2023-09-01 Thread via GitHub
abhijeetk88 commented on code in PR #14113: URL: https://github.com/apache/kafka/pull/14113#discussion_r1312962330 ## core/src/test/java/kafka/log/remote/RemoteLogManagerTest.java: ## @@ -754,6 +755,61 @@ void testCopyLogSegmentsToRemoteShouldNotCopySegmentForFollower() throws

[GitHub] [kafka] showuon commented on pull request #14320: KAFKA-15421: fix network thread leak in testThreadPoolResize

2023-09-01 Thread via GitHub
showuon commented on PR #14320: URL: https://github.com/apache/kafka/pull/14320#issuecomment-1702654646 @clolov @divijvaidya @satishd @kamalcph , call for review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [kafka] showuon commented on a diff in pull request #14320: KAFKA-15421: fix network thread leak in testThreadPoolResize

2023-09-01 Thread via GitHub
showuon commented on code in PR #14320: URL: https://github.com/apache/kafka/pull/14320#discussion_r1312952812 ## core/src/test/scala/unit/kafka/network/SocketServerTest.scala: ## @@ -94,6 +93,7 @@ class SocketServerTest { @BeforeEach def setUp(): Unit = { +server.en

[GitHub] [kafka] showuon opened a new pull request, #14320: KAFKA-15421: fix network thread leak

2023-09-01 Thread via GitHub
showuon opened a new pull request, #14320: URL: https://github.com/apache/kafka/pull/14320 In `SocketServerTest`, we create `SocketServer` and `enableRequestProcessing` on each test class initialization. That's fine since we shutdown it in `@AfterEach`. The issue we have is we disabled 2 te

[jira] [Assigned] (KAFKA-15421) Enable DynamicBrokerReconfigurationTest#testThreadPoolResize test

2023-09-01 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen reassigned KAFKA-15421: - Assignee: Luke Chen > Enable DynamicBrokerReconfigurationTest#testThreadPoolResize test > -

[GitHub] [kafka] showuon commented on pull request #14319: KAFKA-15427: Fix resource leak in integration tests for tiered storage

2023-09-01 Thread via GitHub
showuon commented on PR #14319: URL: https://github.com/apache/kafka/pull/14319#issuecomment-1702637694 @clolov , there is already a JIRA for this issue: [KAFKA-15399](https://issues.apache.org/jira/browse/KAFKA-15399). Maybe you could close one as duplicate. Thanks. -- This is an automa

[GitHub] [kafka] dajac commented on a diff in pull request #14271: KAFKA-14503: Implement ListGroups

2023-09-01 Thread via GitHub
dajac commented on code in PR #14271: URL: https://github.com/apache/kafka/pull/14271#discussion_r1312918434 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupCoordinatorService.java: ## @@ -428,9 +430,44 @@ public CompletableFuture listGroups(

[GitHub] [kafka] satishd commented on a diff in pull request #14319: KAFKA-15427: Fix resource leak in integration tests for tiered storage

2023-09-01 Thread via GitHub
satishd commented on code in PR #14319: URL: https://github.com/apache/kafka/pull/14319#discussion_r1312908602 ## storage/src/test/java/org/apache/kafka/server/log/remote/metadata/storage/TopicBasedRemoteLogMetadataManagerWrapperWithHarness.java: ## @@ -93,7 +93,7 @@ public long

[GitHub] [kafka] yashmayya commented on pull request #14279: KAFKA-15179: Add integration tests for the file sink and source connectors

2023-09-01 Thread via GitHub
yashmayya commented on PR #14279: URL: https://github.com/apache/kafka/pull/14279#issuecomment-1702586926 @C0urante would you be able to take a look at this sometime (for context - we'd discussed this earlier [here](https://github.com/apache/kafka/pull/13945#pullrequestreview-1524345339))?

[GitHub] [kafka] yashmayya commented on a diff in pull request #14304: KAFKA-13328, KAFKA-13329 (1): Add preflight validations for key, value, and header converter classes

2023-09-01 Thread via GitHub
yashmayya commented on code in PR #14304: URL: https://github.com/apache/kafka/pull/14304#discussion_r1312900639 ## connect/runtime/src/main/java/org/apache/kafka/connect/util/InstantiableClassValidator.java: ## @@ -0,0 +1,47 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] yashmayya commented on a diff in pull request #14304: KAFKA-13328, KAFKA-13329 (1): Add preflight validations for key, value, and header converter classes

2023-09-01 Thread via GitHub
yashmayya commented on code in PR #14304: URL: https://github.com/apache/kafka/pull/14304#discussion_r1312897811 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/ConnectorConfig.java: ## @@ -82,17 +86,29 @@ public class ConnectorConfig extends AbstractConfig {

  1   2   >