[GitHub] [kafka] cmccabe commented on a diff in pull request #13826: KAFKA-15060: fix the ApiVersionManager interface

2023-06-15 Thread via GitHub
cmccabe commented on code in PR #13826: URL: https://github.com/apache/kafka/pull/13826#discussion_r1230241309 ## core/src/main/scala/kafka/server/ControllerApis.scala: ## @@ -441,23 +441,14 @@ class ControllerApis(val requestChannel: RequestChannel, if (apiVersionRequest.h

[jira] [Assigned] (KAFKA-15094) Add RemoteIndexCache metrics like misses/evictions/load-failures.

2023-06-15 Thread Satish Duggana (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Satish Duggana reassigned KAFKA-15094: -- Assignee: Abhijeet Kumar > Add RemoteIndexCache metrics like misses/evictions/load-fa

[GitHub] [kafka] satishd commented on pull request #13850: KAFKA-15084: Remove lock contention from RemoteIndexCache

2023-06-15 Thread via GitHub
satishd commented on PR #13850: URL: https://github.com/apache/kafka/pull/13850#issuecomment-1594160765 Thanks for the clarification @ben-manes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[jira] [Created] (KAFKA-15094) Add RemoteIndexCache metrics like misses/evictions/load-failures.

2023-06-15 Thread Satish Duggana (Jira)
Satish Duggana created KAFKA-15094: -- Summary: Add RemoteIndexCache metrics like misses/evictions/load-failures. Key: KAFKA-15094 URL: https://issues.apache.org/jira/browse/KAFKA-15094 Project: Kafka

[GitHub] [kafka] ben-manes commented on pull request #13850: KAFKA-15084: Remove lock contention from RemoteIndexCache

2023-06-15 Thread via GitHub
ben-manes commented on PR #13850: URL: https://github.com/apache/kafka/pull/13850#issuecomment-1594120179 fyi, Caffeine only depends on annotations for static analysis. This allows users to catch mistakes easier, e.g. if using a null checker. Traditionally these could be marked

[GitHub] [kafka] vamossagar12 commented on pull request #13801: KAFKA-15018: Failing offset flush for EOS when secondary offset store writes fails for tombstone records

2023-06-15 Thread via GitHub
vamossagar12 commented on PR #13801: URL: https://github.com/apache/kafka/pull/13801#issuecomment-1594109033 The build was aborted. But the failed tests seem unrelated as such to this PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [kafka] showuon commented on pull request #13828: KAFKA-15066: add "remote.log.metadata.manager.listener.name" config to rlmm

2023-06-15 Thread via GitHub
showuon commented on PR #13828: URL: https://github.com/apache/kafka/pull/13828#issuecomment-1594043903 Still cannot get a healthy CI build results. Try to rebase to the latest trunk. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [kafka] github-actions[bot] commented on pull request #13395: MINOR: don't disconnect stale controller if the network client is res…

2023-06-15 Thread via GitHub
github-actions[bot] commented on PR #13395: URL: https://github.com/apache/kafka/pull/13395#issuecomment-1594037886 This PR is being marked as stale since it has not had any activity in 90 days. If you would like to keep this PR alive, please ask a committer for review. If the PR has merge

[GitHub] [kafka] jolshan commented on a diff in pull request #13591: KAFKA-14831: Illegal state errors should be fatal in transactional producer

2023-06-15 Thread via GitHub
jolshan commented on code in PR #13591: URL: https://github.com/apache/kafka/pull/13591#discussion_r1231739915 ## clients/src/main/java/org/apache/kafka/clients/producer/internals/TransactionManager.java: ## @@ -609,7 +673,7 @@ public synchronized void handleCompletedBatch(Produ

[GitHub] [kafka] Owen-CH-Leung commented on pull request #13773: KAFKA-14712: Produce correct error msg with correct metadataversion

2023-06-15 Thread via GitHub
Owen-CH-Leung commented on PR #13773: URL: https://github.com/apache/kafka/pull/13773#issuecomment-1594025469 I saw there're a few unit test failed in the CI job (e.g. `MirrorConnectorsIntegrationExactlyOnceTest.testReplication()`, but I don't think it's related to my changes ? Can someone

[GitHub] [kafka] kirktrue commented on a diff in pull request #13591: KAFKA-14831: Illegal state errors should be fatal in transactional producer

2023-06-15 Thread via GitHub
kirktrue commented on code in PR #13591: URL: https://github.com/apache/kafka/pull/13591#discussion_r1231563134 ## clients/src/test/java/org/apache/kafka/clients/producer/internals/TransactionManagerTest.java: ## @@ -3405,6 +3405,59 @@ MAX_REQUEST_SIZE, ACKS_ALL, MAX_RETRIES, ne

[GitHub] [kafka] jolshan commented on pull request #13796: KAFKA-14034 Idempotent producer should wait for preceding in-flight b…

2023-06-15 Thread via GitHub
jolshan commented on PR #13796: URL: https://github.com/apache/kafka/pull/13796#issuecomment-1593699560 I suppose my general my concern is along that line. If we are in a state that is non-retriable it seems like it would be more likely to not have a way to recover. That's just my general f

[GitHub] [kafka] jolshan commented on pull request #13721: KAFKA-14782: Implementation Details Different from Documentation (del…

2023-06-15 Thread via GitHub
jolshan commented on PR #13721: URL: https://github.com/apache/kafka/pull/13721#issuecomment-1593691424 Thanks for bringing this up. Will take a look when I can. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [kafka] rishiraj88 commented on pull request #13756: KAFKA-14936: Add On Disk Time Ordered Buffer (1/N)

2023-06-15 Thread via GitHub
rishiraj88 commented on PR #13756: URL: https://github.com/apache/kafka/pull/13756#issuecomment-1593612430 Did we get an answer to the question from @cadonna ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[jira] [Resolved] (KAFKA-14278) Convert INVALID_PRODUCER_EPOCH into PRODUCER_FENCED TxnOffsetCommit

2023-06-15 Thread Lucas Brutschy (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lucas Brutschy resolved KAFKA-14278. Resolution: Fixed > Convert INVALID_PRODUCER_EPOCH into PRODUCER_FENCED TxnOffsetCommit >

[GitHub] [kafka] maniekes commented on pull request #13321: MINOR: added compatibility for MinGW

2023-06-15 Thread via GitHub
maniekes commented on PR #13321: URL: https://github.com/apache/kafka/pull/13321#issuecomment-1593543359 @divijvaidya, sure, it's quite simple. 1. open git bash, checkout trunk 2. build ``` ./gradlew jar -PscalaVersion=2.13.10 ``` 3. checkout trunk before my commits: `git

[jira] [Commented] (KAFKA-15050) Prompts in the quickstarts

2023-06-15 Thread Joobi S B (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17733196#comment-17733196 ] Joobi S B commented on KAFKA-15050: --- Hi [~tombentley] , since this was not picked up i

[GitHub] [kafka] joobisb opened a new pull request, #13862: KAFKA-15050: format the prompts in the quickstart

2023-06-15 Thread via GitHub
joobisb opened a new pull request, #13862: URL: https://github.com/apache/kafka/pull/13862 This patch formats the prompts in quickstart and in kafka streams quickstart ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test

[GitHub] [kafka] stevenbooke commented on pull request #13842: KAFKA-14995: Automate asf.yaml collaborators refresh

2023-06-15 Thread via GitHub
stevenbooke commented on PR #13842: URL: https://github.com/apache/kafka/pull/13842#issuecomment-1593525011 Could someone review this PR and leave some feedback please? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

[GitHub] [kafka] mimaison commented on a diff in pull request #13257: MINOR: Add ZK migration docs to the packaged docs

2023-06-15 Thread via GitHub
mimaison commented on code in PR #13257: URL: https://github.com/apache/kafka/pull/13257#discussion_r1231323517 ## docs/ops.html: ## @@ -3594,22 +3594,218 @@ Deploying Considerations -Kafka server's process.role should be set to either broker or controller but not bo

[GitHub] [kafka] gharris1727 commented on pull request #13816: MINOR: Optimize runtime of MM2 integration tests by batching transactions

2023-06-15 Thread via GitHub
gharris1727 commented on PR #13816: URL: https://github.com/apache/kafka/pull/13816#issuecomment-1593378454 I believe this PR will solve the current MirrorConnectorsIntegrationTransactionsTest#testReplication instability that appears on trunk. I'm getting < 1% failure rate in a 30% CPU test

[GitHub] [kafka] Owen-CH-Leung commented on a diff in pull request #13773: KAFKA-14712: Produce correct error msg with correct metadataversion

2023-06-15 Thread via GitHub
Owen-CH-Leung commented on code in PR #13773: URL: https://github.com/apache/kafka/pull/13773#discussion_r1231250401 ## metadata/src/main/java/org/apache/kafka/image/writer/ImageWriterOptions.java: ## @@ -45,6 +46,7 @@ public Builder setMetadataVersion(MetadataVersion metadataV

[GitHub] [kafka] Owen-CH-Leung commented on a diff in pull request #13773: KAFKA-14712: Produce correct error msg with correct metadataversion

2023-06-15 Thread via GitHub
Owen-CH-Leung commented on code in PR #13773: URL: https://github.com/apache/kafka/pull/13773#discussion_r1231247557 ## metadata/src/main/java/org/apache/kafka/image/writer/ImageWriterOptions.java: ## @@ -29,6 +29,7 @@ public final class ImageWriterOptions { public static

[GitHub] [kafka] hudeqi commented on pull request #13719: MINOR:Fix illogical log in fetchOffsetAndTruncate method

2023-06-15 Thread via GitHub
hudeqi commented on PR #13719: URL: https://github.com/apache/kafka/pull/13719#issuecomment-1593332941 > Also, @hudeqi please don't use force pushes because it makes it impossible to go back to previous commits which sometimes needed in reviews to get the proper context. When we merge a PR

[GitHub] [kafka] satishd commented on pull request #13850: KAFKA-15084: Remove lock contention from RemoteIndexCache

2023-06-15 Thread via GitHub
satishd commented on PR #13850: URL: https://github.com/apache/kafka/pull/13850#issuecomment-1593330285 I will review this PR in a couple of days. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] viktorsomogyi commented on pull request #13719: MINOR:Fix illogical log in fetchOffsetAndTruncate method

2023-06-15 Thread via GitHub
viktorsomogyi commented on PR #13719: URL: https://github.com/apache/kafka/pull/13719#issuecomment-1593317993 Also, @hudeqi please don't use force pushes because it makes it impossible to go back to previous commits which sometimes needed in reviews to get the proper context. When we merge

[jira] [Assigned] (KAFKA-14712) Confusing error when writing downgraded FeatureImage

2023-06-15 Thread Owen C.H. Leung (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Owen C.H. Leung reassigned KAFKA-14712: --- Assignee: Owen C.H. Leung > Confusing error when writing downgraded FeatureImage >

[GitHub] [kafka] hudeqi commented on pull request #13705: MINOR:Refactor the metric names into constants in ReplicaManager

2023-06-15 Thread via GitHub
hudeqi commented on PR #13705: URL: https://github.com/apache/kafka/pull/13705#issuecomment-1593287087 > > It seems that the failure of CI check has nothing to do with this pr. @divijvaidya > > Hmm...not sure about that. There are 99 tests failing and I am not comfortable saying that

[GitHub] [kafka] divijvaidya commented on pull request #13705: MINOR:Refactor the metric names into constants in ReplicaManager

2023-06-15 Thread via GitHub
divijvaidya commented on PR #13705: URL: https://github.com/apache/kafka/pull/13705#issuecomment-1593276292 > It seems that the failure of CI check has nothing to do with this pr. @divijvaidya Hmm...not sure about that. There are 99 tests failing and I am not comfortable saying that

[GitHub] [kafka] viktorsomogyi commented on pull request #13421: KAFKA-14824: ReplicaAlterLogDirsThread may cause serious disk growing in case of potential exception

2023-06-15 Thread via GitHub
viktorsomogyi commented on PR #13421: URL: https://github.com/apache/kafka/pull/13421#issuecomment-1593258910 It seems like @clolov is right, I tested it both in quorum and zk mode, Kafka successfully reconciles the questionable case (when X-1 on B comes back after A has compacted the logs)

[GitHub] [kafka] divijvaidya merged pull request #13861: MINOR: Change file permissions of reviewers.py to make it executable

2023-06-15 Thread via GitHub
divijvaidya merged PR #13861: URL: https://github.com/apache/kafka/pull/13861 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apa

[GitHub] [kafka] mimaison commented on a diff in pull request #13257: MINOR: Add ZK migration docs to the packaged docs

2023-06-15 Thread via GitHub
mimaison commented on code in PR #13257: URL: https://github.com/apache/kafka/pull/13257#discussion_r1231157115 ## docs/ops.html: ## @@ -3550,14 +3550,14 @@ Deploying Considerations -Kafka server's process.role should be set to either broker or controller but not bot

[GitHub] [kafka] divijvaidya commented on pull request #13861: MINOR: Change file permissions of reviewers.py to make it executable

2023-06-15 Thread via GitHub
divijvaidya commented on PR #13861: URL: https://github.com/apache/kafka/pull/13861#issuecomment-1593239019 Merging this in since there are no tests associated with this script. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [kafka] hudeqi commented on pull request #13705: MINOR:Refactor the metric names into constants in ReplicaManager

2023-06-15 Thread via GitHub
hudeqi commented on PR #13705: URL: https://github.com/apache/kafka/pull/13705#issuecomment-1593228168 > I triggered the CI tests again since last run wasn't complete. Will merge this in after they are successful. It seems that the failure of CI check has nothing to do with this pr.

[GitHub] [kafka] divijvaidya commented on pull request #13321: MINOR: added compatibility for MinGW

2023-06-15 Thread via GitHub
divijvaidya commented on PR #13321: URL: https://github.com/apache/kafka/pull/13321#issuecomment-1593219444 @maniekes can you please describe the testing steps and the results of the tests that you did to test the latest version of this change? -- This is an automated message from the Apa

[GitHub] [kafka] mumrah commented on pull request #13257: MINOR: Add ZK migration docs to the packaged docs

2023-06-15 Thread via GitHub
mumrah commented on PR #13257: URL: https://github.com/apache/kafka/pull/13257#issuecomment-1593205135 @mimaison can you take another look at this? Sorry for the excessive delay on this one, I'll open a kafka-site PR after this is merged to trunk and 3.5 -- This is an automated message fr

[GitHub] [kafka] mumrah commented on a diff in pull request #13773: KAFKA-14712: Produce correct error msg with correct metadataversion

2023-06-15 Thread via GitHub
mumrah commented on code in PR #13773: URL: https://github.com/apache/kafka/pull/13773#discussion_r1231113907 ## metadata/src/main/java/org/apache/kafka/image/writer/ImageWriterOptions.java: ## @@ -45,6 +46,7 @@ public Builder setMetadataVersion(MetadataVersion metadataVersion)

[GitHub] [kafka] maniekes commented on pull request #13321: MINOR: added compatibility for MinGW

2023-06-15 Thread via GitHub
maniekes commented on PR #13321: URL: https://github.com/apache/kafka/pull/13321#issuecomment-1593129660 hi @divijvaidya, i pushed changes with new variable name. since in gradlew there is support for MSYS, i have also added it for check. hope this is fine? -- This is an automated mess

[GitHub] [kafka] dajac commented on a diff in pull request #13848: MINOR: Make offsets topic creation more reliable in tests (zk mode)

2023-06-15 Thread via GitHub
dajac commented on code in PR #13848: URL: https://github.com/apache/kafka/pull/13848#discussion_r1231056163 ## core/src/test/scala/unit/kafka/utils/TestUtils.scala: ## @@ -611,11 +611,33 @@ object TestUtils extends Logging { */ def createOffsetsTopic(zkClient: KafkaZkCl

[GitHub] [kafka] dajac commented on a diff in pull request #13848: MINOR: Make offsets topic creation more reliable in tests (zk mode)

2023-06-15 Thread via GitHub
dajac commented on code in PR #13848: URL: https://github.com/apache/kafka/pull/13848#discussion_r1231061065 ## core/src/test/scala/unit/kafka/utils/TestUtils.scala: ## @@ -611,11 +611,33 @@ object TestUtils extends Logging { */ def createOffsetsTopic(zkClient: KafkaZkCl

[GitHub] [kafka] maniekes opened a new pull request, #13321: MINOR: added compatibility for MinGW

2023-06-15 Thread via GitHub
maniekes opened a new pull request, #13321: URL: https://github.com/apache/kafka/pull/13321 kafka class runner is not working from MINGW/Git Bash on Windows: ```bash rafal@rafal-laptok MINGW64 /c/rafal/git/configs (master) $ "$KAFKA_DIR/bin/kafka-server-start.sh" server.propertie

[GitHub] [kafka] dajac commented on a diff in pull request #13848: MINOR: Make offsets topic creation more reliable in tests (zk mode)

2023-06-15 Thread via GitHub
dajac commented on code in PR #13848: URL: https://github.com/apache/kafka/pull/13848#discussion_r1231056163 ## core/src/test/scala/unit/kafka/utils/TestUtils.scala: ## @@ -611,11 +611,33 @@ object TestUtils extends Logging { */ def createOffsetsTopic(zkClient: KafkaZkCl

[GitHub] [kafka] divijvaidya commented on pull request #13852: KAFKA-15086:Set a reasonable segment size upper limit for MM2 internal topics

2023-06-15 Thread via GitHub
divijvaidya commented on PR #13852: URL: https://github.com/apache/kafka/pull/13852#issuecomment-1593109309 I am not familiar with MM2 @hudeqi. I will leave it up to the experts such as @C0urante and @mimaison to look into this one. -- This is an automated message from the Apache Git Serv

[GitHub] [kafka] maniekes closed pull request #13321: MINOR: added compatibility for MinGW

2023-06-15 Thread via GitHub
maniekes closed pull request #13321: MINOR: added compatibility for MinGW URL: https://github.com/apache/kafka/pull/13321 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

[GitHub] [kafka] divijvaidya opened a new pull request, #13861: MINOR: Change file permissions of reviewers.py to make it executable

2023-06-15 Thread via GitHub
divijvaidya opened a new pull request, #13861: URL: https://github.com/apache/kafka/pull/13861 Before this PR the file permissions did not allow this file to be an executable: ``` $ ./reviewers.py zsh: permission denied: ./reviewers.py ``` ``` $ ls -ld reviewers.py |awk

[GitHub] [kafka] mimaison opened a new pull request, #13860: KAFKA-15093: Add 3.4 and 3.5 Streams upgrade system tests

2023-06-15 Thread via GitHub
mimaison opened a new pull request, #13860: URL: https://github.com/apache/kafka/pull/13860 ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgr

[GitHub] [kafka] divijvaidya commented on a diff in pull request #13848: MINOR: Make offsets topic creation more reliable in tests (zk mode)

2023-06-15 Thread via GitHub
divijvaidya commented on code in PR #13848: URL: https://github.com/apache/kafka/pull/13848#discussion_r1231007534 ## core/src/test/scala/unit/kafka/utils/TestUtils.scala: ## @@ -611,11 +611,33 @@ object TestUtils extends Logging { */ def createOffsetsTopic(zkClient: Kaf

[GitHub] [kafka] dajac commented on a diff in pull request #13848: MINOR: Make offsets topic creation more reliable in tests (zk mode)

2023-06-15 Thread via GitHub
dajac commented on code in PR #13848: URL: https://github.com/apache/kafka/pull/13848#discussion_r1230916047 ## core/src/test/scala/unit/kafka/utils/TestUtils.scala: ## @@ -611,11 +611,33 @@ object TestUtils extends Logging { */ def createOffsetsTopic(zkClient: KafkaZkCl

[GitHub] [kafka] divijvaidya merged pull request #13711: KAFKA-14133: Migrate StandbyTaskCreator mock in TaskManagerTest to Mockito

2023-06-15 Thread via GitHub
divijvaidya merged PR #13711: URL: https://github.com/apache/kafka/pull/13711 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apa

[GitHub] [kafka] divijvaidya commented on pull request #13711: KAFKA-14133: Migrate StandbyTaskCreator mock in TaskManagerTest to Mockito

2023-06-15 Thread via GitHub
divijvaidya commented on PR #13711: URL: https://github.com/apache/kafka/pull/13711#issuecomment-1592990499 Failing tests are not related. Merging this code. ``` [Build / JDK 11 and Scala 2.13 / org.apache.kafka.connect.mirror.integration.IdentityReplicationIntegrationTest.testReplicat

[GitHub] [kafka] hudeqi commented on a diff in pull request #13719: MINOR:Fix illogical log in fetchOffsetAndTruncate method

2023-06-15 Thread via GitHub
hudeqi commented on code in PR #13719: URL: https://github.com/apache/kafka/pull/13719#discussion_r1230955995 ## core/src/main/scala/kafka/server/AbstractFetcherThread.scala: ## @@ -680,12 +680,16 @@ abstract class AbstractFetcherThread(name: String, */ val offset

[GitHub] [kafka] dajac commented on a diff in pull request #13812: KAFKA-14462; [18/N] Add GroupCoordinatorService

2023-06-15 Thread via GitHub
dajac commented on code in PR #13812: URL: https://github.com/apache/kafka/pull/13812#discussion_r1230946396 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupCoordinatorServiceTest.java: ## @@ -0,0 +1,269 @@ +/* + * Licensed to the Apache Software Found

[GitHub] [kafka] dajac commented on a diff in pull request #13812: KAFKA-14462; [18/N] Add GroupCoordinatorService

2023-06-15 Thread via GitHub
dajac commented on code in PR #13812: URL: https://github.com/apache/kafka/pull/13812#discussion_r1230942477 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/ReplicatedGroupCoordinatorTest.java: ## @@ -0,0 +1,308 @@ +/* + * Licensed to the Apache Software Fo

[GitHub] [kafka] dajac commented on a diff in pull request #13812: KAFKA-14462; [18/N] Add GroupCoordinatorService

2023-06-15 Thread via GitHub
dajac commented on code in PR #13812: URL: https://github.com/apache/kafka/pull/13812#discussion_r1230940881 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupCoordinatorServiceTest.java: ## @@ -0,0 +1,269 @@ +/* + * Licensed to the Apache Software Found

[GitHub] [kafka] dajac commented on a diff in pull request #13812: KAFKA-14462; [18/N] Add GroupCoordinatorService

2023-06-15 Thread via GitHub
dajac commented on code in PR #13812: URL: https://github.com/apache/kafka/pull/13812#discussion_r1230939154 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupCoordinatorService.java: ## @@ -0,0 +1,581 @@ +/* + * Licensed to the Apache Software Foundatio

[GitHub] [kafka] dajac commented on a diff in pull request #13812: KAFKA-14462; [18/N] Add GroupCoordinatorService

2023-06-15 Thread via GitHub
dajac commented on code in PR #13812: URL: https://github.com/apache/kafka/pull/13812#discussion_r1230937946 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupCoordinatorService.java: ## @@ -0,0 +1,581 @@ +/* + * Licensed to the Apache Software Foundatio

[GitHub] [kafka] dajac commented on a diff in pull request #13812: KAFKA-14462; [18/N] Add GroupCoordinatorService

2023-06-15 Thread via GitHub
dajac commented on code in PR #13812: URL: https://github.com/apache/kafka/pull/13812#discussion_r1230937577 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupCoordinatorService.java: ## @@ -0,0 +1,581 @@ +/* + * Licensed to the Apache Software Foundatio

[GitHub] [kafka] machi1990 commented on pull request #13665: KAFKA-12485: Speed up Consumer#committed by returning cached offsets for owned partitions

2023-06-15 Thread via GitHub
machi1990 commented on PR #13665: URL: https://github.com/apache/kafka/pull/13665#issuecomment-1592956695 > LGTM! Just one minor comment. After addressing it, I'll check the CI build result tomorrow. Thank you for the patch! Thank you so much for the review and help on this @showuon

[GitHub] [kafka] machi1990 commented on a diff in pull request #13665: KAFKA-12485: Speed up Consumer#committed by returning cached offsets for owned partitions

2023-06-15 Thread via GitHub
machi1990 commented on code in PR #13665: URL: https://github.com/apache/kafka/pull/13665#discussion_r1230936981 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinatorTest.java: ## @@ -3114,6 +3149,54 @@ public void testFetchCommittedOffsets() {

[GitHub] [kafka] dajac commented on a diff in pull request #13812: KAFKA-14462; [18/N] Add GroupCoordinatorService

2023-06-15 Thread via GitHub
dajac commented on code in PR #13812: URL: https://github.com/apache/kafka/pull/13812#discussion_r1230936129 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupCoordinatorService.java: ## @@ -0,0 +1,581 @@ +/* + * Licensed to the Apache Software Foundatio

[GitHub] [kafka] showuon commented on pull request #13850: KAFKA-15084: Remove lock contention from RemoteIndexCache

2023-06-15 Thread via GitHub
showuon commented on PR #13850: URL: https://github.com/apache/kafka/pull/13850#issuecomment-1592951877 I'll check it this week. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

[GitHub] [kafka] showuon commented on a diff in pull request #13665: KAFKA-12485: Speed up Consumer#committed by returning cached offsets for owned partitions

2023-06-15 Thread via GitHub
showuon commented on code in PR #13665: URL: https://github.com/apache/kafka/pull/13665#discussion_r1230929414 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinatorTest.java: ## @@ -3114,6 +3149,54 @@ public void testFetchCommittedOffsets() {

[GitHub] [kafka] dengziming commented on a diff in pull request #13562: KAFKA-14581: Moving GetOffsetShell to tools

2023-06-15 Thread via GitHub
dengziming commented on code in PR #13562: URL: https://github.com/apache/kafka/pull/13562#discussion_r1230929224 ## tools/src/main/java/org/apache/kafka/tools/GetOffsetShell.java: ## @@ -0,0 +1,340 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more +

[GitHub] [kafka] dajac commented on a diff in pull request #13719: MINOR:Fix illogical log in fetchOffsetAndTruncate method

2023-06-15 Thread via GitHub
dajac commented on code in PR #13719: URL: https://github.com/apache/kafka/pull/13719#discussion_r1230925717 ## core/src/main/scala/kafka/server/AbstractFetcherThread.scala: ## @@ -680,12 +680,16 @@ abstract class AbstractFetcherThread(name: String, */ val offsetA

[GitHub] [kafka] machi1990 commented on pull request #13665: KAFKA-12485: Speed up Consumer#committed by returning cached offsets for owned partitions

2023-06-15 Thread via GitHub
machi1990 commented on PR #13665: URL: https://github.com/apache/kafka/pull/13665#issuecomment-1592934743 Thanks @showuon for your review on this. I've addressed the comments. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

[GitHub] [kafka] machi1990 commented on a diff in pull request #13665: KAFKA-12485: Speed up Consumer#committed by returning cached offsets for owned partitions

2023-06-15 Thread via GitHub
machi1990 commented on code in PR #13665: URL: https://github.com/apache/kafka/pull/13665#discussion_r1230920423 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinatorTest.java: ## @@ -3818,13 +3899,16 @@ private void gracefulCloseTest(ConsumerC

[GitHub] [kafka] machi1990 commented on a diff in pull request #13665: KAFKA-12485: Speed up Consumer#committed by returning cached offsets for owned partitions

2023-06-15 Thread via GitHub
machi1990 commented on code in PR #13665: URL: https://github.com/apache/kafka/pull/13665#discussion_r1230918544 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinatorTest.java: ## @@ -3114,6 +3142,54 @@ public void testFetchCommittedOffsets() {

[GitHub] [kafka] machi1990 commented on a diff in pull request #13665: KAFKA-12485: Speed up Consumer#committed by returning cached offsets for owned partitions

2023-06-15 Thread via GitHub
machi1990 commented on code in PR #13665: URL: https://github.com/apache/kafka/pull/13665#discussion_r1230916909 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinatorTest.java: ## @@ -2521,11 +2524,36 @@ public void testCommitOffsetMetadata() {

[GitHub] [kafka] dajac commented on a diff in pull request #13848: MINOR: Make offsets topic creation more reliable in tests (zk mode)

2023-06-15 Thread via GitHub
dajac commented on code in PR #13848: URL: https://github.com/apache/kafka/pull/13848#discussion_r1230916047 ## core/src/test/scala/unit/kafka/utils/TestUtils.scala: ## @@ -611,11 +611,33 @@ object TestUtils extends Logging { */ def createOffsetsTopic(zkClient: KafkaZkCl

[GitHub] [kafka] showuon commented on a diff in pull request #13665: KAFKA-12485: Speed up Consumer#committed by returning cached offsets for owned partitions

2023-06-15 Thread via GitHub
showuon commented on code in PR #13665: URL: https://github.com/apache/kafka/pull/13665#discussion_r1230868853 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinatorTest.java: ## @@ -2521,11 +2524,36 @@ public void testCommitOffsetMetadata() {

[GitHub] [kafka] divijvaidya commented on pull request #13782: KAFKA-9693: Kafka latency spikes caused by log segment flush on roll

2023-06-15 Thread via GitHub
divijvaidya commented on PR #13782: URL: https://github.com/apache/kafka/pull/13782#issuecomment-1592865706 @showuon - with this change we don't have consistent data on different flushed files on disk (since earlier they were flushed together but now it's done async). I want to ensure that

[GitHub] [kafka] divijvaidya commented on pull request #13582: MINOR: Fix lossy conversions flagged by Java 20

2023-06-15 Thread via GitHub
divijvaidya commented on PR #13582: URL: https://github.com/apache/kafka/pull/13582#issuecomment-1592854559 @ijuma are we waiting to get additional pair of eyes on this one? I am comfortable with merging this PR in it's current form. -- This is an automated message from the Apache Git Ser

[GitHub] [kafka] divijvaidya commented on pull request #13321: MINOR: added compatibility for MinGW

2023-06-15 Thread via GitHub
divijvaidya commented on PR #13321: URL: https://github.com/apache/kafka/pull/13321#issuecomment-1592850958 Hi @maniekes Seems like, we do indeed want to support MinGW (on a best effort basis since we can't test it) as per https://github.com/apache/kafka/blob/505c7b64875ba05301d933a9b73

[GitHub] [kafka] mimaison opened a new pull request, #13859: KAFKA-15093: Add 3.4 and 3.5 to core upgrade and compatibility tests

2023-06-15 Thread via GitHub
mimaison opened a new pull request, #13859: URL: https://github.com/apache/kafka/pull/13859 ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgr

[jira] [Created] (KAFKA-15093) Add 3.5.0 to broker/client and streams upgrade/compatibility tests

2023-06-15 Thread Mickael Maison (Jira)
Mickael Maison created KAFKA-15093: -- Summary: Add 3.5.0 to broker/client and streams upgrade/compatibility tests Key: KAFKA-15093 URL: https://issues.apache.org/jira/browse/KAFKA-15093 Project: Kafka

[GitHub] [kafka] divijvaidya commented on pull request #13850: KAFKA-15084: Remove lock contention from RemoteIndexCache

2023-06-15 Thread via GitHub
divijvaidya commented on PR #13850: URL: https://github.com/apache/kafka/pull/13850#issuecomment-1592791383 @satishd @showuon please take a look at this when you get a chance. We have identified this as a major problem while running Tiered Storage in production. -- This is an automated m

[GitHub] [kafka] divijvaidya commented on a diff in pull request #13848: MINOR: Make offsets topic creation more reliable in tests (zk mode)

2023-06-15 Thread via GitHub
divijvaidya commented on code in PR #13848: URL: https://github.com/apache/kafka/pull/13848#discussion_r1230801748 ## core/src/test/scala/unit/kafka/utils/TestUtils.scala: ## @@ -611,11 +611,33 @@ object TestUtils extends Logging { */ def createOffsetsTopic(zkClient: Kaf

[GitHub] [kafka] novosibman commented on a diff in pull request #13782: KAFKA-9693: Kafka latency spikes caused by log segment flush on roll

2023-06-15 Thread via GitHub
novosibman commented on code in PR #13782: URL: https://github.com/apache/kafka/pull/13782#discussion_r1230800451 ## clients/src/main/java/org/apache/kafka/common/utils/Utils.java: ## @@ -1007,6 +1007,30 @@ public static void flushDir(Path path) throws IOException { }

[jira] [Commented] (KAFKA-9400) Make Consumer Class Extend AutoCloseable

2023-06-15 Thread Manyanda Chitimbo (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17732983#comment-17732983 ] Manyanda Chitimbo commented on KAFKA-9400: -- The Consumer.java interface extends

[GitHub] [kafka] divijvaidya opened a new pull request, #13858: MINOR: Add more information in assertion failure for non daemon threads

2023-06-15 Thread via GitHub
divijvaidya opened a new pull request, #13858: URL: https://github.com/apache/kafka/pull/13858 ## Problem Sometimes we have flaky test failures with a stack trace such as: ``` org.opentest4j.AssertionFailedError: expected: <0> but was: <1> at app//org.junit.jupiter.api.Asser

[GitHub] [kafka] machi1990 commented on a diff in pull request #13665: KAFKA-12485: Speed up Consumer#committed by returning cached offsets for owned partitions

2023-06-15 Thread via GitHub
machi1990 commented on code in PR #13665: URL: https://github.com/apache/kafka/pull/13665#discussion_r1230786333 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java: ## @@ -351,6 +359,10 @@ private Exception invokePartitionsRevoked(final

[GitHub] [kafka] machi1990 commented on pull request #13665: KAFKA-12485: Speed up Consumer#committed by returning cached offsets for owned partitions

2023-06-15 Thread via GitHub
machi1990 commented on PR #13665: URL: https://github.com/apache/kafka/pull/13665#issuecomment-1592771368 > @machi1990 , thanks for the update, left some more comments. Thanks for the thorough review @showuon I've addressed all the comments. Please have another look when you've some t

[GitHub] [kafka] dajac opened a new pull request, #13857: wip - ignore

2023-06-15 Thread via GitHub
dajac opened a new pull request, #13857: URL: https://github.com/apache/kafka/pull/13857 ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgrade

[GitHub] [kafka] cadonna commented on a diff in pull request #13855: KAFKA-14936: Add Grace period logic to Stream Table Join (2/N)

2023-06-15 Thread via GitHub
cadonna commented on code in PR #13855: URL: https://github.com/apache/kafka/pull/13855#discussion_r1230539039 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamImpl.java: ## @@ -1256,10 +1261,25 @@ private KStream doStreamTableJoin(final KTable table,

[jira] [Commented] (KAFKA-15085) Make Timer.java implement AutoCloseable

2023-06-15 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17732944#comment-17732944 ] Divij Vaidya commented on KAFKA-15085: -- Hey [~owen-leung]  I just provided you with

[GitHub] [kafka] machi1990 commented on a diff in pull request #13665: KAFKA-12485: Speed up Consumer#committed by returning cached offsets for owned partitions

2023-06-15 Thread via GitHub
machi1990 commented on code in PR #13665: URL: https://github.com/apache/kafka/pull/13665#discussion_r1230661381 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinator.java: ## @@ -351,6 +359,10 @@ private Exception invokePartitionsRevoked(final

[GitHub] [kafka] clolov commented on pull request #13711: KAFKA-14133: Migrate StandbyTaskCreator mock in TaskManagerTest to Mockito

2023-06-15 Thread via GitHub
clolov commented on PR #13711: URL: https://github.com/apache/kafka/pull/13711#issuecomment-1592595315 Good morning! I believe that the failing checks are not connected with this change. Is it okay to merge this PR? -- This is an automated message from the Apache Git Service. To respond t

[jira] [Commented] (KAFKA-3881) Remove the replacing logic from "." to "_" in Fetcher

2023-06-15 Thread Tom Bentley (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-3881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17732918#comment-17732918 ] Tom Bentley commented on KAFKA-3881: [~kirktrue] the difficult is that metrics are a

[GitHub] [kafka] machi1990 commented on a diff in pull request #13665: KAFKA-12485: Speed up Consumer#committed by returning cached offsets for owned partitions

2023-06-15 Thread via GitHub
machi1990 commented on code in PR #13665: URL: https://github.com/apache/kafka/pull/13665#discussion_r1230620093 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinatorTest.java: ## @@ -3114,6 +3137,70 @@ public void testFetchCommittedOffsets() {

[GitHub] [kafka] machi1990 commented on a diff in pull request #13665: KAFKA-12485: Speed up Consumer#committed by returning cached offsets for owned partitions

2023-06-15 Thread via GitHub
machi1990 commented on code in PR #13665: URL: https://github.com/apache/kafka/pull/13665#discussion_r1230614601 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinatorTest.java: ## @@ -2521,11 +2521,34 @@ public void testCommitOffsetMetadata() {

[GitHub] [kafka] machi1990 commented on a diff in pull request #13665: KAFKA-12485: Speed up Consumer#committed by returning cached offsets for owned partitions

2023-06-15 Thread via GitHub
machi1990 commented on code in PR #13665: URL: https://github.com/apache/kafka/pull/13665#discussion_r1230614177 ## clients/src/test/java/org/apache/kafka/clients/consumer/KafkaConsumerTest.java: ## @@ -922,13 +922,9 @@ public void testCommitsFetchedDuringAssign() { /