Re: [PR] KAFKA-14132; Replace EasyMock with Mockito in StandaloneHerderTest [kafka]

2023-10-27 Thread via GitHub
mdedetrich commented on code in PR #12728: URL: https://github.com/apache/kafka/pull/12728#discussion_r1374163842 ## connect/runtime/src/test/java/org/apache/kafka/connect/runtime/standalone/StandaloneHerderTest.java: ## @@ -528,72 +483,64 @@ public void testRestartConnectorAndT

Re: [PR] KAFKA-14132; Replace EasyMock with Mockito in StandaloneHerderTest [kafka]

2023-10-27 Thread via GitHub
mdedetrich commented on code in PR #12728: URL: https://github.com/apache/kafka/pull/12728#discussion_r1374165166 ## connect/runtime/src/test/java/org/apache/kafka/connect/runtime/standalone/StandaloneHerderTest.java: ## @@ -528,72 +483,64 @@ public void testRestartConnectorAndT

Re: [PR] KAFKA-14132; Replace EasyMock with Mockito in StandaloneHerderTest [kafka]

2023-10-27 Thread via GitHub
mdedetrich commented on PR #12728: URL: https://github.com/apache/kafka/pull/12728#issuecomment-1782422797 @C0urante Commit pushed resolving your latest comments -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] KAFKA-14132; Replace EasyMock with Mockito in StandaloneHerderTest [kafka]

2023-10-27 Thread via GitHub
mdedetrich commented on code in PR #12728: URL: https://github.com/apache/kafka/pull/12728#discussion_r1374168471 ## connect/runtime/src/test/java/org/apache/kafka/connect/runtime/standalone/StandaloneHerderTest.java: ## @@ -617,8 +616,7 @@ public void testRestartConnectorAndTas

[PR] KAFKA-15695: Update the local log start offset of a log after rebuilding the auxiliary state [kafka]

2023-10-27 Thread via GitHub
nikramakrishnan opened a new pull request, #14649: URL: https://github.com/apache/kafka/pull/14649 In `UnifiedLog.truncateFullyAndStartAt`, local log start offset is not updated after truncating the local log to the given new offset. This means that this offset is not updated when reconstr

[jira] [Updated] (KAFKA-15695) Local log start offset is not updated on the follower after rebuilding remote log auxiliary state

2023-10-27 Thread Nikhil Ramakrishnan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikhil Ramakrishnan updated KAFKA-15695: Description: In 3.6, the local log start offset is not updated when reconstructing

[jira] [Commented] (KAFKA-15695) Local log start offset is not updated on the follower after rebuilding remote log auxiliary state

2023-10-27 Thread Nikhil Ramakrishnan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17780216#comment-17780216 ] Nikhil Ramakrishnan commented on KAFKA-15695: - [~ckamal] I added an integra

Re: [PR] KAFKA-15689: Logging skipped event when expected migration state is wrong [kafka]

2023-10-27 Thread via GitHub
ppatierno commented on PR #14646: URL: https://github.com/apache/kafka/pull/14646#issuecomment-1782455220 @showuon everything should be ok now. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

[PR] MINOR: fix typo [kafka]

2023-10-27 Thread via GitHub
appchemist opened a new pull request, #14650: URL: https://github.com/apache/kafka/pull/14650 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e

[PR] Using INFO level for logging the state transition during the ZooKeeper to KRaft migration [kafka]

2023-10-27 Thread via GitHub
ppatierno opened a new pull request, #14651: URL: https://github.com/apache/kafka/pull/14651 Trivial PR to use the INFO level (instead of DEBUG) for logging the state transition during the ZooKeeper to KRaft migration. I think it's a useful information to be logged without the need for th

Re: [PR] Using INFO level for logging the state transition during the ZooKeeper to KRaft migration [kafka]

2023-10-27 Thread via GitHub
ppatierno commented on PR #14651: URL: https://github.com/apache/kafka/pull/14651#issuecomment-1782462055 @showuon I think this is also useful together with #14646 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

Re: [PR] KAFKA-15562: ensure commit request manager handles errors correctly [kafka]

2023-10-27 Thread via GitHub
lucasbru commented on code in PR #14639: URL: https://github.com/apache/kafka/pull/14639#discussion_r1374244715 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -271,14 +445,13 @@ public void onResponse( private voi

Re: [PR] MINOR: Rewrite the meta.properties handling code in Java and fix some issues [kafka]

2023-10-27 Thread via GitHub
divijvaidya commented on code in PR #14628: URL: https://github.com/apache/kafka/pull/14628#discussion_r1374255248 ## metadata/src/main/java/org/apache/kafka/metadata/properties/PropertiesUtils.java: ## @@ -0,0 +1,102 @@ +/* + * Licensed to the Apache Software Foundation (ASF) u

Re: [PR] KAFKA-14133: Replace Easymock with Mockito in StreamsProducerTest, TopologyMetadataTest & GlobalStateStoreProviderTest [kafka]

2023-10-27 Thread via GitHub
divijvaidya commented on PR #14410: URL: https://github.com/apache/kafka/pull/14410#issuecomment-1782528436 Test failures are unrelated to the tests modified in this PR. ``` [Build / JDK 21 and Scala 2.13 / org.apache.kafka.common.network.SslTransportLayerTest.[3] tlsProtocol=TLSv1.3,

Re: [PR] KAFKA-14133: Replace Easymock with Mockito in StreamsProducerTest, TopologyMetadataTest & GlobalStateStoreProviderTest [kafka]

2023-10-27 Thread via GitHub
divijvaidya merged PR #14410: URL: https://github.com/apache/kafka/pull/14410 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apa

[jira] [Updated] (KAFKA-14133) Remaining EasyMock to Mockito tests

2023-10-27 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya updated KAFKA-14133: - Description: {color:#de350b}There are tests which use both PowerMock and EasyMock. I have put t

Re: [PR] KAFKA-15562: ensure commit request manager handles errors correctly [kafka]

2023-10-27 Thread via GitHub
lucasbru commented on PR #14639: URL: https://github.com/apache/kafka/pull/14639#issuecomment-1782544764 Changes look good to me. Did I understand david correctly that we want to add more changes to this PR? > 1. Mockito: Could you be more specific on how you expect to mock the respo

Re: [PR] KAFKA-15695: Update the local log start offset of a log after rebuilding the auxiliary state [kafka]

2023-10-27 Thread via GitHub
divijvaidya commented on code in PR #14649: URL: https://github.com/apache/kafka/pull/14649#discussion_r1374271856 ## core/src/main/scala/kafka/log/UnifiedLog.scala: ## @@ -1754,6 +1754,7 @@ class UnifiedLog(@volatile var logStartOffset: Long, leaderEpochCache.foreach(_

Re: [PR] KAFKA-15693: Immediately reassign lost connectors and tasks when scheduled rebalance delay is disabled [kafka]

2023-10-27 Thread via GitHub
vamossagar12 commented on code in PR #14647: URL: https://github.com/apache/kafka/pull/14647#discussion_r1374326179 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/IncrementalCooperativeAssignor.java: ## @@ -463,6 +464,14 @@ protected void handleLos

Re: [PR] Kafka 15680 [kafka]

2023-10-27 Thread via GitHub
kumarpritam863 commented on PR #14630: URL: https://github.com/apache/kafka/pull/14630#issuecomment-1782659721 @C0urante As I mentioned this is a very small change just a change in the order of calling the same function which updates the partition count. This involves practically no cod

[jira] [Updated] (KAFKA-15695) Local log start offset is not updated on the follower after rebuilding remote log auxiliary state

2023-10-27 Thread Nikhil Ramakrishnan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikhil Ramakrishnan updated KAFKA-15695: Description: In 3.6, the local log start offset is not updated when reconstructing

Re: [PR] KAFKA-14132; Replace EasyMock with Mockito in StandaloneHerderTest [kafka]

2023-10-27 Thread via GitHub
mimaison commented on code in PR #12728: URL: https://github.com/apache/kafka/pull/12728#discussion_r1374348110 ## connect/runtime/src/test/java/org/apache/kafka/connect/runtime/standalone/StandaloneHerderTest.java: ## @@ -127,94 +134,86 @@ private enum SourceSink { non

[jira] [Updated] (KAFKA-15695) Local log start offset is not updated on the follower after rebuilding remote log auxiliary state

2023-10-27 Thread Nikhil Ramakrishnan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikhil Ramakrishnan updated KAFKA-15695: Description: In 3.6, the local log start offset is not updated when reconstructing

Re: [PR] KAFKA-15668: Adding Opentelmetry shadowed library (KIP-714) [kafka]

2023-10-27 Thread via GitHub
apoorvmittal10 commented on code in PR #14618: URL: https://github.com/apache/kafka/pull/14618#discussion_r1374381174 ## build.gradle: ## @@ -295,7 +296,12 @@ subprojects { } publications { mavenJava(MavenPublication) { - from components.java +

Re: [PR] KAFKA-14132; Replace EasyMock with Mockito in StandaloneHerderTest [kafka]

2023-10-27 Thread via GitHub
mdedetrich commented on code in PR #12728: URL: https://github.com/apache/kafka/pull/12728#discussion_r1374442341 ## connect/runtime/src/test/java/org/apache/kafka/connect/runtime/standalone/StandaloneHerderTest.java: ## @@ -838,39 +744,33 @@ public void testPutConnectorConfig()

Re: [PR] KAFKA-14132; Replace EasyMock with Mockito in StandaloneHerderTest [kafka]

2023-10-27 Thread via GitHub
mdedetrich commented on code in PR #12728: URL: https://github.com/apache/kafka/pull/12728#discussion_r1374443260 ## connect/runtime/src/test/java/org/apache/kafka/connect/runtime/standalone/StandaloneHerderTest.java: ## @@ -817,19 +731,11 @@ public void testAccessors() throws E

Re: [PR] KAFKA-14132; Replace EasyMock with Mockito in StandaloneHerderTest [kafka]

2023-10-27 Thread via GitHub
mdedetrich commented on code in PR #12728: URL: https://github.com/apache/kafka/pull/12728#discussion_r1374442966 ## connect/runtime/src/test/java/org/apache/kafka/connect/runtime/standalone/StandaloneHerderTest.java: ## @@ -127,94 +134,86 @@ private enum SourceSink { n

Re: [PR] KAFKA-15679: Consumer configurations for group protocol [kafka]

2023-10-27 Thread via GitHub
lucasbru commented on code in PR #14642: URL: https://github.com/apache/kafka/pull/14642#discussion_r1374443906 ## clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java: ## @@ -105,6 +105,23 @@ public class ConsumerConfig extends AbstractConfig { publi

Re: [PR] KAFKA-15346: add support for 'single key single timestamp' IQs with versioned state stores (KIP-960) [kafka]

2023-10-27 Thread via GitHub
aliehsaeedii commented on code in PR #14596: URL: https://github.com/apache/kafka/pull/14596#discussion_r1374466490 ## streams/src/main/java/org/apache/kafka/streams/query/VersionedKeyQuery.java: ## @@ -0,0 +1,77 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[jira] [Created] (KAFKA-15702) apiVersion request doesn't send when DNS is not ready at first

2023-10-27 Thread Luke Chen (Jira)
Luke Chen created KAFKA-15702: - Summary: apiVersion request doesn't send when DNS is not ready at first Key: KAFKA-15702 URL: https://issues.apache.org/jira/browse/KAFKA-15702 Project: Kafka Iss

[PR] KAFKA-14467:Fixed an issue where local incorrect snapshot files might occur due to first pulling the snapshot file and the n truncate it [kafka]

2023-10-27 Thread via GitHub
hudeqi opened a new pull request, #14652: URL: https://github.com/apache/kafka/pull/14652 1. As [jira](https://issues.apache.org/jira/browse/KAFKA-14467) described, add a test to validate the replica state after processing the OFFSET_MOVED_TO_TIERED_STORAGE error, especially for the transac

Re: [PR] KAFKA-15695: Update the local log start offset of a log after rebuilding the auxiliary state [kafka]

2023-10-27 Thread via GitHub
kamalcph commented on code in PR #14649: URL: https://github.com/apache/kafka/pull/14649#discussion_r1374549518 ## storage/src/test/java/org/apache/kafka/tiered/storage/integration/LocalLogStartOffsetTest.java: ## @@ -0,0 +1,76 @@ +/* + * Licensed to the Apache Software Foundati

[jira] [Created] (KAFKA-15703) Update Highwatermark while building the remote log auxillary state

2023-10-27 Thread Kamal Chandraprakash (Jira)
Kamal Chandraprakash created KAFKA-15703: Summary: Update Highwatermark while building the remote log auxillary state Key: KAFKA-15703 URL: https://issues.apache.org/jira/browse/KAFKA-15703 Pr

[jira] [Updated] (KAFKA-15703) Update Highwatermark while building the remote log auxiliary state

2023-10-27 Thread Kamal Chandraprakash (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamal Chandraprakash updated KAFKA-15703: - Summary: Update Highwatermark while building the remote log auxiliary state (wa

[jira] [Updated] (KAFKA-15703) Update Highwatermark while building the remote log auxilary state

2023-10-27 Thread Kamal Chandraprakash (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamal Chandraprakash updated KAFKA-15703: - Summary: Update Highwatermark while building the remote log auxilary state (was

Re: [PR] MINOR: Using INFO level for logging the state transition during the ZooKeeper to KRaft migration [kafka]

2023-10-27 Thread via GitHub
ppatierno commented on PR #14651: URL: https://github.com/apache/kafka/pull/14651#issuecomment-1782905077 > You can add "MINOR:" at the beginning of the PR title. Thanks. Thanks @hudeqi just done! -- This is an automated message from the Apache Git Service. To respond to the message

[PR] KAFKA-15703: Update Highwatermark while building the remote log auxiliary state [kafka]

2023-10-27 Thread via GitHub
kamalcph opened a new pull request, #14653: URL: https://github.com/apache/kafka/pull/14653 While building the remote log auxiliary state for the follower, we have to update the high-watermark to the offset where the local-log starts from. Currently, if there are no new incoming mess

Re: [PR] KAFKA-15695: Update the local log start offset of a log after rebuilding the auxiliary state [kafka]

2023-10-27 Thread via GitHub
kamalcph commented on PR #14649: URL: https://github.com/apache/kafka/pull/14649#issuecomment-1782912082 On a related note, we have to update the high-watermark while rebuilding the auxiliary state. Opened #14653 to address this. Will add more details on the PR. -- This is an automated me

Re: [PR] KAFKA-14133: Move StreamsMetricsImplTest to Mockito [kafka]

2023-10-27 Thread via GitHub
ijuma commented on code in PR #14623: URL: https://github.com/apache/kafka/pull/14623#discussion_r1374596662 ## gradle/dependencies.gradle: ## @@ -70,8 +70,9 @@ else // The default strategy (i.e. what `mockito-core` uses) changed to `inline` in 5.x because it works better with

Re: [PR] KAFKA-14133: Move StreamsMetricsImplTest to Mockito [kafka]

2023-10-27 Thread via GitHub
ijuma commented on code in PR #14623: URL: https://github.com/apache/kafka/pull/14623#discussion_r1374596662 ## gradle/dependencies.gradle: ## @@ -70,8 +70,9 @@ else // The default strategy (i.e. what `mockito-core` uses) changed to `inline` in 5.x because it works better with

Re: [PR] KAFKA-15703: Update Highwatermark while building the remote log auxiliary state [kafka]

2023-10-27 Thread via GitHub
nikramakrishnan commented on PR #14653: URL: https://github.com/apache/kafka/pull/14653#issuecomment-1782947677 Question: is this not already handled in [UnifiedLog::truncateFullyAndStartAt](https://github.com/apache/kafka/blob/f0e97397c01241be9d8d05b1642352b58d1ed196/core/src/main/scala/kaf

[jira] [Commented] (KAFKA-15688) Partition leader election not running when disk IO hangs

2023-10-27 Thread Peter Sinoros-Szabo (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17780399#comment-17780399 ] Peter Sinoros-Szabo commented on KAFKA-15688: - > 2) Deploy another process t

[jira] [Commented] (KAFKA-14133) Remaining EasyMock to Mockito tests

2023-10-27 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17780406#comment-17780406 ] Ismael Juma commented on KAFKA-14133: - It looks like all the remaining tests are in

Re: [PR] KAFKA-14133: Move StreamsMetricsImplTest to Mockito [kafka]

2023-10-27 Thread via GitHub
clolov commented on code in PR #14623: URL: https://github.com/apache/kafka/pull/14623#discussion_r1374684030 ## gradle/dependencies.gradle: ## @@ -70,8 +70,9 @@ else // The default strategy (i.e. what `mockito-core` uses) changed to `inline` in 5.x because it works better wit

Re: [PR] KAFKA-15703: Update Highwatermark while building the remote log auxiliary state [kafka]

2023-10-27 Thread via GitHub
kamalcph commented on PR #14653: URL: https://github.com/apache/kafka/pull/14653#issuecomment-1783055220 > Question: is this not already handled in [UnifiedLog::truncateFullyAndStartAt](https://github.com/apache/kafka/blob/f0e97397c01241be9d8d05b1642352b58d1ed196/core/src/main/scala/kafka/lo

Re: [PR] KAFKA-15703: Update Highwatermark while building the remote log auxiliary state [kafka]

2023-10-27 Thread via GitHub
kamalcph closed pull request #14653: KAFKA-15703: Update Highwatermark while building the remote log auxiliary state URL: https://github.com/apache/kafka/pull/14653 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[jira] [Resolved] (KAFKA-15415) In Java-client, backoff should be skipped for retried producer-batch to a new leader

2023-10-27 Thread Mayank Shekhar Narula (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mayank Shekhar Narula resolved KAFKA-15415. --- Resolution: Fixed > In Java-client, backoff should be skipped for retried pr

Re: [PR] KAFKA-14595 Move ReassignPartitionsCommand to java [kafka]

2023-10-27 Thread via GitHub
nizhikov commented on PR #13247: URL: https://github.com/apache/kafka/pull/13247#issuecomment-1783128017 Hello @mimaison Can you, please, share your feeback for PR? Are we ready to merge this? -- This is an automated message from the Apache Git Service. To respond to the message, pl

[jira] [Updated] (KAFKA-14133) Remaining EasyMock to Mockito tests

2023-10-27 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya updated KAFKA-14133: - Description: {color:#de350b}There are tests which use both PowerMock and EasyMock. I have put t

[jira] [Updated] (KAFKA-14133) Remaining EasyMock to Mockito tests

2023-10-27 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya updated KAFKA-14133: - Description: {color:#de350b}There are tests which use both PowerMock and EasyMock. I have put t

Re: [PR] KAFKA-14133: Move StreamsMetricsImplTest to Mockito [kafka]

2023-10-27 Thread via GitHub
ijuma commented on code in PR #14623: URL: https://github.com/apache/kafka/pull/14623#discussion_r1374763427 ## gradle/dependencies.gradle: ## @@ -70,8 +70,9 @@ else // The default strategy (i.e. what `mockito-core` uses) changed to `inline` in 5.x because it works better with

Re: [PR] MINOR: Fix flaky testIdleConnection() test (cherry-pick from apache/kafka #11996) [kafka]

2023-10-27 Thread via GitHub
divijvaidya commented on PR #13299: URL: https://github.com/apache/kafka/pull/13299#issuecomment-1783148515 Hey @SunnyChaturvedi It's unlikely that we will do a release for 3.2 at this stage since it's considered End of Life (EOL) as per https://cwiki.apache.org/confluence/display/KAFKA

Re: [PR] MINOR: Fix flaky testIdleConnection() test (cherry-pick from apache/kafka #11996) [kafka]

2023-10-27 Thread via GitHub
divijvaidya closed pull request #13299: MINOR: Fix flaky testIdleConnection() test (cherry-pick from apache/kafka #11996) URL: https://github.com/apache/kafka/pull/13299 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[jira] [Updated] (KAFKA-14133) Remaining EasyMock to Mockito tests

2023-10-27 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya updated KAFKA-14133: - Description: {color:#de350b}There are tests which use both PowerMock and EasyMock. I have put t

Re: [PR] KAFKA-14133: Migrate ProcessorStateManagerTest and StreamThreadTest to Mockito [kafka]

2023-10-27 Thread via GitHub
divijvaidya commented on PR #13932: URL: https://github.com/apache/kafka/pull/13932#issuecomment-1783154394 @clolov It's been a while since this PR. Is it still relevant? If yes, I can take a look at this next week. Please rebase from trunk. -- This is an automated message from the Apache

[jira] [Commented] (KAFKA-14133) Remaining EasyMock to Mockito tests

2023-10-27 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17780444#comment-17780444 ] Divij Vaidya commented on KAFKA-14133: -- Added PRs to remaining tests. > Remaining

[jira] [Updated] (KAFKA-15700) FetchFromFollowerIntegrationTest is flaky

2023-10-27 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya updated KAFKA-15700: - Labels: flaky-test (was: ) > FetchFromFollowerIntegrationTest is flaky > --

[jira] [Updated] (KAFKA-15700) FetchFromFollowerIntegrationTest is flaky

2023-10-27 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya updated KAFKA-15700: - Component/s: unit tests > FetchFromFollowerIntegrationTest is flaky > --

Re: [PR] KAFKA-15582: Move the clean shutdown file to the storage package [kafka]

2023-10-27 Thread via GitHub
junrao commented on code in PR #14603: URL: https://github.com/apache/kafka/pull/14603#discussion_r1374791457 ## storage/src/main/java/org/apache/kafka/storage/internals/checkpoint/CleanShutdownFileHandler.java: ## @@ -85,17 +86,16 @@ void write(long brokerEpoch, int version) th

[PR] MINOR Set missing ZkMigrationReady field on ControllerRegistrationRequest [kafka]

2023-10-27 Thread via GitHub
mumrah opened a new pull request, #14654: URL: https://github.com/apache/kafka/pull/14654 This field was missed by the initial KIP-919 PR(s). The result is that migrations can't begin since the controllers will never become ready. This patch fixes that as well as pulls over some fixes from

[PR] Change `CreateTopicPolicy.RequestMetadata` to always provide partitions, replication factor and replica assignments. [kafka]

2023-10-27 Thread via GitHub
jaewie opened a new pull request, #14655: URL: https://github.com/apache/kafka/pull/14655 For replica assignments, the interface would change to be the actual assignments that will be used, whether it's user provided or Kafka determined it. This way, implementations can use it to det

Re: [PR] KAFKA-15578: System Tests for running old protocol with new coordinator [kafka]

2023-10-27 Thread via GitHub
rreddy-22 commented on PR #14524: URL: https://github.com/apache/kafka/pull/14524#issuecomment-1783215920 https://jenkins.confluent.io/job/system-test-kafka-branch-builder/5925/ - Passing test after changes -- This is an automated message from the Apache Git Service. To respond to the me

Re: [PR] KAFKA-15582: Move the clean shutdown file to the storage package [kafka]

2023-10-27 Thread via GitHub
CalvinConfluent commented on code in PR #14603: URL: https://github.com/apache/kafka/pull/14603#discussion_r1374835745 ## storage/src/main/java/org/apache/kafka/storage/internals/checkpoint/CleanShutdownFileHandler.java: ## @@ -85,17 +86,16 @@ void write(long brokerEpoch, int ve

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-10-27 Thread via GitHub
junrao commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1374839084 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -53,25 +53,27 @@ object KafkaRequestHandler { * Wrap callback to schedule it on a request thread

Re: [PR] KAFKA-15629: Proposal to introduce IQv2 Query Types: TimestampedKeyQuery and TimestampedRangeQuery [kafka]

2023-10-27 Thread via GitHub
mjsax commented on code in PR #14570: URL: https://github.com/apache/kafka/pull/14570#discussion_r1374842688 ## streams/src/main/java/org/apache/kafka/streams/state/internals/StoreQueryUtils.java: ## @@ -346,6 +346,23 @@ public static Function getDeserializeValue(final StateSe

Re: [PR] KAFKA-15355: Message schema changes [kafka]

2023-10-27 Thread via GitHub
cmccabe commented on code in PR #14290: URL: https://github.com/apache/kafka/pull/14290#discussion_r1374843847 ## clients/src/main/resources/common/message/AssignReplicasToDirsRequest.json: ## @@ -0,0 +1,40 @@ +// Licensed to the Apache Software Foundation (ASF) under one or mor

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-10-27 Thread via GitHub
jolshan commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1374844889 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -53,25 +53,27 @@ object KafkaRequestHandler { * Wrap callback to schedule it on a request threa

Re: [PR] KAFKA-15355: Message schema changes [kafka]

2023-10-27 Thread via GitHub
cmccabe commented on PR #14290: URL: https://github.com/apache/kafka/pull/14290#issuecomment-1783251459 Thanks for the PR, @soarez ! Do you have revisions ready for this one or do you want one of us to jump in and fix these comments like we talked about earlier? I feel like it's really clos

Re: [PR] MINOR Set missing ZkMigrationReady field on ControllerRegistrationRequest [kafka]

2023-10-27 Thread via GitHub
cmccabe commented on code in PR #14654: URL: https://github.com/apache/kafka/pull/14654#discussion_r1374859694 ## core/src/test/scala/integration/kafka/zk/ZkMigrationIntegrationTest.scala: ## @@ -66,7 +66,12 @@ object ZkMigrationIntegrationTest { } def zkClustersForAllMi

[jira] [Created] (KAFKA-15704) ControllerRegistrationRequest must set ZkMigrationReady field if appropriate

2023-10-27 Thread Colin McCabe (Jira)
Colin McCabe created KAFKA-15704: Summary: ControllerRegistrationRequest must set ZkMigrationReady field if appropriate Key: KAFKA-15704 URL: https://issues.apache.org/jira/browse/KAFKA-15704 Project:

Re: [PR] KAFKA-15578: System Tests for running old protocol with new coordinator [kafka]

2023-10-27 Thread via GitHub
dajac merged PR #14524: URL: https://github.com/apache/kafka/pull/14524 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

[jira] [Created] (KAFKA-15705) Add integration tests for Heartbeat API and GroupLeave API

2023-10-27 Thread Dongnuo Lyu (Jira)
Dongnuo Lyu created KAFKA-15705: --- Summary: Add integration tests for Heartbeat API and GroupLeave API Key: KAFKA-15705 URL: https://issues.apache.org/jira/browse/KAFKA-15705 Project: Kafka Issu

[jira] [Assigned] (KAFKA-15705) Add integration tests for Heartbeat API and GroupLeave API

2023-10-27 Thread Dongnuo Lyu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongnuo Lyu reassigned KAFKA-15705: --- Assignee: Dongnuo Lyu > Add integration tests for Heartbeat API and GroupLeave API > --

[jira] [Assigned] (KAFKA-15574) Update states and transitions for membership manager state machine

2023-10-27 Thread Lianet Magrans (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianet Magrans reassigned KAFKA-15574: -- Assignee: Lianet Magrans (was: Kirk True) > Update states and transitions for member

Re: [PR] KAFKA-15661: KIP-951: protocol changes [kafka]

2023-10-27 Thread via GitHub
jolshan commented on PR #14627: URL: https://github.com/apache/kafka/pull/14627#issuecomment-1783300697 @chb2ab sorry for the confusion. I realized that marking the version unstable will probably cause issues if the ibp suggests using that version. Since the tagged fields are the only diffe

[PR] KAFKA-15705: Add integration tests for Heartbeat API and GroupLeave API [kafka]

2023-10-27 Thread via GitHub
dongnuo123 opened a new pull request, #14656: URL: https://github.com/apache/kafka/pull/14656 This pr is based on https://github.com/apache/kafka/pull/14589, Adding integration tests for Heartbeat API and GroupLeave API. ### JIRA https://issues.apache.org/jira/browse/KAFKA-15705

Re: [PR] KAFKA-15705: Add integration tests for Heartbeat API and GroupLeave API [kafka]

2023-10-27 Thread via GitHub
dongnuo123 commented on code in PR #14656: URL: https://github.com/apache/kafka/pull/14656#discussion_r1374900874 ## core/src/test/scala/unit/kafka/server/LeaveGroupRequestTest.scala: ## @@ -0,0 +1,137 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or mor

Re: [PR] KAFKA-15468: Prevent transaction coordinator reloads on already loaded leaders [kafka]

2023-10-27 Thread via GitHub
jsancio commented on PR #14489: URL: https://github.com/apache/kafka/pull/14489#issuecomment-1783315545 @jolshan there are failing tests for this PR. Can you take a look when you have time? -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] KAFKA-15705: Add integration tests for Heartbeat API and GroupLeave API [kafka]

2023-10-27 Thread via GitHub
dongnuo123 commented on code in PR #14656: URL: https://github.com/apache/kafka/pull/14656#discussion_r1374903378 ## core/src/test/scala/unit/kafka/server/LeaveGroupRequestTest.scala: ## @@ -0,0 +1,137 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or mor

[PR] KAFKA-15643: Fix error logged when unload is called on a broker that was never a coordinator. [kafka]

2023-10-27 Thread via GitHub
rreddy-22 opened a new pull request, #14657: URL: https://github.com/apache/kafka/pull/14657 **Problem Statement:** When a new leader is elected for a __consumer_offset partition, the followers are notified to unload the state. However, only the former leader is aware of it. The remainin

Re: [PR] KAFKA-15643: Fix error logged when unload is called on a broker that was never a coordinator. [kafka]

2023-10-27 Thread via GitHub
dajac commented on code in PR #14657: URL: https://github.com/apache/kafka/pull/14657#discussion_r1374948010 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/runtime/CoordinatorRuntime.java: ## @@ -1359,7 +1359,8 @@ public void scheduleUnloadOperation(

Re: [PR] KAFKA-15417 flip joinSpuriousLookBackTimeMs and emit non-joined items [kafka]

2023-10-27 Thread via GitHub
lihaosky commented on code in PR #14426: URL: https://github.com/apache/kafka/pull/14426#discussion_r1374947218 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamKStreamJoin.java: ## @@ -228,7 +231,7 @@ private void emitNonJoinedOuterRecords(

Re: [PR] KAFKA-15643: Fix error logged when unload is called on a broker that was never a coordinator. [kafka]

2023-10-27 Thread via GitHub
dajac commented on code in PR #14657: URL: https://github.com/apache/kafka/pull/14657#discussion_r1374948601 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/runtime/CoordinatorRuntime.java: ## @@ -1359,7 +1359,8 @@ public void scheduleUnloadOperation(

Re: [PR] KAFKA-15643: Fix error logged when unload is called on a broker that was never a coordinator. [kafka]

2023-10-27 Thread via GitHub
dajac commented on code in PR #14657: URL: https://github.com/apache/kafka/pull/14657#discussion_r1374949071 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/runtime/CoordinatorRuntime.java: ## @@ -1370,7 +1371,9 @@ public void scheduleUnloadOperation(

Re: [PR] KAFKA-15679: Consumer configurations for group protocol [kafka]

2023-10-27 Thread via GitHub
dajac commented on code in PR #14642: URL: https://github.com/apache/kafka/pull/14642#discussion_r1374963674 ## clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java: ## @@ -610,6 +628,18 @@ public class ConsumerConfig extends AbstractConfig {

Re: [PR] KAFKA-15679: Consumer configurations for group protocol [kafka]

2023-10-27 Thread via GitHub
dajac commented on code in PR #14642: URL: https://github.com/apache/kafka/pull/14642#discussion_r1374964423 ## clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java: ## @@ -105,6 +105,23 @@ public class ConsumerConfig extends AbstractConfig { public s

Re: [PR] KAFKA-15679: Consumer configurations for group protocol [kafka]

2023-10-27 Thread via GitHub
dajac commented on code in PR #14642: URL: https://github.com/apache/kafka/pull/14642#discussion_r1374964660 ## clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java: ## @@ -105,6 +105,23 @@ public class ConsumerConfig extends AbstractConfig { public s

Re: [PR] KAFKA-15647: Fix the different behavior in error handling between the old and new group coordinator [kafka]

2023-10-27 Thread via GitHub
dajac commented on PR #14589: URL: https://github.com/apache/kafka/pull/14589#issuecomment-1783398978 @dongnuo123 One of the build is still red. You may have to merge trunk to get the latest changes. -- This is an automated message from the Apache Git Service. To respond to the message, p

Re: [PR] KAFKA-15661: KIP-951: protocol changes [kafka]

2023-10-27 Thread via GitHub
chb2ab commented on PR #14627: URL: https://github.com/apache/kafka/pull/14627#issuecomment-1783399735 @jolshan np, I reverted that change. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [PR] MINOR: LeaveGroupResponse v0 - v2 loses its member under certain error conditions [kafka]

2023-10-27 Thread via GitHub
dajac commented on PR #14635: URL: https://github.com/apache/kafka/pull/14635#issuecomment-1783402130 @wolfchimneyrock Thanks again for the PR. We verified and your suggestion makes sense. I have two asks: 1) Could you please file a bug and use the the jira id in the PR? 2) Would it be

Re: [PR] KAFKA-15679: Consumer configurations for group protocol [kafka]

2023-10-27 Thread via GitHub
philipnee commented on code in PR #14642: URL: https://github.com/apache/kafka/pull/14642#discussion_r1374998305 ## clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java: ## @@ -610,6 +628,18 @@ public class ConsumerConfig extends AbstractConfig {

Re: [PR] KAFKA-15679: Consumer configurations for group protocol [kafka]

2023-10-27 Thread via GitHub
philipnee commented on PR #14642: URL: https://github.com/apache/kafka/pull/14642#issuecomment-1783444304 @kirktrue @lucasbru @dajac - Thanks for taking time to review this PR. I've addressed the recent comments. Let me know if there's anything uncleared. -- This is an automated message

Re: [PR] KAFKA-15661: KIP-951: protocol changes [kafka]

2023-10-27 Thread via GitHub
jolshan commented on PR #14627: URL: https://github.com/apache/kafka/pull/14627#issuecomment-1783457546 I will just watch the build now. Thanks @chb2ab -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] KAFKA-15614: Define interfaces and classes for client telemetry [kafka]

2023-10-27 Thread via GitHub
xvrl commented on code in PR #14575: URL: https://github.com/apache/kafka/pull/14575#discussion_r1375013255 ## clients/src/main/java/org/apache/kafka/common/metrics/MetricsReporter.java: ## @@ -24,13 +24,14 @@ import org.apache.kafka.common.Reconfigurable; import org.apache.ka

Re: [PR] MINOR: LeaveGroupResponse v0 - v2 loses its member under certain error conditions [kafka]

2023-10-27 Thread via GitHub
wolfchimneyrock commented on PR #14635: URL: https://github.com/apache/kafka/pull/14635#issuecomment-1783465122 > @wolfchimneyrock Thanks again for the PR. We verified and your suggestion makes sense. I have two asks: > > 1. Could you please file a bug and use the the jira id in the P

Re: [PR] KAFKA-15614: Define interfaces and classes for client telemetry [kafka]

2023-10-27 Thread via GitHub
xvrl commented on code in PR #14575: URL: https://github.com/apache/kafka/pull/14575#discussion_r1375015180 ## clients/src/main/java/org/apache/kafka/server/telemetry/ClientTelemetry.java: ## @@ -0,0 +1,44 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or

Re: [PR] KAFKA-15669: Implement telemetry metric naming strategy (KIP-714) [kafka]

2023-10-27 Thread via GitHub
xvrl commented on code in PR #14619: URL: https://github.com/apache/kafka/pull/14619#discussion_r1375018518 ## clients/src/main/java/org/apache/kafka/common/telemetry/internals/MetricNamingConvention.java: ## @@ -0,0 +1,119 @@ +/* + * Licensed to the Apache Software Foundation (

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-10-27 Thread via GitHub
artemlivshits commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1375016328 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -53,25 +53,27 @@ object KafkaRequestHandler { * Wrap callback to schedule it on a request

Re: [PR] KAFKA-15669: Implement telemetry metric naming strategy (KIP-714) [kafka]

2023-10-27 Thread via GitHub
xvrl commented on code in PR #14619: URL: https://github.com/apache/kafka/pull/14619#discussion_r1375023475 ## clients/src/main/java/org/apache/kafka/common/telemetry/internals/MetricNamingConvention.java: ## @@ -0,0 +1,119 @@ +/* + * Licensed to the Apache Software Foundation (

Re: [PR] KAFKA-15669: Implement telemetry metric naming strategy (KIP-714) [kafka]

2023-10-27 Thread via GitHub
xvrl commented on code in PR #14619: URL: https://github.com/apache/kafka/pull/14619#discussion_r1375025585 ## clients/src/main/java/org/apache/kafka/common/telemetry/internals/MetricNamingConvention.java: ## @@ -0,0 +1,119 @@ +/* + * Licensed to the Apache Software Foundation (

  1   2   >