Re: [PR] KAFKA-13788: Use AdminClient.incrementalAlterConfigs in ConfigCommand [kafka]

2023-12-08 Thread via GitHub
dengziming commented on PR #12065: URL: https://github.com/apache/kafka/pull/12065#issuecomment-1848246407 Thank you for reminding me of this @dajac , I almost forget about this due to the tedious tasks at work, I will take a look at this in the next week, I think a KIP is necessary for

Re: [PR] KAFKA-15276: Implement event plumbing for ConsumerRebalanceListener callbacks [kafka]

2023-12-08 Thread via GitHub
kirktrue commented on code in PR #14640: URL: https://github.com/apache/kafka/pull/14640#discussion_r1421255375 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImplTest.java: ## @@ -790,6 +812,197 @@ public void

Re: [PR] KAFKA-15276: Implement event plumbing for ConsumerRebalanceListener callbacks [kafka]

2023-12-08 Thread via GitHub
kirktrue commented on code in PR #14640: URL: https://github.com/apache/kafka/pull/14640#discussion_r1421255309 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/events/ConsumerRebalanceListenerCallbackCompletedEvent.java: ## @@ -0,0 +1,102 @@ +/* + *

Re: [PR] KAFKA-15276: Implement event plumbing for ConsumerRebalanceListener callbacks [kafka]

2023-12-08 Thread via GitHub
kirktrue commented on code in PR #14640: URL: https://github.com/apache/kafka/pull/14640#discussion_r1421255259 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -1497,6 +1606,66 @@ private void subscribeInternal(Collection

Re: [PR] KAFKA-15276: Implement event plumbing for ConsumerRebalanceListener callbacks [kafka]

2023-12-08 Thread via GitHub
kirktrue commented on code in PR #14640: URL: https://github.com/apache/kafka/pull/14640#discussion_r1421253120 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImplTest.java: ## @@ -790,6 +812,197 @@ public void

Re: [PR] KAFKA-15276: Implement event plumbing for ConsumerRebalanceListener callbacks [kafka]

2023-12-08 Thread via GitHub
kirktrue commented on code in PR #14640: URL: https://github.com/apache/kafka/pull/14640#discussion_r1421201698 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/events/ApplicationEventProcessor.java: ## @@ -113,16 +113,15 @@ public void

Re: [PR] KAFKA-15276: Implement event plumbing for ConsumerRebalanceListener callbacks [kafka]

2023-12-08 Thread via GitHub
kirktrue commented on code in PR #14640: URL: https://github.com/apache/kafka/pull/14640#discussion_r1421200769 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImplTest.java: ## @@ -790,6 +812,197 @@ public void

Re: [PR] KAFKA-15276: Implement event plumbing for ConsumerRebalanceListener callbacks [kafka]

2023-12-08 Thread via GitHub
kirktrue commented on code in PR #14640: URL: https://github.com/apache/kafka/pull/14640#discussion_r1421200092 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -42,7 +42,11 @@ import

Re: [PR] KAFKA-15365: Broker-side replica management changes [kafka]

2023-12-08 Thread via GitHub
OmniaGM commented on code in PR #14881: URL: https://github.com/apache/kafka/pull/14881#discussion_r1421190145 ## metadata/src/main/java/org/apache/kafka/image/TopicDelta.java: ## @@ -147,7 +149,7 @@ public LocalReplicaChanges localChanges(int brokerId) {

Re: [PR] KAFKA-15365: Broker-side replica management changes [kafka]

2023-12-08 Thread via GitHub
OmniaGM commented on code in PR #14881: URL: https://github.com/apache/kafka/pull/14881#discussion_r1421189818 ## metadata/src/main/java/org/apache/kafka/image/TopicDelta.java: ## @@ -158,11 +160,21 @@ public LocalReplicaChanges localChanges(int brokerId) {

Re: [PR] KAFKA-15365: Broker-side replica management changes [kafka]

2023-12-08 Thread via GitHub
OmniaGM commented on PR #14881: URL: https://github.com/apache/kafka/pull/14881#issuecomment-1848035026 > Checkstyle failed: Here is the diff to fix > > ``` > diff --git a/metadata/src/main/java/org/apache/kafka/image/TopicDelta.java

Re: [PR] KAFKA-15276: Implement event plumbing for ConsumerRebalanceListener callbacks [kafka]

2023-12-08 Thread via GitHub
kirktrue commented on code in PR #14640: URL: https://github.com/apache/kafka/pull/14640#discussion_r1421173308 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImplTest.java: ## @@ -790,6 +812,197 @@ public void

Re: [PR] KAFKA-15276: Implement event plumbing for ConsumerRebalanceListener callbacks [kafka]

2023-12-08 Thread via GitHub
kirktrue commented on code in PR #14640: URL: https://github.com/apache/kafka/pull/14640#discussion_r1421165461 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImplTest.java: ## @@ -790,6 +812,197 @@ public void

Re: [PR] KAFKA-15347: implement lazy deserialization for segment [kafka]

2023-12-08 Thread via GitHub
aliehsaeedii commented on code in PR #14957: URL: https://github.com/apache/kafka/pull/14957#discussion_r1421149432 ## streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBVersionedStoreSegmentValueFormatter.java: ## @@ -340,32 +355,67 @@ public

Re: [PR] KAFKA-15347: implement lazy deserialization for segment [kafka]

2023-12-08 Thread via GitHub
aliehsaeedii commented on PR #14957: URL: https://github.com/apache/kafka/pull/14957#issuecomment-1848005381 @mjsax I modified the `find()` method in the best possible way reusing the currently defined `deserIndex` and caches. Here is a brief list of changes and issues. 1. I had to

Re: [PR] KAFKA-15276: Implement event plumbing for ConsumerRebalanceListener callbacks [kafka]

2023-12-08 Thread via GitHub
kirktrue commented on code in PR #14640: URL: https://github.com/apache/kafka/pull/14640#discussion_r1421148323 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/events/ConsumerRebalanceListenerCallbackCompletedEvent.java: ## @@ -0,0 +1,102 @@ +/* + *

Re: [PR] [MINOR] fix: Align LogContext setup for core components [kafka]

2023-12-08 Thread via GitHub
jeqo commented on PR #14348: URL: https://github.com/apache/kafka/pull/14348#issuecomment-1847985030 @divijvaidya I took a stab on refactoring the LogContext by adding a builder and adopting it on most of the source code but stream module as there is a heavy dependency on logPrefix as a

Re: [PR] KAFKA-15662: Add support for clientInstanceIds in Kafka Stream [kafka]

2023-12-08 Thread via GitHub
mjsax commented on code in PR #14936: URL: https://github.com/apache/kafka/pull/14936#discussion_r1421112931 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/DefaultStateUpdater.java: ## @@ -231,6 +240,69 @@ private void restoreTasks(final long now) {

Re: [PR] MINOR: Bug fixes for AssignmentsManager [kafka]

2023-12-08 Thread via GitHub
cmccabe merged PR #14954: URL: https://github.com/apache/kafka/pull/14954 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Created] (KAFKA-15991) Flaky new consumer test testGroupIdNotNullAndValid

2023-12-08 Thread Lianet Magrans (Jira)
Lianet Magrans created KAFKA-15991: -- Summary: Flaky new consumer test testGroupIdNotNullAndValid Key: KAFKA-15991 URL: https://issues.apache.org/jira/browse/KAFKA-15991 Project: Kafka Issue

Re: [PR] KAFKA-15574; [2/N]: Client state machine updates [kafka]

2023-12-08 Thread via GitHub
lianetm commented on PR #14857: URL: https://github.com/apache/kafka/pull/14857#issuecomment-1847956911 Temporarily disabled a flaky test that was recently added only for the new consumer (`testGroupIdNotNullAndValid`). I can make it fail locally in trunk when running it in a loop so

Re: [PR] KAFKA-15816: Fix leaked sockets in core tests [kafka]

2023-12-08 Thread via GitHub
gharris1727 commented on PR #14754: URL: https://github.com/apache/kafka/pull/14754#issuecomment-1847943789 Even through the CI appears to have all of these tests pass, I'm getting some strange failures locally that i'll need to investigate more before merge. -- This is an automated

Re: [PR] KAFKA-15364: Handle log directory failure in the Controller [kafka]

2023-12-08 Thread via GitHub
cmccabe merged PR #14902: URL: https://github.com/apache/kafka/pull/14902 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] MINOR: allow JBOD during ZK migration [kafka]

2023-12-08 Thread via GitHub
cmccabe merged PR #14968: URL: https://github.com/apache/kafka/pull/14968 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

2023-12-08 Thread via GitHub
lianetm commented on PR #14557: URL: https://github.com/apache/kafka/pull/14557#issuecomment-1847909912 hey @dajac, this is ready for review now now, repurposed to include general improvement for the commit/fetch, including support for the new v9 errors too. Thanks! -- This is an

[jira] [Commented] (KAFKA-10457) JsonConverter.toConnectData trims BigInteger to Long for schema-less case

2023-12-08 Thread Waleed Fateem (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17794854#comment-17794854 ] Waleed Fateem commented on KAFKA-10457: --- [~sagarrao] we ran into this issue recently. In our

Re: [PR] KAFKA-15784: Ensure atomicity of in memory update and write when transactionally committing offsets [kafka]

2023-12-08 Thread via GitHub
hachikuji commented on code in PR #14774: URL: https://github.com/apache/kafka/pull/14774#discussion_r1421011528 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -976,6 +975,231 @@ class ReplicaManager(val config: KafkaConfig, } } + /** + * Append

Re: [PR] MINOR: Bug fixes for AssignmentsManager [kafka]

2023-12-08 Thread via GitHub
OmniaGM commented on PR #14954: URL: https://github.com/apache/kafka/pull/14954#issuecomment-1847807559 @rondagostino @pprovenzano @cmccabe integration tests are failing on the trunk because of the following error ``` [2023-12-07 19:44:18,763] ERROR Unexpected error handling

Re: [PR] KAFKA-15363: Broker log directory failure changes [kafka]

2023-12-08 Thread via GitHub
OmniaGM commented on PR #14790: URL: https://github.com/apache/kafka/pull/14790#issuecomment-1847797722 > I noticed PlaintextAdminIntegrationTest failing on all versions. It seems to have started on this commit #14790 Can we please take a look. > > trunk build before:

Re: [PR] KAFKA-15784: Ensure atomicity of in memory update and write when transactionally committing offsets [kafka]

2023-12-08 Thread via GitHub
jolshan commented on code in PR #14774: URL: https://github.com/apache/kafka/pull/14774#discussion_r1420993155 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -976,6 +975,239 @@ class ReplicaManager(val config: KafkaConfig, } } + /** + * Append

Re: [PR] KAFKA-15978: Update member information on HB response [kafka]

2023-12-08 Thread via GitHub
jonchiu commented on PR #14945: URL: https://github.com/apache/kafka/pull/14945#issuecomment-1847788894 Looking at the build details, it appears the tests passed. It's hung up on processing test results. -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] KAFKA-15366: Modify LogDirFailureTest for KRaft [kafka]

2023-12-08 Thread via GitHub
soarez commented on PR #14977: URL: https://github.com/apache/kafka/pull/14977#issuecomment-1847778976 @viktorsomogyi: Thanks for starting this! Right, we'll need #14881, #14902, #14954 and [KAFKA-15893](https://issues.apache.org/jira/browse/KAFKA-15893) to run this. I'm going to

Re: [PR] KAFKA-13328, KAFKA-13329 (1): Add preflight validations for key, value, and header converter classes [kafka]

2023-12-08 Thread via GitHub
C0urante merged PR #14304: URL: https://github.com/apache/kafka/pull/14304 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-13328, KAFKA-13329 (1): Add preflight validations for key, value, and header converter classes [kafka]

2023-12-08 Thread via GitHub
C0urante commented on PR #14304: URL: https://github.com/apache/kafka/pull/14304#issuecomment-1847772256 Thanks Greg! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] KAFKA-15347: implement lazy deserialization for segment [kafka]

2023-12-08 Thread via GitHub
aliehsaeedii commented on code in PR #14957: URL: https://github.com/apache/kafka/pull/14957#discussion_r1420974212 ## streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBVersionedStoreSegmentValueFormatter.java: ## @@ -340,32 +355,67 @@ public

Re: [PR] KAFKA-15906: Emit latest MM2 offset syncs every offset.flush.interval.ms [kafka]

2023-12-08 Thread via GitHub
gharris1727 commented on code in PR #14967: URL: https://github.com/apache/kafka/pull/14967#discussion_r1420972701 ## connect/mirror/src/test/java/org/apache/kafka/connect/mirror/MirrorSourceTaskTest.java: ## @@ -407,10 +407,30 @@ public void testSendSyncEvent() {

Re: [PR] KAFKA-15906: Emit latest MM2 offset syncs every offset.flush.interval.ms [kafka]

2023-12-08 Thread via GitHub
C0urante commented on code in PR #14967: URL: https://github.com/apache/kafka/pull/14967#discussion_r1420968099 ## connect/mirror/src/test/java/org/apache/kafka/connect/mirror/MirrorSourceTaskTest.java: ## @@ -407,10 +407,30 @@ public void testSendSyncEvent() {

Re: [PR] KAFKA-15988: Reuse embedded clusters across test cases in Connect OffsetsApiIntegrationTest suite [kafka]

2023-12-08 Thread via GitHub
C0urante commented on code in PR #14966: URL: https://github.com/apache/kafka/pull/14966#discussion_r1420935251 ## connect/runtime/src/test/java/org/apache/kafka/connect/integration/OffsetsApiIntegrationTest.java: ## @@ -709,104 +756,124 @@ public void

Re: [PR] KAFKA-15988: Reuse embedded clusters across test cases in Connect OffsetsApiIntegrationTest suite [kafka]

2023-12-08 Thread via GitHub
C0urante commented on code in PR #14966: URL: https://github.com/apache/kafka/pull/14966#discussion_r1420935041 ## connect/runtime/src/test/java/org/apache/kafka/connect/integration/OffsetsApiIntegrationTest.java: ## @@ -324,127 +363,151 @@ public void

Re: [PR] KAFKA-15784: Ensure atomicity of in memory update and write when transactionally committing offsets [kafka]

2023-12-08 Thread via GitHub
artemlivshits commented on code in PR #14774: URL: https://github.com/apache/kafka/pull/14774#discussion_r1420911850 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -976,6 +975,231 @@ class ReplicaManager(val config: KafkaConfig, } } + /** + *

Re: [PR] MINOR: allow JBOD during ZK migration [kafka]

2023-12-08 Thread via GitHub
pprovenzano commented on PR #14968: URL: https://github.com/apache/kafka/pull/14968#issuecomment-1847703403 Okay, I'm comfortable with this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] KAFKA-15365: Broker-side replica management changes [kafka]

2023-12-08 Thread via GitHub
soarez commented on code in PR #14881: URL: https://github.com/apache/kafka/pull/14881#discussion_r1420887859 ## metadata/src/main/java/org/apache/kafka/image/TopicDelta.java: ## @@ -147,7 +149,7 @@ public LocalReplicaChanges localChanges(int brokerId) { );

Re: [PR] MINOR: Remove `SubscribedTopicRegex` field from `ConsumerGroupHeartbeatRequest` [kafka]

2023-12-08 Thread via GitHub
jolshan commented on PR #14956: URL: https://github.com/apache/kafka/pull/14956#issuecomment-1847651374 https://github.com/apache/kafka/pull/14790#issuecomment-1847650922 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] KAFKA-15363: Broker log directory failure changes [kafka]

2023-12-08 Thread via GitHub
jolshan commented on PR #14790: URL: https://github.com/apache/kafka/pull/14790#issuecomment-1847650922 I noticed PlaintextAdminIntegrationTest failing on all versions. It seems to have started on this commit https://github.com/apache/kafka/pull/14790 Can we please take a look.

Re: [PR] Minor: JBOD migration [kafka]

2023-12-08 Thread via GitHub
cmccabe commented on PR #14968: URL: https://github.com/apache/kafka/pull/14968#issuecomment-1847647420 @jolshan : I think that is fixed by #14954 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] KAFKA-15365: Broker-side replica management changes [kafka]

2023-12-08 Thread via GitHub
OmniaGM commented on code in PR #14881: URL: https://github.com/apache/kafka/pull/14881#discussion_r1420879420 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -2626,8 +2601,16 @@ class ReplicaManager(val config: KafkaConfig, localLeaders.forKeyValue { (tp,

Re: [PR] KAFKA-15365: Broker-side replica management changes [kafka]

2023-12-08 Thread via GitHub
OmniaGM commented on code in PR #14881: URL: https://github.com/apache/kafka/pull/14881#discussion_r1420878757 ## metadata/src/main/java/org/apache/kafka/image/TopicDelta.java: ## @@ -158,11 +160,21 @@ public LocalReplicaChanges localChanges(int brokerId) {

Re: [PR] KAFKA-15365: Broker-side replica management changes [kafka]

2023-12-08 Thread via GitHub
OmniaGM commented on code in PR #14881: URL: https://github.com/apache/kafka/pull/14881#discussion_r1420878519 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -2626,8 +2601,16 @@ class ReplicaManager(val config: KafkaConfig, localLeaders.forKeyValue { (tp,

Re: [PR] MINOR: Remove `SubscribedTopicRegex` field from `ConsumerGroupHeartbeatRequest` [kafka]

2023-12-08 Thread via GitHub
jolshan commented on PR #14956: URL: https://github.com/apache/kafka/pull/14956#issuecomment-1847637876 Looks to be on trunk itself. Please double check the build for tests though. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] KAFKA-15365: Broker-side replica management changes [kafka]

2023-12-08 Thread via GitHub
soarez commented on code in PR #14881: URL: https://github.com/apache/kafka/pull/14881#discussion_r1420873183 ## metadata/src/main/java/org/apache/kafka/image/TopicDelta.java: ## @@ -158,11 +160,21 @@ public LocalReplicaChanges localChanges(int brokerId) {

Re: [PR] MINOR: Remove `SubscribedTopicRegex` field from `ConsumerGroupHeartbeatRequest` [kafka]

2023-12-08 Thread via GitHub
jolshan commented on PR #14956: URL: https://github.com/apache/kafka/pull/14956#issuecomment-1847636302 I noticed PlaintextIntegrationTest failling on all builds. Is that from this change or something that got merged incorrectly? -- This is an automated message from the Apache Git

Re: [PR] KAFKA-15361: Migrating brokers register with dirs [kafka]

2023-12-08 Thread via GitHub
cmccabe merged PR #14976: URL: https://github.com/apache/kafka/pull/14976 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-15365: Broker-side replica management changes [kafka]

2023-12-08 Thread via GitHub
soarez commented on code in PR #14881: URL: https://github.com/apache/kafka/pull/14881#discussion_r1420866994 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -2626,8 +2601,16 @@ class ReplicaManager(val config: KafkaConfig, localLeaders.forKeyValue { (tp,

Re: [PR] KAFKA-15361: Migrating brokers register with dirs [kafka]

2023-12-08 Thread via GitHub
cmccabe commented on PR #14976: URL: https://github.com/apache/kafka/pull/14976#issuecomment-1847625068 > We can ignore this PR in favor of https://github.com/apache/kafka/pull/14968 This one has a green build so let's merge it to fix the tests. We can merge your PR after (I'll

Re: [PR] KAFKA-15364: Handle log directory failure in the Controller [kafka]

2023-12-08 Thread via GitHub
cmccabe commented on PR #14902: URL: https://github.com/apache/kafka/pull/14902#issuecomment-1847619920 Thanks @soarez . LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] KAFKA-15818: ensure leave group on max poll interval [kafka]

2023-12-08 Thread via GitHub
philipnee commented on PR #14873: URL: https://github.com/apache/kafka/pull/14873#issuecomment-1847607910 @lianetm - Thanks for the feedback and I've made some changes according to your comment. -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] Minor: JBOD migration [kafka]

2023-12-08 Thread via GitHub
cmccabe commented on code in PR #14968: URL: https://github.com/apache/kafka/pull/14968#discussion_r1420859373 ## core/src/main/scala/kafka/server/KafkaConfig.scala: ## @@ -2406,7 +2406,11 @@ class KafkaConfig private(doLog: Boolean, val props: java.util.Map[_, _], dynami

Re: [PR] KAFKA-15662: Add support for clientInstanceIds in Kafka Stream [kafka]

2023-12-08 Thread via GitHub
mjsax merged PR #14935: URL: https://github.com/apache/kafka/pull/14935 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-15818: ensure leave group on max poll interval [kafka]

2023-12-08 Thread via GitHub
philipnee commented on code in PR #14873: URL: https://github.com/apache/kafka/pull/14873#discussion_r1420828282 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -587,6 +599,18 @@ public boolean shouldSkipHeartbeat() {

Re: [PR] Minor: JBOD migration [kafka]

2023-12-08 Thread via GitHub
soarez commented on code in PR #14968: URL: https://github.com/apache/kafka/pull/14968#discussion_r1420824321 ## core/src/main/scala/kafka/server/KafkaServer.scala: ## @@ -416,7 +416,8 @@ class KafkaServer( lifecycleManager = new BrokerLifecycleManager(config,

[jira] [Updated] (KAFKA-15208) Upgrade Jackson dependencies to version 2.15.6

2023-12-08 Thread Said BOUDJELDA (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Said BOUDJELDA updated KAFKA-15208: --- Description: Upgrading the version of Jackson dependencies to the latest stable version

[jira] [Updated] (KAFKA-15208) Upgrade Jackson dependencies to version 2.16.0

2023-12-08 Thread Said BOUDJELDA (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Said BOUDJELDA updated KAFKA-15208: --- Summary: Upgrade Jackson dependencies to version 2.16.0 (was: Upgrade Jackson dependencies

[jira] [Updated] (KAFKA-15208) Upgrade Jackson dependencies to version 2.15.6

2023-12-08 Thread Said BOUDJELDA (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Said BOUDJELDA updated KAFKA-15208: --- Summary: Upgrade Jackson dependencies to version 2.15.6 (was: Upgrade Jackson dependencies

Re: [PR] KAFKA-15208: Upgrade Jackson dependencies to version 2.15.3 [kafka]

2023-12-08 Thread via GitHub
bmscomp commented on PR #13662: URL: https://github.com/apache/kafka/pull/13662#issuecomment-1847567651 @mimaison I already upgraded to version 2.16.0 and updated the pull request -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] KAFKA-15784: Ensure atomicity of in memory update and write when transactionally committing offsets [kafka]

2023-12-08 Thread via GitHub
jolshan commented on code in PR #14774: URL: https://github.com/apache/kafka/pull/14774#discussion_r1420797233 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -976,6 +975,231 @@ class ReplicaManager(val config: KafkaConfig, } } + /** + * Append

Re: [PR] KAFKA-15818: ensure leave group on max poll interval [kafka]

2023-12-08 Thread via GitHub
philipnee commented on code in PR #14873: URL: https://github.com/apache/kafka/pull/14873#discussion_r1420817920 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -587,6 +599,18 @@ public boolean shouldSkipHeartbeat() {

Re: [PR] KAFKA-15818: ensure leave group on max poll interval [kafka]

2023-12-08 Thread via GitHub
philipnee commented on code in PR #14873: URL: https://github.com/apache/kafka/pull/14873#discussion_r1420804236 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -587,6 +599,18 @@ public boolean shouldSkipHeartbeat() {

Re: [PR] KAFKA-15696: Refactor closing consumer [kafka]

2023-12-08 Thread via GitHub
philipnee commented on code in PR #14937: URL: https://github.com/apache/kafka/pull/14937#discussion_r1420798994 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerNetworkThread.java: ## @@ -273,80 +250,28 @@ private void closeInternal(final Duration

Re: [PR] KAFKA-15361: Migrating brokers register with dirs [kafka]

2023-12-08 Thread via GitHub
pprovenzano commented on PR #14976: URL: https://github.com/apache/kafka/pull/14976#issuecomment-1847542364 We can ignore this PR in favor of https://github.com/apache/kafka/pull/14968 -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] KAFKA-15784: Ensure atomicity of in memory update and write when transactionally committing offsets [kafka]

2023-12-08 Thread via GitHub
jolshan commented on code in PR #14774: URL: https://github.com/apache/kafka/pull/14774#discussion_r1420797233 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -976,6 +975,231 @@ class ReplicaManager(val config: KafkaConfig, } } + /** + * Append

[jira] [Updated] (KAFKA-15987) Refactor ReplicaManager code for transaction verification

2023-12-08 Thread Justine Olshan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justine Olshan updated KAFKA-15987: --- Summary: Refactor ReplicaManager code for transaction verification (was: Refactor

Re: [PR] KAFKA-15784: Ensure atomicity of in memory update and write when transactionally committing offsets [kafka]

2023-12-08 Thread via GitHub
jolshan commented on code in PR #14774: URL: https://github.com/apache/kafka/pull/14774#discussion_r1420791705 ## core/src/main/scala/kafka/coordinator/group/GroupMetadataManager.scala: ## @@ -339,156 +340,183 @@ class GroupMetadataManager(brokerId: Int,

Re: [PR] KAFKA-15696: Refactor closing consumer [kafka]

2023-12-08 Thread via GitHub
philipnee commented on code in PR #14937: URL: https://github.com/apache/kafka/pull/14937#discussion_r1420786217 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerNetworkThread.java: ## @@ -178,27 +171,11 @@ static void runAtClose(final Collection>

Re: [PR] KAFKA-15365: Broker-side replica management changes [kafka]

2023-12-08 Thread via GitHub
OmniaGM commented on code in PR #14881: URL: https://github.com/apache/kafka/pull/14881#discussion_r1420746154 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -2626,8 +2601,16 @@ class ReplicaManager(val config: KafkaConfig, localLeaders.forKeyValue { (tp,

Re: [PR] KAFKA-15365: Broker-side replica management changes [kafka]

2023-12-08 Thread via GitHub
soarez commented on code in PR #14881: URL: https://github.com/apache/kafka/pull/14881#discussion_r1420734792 ## metadata/src/main/java/org/apache/kafka/image/TopicDelta.java: ## @@ -158,11 +160,21 @@ public LocalReplicaChanges localChanges(int brokerId) {

Re: [PR] KAFKA-15365: Broker-side replica management changes [kafka]

2023-12-08 Thread via GitHub
soarez commented on code in PR #14881: URL: https://github.com/apache/kafka/pull/14881#discussion_r1420691621 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -2626,8 +2601,12 @@ class ReplicaManager(val config: KafkaConfig, localLeaders.forKeyValue { (tp,

Re: [PR] KAFKA-15365: Broker-side replica management changes [kafka]

2023-12-08 Thread via GitHub
OmniaGM commented on code in PR #14881: URL: https://github.com/apache/kafka/pull/14881#discussion_r1420732241 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -2626,8 +2601,12 @@ class ReplicaManager(val config: KafkaConfig, localLeaders.forKeyValue { (tp,

Re: [PR] KAFKA-15365: Broker-side replica management changes [kafka]

2023-12-08 Thread via GitHub
OmniaGM commented on code in PR #14881: URL: https://github.com/apache/kafka/pull/14881#discussion_r1420729137 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -2666,8 +2648,8 @@ class ReplicaManager(val config: KafkaConfig, // is unavailable. This

Re: [PR] KAFKA-15971: Re-enable consumer integration tests for new consumer [kafka]

2023-12-08 Thread via GitHub
AndrewJSchofield commented on PR #14925: URL: https://github.com/apache/kafka/pull/14925#issuecomment-1847452190 Clean test run for me. Rebased the PR, which shouldn't make any difference to the code being used. -- This is an automated message from the Apache Git Service. To respond to

Re: [PR] KAFKA-15365: Broker-side replica management changes [kafka]

2023-12-08 Thread via GitHub
soarez commented on code in PR #14881: URL: https://github.com/apache/kafka/pull/14881#discussion_r1420691621 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -2626,8 +2601,12 @@ class ReplicaManager(val config: KafkaConfig, localLeaders.forKeyValue { (tp,

Re: [PR] KAFKA-15365: Broker-side replica management changes [kafka]

2023-12-08 Thread via GitHub
soarez commented on code in PR #14881: URL: https://github.com/apache/kafka/pull/14881#discussion_r1420675158 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -2666,8 +2648,8 @@ class ReplicaManager(val config: KafkaConfig, // is unavailable. This is

Re: [PR] KAFKA-15347: implement lazy deserialization for segment [kafka]

2023-12-08 Thread via GitHub
aliehsaeedii commented on code in PR #14957: URL: https://github.com/apache/kafka/pull/14957#discussion_r1420648455 ## streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBVersionedStoreSegmentValueFormatter.java: ## @@ -340,32 +355,67 @@ public

Re: [PR] KAFKA-15347: implement lazy deserialization for segment [kafka]

2023-12-08 Thread via GitHub
aliehsaeedii commented on code in PR #14957: URL: https://github.com/apache/kafka/pull/14957#discussion_r1420641426 ## streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBVersionedStoreSegmentValueFormatter.java: ## @@ -340,32 +355,67 @@ public

[jira] [Commented] (KAFKA-15985) Mirrormaker 2 offset sync is incomplete

2023-12-08 Thread Philipp Dallig (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17794764#comment-17794764 ] Philipp Dallig commented on KAFKA-15985: Hi [~gharris1727] & [~ChrisEgerton], after I changed

[jira] [Resolved] (KAFKA-15985) Mirrormaker 2 offset sync is incomplete

2023-12-08 Thread Philipp Dallig (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philipp Dallig resolved KAFKA-15985. Resolution: Fixed > Mirrormaker 2 offset sync is incomplete >

Re: [PR] KAFKA-15971: Re-enable consumer integration tests for new consumer [kafka]

2023-12-08 Thread via GitHub
AndrewJSchofield commented on PR #14925: URL: https://github.com/apache/kafka/pull/14925#issuecomment-1847345135 Interesting. The consumer integration tests are all over the floor for the new consumer. I got a clean run from trunk earlier this morning. I'll take a look a bit later. --

Re: [PR] Minor: JBOD migration [kafka]

2023-12-08 Thread via GitHub
rondagostino commented on code in PR #14968: URL: https://github.com/apache/kafka/pull/14968#discussion_r1420576804 ## core/src/main/scala/kafka/server/KafkaServer.scala: ## @@ -416,7 +416,8 @@ class KafkaServer( lifecycleManager = new BrokerLifecycleManager(config,

Re: [PR] KAFKA-15818: ensure leave group on max poll interval [kafka]

2023-12-08 Thread via GitHub
lianetm commented on code in PR #14873: URL: https://github.com/apache/kafka/pull/14873#discussion_r1420598917 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -587,6 +599,18 @@ public boolean shouldSkipHeartbeat() {

Re: [PR] KAFKA-15696: Refactor closing consumer [kafka]

2023-12-08 Thread via GitHub
lucasbru commented on code in PR #14937: URL: https://github.com/apache/kafka/pull/14937#discussion_r1420557352 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerNetworkThread.java: ## @@ -178,27 +171,11 @@ static void runAtClose(final Collection>

Re: [PR] KAFKA-15971: Re-enable consumer integration tests for new consumer [kafka]

2023-12-08 Thread via GitHub
dajac commented on PR #14925: URL: https://github.com/apache/kafka/pull/14925#issuecomment-1847317929 @AndrewJSchofield The last run has a lot of failed tests. Do you know what is causing them? -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] KAFKA-15364: Handle log directory failure in the Controller [kafka]

2023-12-08 Thread via GitHub
soarez commented on PR #14902: URL: https://github.com/apache/kafka/pull/14902#issuecomment-1847312525 @rondagostino : it was an edge case in 1bb14ac19757f17311bf7d296a8f3b39259f7438 `updatePartitionDirectories`. I think I've fixed it now. @cmccabe : thanks for making the changes.

Re: [PR] KAFKA-15818: ensure leave group on max poll interval [kafka]

2023-12-08 Thread via GitHub
lianetm commented on code in PR #14873: URL: https://github.com/apache/kafka/pull/14873#discussion_r1420570473 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -587,6 +599,18 @@ public boolean shouldSkipHeartbeat() {

Re: [PR] KAFKA-15818: ensure leave group on max poll interval [kafka]

2023-12-08 Thread via GitHub
lianetm commented on code in PR #14873: URL: https://github.com/apache/kafka/pull/14873#discussion_r1420576356 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -587,6 +599,18 @@ public boolean shouldSkipHeartbeat() {

Re: [PR] KAFKA-15818: ensure leave group on max poll interval [kafka]

2023-12-08 Thread via GitHub
lianetm commented on code in PR #14873: URL: https://github.com/apache/kafka/pull/14873#discussion_r1420570473 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -587,6 +599,18 @@ public boolean shouldSkipHeartbeat() {

Re: [PR] KAFKA-15208: Upgrade Jackson dependencies to version 2.15.3 [kafka]

2023-12-08 Thread via GitHub
bmscomp commented on code in PR #13662: URL: https://github.com/apache/kafka/pull/13662#discussion_r1420566217 ## LICENSE-binary: ## @@ -216,16 +216,16 @@ commons-lang3-3.8.1 commons-logging-1.2 commons-validator-1.7 error_prone_annotations-2.10.0 -jackson-annotations-2.13.5

Re: [PR] KAFKA-15208: Upgrade Jackson dependencies to version 2.15.3 [kafka]

2023-12-08 Thread via GitHub
bmscomp commented on code in PR #13662: URL: https://github.com/apache/kafka/pull/13662#discussion_r1420566217 ## LICENSE-binary: ## @@ -216,16 +216,16 @@ commons-lang3-3.8.1 commons-logging-1.2 commons-validator-1.7 error_prone_annotations-2.10.0 -jackson-annotations-2.13.5

Re: [PR] KAFKA-15364: Handle log directory failure in the Controller [kafka]

2023-12-08 Thread via GitHub
soarez commented on code in PR #14902: URL: https://github.com/apache/kafka/pull/14902#discussion_r1420565421 ## metadata/src/main/java/org/apache/kafka/metadata/BrokerRegistration.java: ## @@ -159,7 +159,7 @@ public static Optional zkBrokerEpoch(long value) { private

Re: [PR] KAFKA-15714: KRaft support in DynamicNumNetworkThreadsTest [kafka]

2023-12-08 Thread via GitHub
phong260702 commented on code in PR #14970: URL: https://github.com/apache/kafka/pull/14970#discussion_r1420556973 ## core/src/test/scala/integration/kafka/network/DynamicNumNetworkThreadsTest.scala: ## @@ -44,19 +48,29 @@ class DynamicNumNetworkThreadsTest extends

Re: [PR] KAFKA-12216; Improve flaky test reporting [kafka]

2023-12-08 Thread via GitHub
dajac commented on PR #14862: URL: https://github.com/apache/kafka/pull/14862#issuecomment-1847277446 We reverted https://github.com/apache/kafka/pull/14961 because it created flakiness in Streams' tests. We should bring it back before we can continue with this one. -- This is an

Re: [PR] KAFKA-15365: Broker-side replica management changes [kafka]

2023-12-08 Thread via GitHub
OmniaGM commented on code in PR #14881: URL: https://github.com/apache/kafka/pull/14881#discussion_r1420552612 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -2666,8 +2648,8 @@ class ReplicaManager(val config: KafkaConfig, // is unavailable. This

  1   2   >