[GitHub] [kafka] KarboniteKream commented on pull request #13679: KAFKA-14291: KRaft controller should return right finalized features in ApiVersionResponse

2023-05-24 Thread via GitHub
KarboniteKream commented on PR #13679: URL: https://github.com/apache/kafka/pull/13679#issuecomment-1562213887 Thank you, I'll check today! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [kafka] dengziming commented on pull request #13679: KAFKA-14291: KRaft controller should return right finalized features in ApiVersionResponse

2023-05-24 Thread via GitHub
dengziming commented on PR #13679: URL: https://github.com/apache/kafka/pull/13679#issuecomment-1562174228 @KarboniteKream I reproduced this in #13761 , you can run the new test case and help investigating if you are interested. -- This is an automated message from the Apache Git

[GitHub] [kafka] dengziming opened a new pull request, #13761: Reproduce KAFKA-14996, Test restart controller

2023-05-24 Thread via GitHub
dengziming opened a new pull request, #13761: URL: https://github.com/apache/kafka/pull/13761 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of

[GitHub] [kafka] jeffkbkim commented on a diff in pull request #13639: KAFKA-14462; [12/N] Add GroupMetadataManager and ConsumerGroup

2023-05-24 Thread via GitHub
jeffkbkim commented on code in PR #13639: URL: https://github.com/apache/kafka/pull/13639#discussion_r1204875457 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupMetadataManagerTest.java: ## @@ -0,0 +1,2017 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] jeffkbkim commented on a diff in pull request #13639: KAFKA-14462; [12/N] Add GroupMetadataManager and ConsumerGroup

2023-05-24 Thread via GitHub
jeffkbkim commented on code in PR #13639: URL: https://github.com/apache/kafka/pull/13639#discussion_r1204874200 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/consumer/ConsumerGroupTest.java: ## @@ -0,0 +1,500 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] jeffkbkim commented on a diff in pull request #13639: KAFKA-14462; [12/N] Add GroupMetadataManager and ConsumerGroup

2023-05-24 Thread via GitHub
jeffkbkim commented on code in PR #13639: URL: https://github.com/apache/kafka/pull/13639#discussion_r1204872093 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/consumer/ConsumerGroupTest.java: ## @@ -0,0 +1,500 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] jeffkbkim commented on a diff in pull request #13639: KAFKA-14462; [12/N] Add GroupMetadataManager and ConsumerGroup

2023-05-24 Thread via GitHub
jeffkbkim commented on code in PR #13639: URL: https://github.com/apache/kafka/pull/13639#discussion_r1194455250 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -0,0 +1,868 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] tinaselenge opened a new pull request, #13760: KAFKA-8982: Add retry of fetching metadata to Admin.deleteRecords

2023-05-24 Thread via GitHub
tinaselenge opened a new pull request, #13760: URL: https://github.com/apache/kafka/pull/13760 The retry of fetching metadata is handled by the AdminApiDriver class. *Summary of testing strategy (including rationale) for the feature or bug fix. Unit and/or integration tests are

[GitHub] [kafka] jeffkbkim commented on a diff in pull request #13267: KAFKA-14694: RPCProducerIdManager should not wait on new block

2023-05-24 Thread via GitHub
jeffkbkim commented on code in PR #13267: URL: https://github.com/apache/kafka/pull/13267#discussion_r1204862724 ## core/src/main/scala/kafka/coordinator/transaction/ProducerIdManager.scala: ## @@ -123,73 +130,107 @@ class ZkProducerIdManager(brokerId: Int, } } - def

[GitHub] [kafka] cmccabe opened a new pull request, #13759: KAFKA-15019: Improve handling of overload situations in the kcontroller

2023-05-24 Thread via GitHub
cmccabe opened a new pull request, #13759: URL: https://github.com/apache/kafka/pull/13759 When the active KRaft controller is overloaded, it will not be able to process broker heartbeat requests. Instead, they will be timed out. When using the default configuration, this will happen

[jira] [Created] (KAFKA-15019) Improve handling of overload situations in the kcontroller

2023-05-24 Thread Colin McCabe (Jira)
Colin McCabe created KAFKA-15019: Summary: Improve handling of overload situations in the kcontroller Key: KAFKA-15019 URL: https://issues.apache.org/jira/browse/KAFKA-15019 Project: Kafka

[jira] [Resolved] (KAFKA-14658) When listening on fixed ports, defer port opening until we're ready

2023-05-24 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-14658. -- Resolution: Fixed > When listening on fixed ports, defer port opening until we're ready >

[jira] [Resolved] (KAFKA-14835) Create ControllerServerMetricsPublisher

2023-05-24 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-14835. -- Resolution: Fixed > Create ControllerServerMetricsPublisher >

[jira] [Resolved] (KAFKA-14943) Fix ClientQuotaControlManager validation

2023-05-24 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-14943. -- Fix Version/s: 3.5.0 Resolution: Fixed > Fix ClientQuotaControlManager validation >

[jira] [Resolved] (KAFKA-14857) Fix some MetadataLoader bugs

2023-05-24 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-14857. -- Resolution: Fixed > Fix some MetadataLoader bugs > > >

[GitHub] [kafka] mumrah opened a new pull request, #13758: KAFKA-15010 ZK migration failover support

2023-05-24 Thread via GitHub
mumrah opened a new pull request, #13758: URL: https://github.com/apache/kafka/pull/13758 Previously, if the KRaft controller failed over while metadata changes was pending in the KRaftMigrationDriver queue, we would lose those writes to ZK. This patch adds snapshot reconciliation so that

[jira] [Commented] (KAFKA-8154) Buffer Overflow exceptions between brokers and with clients

2023-05-24 Thread keith.paulson (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17725975#comment-17725975 ] keith.paulson commented on KAFKA-8154: -- [~tmancill] I used almost the same patch as you did, even

[jira] [Assigned] (KAFKA-15010) KRaft Controller doesn't reconcile with Zookeeper metadata upon becoming new controller while in dual write mode.

2023-05-24 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe reassigned KAFKA-15010: Assignee: David Arthur > KRaft Controller doesn't reconcile with Zookeeper metadata upon

[jira] [Resolved] (KAFKA-15009) New ACLs are not written to ZK during migration

2023-05-24 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-15009. -- Fix Version/s: 3.5.0 Resolution: Fixed > New ACLs are not written to ZK during

[GitHub] [kafka] pprovenzano opened a new pull request, #13757: [WIP] Fix snapshot load during dual write.

2023-05-24 Thread via GitHub
pprovenzano opened a new pull request, #13757: URL: https://github.com/apache/kafka/pull/13757 …the objects that are only in ZK. Then reverse migrate as appropriate *More detailed description of your change, if necessary. The PR title and PR message become the squashed

[GitHub] [kafka] wcarlson5 commented on pull request #13756: KAFKA-14936: Add On Disk Time Ordered Buffer (1/N)

2023-05-24 Thread via GitHub
wcarlson5 commented on PR #13756: URL: https://github.com/apache/kafka/pull/13756#issuecomment-1561954150 @vcrfxia Can you take a look at this when you get a chance? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [kafka] kirktrue commented on a diff in pull request #13591: KAFKA-14831: Illegal state errors should be fatal in transactional producer

2023-05-24 Thread via GitHub
kirktrue commented on code in PR #13591: URL: https://github.com/apache/kafka/pull/13591#discussion_r1204760045 ## clients/src/main/java/org/apache/kafka/clients/producer/internals/TransactionManager.java: ## @@ -626,12 +704,14 @@ public synchronized void

[GitHub] [kafka] mumrah merged pull request #13729: KAFKA-14970: Dual write mode testing for SCRAM and Quota

2023-05-24 Thread via GitHub
mumrah merged PR #13729: URL: https://github.com/apache/kafka/pull/13729 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] wcarlson5 opened a new pull request, #13756: KAFKA-14936: Add On Disk Time Ordered Buffer (1/N)

2023-05-24 Thread via GitHub
wcarlson5 opened a new pull request, #13756: URL: https://github.com/apache/kafka/pull/13756 Add a time ordered key-value buffer stored on dick and implemented using RocksDBTimeOrderedKeyValueSegmentedBytesStore. This will be used in the stream buffered for joins with a grace period.

[GitHub] [kafka] C0urante commented on a diff in pull request #13465: KAFKA-14368: Connect offset write REST API

2023-05-24 Thread via GitHub
C0urante commented on code in PR #13465: URL: https://github.com/apache/kafka/pull/13465#discussion_r1204681794 ## connect/runtime/src/test/java/org/apache/kafka/connect/runtime/distributed/DistributedHerderTest.java: ## @@ -4123,6 +4136,280 @@ public void

[GitHub] [kafka] pprovenzano commented on a diff in pull request #13729: KAFKA-14970: Dual write mode testing for SCRAM and Quota

2023-05-24 Thread via GitHub
pprovenzano commented on code in PR #13729: URL: https://github.com/apache/kafka/pull/13729#discussion_r1204709476 ## core/src/test/scala/integration/kafka/zk/ZkMigrationIntegrationTest.scala: ## @@ -187,13 +188,74 @@ class ZkMigrationIntegrationTest { migrationState =

[GitHub] [kafka] mimaison merged pull request #13752: MINOR: fix typo in Connect Config documentation

2023-05-24 Thread via GitHub
mimaison merged PR #13752: URL: https://github.com/apache/kafka/pull/13752 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] mumrah commented on a diff in pull request #13729: KAFKA-14970: Dual write mode testing for SCRAM and Quota

2023-05-24 Thread via GitHub
mumrah commented on code in PR #13729: URL: https://github.com/apache/kafka/pull/13729#discussion_r1204650002 ## core/src/test/scala/integration/kafka/zk/ZkMigrationIntegrationTest.scala: ## @@ -187,13 +188,74 @@ class ZkMigrationIntegrationTest { migrationState =

[GitHub] [kafka] dajac commented on pull request #13754: MINOR: Hopefully fix flaky FetchFromFollowerIntegrationTest.testRackAwareRangeAssignor

2023-05-24 Thread via GitHub
dajac commented on PR #13754: URL: https://github.com/apache/kafka/pull/13754#issuecomment-1561771385 @rajinisivaram Could you please review this one? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [kafka] jolshan commented on a diff in pull request #13639: KAFKA-14462; [12/N] Add GroupMetadataManager and ConsumerGroup

2023-05-24 Thread via GitHub
jolshan commented on code in PR #13639: URL: https://github.com/apache/kafka/pull/13639#discussion_r1204623123 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/consumer/ConsumerGroupTest.java: ## @@ -0,0 +1,500 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] jolshan commented on a diff in pull request #13639: KAFKA-14462; [12/N] Add GroupMetadataManager and ConsumerGroup

2023-05-24 Thread via GitHub
jolshan commented on code in PR #13639: URL: https://github.com/apache/kafka/pull/13639#discussion_r1204613762 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/consumer/ConsumerGroupTest.java: ## @@ -0,0 +1,500 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] C0urante commented on a diff in pull request #13465: KAFKA-14368: Connect offset write REST API

2023-05-24 Thread via GitHub
C0urante commented on code in PR #13465: URL: https://github.com/apache/kafka/pull/13465#discussion_r1204612735 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/Worker.java: ## @@ -1247,6 +1257,257 @@ void sourceConnectorOffsets(String connName,

[GitHub] [kafka] C0urante commented on a diff in pull request #13465: KAFKA-14368: Connect offset write REST API

2023-05-24 Thread via GitHub
C0urante commented on code in PR #13465: URL: https://github.com/apache/kafka/pull/13465#discussion_r1204612162 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/Worker.java: ## @@ -1247,6 +1257,257 @@ void sourceConnectorOffsets(String connName,

[GitHub] [kafka] jolshan commented on a diff in pull request #13639: KAFKA-14462; [12/N] Add GroupMetadataManager and ConsumerGroup

2023-05-24 Thread via GitHub
jolshan commented on code in PR #13639: URL: https://github.com/apache/kafka/pull/13639#discussion_r1204613085 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/consumer/ConsumerGroupTest.java: ## @@ -0,0 +1,500 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] jolshan commented on a diff in pull request #13639: KAFKA-14462; [12/N] Add GroupMetadataManager and ConsumerGroup

2023-05-24 Thread via GitHub
jolshan commented on code in PR #13639: URL: https://github.com/apache/kafka/pull/13639#discussion_r1204611848 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/consumer/ConsumerGroupTest.java: ## @@ -0,0 +1,500 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] jolshan commented on a diff in pull request #13639: KAFKA-14462; [12/N] Add GroupMetadataManager and ConsumerGroup

2023-05-24 Thread via GitHub
jolshan commented on code in PR #13639: URL: https://github.com/apache/kafka/pull/13639#discussion_r1204607029 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/consumer/ConsumerGroupTest.java: ## @@ -0,0 +1,500 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] C0urante commented on a diff in pull request #13465: KAFKA-14368: Connect offset write REST API

2023-05-24 Thread via GitHub
C0urante commented on code in PR #13465: URL: https://github.com/apache/kafka/pull/13465#discussion_r1204604996 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/DistributedHerder.java: ## @@ -1528,6 +1539,80 @@ public void connectorOffsets(String

[GitHub] [kafka] jolshan commented on a diff in pull request #13639: KAFKA-14462; [12/N] Add GroupMetadataManager and ConsumerGroup

2023-05-24 Thread via GitHub
jolshan commented on code in PR #13639: URL: https://github.com/apache/kafka/pull/13639#discussion_r1204591900 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/consumer/ConsumerGroupTest.java: ## @@ -0,0 +1,500 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] jolshan commented on a diff in pull request #13639: KAFKA-14462; [12/N] Add GroupMetadataManager and ConsumerGroup

2023-05-24 Thread via GitHub
jolshan commented on code in PR #13639: URL: https://github.com/apache/kafka/pull/13639#discussion_r1204599222 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/consumer/ConsumerGroupTest.java: ## @@ -0,0 +1,500 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] jolshan commented on a diff in pull request #13639: KAFKA-14462; [12/N] Add GroupMetadataManager and ConsumerGroup

2023-05-24 Thread via GitHub
jolshan commented on code in PR #13639: URL: https://github.com/apache/kafka/pull/13639#discussion_r1204597635 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/consumer/ConsumerGroupTest.java: ## @@ -0,0 +1,500 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] jolshan commented on a diff in pull request #13639: KAFKA-14462; [12/N] Add GroupMetadataManager and ConsumerGroup

2023-05-24 Thread via GitHub
jolshan commented on code in PR #13639: URL: https://github.com/apache/kafka/pull/13639#discussion_r1204594741 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/consumer/ConsumerGroupTest.java: ## @@ -0,0 +1,500 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] jolshan commented on a diff in pull request #13639: KAFKA-14462; [12/N] Add GroupMetadataManager and ConsumerGroup

2023-05-24 Thread via GitHub
jolshan commented on code in PR #13639: URL: https://github.com/apache/kafka/pull/13639#discussion_r1204594741 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/consumer/ConsumerGroupTest.java: ## @@ -0,0 +1,500 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] jolshan commented on a diff in pull request #13639: KAFKA-14462; [12/N] Add GroupMetadataManager and ConsumerGroup

2023-05-24 Thread via GitHub
jolshan commented on code in PR #13639: URL: https://github.com/apache/kafka/pull/13639#discussion_r1204591900 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/consumer/ConsumerGroupTest.java: ## @@ -0,0 +1,500 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] jeffkbkim commented on a diff in pull request #13639: KAFKA-14462; [12/N] Add GroupMetadataManager and ConsumerGroup

2023-05-24 Thread via GitHub
jeffkbkim commented on code in PR #13639: URL: https://github.com/apache/kafka/pull/13639#discussion_r1204467377 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/ConsumerGroup.java: ## @@ -0,0 +1,633 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] jolshan commented on a diff in pull request #13639: KAFKA-14462; [12/N] Add GroupMetadataManager and ConsumerGroup

2023-05-24 Thread via GitHub
jolshan commented on code in PR #13639: URL: https://github.com/apache/kafka/pull/13639#discussion_r1204582235 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/consumer/ConsumerGroupTest.java: ## @@ -0,0 +1,500 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] jolshan commented on a diff in pull request #13639: KAFKA-14462; [12/N] Add GroupMetadataManager and ConsumerGroup

2023-05-24 Thread via GitHub
jolshan commented on code in PR #13639: URL: https://github.com/apache/kafka/pull/13639#discussion_r1204579244 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/consumer/ConsumerGroupTest.java: ## @@ -0,0 +1,500 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] jolshan commented on a diff in pull request #13639: KAFKA-14462; [12/N] Add GroupMetadataManager and ConsumerGroup

2023-05-24 Thread via GitHub
jolshan commented on code in PR #13639: URL: https://github.com/apache/kafka/pull/13639#discussion_r1204564082 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/consumer/ConsumerGroupTest.java: ## @@ -0,0 +1,500 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] jolshan commented on a diff in pull request #13639: KAFKA-14462; [12/N] Add GroupMetadataManager and ConsumerGroup

2023-05-24 Thread via GitHub
jolshan commented on code in PR #13639: URL: https://github.com/apache/kafka/pull/13639#discussion_r1204556978 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupMetadataManagerTest.java: ## @@ -0,0 +1,2017 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] gharris1727 commented on pull request #13748: [BUGFIX] Bugfixed in KAFKA-8713, but it doesn't work properly.

2023-05-24 Thread via GitHub
gharris1727 commented on PR #13748: URL: https://github.com/apache/kafka/pull/13748#issuecomment-1561666475 @krespo It looks like the `replace.null.with.default` feature of the JsonConverter is [due to be released in

[GitHub] [kafka] jolshan commented on a diff in pull request #13639: KAFKA-14462; [12/N] Add GroupMetadataManager and ConsumerGroup

2023-05-24 Thread via GitHub
jolshan commented on code in PR #13639: URL: https://github.com/apache/kafka/pull/13639#discussion_r1204538473 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupMetadataManagerTest.java: ## @@ -0,0 +1,2017 @@ +/* + * Licensed to the Apache Software

[jira] [Updated] (KAFKA-14996) The KRaft controller should properly handle overly large user operations

2023-05-24 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe updated KAFKA-14996: - Summary: The KRaft controller should properly handle overly large user operations (was:

[jira] [Updated] (KAFKA-14996) CreateTopic falis with UnknownServerException if num partitions >= QuorumController.MAX_RECORDS_PER_BATCH

2023-05-24 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe updated KAFKA-14996: - Affects Version/s: 3.5.0 Priority: Blocker (was: Critical) > CreateTopic falis

[GitHub] [kafka] jolshan commented on a diff in pull request #13639: KAFKA-14462; [12/N] Add GroupMetadataManager and ConsumerGroup

2023-05-24 Thread via GitHub
jolshan commented on code in PR #13639: URL: https://github.com/apache/kafka/pull/13639#discussion_r1204535515 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupMetadataManagerTest.java: ## @@ -0,0 +1,2017 @@ +/* + * Licensed to the Apache Software

[jira] [Updated] (KAFKA-15010) KRaft Controller doesn't reconcile with Zookeeper metadata upon becoming new controller while in dual write mode.

2023-05-24 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe updated KAFKA-15010: - Affects Version/s: 3.5.0 > KRaft Controller doesn't reconcile with Zookeeper metadata upon

[jira] [Commented] (KAFKA-13679) Superfluous node disconnected log messages

2023-05-24 Thread Ondrej Cervinka (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17725882#comment-17725882 ] Ondrej Cervinka commented on KAFKA-13679: - Just adding some observations as I am also having

[GitHub] [kafka] dajac commented on a diff in pull request #13639: KAFKA-14462; [12/N] Add GroupMetadataManager and ConsumerGroup

2023-05-24 Thread via GitHub
dajac commented on code in PR #13639: URL: https://github.com/apache/kafka/pull/13639#discussion_r1204523524 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -0,0 +1,859 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] dajac commented on a diff in pull request #13639: KAFKA-14462; [12/N] Add GroupMetadataManager and ConsumerGroup

2023-05-24 Thread via GitHub
dajac commented on code in PR #13639: URL: https://github.com/apache/kafka/pull/13639#discussion_r1204523103 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -0,0 +1,859 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] dajac commented on a diff in pull request #13639: KAFKA-14462; [12/N] Add GroupMetadataManager and ConsumerGroup

2023-05-24 Thread via GitHub
dajac commented on code in PR #13639: URL: https://github.com/apache/kafka/pull/13639#discussion_r1204522037 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -0,0 +1,865 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] gharris1727 commented on pull request #13165: KAFKA-14654: Connector classes should statically initialize with plugin classloader

2023-05-24 Thread via GitHub
gharris1727 commented on PR #13165: URL: https://github.com/apache/kafka/pull/13165#issuecomment-1561615543 @C0urante I added a static list to all of the Sampling plugins that allow us to inspect the classloader used for all method calls to all instances of each plugin type. This should

[GitHub] [kafka] jolshan commented on a diff in pull request #13639: KAFKA-14462; [12/N] Add GroupMetadataManager and ConsumerGroup

2023-05-24 Thread via GitHub
jolshan commented on code in PR #13639: URL: https://github.com/apache/kafka/pull/13639#discussion_r1204494016 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -0,0 +1,859 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] jolshan commented on a diff in pull request #13639: KAFKA-14462; [12/N] Add GroupMetadataManager and ConsumerGroup

2023-05-24 Thread via GitHub
jolshan commented on code in PR #13639: URL: https://github.com/apache/kafka/pull/13639#discussion_r1204493167 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -0,0 +1,859 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] jolshan commented on a diff in pull request #13639: KAFKA-14462; [12/N] Add GroupMetadataManager and ConsumerGroup

2023-05-24 Thread via GitHub
jolshan commented on code in PR #13639: URL: https://github.com/apache/kafka/pull/13639#discussion_r1204492401 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -0,0 +1,859 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] clayburn commented on a diff in pull request #13676: MINOR: Capture build scans on ge.apache.org to benefit from deep build insights

2023-05-24 Thread via GitHub
clayburn commented on code in PR #13676: URL: https://github.com/apache/kafka/pull/13676#discussion_r1204490523 ## build.gradle: ## @@ -39,7 +39,6 @@ plugins { id "io.swagger.core.v3.swagger-gradle-plugin" version "2.2.8" id "com.github.spotbugs" version '5.0.13' apply

[GitHub] [kafka] Hangleton commented on a diff in pull request #13240: KAFKA-14690: Add topic IDs to the OffsetCommit API version 9

2023-05-24 Thread via GitHub
Hangleton commented on code in PR #13240: URL: https://github.com/apache/kafka/pull/13240#discussion_r1204079800 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinatorTest.java: ## @@ -2707,11 +2742,271 @@ public void

[GitHub] [kafka] gharris1727 commented on a diff in pull request #13165: KAFKA-14654: Connector classes should statically initialize with plugin classloader

2023-05-24 Thread via GitHub
gharris1727 commented on code in PR #13165: URL: https://github.com/apache/kafka/pull/13165#discussion_r1204482479 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/isolation/LoaderSwap.java: ## @@ -24,13 +24,32 @@ public class LoaderSwap implements

[GitHub] [kafka] yashmayya commented on a diff in pull request #13750: MINOR: Handle the config topic read timeout edge case in DistributedHerder's stopConnector method

2023-05-24 Thread via GitHub
yashmayya commented on code in PR #13750: URL: https://github.com/apache/kafka/pull/13750#discussion_r1204446383 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/DistributedHerder.java: ## @@ -1104,7 +1104,9 @@ public void stopConnector(final String

[GitHub] [kafka] yashmayya commented on a diff in pull request #13750: MINOR: Handle the config topic read timeout edge case in DistributedHerder's stopConnector method

2023-05-24 Thread via GitHub
yashmayya commented on code in PR #13750: URL: https://github.com/apache/kafka/pull/13750#discussion_r1204446383 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/DistributedHerder.java: ## @@ -1104,7 +1104,9 @@ public void stopConnector(final String

[GitHub] [kafka] ijuma commented on a diff in pull request #13676: MINOR: Capture build scans on ge.apache.org to benefit from deep build insights

2023-05-24 Thread via GitHub
ijuma commented on code in PR #13676: URL: https://github.com/apache/kafka/pull/13676#discussion_r120291 ## build.gradle: ## @@ -39,7 +39,6 @@ plugins { id "io.swagger.core.v3.swagger-gradle-plugin" version "2.2.8" id "com.github.spotbugs" version '5.0.13' apply

[GitHub] [kafka] vamossagar12 opened a new pull request, #13755: [MINOR] Fixing typo in ExactlyOnceWorkerSourceTask#commitableRecords

2023-05-24 Thread via GitHub
vamossagar12 opened a new pull request, #13755: URL: https://github.com/apache/kafka/pull/13755 Renaming filed commitableRecords to committableRecords in ExactlyOnceWorkerSourceTask. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [kafka] divijvaidya commented on pull request #13135: KAFKA-14633: Reduce data copy & buffer allocation during decompression

2023-05-24 Thread via GitHub
divijvaidya commented on PR #13135: URL: https://github.com/apache/kafka/pull/13135#issuecomment-1561407539 Failing tests are unrelated: ``` Build / JDK 11 and Scala 2.13 / [1] quorum=kraft, isIdempotenceEnabled=true – kafka.api.SaslPlainSslEndToEndAuthorizationTest 14s Build /

[jira] [Resolved] (KAFKA-15016) LICENSE-binary file contains dependencies not included anymore

2023-05-24 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mickael Maison resolved KAFKA-15016. Fix Version/s: 3.6.0 Resolution: Fixed > LICENSE-binary file contains dependencies

[GitHub] [kafka] mimaison merged pull request #13749: KAFKA-15016: Update LICENSE-binary file

2023-05-24 Thread via GitHub
mimaison merged PR #13749: URL: https://github.com/apache/kafka/pull/13749 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] Hangleton commented on a diff in pull request #13240: KAFKA-14690: Add topic IDs to the OffsetCommit API version 9

2023-05-24 Thread via GitHub
Hangleton commented on code in PR #13240: URL: https://github.com/apache/kafka/pull/13240#discussion_r1204403475 ## core/src/test/scala/unit/kafka/server/OffsetCommitRequestTest.scala: ## @@ -0,0 +1,147 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or

[jira] [Commented] (KAFKA-14996) CreateTopic falis with UnknownServerException if num partitions >= QuorumController.MAX_RECORDS_PER_BATCH

2023-05-24 Thread Edoardo Comar (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17725858#comment-17725858 ] Edoardo Comar commented on KAFKA-14996: --- I found another way to get into the error state. 3

[GitHub] [kafka] dajac opened a new pull request, #13754: MINOR: Hopefully fix flaky FetchFromFollowerIntegrationTest.testRackAwareRangeAssignor

2023-05-24 Thread via GitHub
dajac opened a new pull request, #13754: URL: https://github.com/apache/kafka/pull/13754 This test fails regularly in CI ([example](https://ci-builds.apache.org/blue/organizations/jenkins/Kafka%2Fkafka-pr/detail/PR-13753/1/tests)). This is an attempt to stabilize it. ### Committer

[GitHub] [kafka] Hangleton commented on a diff in pull request #13240: KAFKA-14690: Add topic IDs to the OffsetCommit API version 9

2023-05-24 Thread via GitHub
Hangleton commented on code in PR #13240: URL: https://github.com/apache/kafka/pull/13240#discussion_r1204376110 ## core/src/test/scala/unit/kafka/server/OffsetCommitRequestTest.scala: ## @@ -0,0 +1,147 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or

[GitHub] [kafka] Hangleton commented on a diff in pull request #13240: KAFKA-14690: Add topic IDs to the OffsetCommit API version 9

2023-05-24 Thread via GitHub
Hangleton commented on code in PR #13240: URL: https://github.com/apache/kafka/pull/13240#discussion_r1204361177 ## core/src/test/scala/unit/kafka/server/OffsetCommitRequestTest.scala: ## @@ -0,0 +1,147 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or

[GitHub] [kafka] Hangleton commented on a diff in pull request #13240: KAFKA-14690: Add topic IDs to the OffsetCommit API version 9

2023-05-24 Thread via GitHub
Hangleton commented on code in PR #13240: URL: https://github.com/apache/kafka/pull/13240#discussion_r1204356253 ## core/src/test/scala/unit/kafka/server/OffsetCommitRequestTest.scala: ## @@ -0,0 +1,147 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or

[GitHub] [kafka] C0urante commented on pull request #13165: KAFKA-14654: Connector classes should statically initialize with plugin classloader

2023-05-24 Thread via GitHub
C0urante commented on PR #13165: URL: https://github.com/apache/kafka/pull/13165#issuecomment-1561331016 That's a good point about the static initialization taking place directly before the constructor right now, but it's possible that other logic either directly from the Connect framework

[GitHub] [kafka] C0urante commented on a diff in pull request #13165: KAFKA-14654: Connector classes should statically initialize with plugin classloader

2023-05-24 Thread via GitHub
C0urante commented on code in PR #13165: URL: https://github.com/apache/kafka/pull/13165#discussion_r1204337446 ## connect/runtime/src/main/java/org/apache/kafka/connect/cli/AbstractConnectCli.java: ## @@ -119,36 +120,37 @@ public Connect startConnect(Map workerProps,

[GitHub] [kafka] C0urante commented on a diff in pull request #13165: KAFKA-14654: Connector classes should statically initialize with plugin classloader

2023-05-24 Thread via GitHub
C0urante commented on code in PR #13165: URL: https://github.com/apache/kafka/pull/13165#discussion_r1204336495 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/isolation/LoaderSwap.java: ## @@ -24,13 +24,32 @@ public class LoaderSwap implements AutoCloseable {

[GitHub] [kafka] Hangleton commented on a diff in pull request #13240: KAFKA-14690: Add topic IDs to the OffsetCommit API version 9

2023-05-24 Thread via GitHub
Hangleton commented on code in PR #13240: URL: https://github.com/apache/kafka/pull/13240#discussion_r1204312762 ## core/src/test/scala/unit/kafka/server/OffsetCommitRequestTest.scala: ## @@ -0,0 +1,147 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or

[GitHub] [kafka] divijvaidya commented on a diff in pull request #13753: MINOR: Log previous leader when replica transitions from leader to follower or vice versa

2023-05-24 Thread via GitHub
divijvaidya commented on code in PR #13753: URL: https://github.com/apache/kafka/pull/13753#discussion_r1204309801 ## core/src/main/scala/kafka/cluster/Partition.scala: ## @@ -726,7 +726,7 @@ class Partition(val topicPartition: TopicPartition, s"with partition epoch

[GitHub] [kafka] dajac commented on a diff in pull request #13753: MINOR: Log previous leader when replica transitions from leader to follower or vice versa

2023-05-24 Thread via GitHub
dajac commented on code in PR #13753: URL: https://github.com/apache/kafka/pull/13753#discussion_r1204298139 ## core/src/main/scala/kafka/cluster/Partition.scala: ## @@ -726,7 +726,7 @@ class Partition(val topicPartition: TopicPartition, s"with partition epoch

[GitHub] [kafka] divijvaidya commented on a diff in pull request #13753: MINOR: Log previous leader when replica transitions from leader to follower or vice versa

2023-05-24 Thread via GitHub
divijvaidya commented on code in PR #13753: URL: https://github.com/apache/kafka/pull/13753#discussion_r1204246878 ## core/src/main/scala/kafka/cluster/Partition.scala: ## @@ -726,7 +726,7 @@ class Partition(val topicPartition: TopicPartition, s"with partition epoch

[GitHub] [kafka] Hangleton commented on a diff in pull request #13240: KAFKA-14690: Add topic IDs to the OffsetCommit API version 9

2023-05-24 Thread via GitHub
Hangleton commented on code in PR #13240: URL: https://github.com/apache/kafka/pull/13240#discussion_r1204162396 ## core/src/test/scala/unit/kafka/server/OffsetCommitRequestTest.scala: ## @@ -0,0 +1,147 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or

[GitHub] [kafka] Hangleton commented on a diff in pull request #13240: KAFKA-14690: Add topic IDs to the OffsetCommit API version 9

2023-05-24 Thread via GitHub
Hangleton commented on code in PR #13240: URL: https://github.com/apache/kafka/pull/13240#discussion_r1204151150 ## core/src/test/scala/unit/kafka/server/OffsetCommitRequestTest.scala: ## @@ -0,0 +1,147 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or

[GitHub] [kafka] clayburn commented on pull request #13676: MINOR: Capture build scans on ge.apache.org to benefit from deep build insights

2023-05-24 Thread via GitHub
clayburn commented on PR #13676: URL: https://github.com/apache/kafka/pull/13676#issuecomment-1561117359 The test failures appear to come from these two test cases across the following JDKs: * JDK 8: https://ge.apache.org/s/7zk5qyab6i54u/tests/overview?outcome=failed * JDK 11:

[GitHub] [kafka] dajac opened a new pull request, #13753: MINOR: Log previous leader when replica transitions from leader to follower or vice versa

2023-05-24 Thread via GitHub
dajac opened a new pull request, #13753: URL: https://github.com/apache/kafka/pull/13753 I just had a case where this would have been useful so adding it. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage

[GitHub] [kafka] Hangleton commented on a diff in pull request #13240: KAFKA-14690: Add topic IDs to the OffsetCommit API version 9

2023-05-24 Thread via GitHub
Hangleton commented on code in PR #13240: URL: https://github.com/apache/kafka/pull/13240#discussion_r1204087015 ## core/src/test/scala/integration/kafka/api/AuthorizerIntegrationTest.scala: ## @@ -489,24 +489,23 @@ class AuthorizerIntegrationTest extends BaseRequestTest { }

[GitHub] [kafka] Hangleton commented on a diff in pull request #13240: KAFKA-14690: Add topic IDs to the OffsetCommit API version 9

2023-05-24 Thread via GitHub
Hangleton commented on code in PR #13240: URL: https://github.com/apache/kafka/pull/13240#discussion_r1204079800 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinatorTest.java: ## @@ -2707,11 +2742,271 @@ public void

[GitHub] [kafka] divijvaidya commented on a diff in pull request #13703: MINOR: Standardize controller log4j output for replaying records

2023-05-24 Thread via GitHub
divijvaidya commented on code in PR #13703: URL: https://github.com/apache/kafka/pull/13703#discussion_r1204039964 ## metadata/src/main/java/org/apache/kafka/controller/AclControlManager.java: ## @@ -64,12 +66,44 @@ * completed, which is another reason the prepare / complete

[GitHub] [kafka] urbandan commented on a diff in pull request #13591: KAFKA-14831: Illegal state errors should be fatal in transactional producer

2023-05-24 Thread via GitHub
urbandan commented on code in PR #13591: URL: https://github.com/apache/kafka/pull/13591#discussion_r1204028973 ## clients/src/main/java/org/apache/kafka/clients/producer/internals/TransactionManager.java: ## @@ -626,12 +704,14 @@ public synchronized void

[GitHub] [kafka] Hangleton commented on a diff in pull request #13240: KAFKA-14690: Add topic IDs to the OffsetCommit API version 9

2023-05-24 Thread via GitHub
Hangleton commented on code in PR #13240: URL: https://github.com/apache/kafka/pull/13240#discussion_r1203992806 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinatorTest.java: ## @@ -2707,11 +2742,271 @@ public void

[GitHub] [kafka] Hangleton commented on a diff in pull request #13240: KAFKA-14690: Add topic IDs to the OffsetCommit API version 9

2023-05-24 Thread via GitHub
Hangleton commented on code in PR #13240: URL: https://github.com/apache/kafka/pull/13240#discussion_r1203965540 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/ConsumerCoordinatorTest.java: ## @@ -129,16 +150,19 @@ import static

[jira] [Resolved] (KAFKA-12384) Flaky Test ListOffsetsRequestTest.testResponseIncludesLeaderEpoch

2023-05-24 Thread Viktor Somogyi-Vass (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viktor Somogyi-Vass resolved KAFKA-12384. - Fix Version/s: 3.6.0 Resolution: Fixed > Flaky Test

[jira] [Commented] (KAFKA-12384) Flaky Test ListOffsetsRequestTest.testResponseIncludesLeaderEpoch

2023-05-24 Thread Viktor Somogyi-Vass (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17725761#comment-17725761 ] Viktor Somogyi-Vass commented on KAFKA-12384: - Ok, [~dajac] got to it sooner, resolving

[GitHub] [kafka] divijvaidya closed pull request #9256: Fix some Gradle deprecation warnings

2023-05-24 Thread via GitHub
divijvaidya closed pull request #9256: Fix some Gradle deprecation warnings URL: https://github.com/apache/kafka/pull/9256 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [kafka] divijvaidya commented on pull request #9256: Fix some Gradle deprecation warnings

2023-05-24 Thread via GitHub
divijvaidya commented on PR #9256: URL: https://github.com/apache/kafka/pull/9256#issuecomment-1560889897 We are currently running gradle v8 and these warning have been fixed since this PR was opened. I am closing this PR. Please feel free to re-open if you still observe these deprecation

[GitHub] [kafka] divijvaidya commented on pull request #13421: KAFKA-14824: ReplicaAlterLogDirsThread may cause serious disk growing in case of potential exception

2023-05-24 Thread via GitHub
divijvaidya commented on PR #13421: URL: https://github.com/apache/kafka/pull/13421#issuecomment-1560847517 Apologies about the delay in reviewing this. I need another week to look into this part of the code base. -- This is an automated message from the Apache Git Service. To respond to

  1   2   >