Re: [PR] KAFKA-15804: Close SocketServer channels when calling shutdown before enableRequestProcessing [kafka]

2023-11-15 Thread via GitHub
hudeqi commented on code in PR #14729: URL: https://github.com/apache/kafka/pull/14729#discussion_r1395236215 ## core/src/main/scala/kafka/network/SocketServer.scala: ## @@ -1373,6 +1384,9 @@ private[kafka] class Processor( try { beginShutdown() thread.join()

Re: [PR] KAFKA-15804: Close SocketServer channels when calling shutdown before enableRequestProcessing [kafka]

2023-11-15 Thread via GitHub
gharris1727 commented on code in PR #14729: URL: https://github.com/apache/kafka/pull/14729#discussion_r1395228455 ## core/src/main/scala/kafka/network/SocketServer.scala: ## @@ -1373,6 +1384,9 @@ private[kafka] class Processor( try { beginShutdown()

Re: [PR] KAFKA-15778 & KAFKA-15779: Implement metrics manager (KIP-714) [kafka]

2023-11-15 Thread via GitHub
apoorvmittal10 commented on code in PR #14699: URL: https://github.com/apache/kafka/pull/14699#discussion_r1395220263 ## core/src/main/java/kafka/metrics/ClientMetricsInstance.java: ## @@ -0,0 +1,122 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more

Re: [PR] KAFKA-15804: Close SocketServer channels when calling shutdown before enableRequestProcessing [kafka]

2023-11-15 Thread via GitHub
hudeqi commented on code in PR #14729: URL: https://github.com/apache/kafka/pull/14729#discussion_r1395198255 ## core/src/main/scala/kafka/network/SocketServer.scala: ## @@ -1373,6 +1384,9 @@ private[kafka] class Processor( try { beginShutdown() thread.join()

Re: [PR] KAFKA-15174: Ensure CommitAsync propagate the exception to the user [kafka]

2023-11-15 Thread via GitHub
philipnee commented on PR #14680: URL: https://github.com/apache/kafka/pull/14680#issuecomment-1813871310 @lucasbru - I think we probably need to refactor KafkaConsumerTest again because most of the tests don't really apply to the async KafkaConsumer due to different protocol use etc.

Re: [PR] KAFKA-15804: Close SocketServer channels when calling shutdown before enableRequestProcessing [kafka]

2023-11-15 Thread via GitHub
hudeqi commented on code in PR #14729: URL: https://github.com/apache/kafka/pull/14729#discussion_r1395212845 ## core/src/main/scala/kafka/network/SocketServer.scala: ## @@ -1373,6 +1384,9 @@ private[kafka] class Processor( try { beginShutdown() thread.join()

Re: [PR] KAFKA-15804: Close SocketServer channels when calling shutdown before enableRequestProcessing [kafka]

2023-11-15 Thread via GitHub
gharris1727 commented on code in PR #14729: URL: https://github.com/apache/kafka/pull/14729#discussion_r1395204633 ## core/src/main/scala/kafka/network/SocketServer.scala: ## @@ -1373,6 +1384,9 @@ private[kafka] class Processor( try { beginShutdown()

Re: [PR] KAFKA-15778 & KAFKA-15779: Implement metrics manager (KIP-714) [kafka]

2023-11-15 Thread via GitHub
apoorvmittal10 commented on code in PR #14699: URL: https://github.com/apache/kafka/pull/14699#discussion_r1395204343 ## core/src/main/java/kafka/server/ClientMetricsManager.java: ## @@ -30,17 +68,366 @@ public class ClientMetricsManager implements Closeable { private

Re: [PR] KAFKA-15804: Close SocketServer channels when calling shutdown before enableRequestProcessing [kafka]

2023-11-15 Thread via GitHub
hudeqi commented on code in PR #14729: URL: https://github.com/apache/kafka/pull/14729#discussion_r1395198255 ## core/src/main/scala/kafka/network/SocketServer.scala: ## @@ -1373,6 +1384,9 @@ private[kafka] class Processor( try { beginShutdown() thread.join()

Re: [PR] KAFKA-15174: Ensure CommitAsync propagate the exception to the user [kafka]

2023-11-15 Thread via GitHub
philipnee commented on PR #14680: URL: https://github.com/apache/kafka/pull/14680#issuecomment-1813848500 I think the test failures was probably caused by heartbeat bubbled up to the consumer subsequently throwing an exception -- This is an automated message from the Apache Git Service.

Re: [PR] MINOR: Query before creating the internal remote log metadata topic [kafka]

2023-11-15 Thread via GitHub
kamalcph commented on PR #14755: URL: https://github.com/apache/kafka/pull/14755#issuecomment-1813827068 @satishd Addressed your review comments. PTAL. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] MINOR: Query before creating the internal remote log metadata topic [kafka]

2023-11-15 Thread via GitHub
kamalcph commented on code in PR #14755: URL: https://github.com/apache/kafka/pull/14755#discussion_r1395176822 ## storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/TopicBasedRemoteLogMetadataManager.java: ## @@ -477,20 +492,22 @@ private NewTopic

Re: [PR] KAFKA-15778 & KAFKA-15779: Implement metrics manager (KIP-714) [kafka]

2023-11-15 Thread via GitHub
apoorvmittal10 commented on code in PR #14699: URL: https://github.com/apache/kafka/pull/14699#discussion_r1395171995 ## core/src/test/java/kafka/server/ClientMetricsManagerTest.java: ## @@ -0,0 +1,921 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or

Re: [PR] KAFKA-15778 & KAFKA-15779: Implement metrics manager (KIP-714) [kafka]

2023-11-15 Thread via GitHub
apoorvmittal10 commented on code in PR #14699: URL: https://github.com/apache/kafka/pull/14699#discussion_r1395166691 ## core/src/test/java/kafka/server/ClientMetricsManagerTest.java: ## @@ -0,0 +1,921 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or

Re: [PR] KAFKA-15804: Close SocketServer channels when calling shutdown before enableRequestProcessing [kafka]

2023-11-15 Thread via GitHub
gharris1727 commented on code in PR #14729: URL: https://github.com/apache/kafka/pull/14729#discussion_r1395163631 ## core/src/main/scala/kafka/network/SocketServer.scala: ## @@ -1373,6 +1384,9 @@ private[kafka] class Processor( try { beginShutdown()

Re: [PR] KAFKA-15804: Close SocketServer channels when calling shutdown before enableRequestProcessing [kafka]

2023-11-15 Thread via GitHub
gharris1727 commented on code in PR #14729: URL: https://github.com/apache/kafka/pull/14729#discussion_r1395163631 ## core/src/main/scala/kafka/network/SocketServer.scala: ## @@ -1373,6 +1384,9 @@ private[kafka] class Processor( try { beginShutdown()

Re: [PR] KAFKA-15778 & KAFKA-15779: Implement metrics manager (KIP-714) [kafka]

2023-11-15 Thread via GitHub
apoorvmittal10 commented on code in PR #14699: URL: https://github.com/apache/kafka/pull/14699#discussion_r1395156877 ## core/src/test/java/kafka/metrics/ClientMetricsInstanceTest.java: ## @@ -0,0 +1,104 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or

[jira] [Updated] (KAFKA-13447) Consumer should not reuse committed offset after topic recreation

2023-11-15 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee updated KAFKA-13447: --- Labels: consumer needs-kip (was: needs-kip) > Consumer should not reuse committed offset after

[jira] [Commented] (KAFKA-15325) Integrate topicId in OffsetFetch and OffsetCommit async consumer calls

2023-11-15 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17786609#comment-17786609 ] Philip Nee commented on KAFKA-15325: [~lianetm] a relevant ticket

Re: [PR] KAFKA-15574; [1/N]: Client state machine updates [kafka]

2023-11-15 Thread via GitHub
lianetm commented on code in PR #14690: URL: https://github.com/apache/kafka/pull/14690#discussion_r1395143611 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -181,63 +348,560 @@ public void

[jira] [Assigned] (KAFKA-15839) Review topic recreation handling in client reconciliation

2023-11-15 Thread Lianet Magrans (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianet Magrans reassigned KAFKA-15839: -- Assignee: Lianet Magrans > Review topic recreation handling in client reconciliation

[jira] [Created] (KAFKA-15839) Review topic recreation handling in client reconciliation

2023-11-15 Thread Lianet Magrans (Jira)
Lianet Magrans created KAFKA-15839: -- Summary: Review topic recreation handling in client reconciliation Key: KAFKA-15839 URL: https://issues.apache.org/jira/browse/KAFKA-15839 Project: Kafka

Re: [PR] KAFKA-15574; [1/N]: Client state machine updates [kafka]

2023-11-15 Thread via GitHub
lianetm commented on code in PR #14690: URL: https://github.com/apache/kafka/pull/14690#discussion_r1395138981 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MemberState.java: ## @@ -17,57 +17,113 @@ package org.apache.kafka.clients.consumer.internals;

Re: [PR] KAFKA-15574; [1/N]: Client state machine updates [kafka]

2023-11-15 Thread via GitHub
lianetm commented on code in PR #14690: URL: https://github.com/apache/kafka/pull/14690#discussion_r1395138233 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MemberState.java: ## @@ -17,57 +17,113 @@ package org.apache.kafka.clients.consumer.internals;

Re: [PR] KAFKA-15574; [1/N]: Client state machine updates [kafka]

2023-11-15 Thread via GitHub
lianetm commented on code in PR #14690: URL: https://github.com/apache/kafka/pull/14690#discussion_r1395132794 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -137,39 +135,104 @@ public HeartbeatRequestManager( }

Re: [PR] KAFKA-15574; [1/N]: Client state machine updates [kafka]

2023-11-15 Thread via GitHub
lianetm commented on code in PR #14690: URL: https://github.com/apache/kafka/pull/14690#discussion_r1395130865 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -137,39 +135,104 @@ public HeartbeatRequestManager( }

[jira] [Updated] (KAFKA-15838) [Connect] ExtractField and InsertField NULL Values are replaced by default value even in NULLABLE fields

2023-11-15 Thread Eric Pangiawan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Pangiawan updated KAFKA-15838: --- Description: ExtractField: Line 116-119

[jira] [Updated] (KAFKA-15838) [Connect] ExtractField and InsertField NULL Values are replaced by default value even in NULLABLE fields

2023-11-15 Thread Eric Pangiawan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Pangiawan updated KAFKA-15838: --- Description: ExtractField: Line 116-119

[jira] [Created] (KAFKA-15838) [Connect] ExtractField and InsertField NULL Values are replaced by default value even in NULLABLE fields

2023-11-15 Thread Eric Pangiawan (Jira)
Eric Pangiawan created KAFKA-15838: -- Summary: [Connect] ExtractField and InsertField NULL Values are replaced by default value even in NULLABLE fields Key: KAFKA-15838 URL:

Re: [PR] MINOR: Query before creating the internal remote log metadata topic [kafka]

2023-11-15 Thread via GitHub
satishd commented on code in PR #14755: URL: https://github.com/apache/kafka/pull/14755#discussion_r1395121310 ## storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/TopicBasedRemoteLogMetadataManager.java: ## @@ -477,20 +492,22 @@ private NewTopic

Re: [PR] KAFKA-15804: Close SocketServer channels when calling shutdown before enableRequestProcessing [kafka]

2023-11-15 Thread via GitHub
hudeqi commented on code in PR #14729: URL: https://github.com/apache/kafka/pull/14729#discussion_r1395121316 ## core/src/main/scala/kafka/network/SocketServer.scala: ## @@ -1373,6 +1384,9 @@ private[kafka] class Processor( try { beginShutdown() thread.join()

Re: [PR] KAFKA-15574; [1/N]: Client state machine updates [kafka]

2023-11-15 Thread via GitHub
lianetm commented on code in PR #14690: URL: https://github.com/apache/kafka/pull/14690#discussion_r1395108554 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -181,63 +348,560 @@ public void

[PR] [MINOR] Removing unused variables from StreamThreadTest [kafka]

2023-11-15 Thread via GitHub
vamossagar12 opened a new pull request, #14777: URL: https://github.com/apache/kafka/pull/14777 While working on https://github.com/apache/kafka/pull/13283 I found a few variables which aren't being used anymore but still exist. I believe this is happening due to some recent refactoring

Re: [PR] KAFKA-15574; [1/N]: Client state machine updates [kafka]

2023-11-15 Thread via GitHub
lianetm commented on code in PR #14690: URL: https://github.com/apache/kafka/pull/14690#discussion_r1395108554 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -181,63 +348,560 @@ public void

Re: [PR] KAFKA-15802: Validate remote segment state before fetching index [kafka]

2023-11-15 Thread via GitHub
satishd merged PR #14759: URL: https://github.com/apache/kafka/pull/14759 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-15802: Validate remote segment state before fetching index [kafka]

2023-11-15 Thread via GitHub
satishd commented on PR #14759: URL: https://github.com/apache/kafka/pull/14759#issuecomment-1813729407 There are a few unrelated test failures, merging it to trunk. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] KAFKA-13152: Kip 770 buffer size fix [kafka]

2023-11-15 Thread via GitHub
vamossagar12 commented on PR #13283: URL: https://github.com/apache/kafka/pull/13283#issuecomment-1813728802 hey @ableegoldman , quick question. I am noticing that tests like `shouldRecoverFromInvalidOffsetExceptionOnRestoreAndFinishRestore` are failing with an error like ```

[jira] [Commented] (KAFKA-15821) Active topics for deleted connectors are not reset in standalone mode

2023-11-15 Thread David Jameson (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17786575#comment-17786575 ] David Jameson commented on KAFKA-15821: --- Had a go at this one because it looked like an easy fix

Re: [PR] KAFKA-14519; [2/N] New coordinator metrics [kafka]

2023-11-15 Thread via GitHub
jeffkbkim commented on PR #14387: URL: https://github.com/apache/kafka/pull/14387#issuecomment-1813664706 @jolshan the other test failures look unrelated -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[PR] KAFKA-15460: Add group type filter to the List Groups API [kafka]

2023-11-15 Thread via GitHub
rreddy-22 opened a new pull request, #14775: URL: https://github.com/apache/kafka/pull/14775 KIP-848 introduces the concept of "group types" in the context of the ListGroups API. The ListGroups API has been extended to support both filtering on the group types and returning the group types

Re: [PR] PROPOSAL: support async event execution in group coordinator [kafka]

2023-11-15 Thread via GitHub
artemlivshits commented on code in PR #14705: URL: https://github.com/apache/kafka/pull/14705#discussion_r1395062167 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/runtime/CoordinatorRuntime.java: ## @@ -659,9 +660,21 @@ public TopicPartition key() {

Re: [PR] WIP KAFKA-15784: Ensure atomicity of in memory update and write when transactionally committing offsets [kafka]

2023-11-15 Thread via GitHub
jolshan commented on code in PR #14774: URL: https://github.com/apache/kafka/pull/14774#discussion_r1395060812 ## core/src/main/scala/kafka/coordinator/group/GroupMetadataManager.scala: ## @@ -469,19 +469,35 @@ class GroupMetadataManager(brokerId: Int,

[PR] WIP KAFKA-15784: Ensure atomicity of in memory update and write when transactionally committing offsets [kafka]

2023-11-15 Thread via GitHub
jolshan opened a new pull request, #14774: URL: https://github.com/apache/kafka/pull/14774 Rewrote the verification flow to have 3 stages 1. collect the partitions that need verification 2. with the callback result separate results into errors and verified partitions 3. Perform the action

Re: [PR] KAFKA-15799 Handle full metadata updates on ZK brokers [kafka]

2023-11-15 Thread via GitHub
mumrah commented on code in PR #14719: URL: https://github.com/apache/kafka/pull/14719#discussion_r1394884307 ## core/src/main/scala/kafka/controller/ControllerChannelManager.scala: ## @@ -567,7 +567,6 @@ abstract class AbstractControllerBrokerRequestBatch(config: KafkaConfig,

[jira] [Resolved] (KAFKA-15346) Single-Key_single-timestamp IQs with versioned state stores

2023-11-15 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-15346. - Fix Version/s: 3.7.0 Resolution: Fixed > Single-Key_single-timestamp IQs with

Re: [PR] KAFKA-15346: add support for 'single key single timestamp' IQs with versioned state stores (KIP-960) [kafka]

2023-11-15 Thread via GitHub
mjsax commented on PR #14596: URL: https://github.com/apache/kafka/pull/14596#issuecomment-1813605452 Merged to `trunk`. Congrats to your first PR and KIP!  -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] KAFKA-15346: add support for 'single key single timestamp' IQs with versioned state stores (KIP-960) [kafka]

2023-11-15 Thread via GitHub
mjsax merged PR #14596: URL: https://github.com/apache/kafka/pull/14596 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (KAFKA-15834) Subscribing to non-existent topic blocks StreamThread from stopping

2023-11-15 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17786564#comment-17786564 ] Matthias J. Sax commented on KAFKA-15834: - Thank for reporting and such a detailed analysis,

[jira] [Updated] (KAFKA-15836) KafkaConsumer subscribes to multiple topics does not respect max.poll.records

2023-11-15 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma updated KAFKA-15836: Fix Version/s: 3.6.1 > KafkaConsumer subscribes to multiple topics does not respect

[PR] KAFKA-15837: Throw error on use of Consumer.poll(long timeout) [kafka]

2023-11-15 Thread via GitHub
kirktrue opened a new pull request, #14773: URL: https://github.com/apache/kafka/pull/14773 Per KIP-266, the `Consumer.poll(long timeout)` method was deprecated in 2.0.0. The default implementation of `Consumer.poll(long timeout)` now throws a `KafkaException`, and the known

Re: [PR] PROPOSAL: support async event execution in group coordinator [kafka]

2023-11-15 Thread via GitHub
junrao commented on code in PR #14705: URL: https://github.com/apache/kafka/pull/14705#discussion_r1395031179 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/runtime/CoordinatorRuntime.java: ## @@ -659,9 +660,21 @@ public TopicPartition key() {

Re: [PR] KAFKA-15799 Handle full metadata updates on ZK brokers [kafka]

2023-11-15 Thread via GitHub
mumrah commented on code in PR #14719: URL: https://github.com/apache/kafka/pull/14719#discussion_r1395024903 ## core/src/main/scala/kafka/server/metadata/ZkMetadataCache.scala: ## @@ -55,6 +56,64 @@ trait ZkFinalizedFeatureCache { def getFeatureOption: Option[Features] }

[jira] [Created] (KAFKA-15837) Throw error on use of Consumer.poll(long timeout)

2023-11-15 Thread Kirk True (Jira)
Kirk True created KAFKA-15837: - Summary: Throw error on use of Consumer.poll(long timeout) Key: KAFKA-15837 URL: https://issues.apache.org/jira/browse/KAFKA-15837 Project: Kafka Issue Type:

Re: [PR] KAFKA-14438: Throw error when consumer configured with empty/whitespace-only group.id [kafka]

2023-11-15 Thread via GitHub
kirktrue commented on code in PR #14768: URL: https://github.com/apache/kafka/pull/14768#discussion_r1395013498 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/LegacyKafkaConsumer.java: ## @@ -150,11 +150,9 @@ public class LegacyKafkaConsumer implements

[jira] [Updated] (KAFKA-15836) KafkaConsumer subscribes to multiple topics does not respect max.poll.records

2023-11-15 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee updated KAFKA-15836: --- Labels: consumer (was: ) > KafkaConsumer subscribes to multiple topics does not respect

[jira] [Updated] (KAFKA-15836) KafkaConsumer subscribes to multiple topics does not respect max.poll.records

2023-11-15 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee updated KAFKA-15836: --- Summary: KafkaConsumer subscribes to multiple topics does not respect max.poll.records (was:

[jira] [Created] (KAFKA-15836) KafkaConsumer subscribe to multiple topics does not respect max.poll.records

2023-11-15 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15836: -- Summary: KafkaConsumer subscribe to multiple topics does not respect max.poll.records Key: KAFKA-15836 URL: https://issues.apache.org/jira/browse/KAFKA-15836 Project:

[PR] KafkaConsumer not respecting max.poll.record when subscribing to more than 1 topics. [kafka]

2023-11-15 Thread via GitHub
philipnee opened a new pull request, #14772: URL: https://github.com/apache/kafka/pull/14772 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of

[PR] KAFKA-15816: Fix leaked sockets in trogdor tests [kafka]

2023-11-15 Thread via GitHub
gharris1727 opened a new pull request, #14771: URL: https://github.com/apache/kafka/pull/14771 The AgentTest leaks the agent under test due to typos. Follow the existing style in the test to stop it. ### Committer Checklist (excluded from commit message) - [ ] Verify design and

[jira] [Comment Edited] (KAFKA-15341) Enabling TS for a topic during rolling restart causes problems

2023-11-15 Thread Phuc Hong Tran (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17786537#comment-17786537 ] Phuc Hong Tran edited comment on KAFKA-15341 at 11/15/23 11:26 PM: ---

Re: [PR] KIP-978: Allow dynamic reloading of certificates with different DN / SANs [kafka]

2023-11-15 Thread via GitHub
scholzj commented on code in PR #14756: URL: https://github.com/apache/kafka/pull/14756#discussion_r1394966608 ## clients/src/main/java/org/apache/kafka/common/security/ssl/SslFactory.java: ## @@ -185,6 +188,18 @@ private SslEngineFactory createNewSslEngineFactory(Map

[jira] [Comment Edited] (KAFKA-15341) Enabling TS for a topic during rolling restart causes problems

2023-11-15 Thread Phuc Hong Tran (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17786537#comment-17786537 ] Phuc Hong Tran edited comment on KAFKA-15341 at 11/15/23 11:09 PM: ---

Re: [PR] KAFKA-15826: Close consumer when sink task is cancelled [kafka]

2023-11-15 Thread via GitHub
gharris1727 commented on PR #14762: URL: https://github.com/apache/kafka/pull/14762#issuecomment-1813403690 The only problem I have with this approach is that the [official documentation](https://kafka.apache.org/36/javadoc/org/apache/kafka/clients/consumer/KafkaConsumer.html#multithreaded)

Re: [PR] MINOR: Always send cumulative failed dirs in HB request [kafka]

2023-11-15 Thread via GitHub
soarez commented on PR #14770: URL: https://github.com/apache/kafka/pull/14770#issuecomment-1813400599 @pprovenzano @cmccabe -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[PR] MINOR: Always send cumulative failed dirs in HB request [kafka]

2023-11-15 Thread via GitHub
soarez opened a new pull request, #14770: URL: https://github.com/apache/kafka/pull/14770 Instead of only sending failed log directory UUIDs in the heartbeat request until a successful response is received, the broker sends the full cumulative set of failed directories since startup time.

Re: [PR] KAFKA-15215: [KIP-954] support custom DSL store providers [kafka]

2023-11-15 Thread via GitHub
agavra commented on code in PR #14648: URL: https://github.com/apache/kafka/pull/14648#discussion_r1394939779 ## streams/src/main/java/org/apache/kafka/streams/TopologyConfig.java: ## @@ -216,12 +232,38 @@ public TopologyConfig(final String topologyName, final StreamsConfig

Re: [PR] KAFKA-15174: Ensure CommitAsync propagate the exception to the user [kafka]

2023-11-15 Thread via GitHub
philipnee commented on PR #14680: URL: https://github.com/apache/kafka/pull/14680#issuecomment-1813394707 I rebased the trunk and was unable to reproduce the failure locally. I wonder if the tests is flaky and was caused by the delegate PR. -- This is an automated message from the

[jira] [Commented] (KAFKA-15341) Enabling TS for a topic during rolling restart causes problems

2023-11-15 Thread Phuc Hong Tran (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17786537#comment-17786537 ] Phuc Hong Tran commented on KAFKA-15341: thanks [~divijvaidya] [~showuon] [~satishd]. May anyone

[PR] KAFKA-15174: Ensure CommitAsync propagate the exception to the user [kafka]

2023-11-15 Thread via GitHub
philipnee opened a new pull request, #14680: URL: https://github.com/apache/kafka/pull/14680 The PR covers a few important points: 1. Exception handling: We should be thrown RetriableCommitException when the commit exception is retriable. We should throw FencedIdException on commit and

Re: [PR] KAFKA-15174: Ensure CommitAsync propagate the exception to the user [kafka]

2023-11-15 Thread via GitHub
philipnee closed pull request #14680: KAFKA-15174: Ensure CommitAsync propagate the exception to the user URL: https://github.com/apache/kafka/pull/14680 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[PR] KAFKA-15816: Fix leaked sockets in streams tests [kafka]

2023-11-15 Thread via GitHub
gharris1727 opened a new pull request, #14769: URL: https://github.com/apache/kafka/pull/14769 These tests leak sockets via Kafka clients, either directly or via KafkaStreams instances. The tests should close these resources when appropriate. ### Committer Checklist (excluded from

Re: [PR] [Draft] KIP-994 (Part 1) Minor Enhancements to ListTransactionsRequest [kafka]

2023-11-15 Thread via GitHub
artemlivshits commented on code in PR #14731: URL: https://github.com/apache/kafka/pull/14731#discussion_r1394891315 ## core/src/main/scala/kafka/coordinator/transaction/TransactionStateManager.scala: ## @@ -327,6 +328,7 @@ class TransactionStateManager(brokerId: Int,

Re: [PR] KAFKA-15799 Handle full metadata updates on ZK brokers [kafka]

2023-11-15 Thread via GitHub
mumrah commented on code in PR #14719: URL: https://github.com/apache/kafka/pull/14719#discussion_r1394886188 ## core/src/main/scala/kafka/server/metadata/ZkMetadataCache.scala: ## @@ -376,6 +436,24 @@ class ZkMetadataCache( // This method returns the deleted TopicPartitions

Re: [PR] KAFKA-15799 Handle full metadata updates on ZK brokers [kafka]

2023-11-15 Thread via GitHub
mumrah commented on code in PR #14719: URL: https://github.com/apache/kafka/pull/14719#discussion_r1394884307 ## core/src/main/scala/kafka/controller/ControllerChannelManager.scala: ## @@ -567,7 +567,6 @@ abstract class AbstractControllerBrokerRequestBatch(config: KafkaConfig,

Re: [PR] KAFKA-15174: Ensure CommitAsync propagate the exception to the user [kafka]

2023-11-15 Thread via GitHub
philipnee commented on PR #14680: URL: https://github.com/apache/kafka/pull/14680#issuecomment-1813338012 hm - why is it sending heartbeat? looking -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] KAFKA-15174: Ensure CommitAsync propagate the exception to the user [kafka]

2023-11-15 Thread via GitHub
lucasbru commented on PR #14680: URL: https://github.com/apache/kafka/pull/14680#issuecomment-1813330936 @philipnee there is a suspicious test failure for `org.apache.kafka.clients.consumer.KafkaConsumerTest`, could you have a look? -- This is an automated message from the Apache Git

Re: [PR] KAFKA-15799 Handle full metadata updates on ZK brokers [kafka]

2023-11-15 Thread via GitHub
cmccabe commented on code in PR #14719: URL: https://github.com/apache/kafka/pull/14719#discussion_r1394832061 ## core/src/main/scala/kafka/server/metadata/ZkMetadataCache.scala: ## @@ -376,6 +436,24 @@ class ZkMetadataCache( // This method returns the deleted

Re: [PR] KAFKA-15799 Handle full metadata updates on ZK brokers [kafka]

2023-11-15 Thread via GitHub
cmccabe commented on code in PR #14719: URL: https://github.com/apache/kafka/pull/14719#discussion_r1394828896 ## core/src/main/scala/kafka/server/metadata/ZkMetadataCache.scala: ## @@ -376,6 +436,24 @@ class ZkMetadataCache( // This method returns the deleted

Re: [PR] KAFKA-15799 Handle full metadata updates on ZK brokers [kafka]

2023-11-15 Thread via GitHub
cmccabe commented on code in PR #14719: URL: https://github.com/apache/kafka/pull/14719#discussion_r1394827643 ## core/src/main/scala/kafka/server/metadata/ZkMetadataCache.scala: ## @@ -55,6 +56,64 @@ trait ZkFinalizedFeatureCache { def getFeatureOption: Option[Features] }

Re: [PR] KAFKA-14438: Throw error when consumer configured with empty/whitespace-only group.id [kafka]

2023-11-15 Thread via GitHub
AndrewJSchofield commented on code in PR #14768: URL: https://github.com/apache/kafka/pull/14768#discussion_r1394821601 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/LegacyKafkaConsumer.java: ## @@ -150,11 +150,9 @@ public class LegacyKafkaConsumer

Re: [PR] KAFKA-15799 Handle full metadata updates on ZK brokers [kafka]

2023-11-15 Thread via GitHub
cmccabe commented on code in PR #14719: URL: https://github.com/apache/kafka/pull/14719#discussion_r1394820962 ## core/src/main/scala/kafka/server/metadata/ZkMetadataCache.scala: ## @@ -55,6 +56,64 @@ trait ZkFinalizedFeatureCache { def getFeatureOption: Option[Features] }

Re: [PR] KAFKA-15799 Handle full metadata updates on ZK brokers [kafka]

2023-11-15 Thread via GitHub
cmccabe commented on code in PR #14719: URL: https://github.com/apache/kafka/pull/14719#discussion_r1394818560 ## core/src/main/scala/kafka/controller/ControllerChannelManager.scala: ## @@ -567,7 +567,6 @@ abstract class AbstractControllerBrokerRequestBatch(config:

Re: [PR] KAFKA-14438: Throw error when consumer configured with empty/whitespace-only group.id [kafka]

2023-11-15 Thread via GitHub
kirktrue commented on PR #14768: URL: https://github.com/apache/kafka/pull/14768#issuecomment-1813279513 @philipnee—can you add the `ctr` and `KIP-848` labels? Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[PR] KAFKA-14438: Throw error when consumer configured with empty/whitespace-only group.id [kafka]

2023-11-15 Thread via GitHub
kirktrue opened a new pull request, #14768: URL: https://github.com/apache/kafka/pull/14768 Per [KIP-289 (Improve the default group id behavior in `KafkaConsumer`)](https://cwiki.apache.org/confluence/display/KAFKA/KIP-289%3A+Improve+the+default+group+id+behavior+in+KafkaConsumer), the

[jira] [Updated] (KAFKA-14438) Throw error when consumer configured with empty/whitespace-only group.id

2023-11-15 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-14438: -- Summary: Throw error when consumer configured with empty/whitespace-only group.id (was: Throw

Re: [PR] KAFKA-15574; [1/N]: Client state machine updates [kafka]

2023-11-15 Thread via GitHub
lianetm commented on code in PR #14690: URL: https://github.com/apache/kafka/pull/14690#discussion_r1394789418 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -181,63 +348,560 @@ public void

Re: [PR] KAFKA-15574; [1/N]: Client state machine updates [kafka]

2023-11-15 Thread via GitHub
lianetm commented on code in PR #14690: URL: https://github.com/apache/kafka/pull/14690#discussion_r1394786770 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -181,63 +348,560 @@ public void

[jira] [Assigned] (KAFKA-15835) Group commit/callbacks triggering logic

2023-11-15 Thread Lianet Magrans (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianet Magrans reassigned KAFKA-15835: -- Assignee: Lianet Magrans > Group commit/callbacks triggering logic >

[jira] [Created] (KAFKA-15835) Group commit/callbacks triggering logic

2023-11-15 Thread Lianet Magrans (Jira)
Lianet Magrans created KAFKA-15835: -- Summary: Group commit/callbacks triggering logic Key: KAFKA-15835 URL: https://issues.apache.org/jira/browse/KAFKA-15835 Project: Kafka Issue Type:

Re: [PR] KAFKA-15574; [1/N]: Client state machine updates [kafka]

2023-11-15 Thread via GitHub
lianetm commented on code in PR #14690: URL: https://github.com/apache/kafka/pull/14690#discussion_r1394774045 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -181,63 +348,560 @@ public void

Re: [PR] KAFKA-15574; [1/N]: Client state machine updates [kafka]

2023-11-15 Thread via GitHub
lianetm commented on code in PR #14690: URL: https://github.com/apache/kafka/pull/14690#discussion_r1394759427 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -168,10 +308,37 @@ public void

Re: [PR] KAFKA-15574; [1/N]: Client state machine updates [kafka]

2023-11-15 Thread via GitHub
lianetm commented on code in PR #14690: URL: https://github.com/apache/kafka/pull/14690#discussion_r1394753399 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -77,32 +138,111 @@ public class MembershipManagerImpl implements

[jira] [Updated] (KAFKA-14438) Throw InvalidGroupIdException when consumer configured with empty/whitespace-only group.id

2023-11-15 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-14438: -- Summary: Throw InvalidGroupIdException when consumer configured with empty/whitespace-only group.id

[jira] [Updated] (KAFKA-14438) Stop supporting empty consumer group.id

2023-11-15 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-14438: -- Labels: kip-848-client-support kip-848-e2e kip-848-preview (was: ) > Stop supporting empty consumer

[jira] [Updated] (KAFKA-14438) Stop supporting empty consumer group.id

2023-11-15 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-14438: -- Description: Currently, a warning message is logged upon using an empty consumer groupId. In the

Re: [PR] KAFKA-15574; [1/N]: Client state machine updates [kafka]

2023-11-15 Thread via GitHub
lianetm commented on code in PR #14690: URL: https://github.com/apache/kafka/pull/14690#discussion_r1394746271 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -181,63 +348,560 @@ public void

[jira] [Updated] (KAFKA-14438) Stop supporting empty consumer group.id

2023-11-15 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-14438: -- Summary: Stop supporting empty consumer group.id (was: Stop supporting empty consumer groupId) >

Re: [PR] KAFKA-15778 & KAFKA-15779: Implement metrics manager (KIP-714) [kafka]

2023-11-15 Thread via GitHub
AndrewJSchofield commented on code in PR #14699: URL: https://github.com/apache/kafka/pull/14699#discussion_r1394744738 ## core/src/main/java/kafka/metrics/ClientMetricsConfigs.java: ## @@ -80,6 +82,11 @@ public class ClientMetricsConfigs { public static final String

[jira] [Assigned] (KAFKA-14438) Stop supporting empty consumer groupId

2023-11-15 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True reassigned KAFKA-14438: - Assignee: Kirk True > Stop supporting empty consumer groupId >

Re: [PR] KAFKA-15574; [1/N]: Client state machine updates [kafka]

2023-11-15 Thread via GitHub
lianetm commented on code in PR #14690: URL: https://github.com/apache/kafka/pull/14690#discussion_r1394742401 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -181,63 +348,560 @@ public void

  1   2   >