Re: [PR] MINOR: Fix GroupCoordinatorShardTest stubbing [kafka]

2023-11-14 Thread via GitHub
dajac merged PR #14637: URL: https://github.com/apache/kafka/pull/14637 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (KAFKA-15653) NPE in ChunkedByteStream

2023-11-14 Thread Henry Cai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17786206#comment-17786206 ] Henry Cai commented on KAFKA-15653: --- Is this Jira resolved?  I saw all the three linked PRs are merged

[jira] [Commented] (KAFKA-15809) Adding TS to broker's features list and updating broker's metadata schema to include TS enable status

2023-11-14 Thread Phuc Hong Tran (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17786204#comment-17786204 ] Phuc Hong Tran commented on KAFKA-15809: Nvm I got the answer > Adding TS to broker's features

Re: [PR] KAFKA-15470: Allow creating connectors in a stopped state [kafka]

2023-11-14 Thread via GitHub
yashmayya merged PR #14704: URL: https://github.com/apache/kafka/pull/14704 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-15470: Allow creating connectors in a stopped state [kafka]

2023-11-14 Thread via GitHub
yashmayya commented on PR #14704: URL: https://github.com/apache/kafka/pull/14704#issuecomment-1811862334 Thanks Chris! Test failures appear unrelated, merging to `trunk`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] KAFKA-15470: Allow creating connectors in a stopped state [kafka]

2023-11-14 Thread via GitHub
yashmayya commented on code in PR #14704: URL: https://github.com/apache/kafka/pull/14704#discussion_r1393673421 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/Herder.java: ## @@ -108,6 +108,19 @@ public interface Herder { */ void

Re: [PR] KAFKA-15445: Add JVM Docker image [kafka]

2023-11-14 Thread via GitHub
VedarthConfluent commented on code in PR #14552: URL: https://github.com/apache/kafka/pull/14552#discussion_r1393649057 ## docker/docker_release.py: ## @@ -0,0 +1,80 @@ +#!/usr/bin/env python + +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

Re: [PR] KAFKA-15585: Add DescribeTopics API server side support [kafka]

2023-11-14 Thread via GitHub
CalvinConfluent commented on code in PR #14612: URL: https://github.com/apache/kafka/pull/14612#discussion_r1393640277 ## core/src/main/scala/kafka/server/KafkaApis.scala: ## @@ -1355,6 +1402,60 @@ class KafkaApis(val requestChannel: RequestChannel, )) } + def

Re: [PR] KAFKA-15585: Add DescribeTopics API server side support [kafka]

2023-11-14 Thread via GitHub
CalvinConfluent commented on code in PR #14612: URL: https://github.com/apache/kafka/pull/14612#discussion_r1393639171 ## clients/src/main/resources/common/message/DescribeTopicsRequest.json: ## @@ -0,0 +1,34 @@ +// Licensed to the Apache Software Foundation (ASF) under one or

Re: [PR] KAFKA-15585: Add DescribeTopics API server side support [kafka]

2023-11-14 Thread via GitHub
CalvinConfluent commented on code in PR #14612: URL: https://github.com/apache/kafka/pull/14612#discussion_r1393638945 ## clients/src/main/resources/common/message/DescribeTopicsResponse.json: ## @@ -0,0 +1,61 @@ +// Licensed to the Apache Software Foundation (ASF) under one or

Re: [PR] KAFKA-15585: Add DescribeTopics API server side support [kafka]

2023-11-14 Thread via GitHub
CalvinConfluent commented on code in PR #14612: URL: https://github.com/apache/kafka/pull/14612#discussion_r1393636842 ## clients/src/main/java/org/apache/kafka/common/requests/DescribeTopicsRequest.java: ## @@ -0,0 +1,96 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] KAFKA-15585: Add DescribeTopics API server side support [kafka]

2023-11-14 Thread via GitHub
CalvinConfluent commented on code in PR #14612: URL: https://github.com/apache/kafka/pull/14612#discussion_r1393636378 ## clients/src/main/java/org/apache/kafka/common/protocol/Errors.java: ## @@ -386,7 +387,8 @@ public enum Errors { STALE_MEMBER_EPOCH(113, "The member

Re: [PR] KAFKA-15585: Add DescribeTopics API server side support [kafka]

2023-11-14 Thread via GitHub
CalvinConfluent commented on code in PR #14612: URL: https://github.com/apache/kafka/pull/14612#discussion_r1393634656 ## clients/src/main/java/org/apache/kafka/common/errors/RequestLimitReachedException.java: ## @@ -0,0 +1,23 @@ +/* + * Licensed to the Apache Software

Re: [PR] KAFKA-15585: Add DescribeTopics API server side support [kafka]

2023-11-14 Thread via GitHub
CalvinConfluent commented on code in PR #14612: URL: https://github.com/apache/kafka/pull/14612#discussion_r1393633978 ## clients/src/main/java/org/apache/kafka/common/protocol/ApiKeys.java: ## @@ -114,6 +114,7 @@ public enum ApiKeys {

Re: [PR] KAFKA-15585: Add DescribeTopics API server side support [kafka]

2023-11-14 Thread via GitHub
CalvinConfluent commented on code in PR #14612: URL: https://github.com/apache/kafka/pull/14612#discussion_r1393633762 ## core/src/main/scala/kafka/server/KafkaApis.scala: ## @@ -116,6 +117,7 @@ class KafkaApis(val requestChannel: RequestChannel, val requestHelper = new

Re: [PR] WIP: initial simple and incomplete implementation [kafka]

2023-11-14 Thread via GitHub
github-actions[bot] commented on PR #14144: URL: https://github.com/apache/kafka/pull/14144#issuecomment-1811755757 This PR is being marked as stale since it has not had any activity in 90 days. If you would like to keep this PR alive, please ask a committer for review. If the PR has

Re: [PR] [DRAFT] EXAMPLE: dual write recipe example for KIP-939 [kafka]

2023-11-14 Thread via GitHub
github-actions[bot] commented on PR #14231: URL: https://github.com/apache/kafka/pull/14231#issuecomment-1811755708 This PR is being marked as stale since it has not had any activity in 90 days. If you would like to keep this PR alive, please ask a committer for review. If the PR has

[jira] [Created] (KAFKA-15829) How to build Kafka 2.7 with maven instead of gradle?

2023-11-14 Thread rain.liang (Jira)
rain.liang created KAFKA-15829: -- Summary: How to build Kafka 2.7 with maven instead of gradle? Key: KAFKA-15829 URL: https://issues.apache.org/jira/browse/KAFKA-15829 Project: Kafka Issue Type:

Re: [PR] KAFKA-15346: add support for 'single key single timestamp' IQs with versioned state stores (KIP-960) [kafka]

2023-11-14 Thread via GitHub
mjsax commented on PR #14596: URL: https://github.com/apache/kafka/pull/14596#issuecomment-1811669805 Jenkins did crash. Re-started the build. Can merge after we got a green run. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] KAFKA-15174: Ensure CommitAsync propagate the exception to the user [kafka]

2023-11-14 Thread via GitHub
kirktrue commented on PR #14680: URL: https://github.com/apache/kafka/pull/14680#issuecomment-1811620894 @lucasbru As @philipnee said, the two mechanisms appear similar and do have overlap. However, not only is it "awkward" to mix them, but it raises correctness questions. `poll()`

Re: [PR] KAFKA-15585: Add DescribeTopics API server side support [kafka]

2023-11-14 Thread via GitHub
hachikuji commented on code in PR #14612: URL: https://github.com/apache/kafka/pull/14612#discussion_r1393479827 ## clients/src/main/java/org/apache/kafka/common/requests/DescribeTopicsRequest.java: ## @@ -0,0 +1,96 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[jira] [Updated] (KAFKA-15828) Protect clients from broker hostname reuse

2023-11-14 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15828: Labels: needs-kip (was: ) > Protect clients from broker hostname reuse >

[jira] [Updated] (KAFKA-15828) Protect clients from broker hostname reuse

2023-11-14 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15828: Component/s: clients consumer producer > Protect

Re: [PR] KAFKA-15544: Enable integration tests for new consumer [kafka]

2023-11-14 Thread via GitHub
kirktrue commented on PR #14758: URL: https://github.com/apache/kafka/pull/14758#issuecomment-1811599069 @philipnee can you tag with `ctr` and `KIP-848` 梁 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[jira] [Updated] (KAFKA-15276) Implement event plumbing for ConsumerRebalanceListener callbacks

2023-11-14 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15276: -- Summary: Implement event plumbing for ConsumerRebalanceListener callbacks (was: Implement partition

[jira] [Updated] (KAFKA-15276) Implement partition assignment reconciliation callbacks

2023-11-14 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15276: -- Summary: Implement partition assignment reconciliation callbacks (was: Complete partition assignment

Re: [PR] MINOR: Small LogValidator clean ups [kafka]

2023-11-14 Thread via GitHub
hachikuji commented on PR #14697: URL: https://github.com/apache/kafka/pull/14697#issuecomment-1811585213 @ex172000 @junrao Thanks for reviewing. Please take another look when you have time. -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] MINOR Re-add action queue parameter removed from appendRecords [kafka]

2023-11-14 Thread via GitHub
jolshan merged PR #14753: URL: https://github.com/apache/kafka/pull/14753 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-15277: Design & implement support for internal Consumer delegates [kafka]

2023-11-14 Thread via GitHub
kirktrue closed pull request #14670: KAFKA-15277: Design & implement support for internal Consumer delegates URL: https://github.com/apache/kafka/pull/14670 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] KAFKA-15277: Design & implement support for internal Consumer delegates [kafka]

2023-11-14 Thread via GitHub
kirktrue commented on PR #14670: URL: https://github.com/apache/kafka/pull/14670#issuecomment-1811579631 Update on test failures: - `kafka.api.ClientIdQuotaTest` - `testQuotaOverrideDelete`: ran 50 times locally with no errors - `testThrottledProducerConsumer`: ran 75 times

Re: [PR] MINOR Re-add action queue parameter removed from appendRecords [kafka]

2023-11-14 Thread via GitHub
jolshan commented on PR #14753: URL: https://github.com/apache/kafka/pull/14753#issuecomment-1811575702 Given the scope of the change, these test failures are not related. Will followup elsewhere. -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] MINOR: Small LogValidator clean ups [kafka]

2023-11-14 Thread via GitHub
hachikuji commented on code in PR #14697: URL: https://github.com/apache/kafka/pull/14697#discussion_r1393453106 ## core/src/main/scala/kafka/log/UnifiedLog.scala: ## @@ -818,9 +818,7 @@ class UnifiedLog(@volatile var logStartOffset: Long,

Re: [PR] KAFKA-15824: SubscriptionState's maybeValidatePositionForCurrentLeader should handle partition which isn't subscribed yet [kafka]

2023-11-14 Thread via GitHub
wcarlson5 merged PR #14757: URL: https://github.com/apache/kafka/pull/14757 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-15824: SubscriptionState's maybeValidatePositionForCurrentLeader should handle partition which isn't subscribed yet [kafka]

2023-11-14 Thread via GitHub
wcarlson5 commented on PR #14757: URL: https://github.com/apache/kafka/pull/14757#issuecomment-1811568924 Execution failed for task ':streams:upgrade-system-tests-0101:test'. is unrelated to this PR. -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] MINOR: Small LogValidator clean ups [kafka]

2023-11-14 Thread via GitHub
hachikuji commented on code in PR #14697: URL: https://github.com/apache/kafka/pull/14697#discussion_r1393447344 ## core/src/test/scala/unit/kafka/log/LogValidatorTest.scala: ## @@ -399,12 +402,20 @@ class LogValidatorTest { assertEquals(i, offsetCounter.value);

[PR] KAFKA-15816: Fix leaked sockets in runtime tests [kafka]

2023-11-14 Thread via GitHub
gharris1727 opened a new pull request, #14764: URL: https://github.com/apache/kafka/pull/14764 These tests contain typos that leak sockets, mostly through clients that are left open. The WorkerTest had rather involved resource leaks, which arose because the typical lifetime of the

Re: [PR] KAFKA-15544: Enable integration tests for new consumer [kafka]

2023-11-14 Thread via GitHub
AndrewJSchofield commented on code in PR #14758: URL: https://github.com/apache/kafka/pull/14758#discussion_r1393368519 ## core/src/test/scala/integration/kafka/api/PlaintextConsumerTest.scala: ## @@ -1805,17 +1909,14 @@ class PlaintextConsumerTest extends BaseConsumerTest {

Re: [PR] KAFKA-15778 & KAFKA-15779: Implement metrics manager (KIP-714) [kafka]

2023-11-14 Thread via GitHub
junrao commented on code in PR #14699: URL: https://github.com/apache/kafka/pull/14699#discussion_r1391737904 ## clients/src/main/java/org/apache/kafka/common/requests/PushTelemetryRequest.java: ## @@ -60,17 +62,39 @@ public PushTelemetryRequest(PushTelemetryRequestData data,

Re: [PR] KAFKA-15277: Design & implement support for internal Consumer delegates [kafka]

2023-11-14 Thread via GitHub
kirktrue commented on PR #14670: URL: https://github.com/apache/kafka/pull/14670#issuecomment-1811460391 @dajac—here are the unique test failures with notes: - integration.kafka.server.FetchFromFollowerIntegrationTest: testRackAwareRangeAssignor(String).quorum=kraft -

Re: [PR] KAFKA-15544: Enable integration tests for new consumer [kafka]

2023-11-14 Thread via GitHub
AndrewJSchofield commented on code in PR #14758: URL: https://github.com/apache/kafka/pull/14758#discussion_r1393368519 ## core/src/test/scala/integration/kafka/api/PlaintextConsumerTest.scala: ## @@ -1805,17 +1909,14 @@ class PlaintextConsumerTest extends BaseConsumerTest {

Re: [PR] KAFKA-15544: Enable integration tests for new consumer [kafka]

2023-11-14 Thread via GitHub
AndrewJSchofield commented on code in PR #14758: URL: https://github.com/apache/kafka/pull/14758#discussion_r1393368202 ## core/src/test/scala/integration/kafka/api/PlaintextConsumerTest.scala: ## @@ -1805,17 +1909,14 @@ class PlaintextConsumerTest extends BaseConsumerTest {

Re: [PR] KAFKA-15544: Enable integration tests for new consumer [kafka]

2023-11-14 Thread via GitHub
AndrewJSchofield commented on code in PR #14758: URL: https://github.com/apache/kafka/pull/14758#discussion_r1393367240 ## core/src/test/scala/integration/kafka/api/BaseConsumerTest.scala: ## @@ -34,13 +35,15 @@ import scala.collection.Seq */ abstract class BaseConsumerTest

Re: [PR] KAFKA-15544: Enable integration tests for new consumer [kafka]

2023-11-14 Thread via GitHub
AndrewJSchofield commented on code in PR #14758: URL: https://github.com/apache/kafka/pull/14758#discussion_r1393365674 ## core/src/test/scala/integration/kafka/api/PlaintextConsumerTest.scala: ## @@ -1657,35 +1759,37 @@ class PlaintextConsumerTest extends BaseConsumerTest {

Re: [PR] KAFKA-15756: Migrate existing integration tests to run old protocol in new coordinator [kafka]

2023-11-14 Thread via GitHub
dongnuo123 commented on code in PR #14675: URL: https://github.com/apache/kafka/pull/14675#discussion_r1393317417 ## core/src/test/scala/integration/kafka/api/PlaintextConsumerTest.scala: ## @@ -547,7 +565,7 @@ class PlaintextConsumerTest extends BaseConsumerTest {

[jira] [Created] (KAFKA-15828) Protect clients from broker hostname reuse

2023-11-14 Thread Jason Gustafson (Jira)
Jason Gustafson created KAFKA-15828: --- Summary: Protect clients from broker hostname reuse Key: KAFKA-15828 URL: https://issues.apache.org/jira/browse/KAFKA-15828 Project: Kafka Issue Type:

Re: [PR] KAFKA-15756: Migrate existing integration tests to run old protocol in new coordinator [kafka]

2023-11-14 Thread via GitHub
dongnuo123 commented on code in PR #14675: URL: https://github.com/apache/kafka/pull/14675#discussion_r1393317417 ## core/src/test/scala/integration/kafka/api/PlaintextConsumerTest.scala: ## @@ -547,7 +565,7 @@ class PlaintextConsumerTest extends BaseConsumerTest {

Re: [PR] KAFKA-15544: Enable integration tests for new consumer [kafka]

2023-11-14 Thread via GitHub
kirktrue commented on code in PR #14758: URL: https://github.com/apache/kafka/pull/14758#discussion_r1393305110 ## core/src/test/scala/integration/kafka/api/BaseConsumerTest.scala: ## @@ -34,13 +35,15 @@ import scala.collection.Seq */ abstract class BaseConsumerTest extends

[PR] KAFKA-15827: Prevent KafkaBasedLog subclasses from leaking passed-in clients [kafka]

2023-11-14 Thread via GitHub
gharris1727 opened a new pull request, #14763: URL: https://github.com/apache/kafka/pull/14763 The KafkaBasedLog normally creates clients during start() and closes them in stop(). Some KafkaBasedLog subclasses accept already-created clients, and close them in stop() if start() is called

[jira] [Created] (KAFKA-15827) KafkaBasedLog.withExistingClients leaks clients if start is not called

2023-11-14 Thread Greg Harris (Jira)
Greg Harris created KAFKA-15827: --- Summary: KafkaBasedLog.withExistingClients leaks clients if start is not called Key: KAFKA-15827 URL: https://issues.apache.org/jira/browse/KAFKA-15827 Project: Kafka

Re: [PR] KAFKA-15448: Streams Standby Update Listener (KIP-988) [kafka]

2023-11-14 Thread via GitHub
eduwercamacaro commented on code in PR #14735: URL: https://github.com/apache/kafka/pull/14735#discussion_r1393218354 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java: ## @@ -701,7 +706,15 @@ private StandbyTask convertActiveToStandby(final

Re: [PR] KAFKA-15215: [KIP-954] support custom DSL store providers [kafka]

2023-11-14 Thread via GitHub
agavra commented on code in PR #14648: URL: https://github.com/apache/kafka/pull/14648#discussion_r1393203579 ## streams/src/main/java/org/apache/kafka/streams/kstream/Materialized.java: ## @@ -97,21 +128,21 @@ protected Materialized(final Materialized materialized) {

Re: [PR] KAFKA-15215: [KIP-954] support custom DSL store providers [kafka]

2023-11-14 Thread via GitHub
agavra commented on code in PR #14648: URL: https://github.com/apache/kafka/pull/14648#discussion_r1393194762 ## streams/src/main/java/org/apache/kafka/streams/StreamsConfig.java: ## @@ -536,6 +537,11 @@ public class StreamsConfig extends AbstractConfig { public static

Re: [PR] KAFKA-13152: Kip 770 buffer size fix [kafka]

2023-11-14 Thread via GitHub
vamossagar12 commented on PR #13283: URL: https://github.com/apache/kafka/pull/13283#issuecomment-1811121235 Hey @sean-rossignol , thanks for explaining your use case and how this can be useful for you! I hope to push it in the upcoming 3.7 release. I haven't gotten the chance to resolve

[jira] [Commented] (KAFKA-15513) KRaft cluster fails with SCRAM authentication enabled for control-plane

2023-11-14 Thread Proven Provenzano (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17786046#comment-17786046 ] Proven Provenzano commented on KAFKA-15513: --- The role of "{{{}kafka-storage --add-scram{}}}"

[jira] [Updated] (KAFKA-15825) KRaft controller writes empty state to ZK after migration

2023-11-14 Thread David Arthur (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Arthur updated KAFKA-15825: - Component/s: kraft > KRaft controller writes empty state to ZK after migration >

[jira] [Updated] (KAFKA-15818) Implement max poll internval

2023-11-14 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee updated KAFKA-15818: --- Labels: consumer consumer-threading-refactor (was: ) > Implement max poll internval >

[PR] KAFKA-15826: Close consumer when sink task is cancelled [kafka]

2023-11-14 Thread via GitHub
gharris1727 opened a new pull request, #14762: URL: https://github.com/apache/kafka/pull/14762 Currently the only place that the consumer is closed is on the task thread, which may be blocked indefinitely by the task plugin. Similar to AbstractWorkerSourceTask, we should close the consumer

Re: [PR] MINOR Re-add action queue parameter removed from appendRecords [kafka]

2023-11-14 Thread via GitHub
jolshan commented on code in PR #14753: URL: https://github.com/apache/kafka/pull/14753#discussion_r1393093702 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -898,20 +900,19 @@ class ReplicaManager(val config: KafkaConfig, } actionQueue.add { -

[PR] KAFKA-15816: Fix leaked sockets in mirror [kafka]

2023-11-14 Thread via GitHub
gharris1727 opened a new pull request, #14761: URL: https://github.com/apache/kafka/pull/14761 These socket leaks were caused by typos, either an extra call to start() or a missing call to close(). ### Committer Checklist (excluded from commit message) - [ ] Verify design and

[jira] [Created] (KAFKA-15826) WorkerSinkTask leaks Consumer if plugin start or stop blocks indefinitely

2023-11-14 Thread Greg Harris (Jira)
Greg Harris created KAFKA-15826: --- Summary: WorkerSinkTask leaks Consumer if plugin start or stop blocks indefinitely Key: KAFKA-15826 URL: https://issues.apache.org/jira/browse/KAFKA-15826 Project:

[jira] [Resolved] (KAFKA-15825) KRaft controller writes empty state to ZK after migration

2023-11-14 Thread David Arthur (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Arthur resolved KAFKA-15825. -- Resolution: Fixed This bug was fixed as part of KAFKA-15605 > KRaft controller writes empty

[jira] [Created] (KAFKA-15825) KRaft controller writes empty state to ZK after migration

2023-11-14 Thread David Arthur (Jira)
David Arthur created KAFKA-15825: Summary: KRaft controller writes empty state to ZK after migration Key: KAFKA-15825 URL: https://issues.apache.org/jira/browse/KAFKA-15825 Project: Kafka

[jira] [Resolved] (KAFKA-15605) Topics marked for deletion in ZK are incorrectly migrated to KRaft

2023-11-14 Thread David Arthur (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Arthur resolved KAFKA-15605. -- Fix Version/s: 3.7.0 Resolution: Fixed > Topics marked for deletion in ZK are

[PR] KAFKA-15823: disconnect from controller on AuthenticationException [kafka]

2023-11-14 Thread via GitHub
gaurav-narula opened a new pull request, #14760: URL: https://github.com/apache/kafka/pull/14760 This PR changes the handling of authenticationException on a request from the node to the controller. We disconnect controller connection and invalidate the cache so that the next run of

Re: [PR] KAFKA-15681: Add support of client-metrics in kafka-configs.sh (KIP-714) [kafka]

2023-11-14 Thread via GitHub
junrao commented on PR #14632: URL: https://github.com/apache/kafka/pull/14632#issuecomment-1810855040 @apoorvmittal10 : Thanks for looking into the test failures. There is an ongoing discussion on requiring a green build before merging the PR. I will need to wait for the result of that

Re: [PR] KAFKA-15555: Ensure wakeups are handled correctly in poll() [kafka]

2023-11-14 Thread via GitHub
philipnee commented on PR #14746: URL: https://github.com/apache/kafka/pull/14746#issuecomment-1810853645 Basically future.get() API only return 3 types of exceptions: ExecutionException, InterruptedException, and Cancellation per documentation. -- This is an automated message from the

Re: [PR] KAFKA-15555: Ensure wakeups are handled correctly in poll() [kafka]

2023-11-14 Thread via GitHub
philipnee commented on PR #14746: URL: https://github.com/apache/kafka/pull/14746#issuecomment-1810851585 Hi @cadonna - When the consumer is woken up. The WakeupTrigger should complete the future exceptionally with WakeupException. To rethrow that exception during future.get(), you will

Re: [PR] KAFKA-15802: Validate remote segment state before fetching index [kafka]

2023-11-14 Thread via GitHub
jeqo commented on PR #14759: URL: https://github.com/apache/kafka/pull/14759#issuecomment-1810780948 cc @satishd -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] KAFKA-15174: Ensure CommitAsync propagate the exception to the user [kafka]

2023-11-14 Thread via GitHub
philipnee commented on PR #14680: URL: https://github.com/apache/kafka/pull/14680#issuecomment-1810775229 Hi @lucasbru - Yes. It is not quite feasible to use the backgroundEventQueue because of the behavior of the current API, i.e., the callback needs to be invoked ouside of the poll call.

Re: [PR] MINOR Re-add action queue parameter removed from appendRecords [kafka]

2023-11-14 Thread via GitHub
hachikuji commented on code in PR #14753: URL: https://github.com/apache/kafka/pull/14753#discussion_r1392979091 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -898,20 +900,19 @@ class ReplicaManager(val config: KafkaConfig, } actionQueue.add { -

[PR] KAFKA-15802: Validate remote segment state before fetching index [kafka]

2023-11-14 Thread via GitHub
jeqo opened a new pull request, #14759: URL: https://github.com/apache/kafka/pull/14759 Cherrypick from https://github.com/apache/kafka/pull/14727 into trunk ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage

Re: [PR] MINOR Re-add action queue parameter removed from appendRecords [kafka]

2023-11-14 Thread via GitHub
jolshan commented on code in PR #14753: URL: https://github.com/apache/kafka/pull/14753#discussion_r1392978203 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -813,7 +813,8 @@ class ReplicaManager(val config: KafkaConfig,

Re: [PR] MINOR: WakeupTrigger cleanup [kafka]

2023-11-14 Thread via GitHub
philipnee commented on PR #14752: URL: https://github.com/apache/kafka/pull/14752#issuecomment-1810767025 Hi @kirktrue Thanks for taking time to clean up the task. I left some comments there. -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] MINOR: WakeupTrigger cleanup [kafka]

2023-11-14 Thread via GitHub
philipnee commented on code in PR #14752: URL: https://github.com/apache/kafka/pull/14752#discussion_r1392968241 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/WakeupTrigger.java: ## @@ -24,85 +25,133 @@ import java.util.concurrent.atomic.AtomicReference;

Re: [PR] MINOR: WakeupTrigger cleanup [kafka]

2023-11-14 Thread via GitHub
philipnee commented on code in PR #14752: URL: https://github.com/apache/kafka/pull/14752#discussion_r1392964486 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/WakeupTrigger.java: ## @@ -24,85 +25,133 @@ import java.util.concurrent.atomic.AtomicReference;

Re: [PR] MINOR: WakeupTrigger cleanup [kafka]

2023-11-14 Thread via GitHub
philipnee commented on code in PR #14752: URL: https://github.com/apache/kafka/pull/14752#discussion_r1392962733 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/WakeupTrigger.java: ## @@ -24,85 +25,133 @@ import java.util.concurrent.atomic.AtomicReference;

Re: [PR] KAFKA-15574; [1/N]: Client state machine updates [kafka]

2023-11-14 Thread via GitHub
lianetm commented on code in PR #14690: URL: https://github.com/apache/kafka/pull/14690#discussion_r1392954694 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -181,63 +347,537 @@ public void

Re: [PR] MINOR Re-add action queue parameter removed from appendRecords [kafka]

2023-11-14 Thread via GitHub
jolshan commented on code in PR #14753: URL: https://github.com/apache/kafka/pull/14753#discussion_r1392953096 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -813,7 +813,8 @@ class ReplicaManager(val config: KafkaConfig,

Re: [PR] MINOR Re-add action queue parameter removed from appendRecords [kafka]

2023-11-14 Thread via GitHub
artemlivshits commented on code in PR #14753: URL: https://github.com/apache/kafka/pull/14753#discussion_r1392946436 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -813,7 +813,8 @@ class ReplicaManager(val config: KafkaConfig,

Re: [PR] KAFKA-15357: Aggregate and propagate assignments [kafka]

2023-11-14 Thread via GitHub
soarez commented on PR #14369: URL: https://github.com/apache/kafka/pull/14369#issuecomment-1810719729 > avoids calling exit(1) in junit tests, which will kill Jenkins dead (even after 3 decades of Java, we don't have the technology to intercept exit() in unit testrs >:( ) @cmccabe

[PR] KAFKA-15544: Enable integration tests for new consumer [kafka]

2023-11-14 Thread via GitHub
AndrewJSchofield opened a new pull request, #14758: URL: https://github.com/apache/kafka/pull/14758 This PR parameterizes the consumer integration tests so they can be run against the existing "generic" group protocol and the new "consumer" group protocol introduced in KIP-848.

[jira] [Updated] (KAFKA-15824) SubscriptionState's maybeValidatePositionForCurrentLeader should handle partition which isn't subscribed yet

2023-11-14 Thread Mayank Shekhar Narula (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mayank Shekhar Narula updated KAFKA-15824: -- Reviewer: (was: Walker Carlson) > SubscriptionState's

[jira] [Updated] (KAFKA-15824) SubscriptionState's maybeValidatePositionForCurrentLeader should handle partition which isn't subscribed yet

2023-11-14 Thread Mayank Shekhar Narula (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mayank Shekhar Narula updated KAFKA-15824: -- Description: As can be seen [here|#L453],

[PR] Kip 951 SubscriptionState's maybeValidatePositionForCurrentLeader should handle partition which isn't subscribed yet [kafka]

2023-11-14 Thread via GitHub
msn-tldr opened a new pull request, #14757: URL: https://github.com/apache/kafka/pull/14757 See the motivation in jira description https://issues.apache.org/jira/browse/KAFKA-15824 Noticed this was discovered as `ReassignReplicaShrinkTest` started to fail with KIP-951 changes.

[jira] [Updated] (KAFKA-15824) SubscriptionState's maybeValidatePositionForCurrentLeader should handle partition which isn't subscribed yet

2023-11-14 Thread Mayank Shekhar Narula (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mayank Shekhar Narula updated KAFKA-15824: -- Description: As can be seen [here|#L453]],

[jira] [Updated] (KAFKA-15824) SubscriptionState's maybeValidatePositionForCurrentLeader should handle partition which isn't subscribed yet

2023-11-14 Thread Mayank Shekhar Narula (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mayank Shekhar Narula updated KAFKA-15824: -- Description: As can be seen [here|#L453]],

[jira] [Comment Edited] (KAFKA-15376) Explore options of removing data earlier to the current leader's leader epoch lineage for topics enabled with tiered storage.

2023-11-14 Thread Kamal Chandraprakash (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17785956#comment-17785956 ] Kamal Chandraprakash edited comment on KAFKA-15376 at 11/14/23 4:29 PM:

Re: [PR] KAFKA-15802: validate remote segment state before fetching index [kafka]

2023-11-14 Thread via GitHub
satishd merged PR #14727: URL: https://github.com/apache/kafka/pull/14727 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (KAFKA-15376) Explore options of removing data earlier to the current leader's leader epoch lineage for topics enabled with tiered storage.

2023-11-14 Thread Kamal Chandraprakash (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17785962#comment-17785962 ] Kamal Chandraprakash commented on KAFKA-15376: -- With unclean-leader-election enabled, there

[jira] [Commented] (KAFKA-15513) KRaft cluster fails with SCRAM authentication enabled for control-plane

2023-11-14 Thread Christian Lefebvre (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17785958#comment-17785958 ] Christian Lefebvre commented on KAFKA-15513: Hi [~pprovenzano] ,   isn't it the role of

[jira] [Updated] (KAFKA-15824) SubscriptionState's maybeValidatePositionForCurrentLeader should handle partition which isn't subscribed yet

2023-11-14 Thread Mayank Shekhar Narula (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mayank Shekhar Narula updated KAFKA-15824: -- Description: As can be seen

Re: [PR] KAFKA-15802: validate remote segment state before fetching index [kafka]

2023-11-14 Thread via GitHub
satishd commented on PR #14727: URL: https://github.com/apache/kafka/pull/14727#issuecomment-1810604177 There are a few unrelated test failures, merging the changes to trunk and 3.6 branches. -- This is an automated message from the Apache Git Service. To respond to the message, please

[jira] [Updated] (KAFKA-15824) SubscriptionState's maybeValidatePositionForCurrentLeader should handle partition which isn't subscribed yet

2023-11-14 Thread Mayank Shekhar Narula (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mayank Shekhar Narula updated KAFKA-15824: -- Fix Version/s: (was: 3.6.1) > SubscriptionState's

[jira] [Updated] (KAFKA-15824) SubscriptionState's maybeValidatePositionForCurrentLeader should handle partition which isn't subscribed yet

2023-11-14 Thread Mayank Shekhar Narula (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mayank Shekhar Narula updated KAFKA-15824: -- Issue Type: Bug (was: Improvement) > SubscriptionState's

[jira] [Created] (KAFKA-15824) SubscriptionState's maybeValidatePositionForCurrentLeader should handle partition which isn't subscribed yet

2023-11-14 Thread Mayank Shekhar Narula (Jira)
Mayank Shekhar Narula created KAFKA-15824: - Summary: SubscriptionState's maybeValidatePositionForCurrentLeader should handle partition which isn't subscribed yet Key: KAFKA-15824 URL:

[jira] [Commented] (KAFKA-15376) Explore options of removing data earlier to the current leader's leader epoch lineage for topics enabled with tiered storage.

2023-11-14 Thread Kamal Chandraprakash (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17785956#comment-17785956 ] Kamal Chandraprakash commented on KAFKA-15376: -- [~divijvaidya]  The

Re: [PR] KAFKA-15448: Streams Standby Update Listener (KIP-988) [kafka]

2023-11-14 Thread via GitHub
coltmcnealy-lh commented on code in PR #14735: URL: https://github.com/apache/kafka/pull/14735#discussion_r1392849939 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/StoreChangelogReader.java: ## @@ -1012,6 +1022,8 @@ private void prepareChangelogs(final

Re: [PR] KAFKA-15574; [1/N]: Client state machine updates [kafka]

2023-11-14 Thread via GitHub
kirktrue commented on code in PR #14690: URL: https://github.com/apache/kafka/pull/14690#discussion_r1392830639 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -77,32 +138,110 @@ public class MembershipManagerImpl

Re: [PR] KAFKA-15574; [1/N]: Client state machine updates [kafka]

2023-11-14 Thread via GitHub
kirktrue commented on code in PR #14690: URL: https://github.com/apache/kafka/pull/14690#discussion_r1392829031 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/events/ApplicationEvent.java: ## @@ -25,7 +25,8 @@ public abstract class ApplicationEvent {

Re: [PR] KAFKA-15574; [1/N]: Client state machine updates [kafka]

2023-11-14 Thread via GitHub
kirktrue commented on code in PR #14690: URL: https://github.com/apache/kafka/pull/14690#discussion_r1392827512 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/events/UnsubscribeApplicationEvent.java: ## @@ -0,0 +1,32 @@ +/* + * Licensed to the Apache

  1   2   >