Re: [PR] KAFKA-15608: Assign lastet leader eopch and offset checkpoint to future log when replacing current log [kafka]

2023-10-24 Thread via GitHub
drawxy closed pull request #14553: KAFKA-15608: Assign lastet leader eopch and offset checkpoint to future log when replacing current log URL: https://github.com/apache/kafka/pull/14553 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] KAFKA-15608: Assign lastet leader eopch and offset checkpoint to future log when replacing current log [kafka]

2023-10-24 Thread via GitHub
drawxy commented on PR #14553: URL: https://github.com/apache/kafka/pull/14553#issuecomment-1778559531 > I tested this process on the latest trunk branch and it did not reproduce. My process is: create a topic with 1 partition, then write traffic and stop writing. For example, the current

[PR] Kafka 15680 [kafka]

2023-10-24 Thread via GitHub
kumarpritam863 opened a new pull request, #14630: URL: https://github.com/apache/kafka/pull/14630 * In ICR(Incremental Cooperative Rebalancing) mode, whenever a new worker, say Worker 3 joins, a new global assignment is computed by the leader, say Worker1, that results in the revocation of

[jira] [Created] (KAFKA-15680) Partition-Count is not getting updated Correctly in the Incremental Co-operative Rebalancing(ICR) Mode of Rebalancing

2023-10-24 Thread Pritam Kumar (Jira)
Pritam Kumar created KAFKA-15680: Summary: Partition-Count is not getting updated Correctly in the Incremental Co-operative Rebalancing(ICR) Mode of Rebalancing Key: KAFKA-15680 URL:

Re: [PR] KAFKA-15584: Leader election with ELR [kafka]

2023-10-24 Thread via GitHub
CalvinConfluent commented on code in PR #14593: URL: https://github.com/apache/kafka/pull/14593#discussion_r1371143824 ## metadata/src/main/java/org/apache/kafka/controller/PartitionChangeBuilder.java: ## @@ -361,11 +410,12 @@ public Optional build() {

Re: [PR] KAFKA-15584: Leader election with ELR [kafka]

2023-10-24 Thread via GitHub
CalvinConfluent commented on code in PR #14593: URL: https://github.com/apache/kafka/pull/14593#discussion_r1371143671 ## metadata/src/main/java/org/apache/kafka/controller/PartitionChangeBuilder.java: ## @@ -93,6 +93,9 @@ public enum Election { private boolean

Re: [PR] KAFKA-15608: Assign lastet leader eopch and offset checkpoint to future log when replacing current log [kafka]

2023-10-24 Thread via GitHub
hudeqi commented on PR #14553: URL: https://github.com/apache/kafka/pull/14553#issuecomment-1778486603 I tested this process on the latest trunk branch and it did not reproduce. My process is: create a topic with 1 partition, then write traffic and stop writing. For example, the current

Re: [PR] KAFKA-15608: Assign lastet leader eopch and offset checkpoint to future log when replacing current log [kafka]

2023-10-24 Thread via GitHub
hudeqi commented on code in PR #14553: URL: https://github.com/apache/kafka/pull/14553#discussion_r1371126756 ## core/src/main/scala/kafka/log/LogManager.scala: ## @@ -1156,6 +1156,9 @@ class LogManager(logDirs: Seq[File], // we will add metrics back after sourceLog

Re: [PR] KAFKA-15673: Adding client metrics resource types (KIP-714) [kafka]

2023-10-24 Thread via GitHub
philipnee commented on code in PR #14621: URL: https://github.com/apache/kafka/pull/14621#discussion_r1371125365 ## core/src/main/scala/kafka/metrics/ClientMetricsConfig.scala: ## @@ -0,0 +1,141 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + *

Re: [PR] KAFKA-15673: Adding client metrics resource types (KIP-714) [kafka]

2023-10-24 Thread via GitHub
philipnee commented on code in PR #14621: URL: https://github.com/apache/kafka/pull/14621#discussion_r1371110225 ## core/src/main/scala/kafka/metrics/ClientMetricsMetadata.scala: ## @@ -0,0 +1,130 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more +

Re: [PR] KAFKA-15673: Adding client metrics resource types (KIP-714) [kafka]

2023-10-24 Thread via GitHub
philipnee commented on code in PR #14621: URL: https://github.com/apache/kafka/pull/14621#discussion_r1371108662 ## core/src/main/scala/kafka/metrics/ClientMetricsMetadata.scala: ## @@ -0,0 +1,130 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more +

Re: [PR] KAFKA-15582: Move the clean shutdown file to the storage package [kafka]

2023-10-24 Thread via GitHub
CalvinConfluent commented on code in PR #14603: URL: https://github.com/apache/kafka/pull/14603#discussion_r1371113716 ## storage/src/main/java/org/apache/kafka/storage/internals/log/CleanShutdownFileHandler.java: ## @@ -85,17 +83,16 @@ void write(long brokerEpoch, int version)

Re: [PR] KAFKA-15582: Move the clean shutdown file to the storage package [kafka]

2023-10-24 Thread via GitHub
CalvinConfluent commented on code in PR #14603: URL: https://github.com/apache/kafka/pull/14603#discussion_r1371113599 ## storage/src/main/java/org/apache/kafka/storage/internals/log/CleanShutdownFileHandler.java: ## @@ -85,17 +83,16 @@ void write(long brokerEpoch, int version)

Re: [PR] KAFKA-15673: Adding client metrics resource types (KIP-714) [kafka]

2023-10-24 Thread via GitHub
philipnee commented on code in PR #14621: URL: https://github.com/apache/kafka/pull/14621#discussion_r1371112996 ## core/src/test/scala/kafka/metrics/ClientMetricsTestUtils.scala: ## @@ -0,0 +1,39 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more +

Re: [PR] KAFKA-15673: Adding client metrics resource types (KIP-714) [kafka]

2023-10-24 Thread via GitHub
philipnee commented on code in PR #14621: URL: https://github.com/apache/kafka/pull/14621#discussion_r1371110225 ## core/src/main/scala/kafka/metrics/ClientMetricsMetadata.scala: ## @@ -0,0 +1,130 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more +

Re: [PR] KAFKA-15673: Adding client metrics resource types (KIP-714) [kafka]

2023-10-24 Thread via GitHub
philipnee commented on code in PR #14621: URL: https://github.com/apache/kafka/pull/14621#discussion_r1371108662 ## core/src/main/scala/kafka/metrics/ClientMetricsMetadata.scala: ## @@ -0,0 +1,130 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more +

Re: [PR] KAFKA-15673: Adding client metrics resource types (KIP-714) [kafka]

2023-10-24 Thread via GitHub
philipnee commented on code in PR #14621: URL: https://github.com/apache/kafka/pull/14621#discussion_r1371106132 ## core/src/main/scala/kafka/metrics/ClientMetricsMetadata.scala: ## @@ -0,0 +1,130 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more +

Re: [PR] KAFKA-14585: Moving StorageTool from core to tools module [kafka]

2023-10-24 Thread via GitHub
github-actions[bot] commented on PR #13417: URL: https://github.com/apache/kafka/pull/13417#issuecomment-1778451254 This PR is being marked as stale since it has not had any activity in 90 days. If you would like to keep this PR alive, please ask a committer for review. If the PR has

Re: [PR] KAFKA-15582: Move the clean shutdown file to the storage package [kafka]

2023-10-24 Thread via GitHub
ijuma commented on code in PR #14603: URL: https://github.com/apache/kafka/pull/14603#discussion_r1371089424 ## storage/src/main/java/org/apache/kafka/storage/internals/log/CleanShutdownFileHandler.java: ## @@ -85,17 +83,16 @@ void write(long brokerEpoch, int version) throws

Re: [PR] KAFKA-15582: Move the clean shutdown file to the storage package [kafka]

2023-10-24 Thread via GitHub
ijuma commented on code in PR #14603: URL: https://github.com/apache/kafka/pull/14603#discussion_r1371088934 ## storage/src/main/java/org/apache/kafka/storage/internals/log/CleanShutdownFileHandler.java: ## @@ -85,17 +83,16 @@ void write(long brokerEpoch, int version) throws

Re: [PR] KAFKA-15582: Move the clean shutdown file to the storage package [kafka]

2023-10-24 Thread via GitHub
CalvinConfluent commented on code in PR #14603: URL: https://github.com/apache/kafka/pull/14603#discussion_r1371084764 ## storage/src/main/java/org/apache/kafka/storage/internals/log/CleanShutdownFileHandler.java: ## @@ -85,17 +83,16 @@ void write(long brokerEpoch, int version)

Re: [PR] MINOR: Rewrite the meta.properties handling code in Java and fix some issues [kafka]

2023-10-24 Thread via GitHub
pprovenzano commented on code in PR #14628: URL: https://github.com/apache/kafka/pull/14628#discussion_r1371083468 ## core/src/test/java/kafka/testkit/KafkaClusterTestKit.java: ## @@ -227,8 +231,12 @@ public KafkaClusterTestKit build() throws Exception {

Re: [PR] KAFKA-15582: Move the clean shutdown file to the storage package [kafka]

2023-10-24 Thread via GitHub
CalvinConfluent commented on code in PR #14603: URL: https://github.com/apache/kafka/pull/14603#discussion_r1371084404 ## storage/src/main/java/org/apache/kafka/storage/internals/log/CleanShutdownFileHandler.java: ## @@ -49,13 +47,15 @@ public class CleanShutdownFileHandler {

Re: [PR] KAFKA-15582: Move the clean shutdown file to the storage package [kafka]

2023-10-24 Thread via GitHub
CalvinConfluent commented on code in PR #14603: URL: https://github.com/apache/kafka/pull/14603#discussion_r1371084249 ## storage/src/main/java/org/apache/kafka/storage/internals/log/CleanShutdownFileHandler.java: ## @@ -15,7 +15,7 @@ * limitations under the License. */

Re: [PR] MINOR: Fix flaky testFollowerCompleteDelayedFetchesOnReplication [kafka]

2023-10-24 Thread via GitHub
dengziming merged PR #14616: URL: https://github.com/apache/kafka/pull/14616 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-15578: System Tests for running old protocol with new coordinator [kafka]

2023-10-24 Thread via GitHub
rreddy-22 commented on code in PR #14524: URL: https://github.com/apache/kafka/pull/14524#discussion_r1370533091 ## tests/kafkatest/services/kafka/kafka.py: ## @@ -277,6 +279,22 @@ def __init__(self, context, num_nodes, zk, security_protocol=SecurityConfig.PLAI

Re: [PR] KAFKA-15578: System Tests for running old protocol with new coordinator [kafka]

2023-10-24 Thread via GitHub
rreddy-22 commented on PR #14524: URL: https://github.com/apache/kafka/pull/14524#issuecomment-1778311241 https://jenkins.confluent.io/job/system-test-kafka-branch-builder/5907/ - Latest test build -- This is an automated message from the Apache Git Service. To respond to the message,

[jira] [Commented] (KAFKA-15602) Breaking change in 3.4.0 ByteBufferSerializer

2023-10-24 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17779278#comment-17779278 ] Philip Nee commented on KAFKA-15602: Hi [~luke.kirby] - Here's the guide to create a KIP:

[jira] [Commented] (KAFKA-15602) Breaking change in 3.4.0 ByteBufferSerializer

2023-10-24 Thread Luke Kirby (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17779276#comment-17779276 ] Luke Kirby commented on KAFKA-15602: >Well, in example one, the position is at 4, and if we say we

[jira] [Updated] (KAFKA-15455) Add support for OffsetCommit version 9 in consumer

2023-10-24 Thread Yi Ding (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Ding updated KAFKA-15455: Labels: kip-848 kip-848-client-support kip-848-e2e kip-848-preview (was: kip-848 kip-848-client-support

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-10-24 Thread via GitHub
jolshan commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1370909940 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -55,23 +55,23 @@ object KafkaRequestHandler { * @param fun Callback function to execute *

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-10-24 Thread via GitHub
jolshan commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1370909940 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -55,23 +55,23 @@ object KafkaRequestHandler { * @param fun Callback function to execute *

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-10-24 Thread via GitHub
jolshan commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1370901998 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -55,23 +55,23 @@ object KafkaRequestHandler { * @param fun Callback function to execute *

Re: [PR] KAFKA-15614: Define interfaces and classes for client telemetry [kafka]

2023-10-24 Thread via GitHub
junrao commented on code in PR #14575: URL: https://github.com/apache/kafka/pull/14575#discussion_r1370888347 ## clients/src/main/java/org/apache/kafka/common/telemetry/ClientTelemetry.java: ## @@ -0,0 +1,44 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-10-24 Thread via GitHub
jolshan commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1370892267 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -55,23 +55,23 @@ object KafkaRequestHandler { * @param fun Callback function to execute *

Re: [PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-10-24 Thread via GitHub
jolshan commented on code in PR #14629: URL: https://github.com/apache/kafka/pull/14629#discussion_r1370892267 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -55,23 +55,23 @@ object KafkaRequestHandler { * @param fun Callback function to execute *

Re: [PR] KAFKA-15629: Proposal to introduce IQv2 Query Types: TimestampedKeyQuery and TimestampedRangeQuery [kafka]

2023-10-24 Thread via GitHub
hanyuzheng7 commented on code in PR #14570: URL: https://github.com/apache/kafka/pull/14570#discussion_r1370877692 ## streams/src/test/java/org/apache/kafka/streams/integration/IQv2StoreIntegrationTest.java: ## @@ -776,64 +779,55 @@ public void verifyStore() {

[PR] KAFKA-15653: Pass requestLocal as argument to callback so we use the correct one for the thread [kafka]

2023-10-24 Thread via GitHub
jolshan opened a new pull request, #14629: URL: https://github.com/apache/kafka/pull/14629 With the new callback mechanism we were accidentally passing context with the wrong request local. Now include a RequestLocal as an explicit argument to the callback. Also make the arguments

[PR] MINOR: Rewrite the meta.properties code in Java [kafka]

2023-10-24 Thread via GitHub
cmccabe opened a new pull request, #14628: URL: https://github.com/apache/kafka/pull/14628 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [PR] KAFKA-15582: Move the clean shutdown file to the storage package [kafka]

2023-10-24 Thread via GitHub
junrao commented on code in PR #14603: URL: https://github.com/apache/kafka/pull/14603#discussion_r1370850704 ## storage/src/main/java/org/apache/kafka/storage/internals/log/CleanShutdownFileHandler.java: ## @@ -15,7 +15,7 @@ * limitations under the License. */ -package

Re: [PR] KAFKA-15584: Leader election with ELR [kafka]

2023-10-24 Thread via GitHub
artemlivshits commented on code in PR #14593: URL: https://github.com/apache/kafka/pull/14593#discussion_r1370847603 ## metadata/src/main/java/org/apache/kafka/controller/PartitionChangeBuilder.java: ## @@ -361,11 +410,12 @@ public Optional build() {

Re: [PR] KAFKA-15673: Adding client metrics resource types (KIP-714) [kafka]

2023-10-24 Thread via GitHub
AndrewJSchofield commented on code in PR #14621: URL: https://github.com/apache/kafka/pull/14621#discussion_r1370842597 ## core/src/main/scala/kafka/server/ZkConfigManager.scala: ## @@ -38,7 +38,8 @@ object ConfigType { val User = "users" val Broker = "brokers" val Ip

[jira] [Resolved] (KAFKA-15304) CompletableApplicationEvents aren't being completed when the consumer is closing

2023-10-24 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True resolved KAFKA-15304. --- Fix Version/s: 3.7.0 Reviewer: Jun Rao Resolution: Fixed Fixed as part of PR

[PR] KIP-951: protocol changes [kafka]

2023-10-24 Thread via GitHub
chb2ab opened a new pull request, #14627: URL: https://github.com/apache/kafka/pull/14627 Separating out the protocol changes from https://github.com/apache/kafka/pull/1 in an effort to more quickly unblock the client side PR.

[jira] [Updated] (KAFKA-15188) Implement more of the remaining PrototypeAsyncConsumer APIs

2023-10-24 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15188: -- Description: There are several {{Consumer}} APIs that only touch the {{ConsumerMetadata}} and/or

[jira] [Comment Edited] (KAFKA-15184) New consumer internals refactoring and clean up

2023-10-24 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17779252#comment-17779252 ] Kirk True edited comment on KAFKA-15184 at 10/24/23 9:17 PM: - See pull

[PR] KAFKA-15347: add support for 'single key multi timestamp' IQs with versioned state stores (KIP-968) [kafka]

2023-10-24 Thread via GitHub
aliehsaeedii opened a new pull request, #14626: URL: https://github.com/apache/kafka/pull/14626 This is a POC PR for KIP-968. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[jira] [Updated] (KAFKA-15456) Add support for OffsetFetch version 9 in consumer

2023-10-24 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15456: -- Labels: kip-848 kip-848-client-support kip-848-e2e kip-848-preview (was: kip-848

Re: [PR] KAFKA-14274 [6, 7]: Introduction of fetch request manager [kafka]

2023-10-24 Thread via GitHub
junrao merged PR #14406: URL: https://github.com/apache/kafka/pull/14406 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Commented] (KAFKA-15602) Breaking change in 3.4.0 ByteBufferSerializer

2023-10-24 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17779216#comment-17779216 ] Matthias J. Sax commented on KAFKA-15602: - {quote}I'm not entirely sure I understand Matthias's

Re: [PR] KAFKA-14274 [6, 7]: Introduction of fetch request manager [kafka]

2023-10-24 Thread via GitHub
kirktrue commented on PR #14406: URL: https://github.com/apache/kafka/pull/14406#issuecomment-1777892441 @junrao—yes. Here's a brief history for JDK 21, starting with the most recent build (build 74): * [build

Re: [PR] KAFKA-14274 [6, 7]: Introduction of fetch request manager [kafka]

2023-10-24 Thread via GitHub
junrao commented on PR #14406: URL: https://github.com/apache/kafka/pull/14406#issuecomment-1777855712 @kirktrue : The build for JDK 21 and Scala 2.13 failed this time. Did it succeed before? -- This is an automated message from the Apache Git Service. To respond to the message, please

[jira] [Updated] (KAFKA-15678) [Tiered Storage] Stall remote reads with long-spanning transactions

2023-10-24 Thread Alexandre Dupriez (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexandre Dupriez updated KAFKA-15678: -- Description: I am facing an issue on the remote data path for uncommitted reads. As

[PR] MINOR: avoid blocking for randomness in DefaultRecordBatchTest [kafka]

2023-10-24 Thread via GitHub
gaurav-narula opened a new pull request, #14625: URL: https://github.com/apache/kafka/pull/14625 PR #13135 introduced tests in DefaultRecordBatchTest which require filling buffers with random data. Using `SecureRandom.getInstanceStrong()` results in using `/dev/random` which is known to

Re: [PR] HOTFIX: close iterator to avoid resource leak [kafka]

2023-10-24 Thread via GitHub
bbejeck commented on PR #14624: URL: https://github.com/apache/kafka/pull/14624#issuecomment-1777825636 @mjsax it looks like the build is failing with a checkstyle error though -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] HOTFIX: close iterator to avoid resource leak [kafka]

2023-10-24 Thread via GitHub
lihaosky commented on code in PR #14624: URL: https://github.com/apache/kafka/pull/14624#discussion_r1370675126 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamSessionWindowAggregate.java: ## @@ -264,23 +264,25 @@ private void fetchAndEmit(final

Re: [PR] KAFKA-15582: Move the clean shutdown file to the storage package [kafka]

2023-10-24 Thread via GitHub
CalvinConfluent commented on code in PR #14603: URL: https://github.com/apache/kafka/pull/14603#discussion_r1370658110 ## storage/src/main/java/org/apache/kafka/storage/internals/util/CleanShutdownFileHandler.java: ## @@ -85,6 +85,7 @@ void write(long brokerEpoch, int version)

Re: [PR] KAFKA-15582: Move the clean shutdown file to the storage package [kafka]

2023-10-24 Thread via GitHub
CalvinConfluent commented on code in PR #14603: URL: https://github.com/apache/kafka/pull/14603#discussion_r1370657796 ## storage/src/main/java/org/apache/kafka/storage/internals/util/CleanShutdownFileHandler.java: ## @@ -15,7 +15,7 @@ * limitations under the License. */

[jira] [Updated] (KAFKA-15676) Scheduled rebalance delay for Connect is unnecessarily triggered when group coordinator bounces

2023-10-24 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton updated KAFKA-15676: -- Description: When a Connect worker loses contact with the group coordinator, it voluntarily

[PR] HOTFIX: close iterator to avoid resource leak [kafka]

2023-10-24 Thread via GitHub
mjsax opened a new pull request, #14624: URL: https://github.com/apache/kafka/pull/14624 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

Re: [PR] KAFKA-15628: Refactor ConsumerRebalanceListener invocation for reuse [kafka]

2023-10-24 Thread via GitHub
kirktrue commented on code in PR #14565: URL: https://github.com/apache/kafka/pull/14565#discussion_r1370618922 ## clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java: ## @@ -976,15 +1053,14 @@ public void subscribe(Collection topics) { * Group

Re: [PR] KAFKA-15628: Refactor ConsumerRebalanceListener invocation for reuse [kafka]

2023-10-24 Thread via GitHub
kirktrue commented on code in PR #14565: URL: https://github.com/apache/kafka/pull/14565#discussion_r1370615968 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerRebalanceListenerInvoker.java: ## @@ -0,0 +1,128 @@ +/* + * Licensed to the Apache

[jira] [Updated] (KAFKA-15678) [Tiered Storage] Stall remote reads with long-spanning transactions

2023-10-24 Thread Alexandre Dupriez (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexandre Dupriez updated KAFKA-15678: -- Description: I am facing an issue on the remote data path for uncommitted reads. As

[jira] [Updated] (KAFKA-15678) [Tiered Storage] Stall remote reads with long-spanning transactions

2023-10-24 Thread Alexandre Dupriez (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexandre Dupriez updated KAFKA-15678: -- Description: I am facing an issue on the remote data path for uncommitted reads. As

[jira] [Updated] (KAFKA-15679) Client support for new consumer configs

2023-10-24 Thread Lianet Magrans (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianet Magrans updated KAFKA-15679: --- Summary: Client support for new consumer configs (was: Client support fore new consumer

[jira] [Updated] (KAFKA-15301) [Tiered Storage] Historically compacted topics send request to remote for active segment during consume

2023-10-24 Thread Alexandre Dupriez (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexandre Dupriez updated KAFKA-15301: -- Description: I have a use case where tiered storage plugin received requests for

Re: [PR] KAFKA-15627: KIP-951's Leader discovery optimisations on the client [kafka]

2023-10-24 Thread via GitHub
msn-tldr commented on code in PR #14564: URL: https://github.com/apache/kafka/pull/14564#discussion_r1370368421 ## clients/src/main/java/org/apache/kafka/clients/MetadataCache.java: ## @@ -150,7 +150,7 @@ MetadataCache mergeWith(String newClusterId, // We want the most

[jira] [Updated] (KAFKA-15678) [Tiered Storage] Stall remote reads with long-spanning transactions

2023-10-24 Thread Alexandre Dupriez (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexandre Dupriez updated KAFKA-15678: -- Description: I am facing an issue on the remote data path for uncommitted reads. As

[jira] [Updated] (KAFKA-15678) [Tiered Storage] Stall remote reads with long-spanning transactions

2023-10-24 Thread Alexandre Dupriez (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexandre Dupriez updated KAFKA-15678: -- Description: I am facing an issue on the remote data path for uncommitted reads. As

[jira] [Updated] (KAFKA-15679) Client support fore new consumer configs

2023-10-24 Thread Lianet Magrans (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianet Magrans updated KAFKA-15679: --- Summary: Client support fore new consumer configs (was: Client support new consumer

[jira] [Updated] (KAFKA-15678) [Tiered Storage] Stall remote reads with long-spanning transactions

2023-10-24 Thread Alexandre Dupriez (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexandre Dupriez updated KAFKA-15678: -- Description: I am facing an issue on the remote data path for uncommitted reads. As

[jira] [Updated] (KAFKA-15678) [Tiered Storage] Stall remote reads with long-spanning transactions

2023-10-24 Thread Alexandre Dupriez (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexandre Dupriez updated KAFKA-15678: -- Labels: KIP-405 (was: ) > [Tiered Storage] Stall remote reads with long-spanning

[jira] [Commented] (KAFKA-15562) Ensure fetch offset and commit offset handler handles both timeout and various error types

2023-10-24 Thread Grigorii Piskunov (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17779181#comment-17779181 ] Grigorii Piskunov commented on KAFKA-15562: --- Hi [~pnee] yes, sure. Please share available. I

[jira] [Created] (KAFKA-15679) Client support new consumer configs

2023-10-24 Thread Lianet Magrans (Jira)
Lianet Magrans created KAFKA-15679: -- Summary: Client support new consumer configs Key: KAFKA-15679 URL: https://issues.apache.org/jira/browse/KAFKA-15679 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-15678) [Tiered Storage] Stall remote reads with long-spanning transactions

2023-10-24 Thread Alexandre Dupriez (Jira)
Alexandre Dupriez created KAFKA-15678: - Summary: [Tiered Storage] Stall remote reads with long-spanning transactions Key: KAFKA-15678 URL: https://issues.apache.org/jira/browse/KAFKA-15678

Re: [PR] MINOR: added compatibility for MinGW [kafka]

2023-10-24 Thread via GitHub
maniekes commented on PR #13321: URL: https://github.com/apache/kafka/pull/13321#issuecomment-1777659808 @divijvaidya looks like it's stale PR :) shall we merge it or close? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[jira] [Updated] (KAFKA-15600) KIP-990: Capability to PAUSE Tasks on DeserializationException

2023-10-24 Thread Nicholas Telford (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Telford updated KAFKA-15600: - Summary: KIP-990: Capability to PAUSE Tasks on DeserializationException (was: KIP-990:

[jira] [Assigned] (KAFKA-15600) KIP-990: Capability to PAUSE Tasks on DeserializationException

2023-10-24 Thread Nicholas Telford (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Telford reassigned KAFKA-15600: Assignee: Nicholas Telford > KIP-990: Capability to PAUSE Tasks on

[jira] [Updated] (KAFKA-15508) Method always return the same value ApplicationEventProcessor.java

2023-10-24 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee updated KAFKA-15508: --- Priority: Minor (was: Blocker) > Method always return the same value

[jira] [Commented] (KAFKA-15508) Method always return the same value ApplicationEventProcessor.java

2023-10-24 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17779175#comment-17779175 ] Philip Nee commented on KAFKA-15508: This is not a blocker right? > Method always return the same

Re: [PR] KAFKA-14516: [1/N] Static Member leave, join, re-join request using ConsumerGroupHeartbeats [kafka]

2023-10-24 Thread via GitHub
vamossagar12 commented on PR #14432: URL: https://github.com/apache/kafka/pull/14432#issuecomment-1777654947 Thanks @dajac , I have addressed all the comments. Have added a couple of more questions. Thanks for all the pointers, the changes look cleaner than the previous iteration (to me

Re: [PR] KAFKA-14516: [1/N] Static Member leave, join, re-join request using ConsumerGroupHeartbeats [kafka]

2023-10-24 Thread via GitHub
vamossagar12 commented on code in PR #14432: URL: https://github.com/apache/kafka/pull/14432#discussion_r1370542261 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupMetadataManagerTest.java: ## @@ -1908,6 +1909,715 @@ public void

Re: [PR] KAFKA-14516: [1/N] Static Member leave, join, re-join request using ConsumerGroupHeartbeats [kafka]

2023-10-24 Thread via GitHub
vamossagar12 commented on code in PR #14432: URL: https://github.com/apache/kafka/pull/14432#discussion_r1370539621 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/ConsumerGroup.java: ## @@ -285,6 +302,11 @@ public ConsumerGroupMember

Re: [PR] KAFKA-14516: [1/N] Static Member leave, join, re-join request using ConsumerGroupHeartbeats [kafka]

2023-10-24 Thread via GitHub
vamossagar12 commented on code in PR #14432: URL: https://github.com/apache/kafka/pull/14432#discussion_r1370538866 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -983,27 +1052,125 @@ private CoordinatorResult consumerGr

Re: [PR] KAFKA-14516: [1/N] Static Member leave, join, re-join request using ConsumerGroupHeartbeats [kafka]

2023-10-24 Thread via GitHub
vamossagar12 commented on code in PR #14432: URL: https://github.com/apache/kafka/pull/14432#discussion_r1370533114 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/ConsumerGroup.java: ## @@ -299,6 +321,9 @@ public void

Re: [PR] KAFKA-15578: System Tests for running old protocol with new coordinator [kafka]

2023-10-24 Thread via GitHub
rreddy-22 commented on code in PR #14524: URL: https://github.com/apache/kafka/pull/14524#discussion_r1370533091 ## tests/kafkatest/services/kafka/kafka.py: ## @@ -277,6 +279,22 @@ def __init__(self, context, num_nodes, zk, security_protocol=SecurityConfig.PLAI

Re: [PR] KAFKA-14516: [1/N] Static Member leave, join, re-join request using ConsumerGroupHeartbeats [kafka]

2023-10-24 Thread via GitHub
vamossagar12 commented on code in PR #14432: URL: https://github.com/apache/kafka/pull/14432#discussion_r1370532536 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -983,27 +1052,125 @@ private CoordinatorResult consumerGr

Re: [PR] KAFKA-14516: [1/N] Static Member leave, join, re-join request using ConsumerGroupHeartbeats [kafka]

2023-10-24 Thread via GitHub
vamossagar12 commented on code in PR #14432: URL: https://github.com/apache/kafka/pull/14432#discussion_r1370531600 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -983,27 +1052,125 @@ private CoordinatorResult consumerGr

Re: [PR] KAFKA-14516: [1/N] Static Member leave, join, re-join request using ConsumerGroupHeartbeats [kafka]

2023-10-24 Thread via GitHub
vamossagar12 commented on code in PR #14432: URL: https://github.com/apache/kafka/pull/14432#discussion_r1370523927 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -983,27 +1052,125 @@ private CoordinatorResult consumerGr

Re: [PR] KAFKA-14516: [1/N] Static Member leave, join, re-join request using ConsumerGroupHeartbeats [kafka]

2023-10-24 Thread via GitHub
vamossagar12 commented on code in PR #14432: URL: https://github.com/apache/kafka/pull/14432#discussion_r1370523697 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -747,6 +798,11 @@ private void throwIfMemberEpochIsInvalid(

Re: [PR] KAFKA-15628: Refactor ConsumerRebalanceListener invocation for reuse [kafka]

2023-10-24 Thread via GitHub
philipnee commented on code in PR #14565: URL: https://github.com/apache/kafka/pull/14565#discussion_r1370505973 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerRebalanceListenerInvoker.java: ## @@ -0,0 +1,128 @@ +/* + * Licensed to the Apache

[jira] [Updated] (KAFKA-15676) Scheduled rebalance delay for Connect is unnecessarily triggered when group coordinator bounces

2023-10-24 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton updated KAFKA-15676: -- Description: When a Connect worker loses contact with the group coordinator, it voluntarily

Re: [PR] KAFKA-15628: Refactor ConsumerRebalanceListener invocation for reuse [kafka]

2023-10-24 Thread via GitHub
philipnee commented on code in PR #14565: URL: https://github.com/apache/kafka/pull/14565#discussion_r1370492706 ## clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java: ## @@ -976,15 +1053,14 @@ public void subscribe(Collection topics) { * Group

Re: [PR] KAFKA-14274 [6, 7]: Introduction of fetch request manager [kafka]

2023-10-24 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1370486899 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/PrototypeAsyncConsumer.java: ## @@ -401,12 +524,12 @@ private void

[jira] [Created] (KAFKA-15677) Add support for re2/j regex parsing for patterns

2023-10-24 Thread Apoorv Mittal (Jira)
Apoorv Mittal created KAFKA-15677: - Summary: Add support for re2/j regex parsing for patterns Key: KAFKA-15677 URL: https://issues.apache.org/jira/browse/KAFKA-15677 Project: Kafka Issue

[jira] [Commented] (KAFKA-15562) Ensure fetch offset and commit offset handler handles both timeout and various error types

2023-10-24 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17779161#comment-17779161 ] Philip Nee commented on KAFKA-15562: Hi [~grigorii] - I'm already in progress working on this issue. 

Re: [PR] KAFKA-14274 [6, 7]: Introduction of fetch request manager [kafka]

2023-10-24 Thread via GitHub
lianetm commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1370452938 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/PrototypeAsyncConsumer.java: ## @@ -401,12 +524,12 @@ private void

Re: [PR] KAFKA-15633: Fix overwrite of meta.properties at startup to handle JBOD. [kafka]

2023-10-24 Thread via GitHub
pprovenzano commented on PR #14578: URL: https://github.com/apache/kafka/pull/14578#issuecomment-1777540934 > I think we should add a test for this. Perhaps we can add or extend one of the existing integration tests? I'd like to do that as a separate PR. -- This is an automated

[jira] [Updated] (KAFKA-15676) Scheduled rebalance delay for Connect is unnecessarily triggered when group coordinator bounces

2023-10-24 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton updated KAFKA-15676: -- Summary: Scheduled rebalance delay for Connect is unnecessarily triggered when group

[jira] [Commented] (KAFKA-15676) Scheduled rebalance delay for Connect is unnecessarily triggered when Kafka cluster bounces

2023-10-24 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17779144#comment-17779144 ] Chris Egerton commented on KAFKA-15676: --- One possible fix for this is to simply wipe the state for

  1   2   >