[GitHub] [kafka] showuon closed pull request #11578: KAFKA-13514: Fix flakey testLargeAssignmentAndGroupWithNonEqualSubscription

2022-01-05 Thread GitBox
showuon closed pull request #11578: URL: https://github.com/apache/kafka/pull/11578 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr.

[GitHub] [kafka] showuon commented on pull request #11578: KAFKA-13514: Fix flakey testLargeAssignmentAndGroupWithNonEqualSubscription

2022-01-05 Thread GitBox
showuon commented on pull request #11578: URL: https://github.com/apache/kafka/pull/11578#issuecomment-1006341423 @ableegoldman , yes, you're right. I didn't check the logs clearly. It looks like there's a bug in general assignor that assign the same partition to multiple consumers. I'll s

[jira] [Commented] (KAFKA-13514) Flakey test StickyAssignorTest

2022-01-05 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17469743#comment-17469743 ] Luke Chen commented on KAFKA-13514: --- [~ableegoldman] [~mjsax] , It looks like the assi

[GitHub] [kafka] showuon commented on pull request #11613: MINOR: Update streamResetter option description

2022-01-05 Thread GitBox
showuon commented on pull request #11613: URL: https://github.com/apache/kafka/pull/11613#issuecomment-1006329948 @mjsax , please take a look when available. Thank you. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [kafka] cmccabe merged pull request #11637: MINOR: enable KRaft in MetadataRequestTest

2022-01-05 Thread GitBox
cmccabe merged pull request #11637: URL: https://github.com/apache/kafka/pull/11637 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr.

[GitHub] [kafka] cmccabe commented on pull request #11637: MINOR: enable KRaft in MetadataRequestTest

2022-01-05 Thread GitBox
cmccabe commented on pull request #11637: URL: https://github.com/apache/kafka/pull/11637#issuecomment-1006304214 Failed tests are `org.apache.kafka.streams.integration.StoreQueryIntegrationTest` and kafka.admin.LeaderElectionCommandTest`, which are not related. -- This is an automated

[jira] [Resolved] (KAFKA-13476) Streams crashes when non Base64 Offset Metadata is found

2022-01-05 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-13476. - Fix Version/s: 3.2.0 Resolution: Fixed > Streams crashes when non Base64 Offset M

[GitHub] [kafka] mjsax commented on pull request #11535: KAFKA-13476: Increase resilience timestamp decoding Kafka Streams

2022-01-05 Thread GitBox
mjsax commented on pull request #11535: URL: https://github.com/apache/kafka/pull/11535#issuecomment-1006299533 Thanks for the PR! Merged to `trunk`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [kafka] mjsax merged pull request #11535: KAFKA-13476: Increase resilience timestamp decoding Kafka Streams

2022-01-05 Thread GitBox
mjsax merged pull request #11535: URL: https://github.com/apache/kafka/pull/11535 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...

[GitHub] [kafka] vvcephei merged pull request #11650: KAFKA-13553: Add PAPI Window and Session store tests for IQv2

2022-01-05 Thread GitBox
vvcephei merged pull request #11650: URL: https://github.com/apache/kafka/pull/11650 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr

[GitHub] [kafka] dongjinleekr commented on pull request #7898: KAFKA-9366: Change log4j dependency into log4j2

2022-01-05 Thread GitBox
dongjinleekr commented on pull request #7898: URL: https://github.com/apache/kafka/pull/7898#issuecomment-1006282789 @gabrieljones > How hard is it to backport this all the way back to Kafka v2.3.x? Almost impossible. There are too many conflicts. -- This is an automated mes

[GitHub] [kafka] showuon commented on pull request #11340: KAFKA-13310 : KafkaConsumer cannot jump out of the poll method, and the…

2022-01-05 Thread GitBox
showuon commented on pull request #11340: URL: https://github.com/apache/kafka/pull/11340#issuecomment-1006251125 The failed tests are flaky and also failed sometimes in recent trunk jenkins build. Looks good. ``` Build / ARM / kafka.server.ReplicaManagerTest.[1] usesTopicIds=true

[GitHub] [kafka] vvcephei merged pull request #11624: KAFKA-13553: add PAPI KV store tests for IQv2

2022-01-05 Thread GitBox
vvcephei merged pull request #11624: URL: https://github.com/apache/kafka/pull/11624 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr

[GitHub] [kafka] vvcephei commented on pull request #11624: KAFKA-13553: add PAPI KV store tests for IQv2

2022-01-05 Thread GitBox
vvcephei commented on pull request #11624: URL: https://github.com/apache/kafka/pull/11624#issuecomment-1006246786 Unrelated test failure: ``` Build / JDK 17 and Scala 2.13 / testTopicIdUpgradeAfterReassigningPartitions() – kafka.controller.ControllerIntegrationTest ``` -- This

[GitHub] [kafka] vvcephei commented on pull request #11652: MINOR: Add num threads logging upon shutdown

2022-01-05 Thread GitBox
vvcephei commented on pull request #11652: URL: https://github.com/apache/kafka/pull/11652#issuecomment-1006244975 Unrelated test failures: ``` Build / ARM / kafka.server.ReplicaManagerTest.[1] usesTopicIds=true Build / JDK 8 and Scala 2.12 / kafka.api.ConsumerBounceTest.testCloseD

[GitHub] [kafka] dengziming commented on pull request #11629: MINOR: enable KRaft mode in CreateTopicsRequestTest

2022-01-05 Thread GitBox
dengziming commented on pull request #11629: URL: https://github.com/apache/kafka/pull/11629#issuecomment-1006228274 > Hmm, I'm not sure I follow. What makes you think the metadata change is not getting propagated? The QA Test failure is related to this PR, and I run this test locall

[GitHub] [kafka] cmccabe commented on a change in pull request #11629: MINOR: enable KRaft mode in CreateTopicsRequestTest

2022-01-05 Thread GitBox
cmccabe commented on a change in pull request #11629: URL: https://github.com/apache/kafka/pull/11629#discussion_r779255324 ## File path: core/src/test/scala/integration/kafka/api/IntegrationTestHarness.scala ## @@ -80,6 +81,19 @@ abstract class IntegrationTestHarness extends

[GitHub] [kafka] junrao merged pull request #11641: MINOR: update the comment for Utils.atomicMoveWithFallback

2022-01-05 Thread GitBox
junrao merged pull request #11641: URL: https://github.com/apache/kafka/pull/11641 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr..

[GitHub] [kafka] junrao commented on a change in pull request #11390: [KAFKA-13369] Follower fetch protocol changes for tiered storage.

2022-01-05 Thread GitBox
junrao commented on a change in pull request #11390: URL: https://github.com/apache/kafka/pull/11390#discussion_r779191474 ## File path: core/src/main/scala/kafka/server/ReplicaFetcherThread.scala ## @@ -386,11 +397,109 @@ class ReplicaFetcherThread(name: String, } /**

[GitHub] [kafka] cmccabe commented on pull request #11629: MINOR: enable KRaft mode in CreateTopicsRequestTest

2022-01-05 Thread GitBox
cmccabe commented on pull request #11629: URL: https://github.com/apache/kafka/pull/11629#issuecomment-1006197493 > It seems that metadata change is not propagated to KRaftMetadataCache timely, we shoud add an assertion before sending metadata request in AbstractCreateTopicsRequestTest.val

[GitHub] [kafka] cmccabe commented on a change in pull request #11629: MINOR: enable KRaft mode in CreateTopicsRequestTest

2022-01-05 Thread GitBox
cmccabe commented on a change in pull request #11629: URL: https://github.com/apache/kafka/pull/11629#discussion_r779227016 ## File path: core/src/test/scala/integration/kafka/api/IntegrationTestHarness.scala ## @@ -80,6 +81,19 @@ abstract class IntegrationTestHarness extends

[GitHub] [kafka] cmccabe commented on a change in pull request #11629: MINOR: enable KRaft mode in CreateTopicsRequestTest

2022-01-05 Thread GitBox
cmccabe commented on a change in pull request #11629: URL: https://github.com/apache/kafka/pull/11629#discussion_r779226683 ## File path: core/src/test/scala/unit/kafka/server/CreateTopicsRequestTest.scala ## @@ -99,29 +101,33 @@ class CreateTopicsRequestTest extends AbstractC

[GitHub] [kafka] cmccabe commented on pull request #11637: MINOR: enable KRaft in MetadataRequestTest

2022-01-05 Thread GitBox
cmccabe commented on pull request #11637: URL: https://github.com/apache/kafka/pull/11637#issuecomment-1006185759 Oops, the quorum parameter should be present for all of these. Fixed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [kafka] hachikuji commented on a change in pull request #11629: MINOR: enable KRaft mode in CreateTopicsRequestTest

2022-01-05 Thread GitBox
hachikuji commented on a change in pull request #11629: URL: https://github.com/apache/kafka/pull/11629#discussion_r779218571 ## File path: core/src/test/scala/integration/kafka/api/IntegrationTestHarness.scala ## @@ -80,6 +81,19 @@ abstract class IntegrationTestHarness extend

[jira] [Commented] (KAFKA-13435) Group won't consume partitions added after static member restart

2022-01-05 Thread Jason Gustafson (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17469592#comment-17469592 ] Jason Gustafson commented on KAFKA-13435: - [~dajac] [~rleslie] [~guozhang] Thank

[GitHub] [kafka] ccding commented on a change in pull request #11390: [KAFKA-13369] Follower fetch protocol changes for tiered storage.

2022-01-05 Thread GitBox
ccding commented on a change in pull request #11390: URL: https://github.com/apache/kafka/pull/11390#discussion_r779162870 ## File path: core/src/main/scala/kafka/log/remote/RemoteIndexCache.scala ## @@ -0,0 +1,225 @@ +/** + * Licensed to the Apache Software Foundation (ASF) un

[GitHub] [kafka] ccding commented on a change in pull request #11390: [KAFKA-13369] Follower fetch protocol changes for tiered storage.

2022-01-05 Thread GitBox
ccding commented on a change in pull request #11390: URL: https://github.com/apache/kafka/pull/11390#discussion_r778990618 ## File path: clients/src/main/java/org/apache/kafka/common/requests/ListOffsetsRequest.java ## @@ -42,6 +42,11 @@ public static final long LATEST_TI

[GitHub] [kafka] ccding commented on a change in pull request #11390: [KAFKA-13369] Follower fetch protocol changes for tiered storage.

2022-01-05 Thread GitBox
ccding commented on a change in pull request #11390: URL: https://github.com/apache/kafka/pull/11390#discussion_r778999529 ## File path: clients/src/main/resources/common/message/ListOffsetsResponse.json ## @@ -31,7 +31,10 @@ // Version 6 enables flexible versions. //

[GitHub] [kafka] vvcephei commented on a change in pull request #11650: KAFKA-13553: Add PAPI Window and Session store tests for IQv2

2022-01-05 Thread GitBox
vvcephei commented on a change in pull request #11650: URL: https://github.com/apache/kafka/pull/11650#discussion_r779188902 ## File path: streams/src/test/java/org/apache/kafka/streams/integration/IQv2StoreIntegrationTest.java ## @@ -626,6 +627,118 @@ public void process(fina

[GitHub] [kafka] lbradstreet opened a new pull request #11653: MINOR: greatly improve test runtime by unblocking purgatory and quota manager threads

2022-01-05 Thread GitBox
lbradstreet opened a new pull request #11653: URL: https://github.com/apache/kafka/pull/11653 We essentially end up waiting far too long to shutdown the purgatory and quota manager ShutdownableThreads. -- This is an automated message from the Apache Git Service. To respond to the message

[GitHub] [kafka] guozhangwang commented on a change in pull request #11650: KAFKA-13553: Add PAPI Window and Session store tests for IQv2

2022-01-05 Thread GitBox
guozhangwang commented on a change in pull request #11650: URL: https://github.com/apache/kafka/pull/11650#discussion_r779180990 ## File path: streams/src/test/java/org/apache/kafka/streams/integration/IQv2StoreIntegrationTest.java ## @@ -626,6 +627,118 @@ public void process(

[GitHub] [kafka] guozhangwang commented on pull request #11652: MINOR: Add num threads logging upon shutdown

2022-01-05 Thread GitBox
guozhangwang commented on pull request #11652: URL: https://github.com/apache/kafka/pull/11652#issuecomment-1006129355 Suspect there are some race conditions in `KafkaStreams#close` that would leak threads. -- This is an automated message from the Apache Git Service. To respond to the me

[GitHub] [kafka] guozhangwang opened a new pull request #11652: MINOR: Add num threads logging upon shutdown

2022-01-05 Thread GitBox
guozhangwang opened a new pull request #11652: URL: https://github.com/apache/kafka/pull/11652 1. Add num of threads logging upon shutdown. 2. Prefix the shutdown thread with client id. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation

[jira] [Commented] (KAFKA-12537) Single Threaded applications will not work with SHUTDOWN_APPLICATION

2022-01-05 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17469566#comment-17469566 ] Guozhang Wang commented on KAFKA-12537: --- Hey [~wcarlson5] just for my own understa

[GitHub] [kafka] lbradstreet opened a new pull request #11651: MINOR: allow test reruns to be merged with original runs

2022-01-05 Thread GitBox
lbradstreet opened a new pull request #11651: URL: https://github.com/apache/kafka/pull/11651 This will need some better jenkins reporting in order to be mergeable. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [kafka] vvcephei commented on a change in pull request #11650: MINOR: Add PAPI Window and Session store tests for IQv2

2022-01-05 Thread GitBox
vvcephei commented on a change in pull request #11650: URL: https://github.com/apache/kafka/pull/11650#discussion_r779160412 ## File path: streams/src/test/java/org/apache/kafka/streams/integration/IQv2StoreIntegrationTest.java ## @@ -800,13 +914,13 @@ public void verifyStore(

[GitHub] [kafka] vvcephei opened a new pull request #11650: MINOR: Add PAPI Window and Session store tests for IQv2

2022-01-05 Thread GitBox
vvcephei opened a new pull request #11650: URL: https://github.com/apache/kafka/pull/11650 During some recent reviews, @mjsax pointed out that StateStore layers are constructed differently the stores are added via the PAPI vs. the DSL. This PR adds PAPI construction for Window and

[GitHub] [kafka] mjsax merged pull request #11648: MINOR: add default-replication-factor to MockAdminClient

2022-01-05 Thread GitBox
mjsax merged pull request #11648: URL: https://github.com/apache/kafka/pull/11648 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...

[GitHub] [kafka] gabrieljones edited a comment on pull request #7898: KAFKA-9366: Change log4j dependency into log4j2

2022-01-05 Thread GitBox
gabrieljones edited a comment on pull request #7898: URL: https://github.com/apache/kafka/pull/7898#issuecomment-1005938463 How hard is it to backport this all the way back to Kafka v2.3.x? -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [kafka] gabrieljones commented on pull request #7898: KAFKA-9366: Change log4j dependency into log4j2

2022-01-05 Thread GitBox
gabrieljones commented on pull request #7898: URL: https://github.com/apache/kafka/pull/7898#issuecomment-1005938463 How hard would it be to backport this all the way back to kafka 2.3.x? -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [kafka] junrao commented on a change in pull request #11641: MINOR: update the comment for Utils.atomicMoveWithFallback

2022-01-05 Thread GitBox
junrao commented on a change in pull request #11641: URL: https://github.com/apache/kafka/pull/11641#discussion_r778987140 ## File path: clients/src/main/java/org/apache/kafka/common/utils/Utils.java ## @@ -924,7 +924,7 @@ public static void atomicMoveWithFallback(Path source,

[GitHub] [kafka] vamossagar12 commented on pull request #11424: KAFKA-13152: Replace "buffered.records.per.partition" with "input.buffer.max.bytes"

2022-01-05 Thread GitBox
vamossagar12 commented on pull request #11424: URL: https://github.com/apache/kafka/pull/11424#issuecomment-1005895064 hey @guozhangwang , For `CACHE_MAX_BYTES_BUFFERING_CONFIG` , I can replace but as per one of the comments here => https://github.com/apache/kafka/pull/11424#discussion_r7

[GitHub] [kafka] mimaison commented on pull request #11560: KAFKA-7589: Allow configuring network threads per listener

2022-01-05 Thread GitBox
mimaison commented on pull request #11560: URL: https://github.com/apache/kafka/pull/11560#issuecomment-1005536732 @dajac @rajinisivaram Do you also want to take a look? If not, I'll merge later this week. Thanks -- This is an automated message from the Apache Git Service. To respond to

[jira] [Updated] (KAFKA-13576) Processor.ConnectionQueueSize provides configuration & metrics, SelectorMetrics adds connection-register related metrics

2022-01-05 Thread RivenSun (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] RivenSun updated KAFKA-13576: - Description: h1. Problem: After all client machines are switched to the company's private BYOIP, produ

[jira] [Commented] (KAFKA-13570) Fallback of unsupported versions of ApiVersionsRequest and RequestHeader?

2022-01-05 Thread Fredrik Arvidsson (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17469134#comment-17469134 ] Fredrik Arvidsson commented on KAFKA-13570: --- Would have been nice having such

[jira] [Commented] (KAFKA-13576) Processor.ConnectionQueueSize provides configuration & metrics, SelectorMetrics adds connection-register related metrics

2022-01-05 Thread RivenSun (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17469124#comment-17469124 ] RivenSun commented on KAFKA-13576: -- Hi [~showuon]  [~guozhang]  Please help check this

[jira] [Created] (KAFKA-13576) Processor.ConnectionQueueSize provides configuration & metrics, SelectorMetrics adds connection-register related metrics

2022-01-05 Thread RivenSun (Jira)
RivenSun created KAFKA-13576: Summary: Processor.ConnectionQueueSize provides configuration & metrics, SelectorMetrics adds connection-register related metrics Key: KAFKA-13576 URL: https://issues.apache.org/jira/brow

[jira] [Comment Edited] (KAFKA-13575) Unwanted cleanup since log.retention.hours=-1 does not work

2022-01-05 Thread Alok Kumar Singh (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17469101#comment-17469101 ] Alok Kumar Singh edited comment on KAFKA-13575 at 1/5/22, 8:08 AM: ---

[jira] [Commented] (KAFKA-13575) Unwanted cleanup since log.retention.hours=-1 does not work

2022-01-05 Thread Alok Kumar Singh (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17469101#comment-17469101 ] Alok Kumar Singh commented on KAFKA-13575: -- If someone picks this we have a run