[GitHub] [kafka] rohitrmd commented on a change in pull request #10276: KAFKA-12253: Add tests that cover all of the cases for ReplicatedLog's validateOffsetAndEpoch

rohitrmd commented on a change in pull request #10276: URL: https://github.com/apache/kafka/pull/10276#discussion_r597158051 ## File path: raft/src/main/java/org/apache/kafka/raft/ValidOffsetAndEpoch.java ## @@ -16,40 +16,56 @@ */ package org.apache.kafka.raft; +import jav

[GitHub] [kafka] vvcephei commented on pull request #10354: MINOR: Exclude KIP-500.md from rat check

vvcephei commented on pull request #10354: URL: https://github.com/apache/kafka/pull/10354#issuecomment-802212427 Thanks, @jolshan ! Will this need to be cherry-picked to 2.8? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

[GitHub] [kafka] cmccabe commented on pull request #10354: MINOR: Exclude KIP-500.md from rat check

cmccabe commented on pull request #10354: URL: https://github.com/apache/kafka/pull/10354#issuecomment-802214287 Thanks, @jolshan -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

[jira] [Created] (KAFKA-12500) Cache memory is leaked after removing/replacing a StreamThread

A. Sophie Blee-Goldman created KAFKA-12500: -- Summary: Cache memory is leaked after removing/replacing a StreamThread Key: KAFKA-12500 URL: https://issues.apache.org/jira/browse/KAFKA-12500 Pr

[GitHub] [kafka] mumrah commented on pull request #10354: MINOR: Exclude KIP-500.md from rat check

mumrah commented on pull request #10354: URL: https://github.com/apache/kafka/pull/10354#issuecomment-802229547 Maybe we should exclude all markdown files from the license check? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[jira] [Assigned] (KAFKA-12500) Cache memory is leaked after removing/replacing a StreamThread

[ https://issues.apache.org/jira/browse/KAFKA-12500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman reassigned KAFKA-12500: -- Assignee: A. Sophie Blee-Goldman > Cache memory is leaked after remov

[GitHub] [kafka] hachikuji commented on a change in pull request #10275: KAFKA-12434; Admin support for `DescribeProducers` API

hachikuji commented on a change in pull request #10275: URL: https://github.com/apache/kafka/pull/10275#discussion_r597193539 ## File path: clients/src/main/java/org/apache/kafka/clients/admin/internals/AdminApiLookupStrategy.java ## @@ -0,0 +1,94 @@ +/* + * Licensed to the Ap

[jira] [Created] (KAFKA-12501) KafkaClusterTestKit should support mixed-mode testing

David Arthur created KAFKA-12501: Summary: KafkaClusterTestKit should support mixed-mode testing Key: KAFKA-12501 URL: https://issues.apache.org/jira/browse/KAFKA-12501 Project: Kafka Issue T

[GitHub] [kafka] mumrah commented on a change in pull request #10220: KAFKA-12383: Get RaftClusterTest.java and other KIP-500 junit tests working

mumrah commented on a change in pull request #10220: URL: https://github.com/apache/kafka/pull/10220#discussion_r597202938 ## File path: core/src/test/java/kafka/testkit/KafkaClusterTestKit.java ## @@ -0,0 +1,499 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] [kafka] ableegoldman opened a new pull request #10355: KAFKA-12500: fix memory leak in thread cache

ableegoldman opened a new pull request #10355: URL: https://github.com/apache/kafka/pull/10355 Need to exclude threads in `PENDING_SHUTDOWN` from the num live threads computation used to compute the new cache size per thread. Also adds some logging to help follow what's happening whe

[GitHub] [kafka] ableegoldman commented on pull request #10355: KAFKA-12500: fix memory leak in thread cache

ableegoldman commented on pull request #10355: URL: https://github.com/apache/kafka/pull/10355#issuecomment-802252651 @wcarlson5 @cadonna @lct45 Should be cherrypicked to 2.8 cc @vvcephei -- This is an automated message from the Apache Git Service. To respond to the message, ple

[GitHub] [kafka] wcarlson5 commented on a change in pull request #10355: KAFKA-12500: fix memory leak in thread cache

wcarlson5 commented on a change in pull request #10355: URL: https://github.com/apache/kafka/pull/10355#discussion_r597208785 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -971,6 +972,7 @@ private int getNumStreamThreads(final boolean has

[jira] [Commented] (KAFKA-12385) Remove FetchResponse#responseData

[ https://issues.apache.org/jira/browse/KAFKA-12385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17304418#comment-17304418 ] Chun-Hao Tang commented on KAFKA-12385: --- [~chia7712] Can I work on this? > Remove

[GitHub] [kafka] ijuma commented on pull request #10354: MINOR: Exclude KIP-500.md from rat check

ijuma commented on pull request #10354: URL: https://github.com/apache/kafka/pull/10354#issuecomment-802259895 Merged to unbreak the build, we can improve further in future PRs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [kafka] ijuma merged pull request #10354: MINOR: Exclude KIP-500.md from rat check

ijuma merged pull request #10354: URL: https://github.com/apache/kafka/pull/10354 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please c

[GitHub] [kafka] mumrah commented on a change in pull request #10220: KAFKA-12383: Get RaftClusterTest.java and other KIP-500 junit tests working

mumrah commented on a change in pull request #10220: URL: https://github.com/apache/kafka/pull/10220#discussion_r597214017 ## File path: core/src/test/java/kafka/testkit/KafkaClusterTestKit.java ## @@ -0,0 +1,499 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] [kafka] ableegoldman commented on a change in pull request #10355: KAFKA-12500: fix memory leak in thread cache

ableegoldman commented on a change in pull request #10355: URL: https://github.com/apache/kafka/pull/10355#discussion_r597215093 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -1097,13 +1102,17 @@ private int getNumStreamThreads(final boole

[GitHub] [kafka] ableegoldman commented on a change in pull request #10355: KAFKA-12500: fix memory leak in thread cache

ableegoldman commented on a change in pull request #10355: URL: https://github.com/apache/kafka/pull/10355#discussion_r597217235 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -1097,13 +1102,17 @@ private int getNumStreamThreads(final boole

[jira] [Updated] (KAFKA-12501) KafkaClusterTestKit should support mixed-mode testing

[ https://issues.apache.org/jira/browse/KAFKA-12501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Arthur updated KAFKA-12501: - Description: Follow-on from https://github.com/apache/kafka/pull/10220 We should allow for mixe

[GitHub] [kafka] jolshan commented on pull request #10351: MINOR: use new method to get number of topics in DeleteTopicsRequest

jolshan commented on pull request #10351: URL: https://github.com/apache/kafka/pull/10351#issuecomment-802271057 Build was broken due to an unrelated change, but should be fixed now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

[GitHub] [kafka] ijuma commented on pull request #10354: MINOR: Exclude KIP-500.md from rat check

ijuma commented on pull request #10354: URL: https://github.com/apache/kafka/pull/10354#issuecomment-802273720 Also cherry-picked to 2.8. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

[jira] [Created] (KAFKA-12502) Quorum controller should return topic configs in CreateTopic response

Jason Gustafson created KAFKA-12502: --- Summary: Quorum controller should return topic configs in CreateTopic response Key: KAFKA-12502 URL: https://issues.apache.org/jira/browse/KAFKA-12502 Project:

[GitHub] [kafka] ableegoldman commented on pull request #10355: KAFKA-12500: fix memory leak in thread cache

ableegoldman commented on pull request #10355: URL: https://github.com/apache/kafka/pull/10355#issuecomment-802298961 @vvcephei yeah I'm on it, just wanted to see if there were any concerns about the proposed fix first. I'll ping you again when it's ready -- This is an automated message

[jira] [Commented] (KAFKA-12463) Update default consumer partition assignor for sink tasks

[ https://issues.apache.org/jira/browse/KAFKA-12463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17304452#comment-17304452 ] A. Sophie Blee-Goldman commented on KAFKA-12463: Out of curiosity, why u

[GitHub] [kafka] wcarlson5 opened a new pull request #10356: cache size set by the thread

wcarlson5 opened a new pull request #10356: URL: https://github.com/apache/kafka/pull/10356 Make is so threads do not directly resize other threads caches ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage an

[GitHub] [kafka] hachikuji commented on a change in pull request #10276: KAFKA-12253: Add tests that cover all of the cases for ReplicatedLog's validateOffsetAndEpoch

hachikuji commented on a change in pull request #10276: URL: https://github.com/apache/kafka/pull/10276#discussion_r597260954 ## File path: core/src/test/scala/kafka/raft/KafkaMetadataLogTest.scala ## @@ -485,6 +485,170 @@ final class KafkaMetadataLogTest { batchBuilder.bu

[GitHub] [kafka] hachikuji commented on a change in pull request #10289: KAFKA-12440: ClusterId validation for Vote, BeginQuorum, EndQuorum and FetchSnapshot

hachikuji commented on a change in pull request #10289: URL: https://github.com/apache/kafka/pull/10289#discussion_r597274570 ## File path: checkstyle/suppressions.xml ## @@ -68,7 +68,7 @@ files="(Utils|Topic|KafkaLZ4BlockOutputStream|AclData|JoinGroupRequest).j

[GitHub] [kafka] hachikuji commented on a change in pull request #10289: KAFKA-12440: ClusterId validation for Vote, BeginQuorum, EndQuorum and FetchSnapshot

hachikuji commented on a change in pull request #10289: URL: https://github.com/apache/kafka/pull/10289#discussion_r597278840 ## File path: raft/src/main/java/org/apache/kafka/raft/KafkaRaftClient.java ## @@ -814,6 +822,10 @@ private EndQuorumEpochResponseData handleEndQuorumE

[GitHub] [kafka] ableegoldman commented on a change in pull request #10342: KAFKA-12288: remove task-level filesystem locks

ableegoldman commented on a change in pull request #10342: URL: https://github.com/apache/kafka/pull/10342#discussion_r597278900 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StateDirectory.java ## @@ -288,50 +277,31 @@ private String logPref

[GitHub] [kafka] rondagostino opened a new pull request #10357: MINOR: KIP-500 Cluster ID is a String

rondagostino opened a new pull request #10357: URL: https://github.com/apache/kafka/pull/10357 It is possible that an existing Cluster ID in Zookeeper is not convertible to a UUID. KIP-500 cannot constrain Cluster IDs to be convertible to a UUID and still expect such a cluster to be able

[GitHub] [kafka] rondagostino commented on pull request #10357: MINOR: KIP-500 Cluster ID is a String

rondagostino commented on pull request #10357: URL: https://github.com/apache/kafka/pull/10357#issuecomment-802344461 This needs to be cherry-picked to 2.8. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[GitHub] [kafka] ableegoldman commented on a change in pull request #10342: KAFKA-12288: remove task-level filesystem locks

ableegoldman commented on a change in pull request #10342: URL: https://github.com/apache/kafka/pull/10342#discussion_r597281724 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StateDirectory.java ## @@ -477,15 +441,7 @@ private void cleanRemov

[GitHub] [kafka] ableegoldman commented on pull request #10342: KAFKA-12288: remove task-level filesystem locks

ableegoldman commented on pull request #10342: URL: https://github.com/apache/kafka/pull/10342#issuecomment-802350667 @wcarlson5 that's a good question, and I totally agree we should be more careful about that. In theory we should always clean up all state including unlock task directories

[GitHub] [kafka] hachikuji commented on a change in pull request #10357: MINOR: KIP-500 Cluster ID is a String

hachikuji commented on a change in pull request #10357: URL: https://github.com/apache/kafka/pull/10357#discussion_r597285632 ## File path: core/src/main/scala/kafka/server/BrokerMetadataCheckpoint.scala ## @@ -106,14 +105,8 @@ object MetaProperties { value.getOrElse(throw

[jira] [Created] (KAFKA-12503) Resizing the thread cache in a non thread safe way can cause records to be redirected throughout the topology

Walker Carlson created KAFKA-12503: -- Summary: Resizing the thread cache in a non thread safe way can cause records to be redirected throughout the topology Key: KAFKA-12503 URL: https://issues.apache.org/jira/bro

[jira] [Commented] (KAFKA-12503) Resizing the thread cache in a non thread safe way can cause records to be redirected throughout the topology

[ https://issues.apache.org/jira/browse/KAFKA-12503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17304518#comment-17304518 ] A. Sophie Blee-Goldman commented on KAFKA-12503: Thanks for the writeup!

[jira] [Commented] (KAFKA-12463) Update default consumer partition assignor for sink tasks

[ https://issues.apache.org/jira/browse/KAFKA-12463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17304519#comment-17304519 ] Chris Egerton commented on KAFKA-12463: --- Thanks Randall-- {quote}I agree with you

[jira] [Commented] (KAFKA-12503) Resizing the thread cache in a non thread safe way can cause records to be redirected throughout the topology

[ https://issues.apache.org/jira/browse/KAFKA-12503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17304523#comment-17304523 ] Guozhang Wang commented on KAFKA-12503: --- Thanks for the summary [~wcarlson5] [~abl

[jira] [Updated] (KAFKA-12463) Update default consumer partition assignor for sink tasks

[ https://issues.apache.org/jira/browse/KAFKA-12463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton updated KAFKA-12463: -- Description: Kafka consumers have a pluggable [partition assignment interface|https://kafka.a

[jira] [Created] (KAFKA-12504) KafkaMetadataLog should check clean shutdown file

Jason Gustafson created KAFKA-12504: --- Summary: KafkaMetadataLog should check clean shutdown file Key: KAFKA-12504 URL: https://issues.apache.org/jira/browse/KAFKA-12504 Project: Kafka Issue

[jira] [Updated] (KAFKA-12463) Update default consumer partition assignor for sink tasks

[ https://issues.apache.org/jira/browse/KAFKA-12463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton updated KAFKA-12463: -- Description: Kafka consumers have a pluggable [partition assignment interface|https://kafka.a

[jira] [Assigned] (KAFKA-12503) Resizing the thread cache in a non thread safe way can cause records to be redirected throughout the topology

[ https://issues.apache.org/jira/browse/KAFKA-12503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman reassigned KAFKA-12503: -- Assignee: Walker Carlson > Resizing the thread cache in a non thread

[jira] [Updated] (KAFKA-10563) Make sure task directories don't remain locked by dead threads

[ https://issues.apache.org/jira/browse/KAFKA-10563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman updated KAFKA-10563: --- Fix Version/s: (was: 2.8.0) > Make sure task directories don't remain lo

[GitHub] [kafka] rohitrmd commented on a change in pull request #10276: KAFKA-12253: Add tests that cover all of the cases for ReplicatedLog's validateOffsetAndEpoch

rohitrmd commented on a change in pull request #10276: URL: https://github.com/apache/kafka/pull/10276#discussion_r597309028 ## File path: core/src/test/scala/kafka/raft/KafkaMetadataLogTest.scala ## @@ -485,6 +485,170 @@ final class KafkaMetadataLogTest { batchBuilder.bui

[jira] [Updated] (KAFKA-12504) KafkaMetadataLog should check clean shutdown file

[ https://issues.apache.org/jira/browse/KAFKA-12504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson updated KAFKA-12504: Description: When a `Log` is opened, we will skip recovery if the clean shutdown file is p

[GitHub] [kafka] wcarlson5 commented on pull request #10356: KAFKA-12503: cache size set by the thread

wcarlson5 commented on pull request #10356: URL: https://github.com/apache/kafka/pull/10356#issuecomment-802393840 @guozhangwang @ableegoldman Can you take a look? @vvcephei I think this needs to be picked to 2.8 as well -- This is an automated message from the Apache Git Service.

[GitHub] [kafka] rondagostino commented on a change in pull request #10357: MINOR: KIP-500 Cluster ID is a String

rondagostino commented on a change in pull request #10357: URL: https://github.com/apache/kafka/pull/10357#discussion_r597324486 ## File path: core/src/main/scala/kafka/tools/StorageTool.scala ## @@ -198,7 +198,7 @@ object StorageTool extends Logging { s"does not appea

[GitHub] [kafka] hachikuji commented on a change in pull request #10357: MINOR: KIP-500 Cluster ID is a String

hachikuji commented on a change in pull request #10357: URL: https://github.com/apache/kafka/pull/10357#discussion_r597325275 ## File path: core/src/main/scala/kafka/tools/StorageTool.scala ## @@ -198,7 +198,7 @@ object StorageTool extends Logging { s"does not appear t

[jira] [Commented] (KAFKA-12499) Adjust transaction timeout according to commit interval on Streams EOS

[ https://issues.apache.org/jira/browse/KAFKA-12499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17304537#comment-17304537 ] A. Sophie Blee-Goldman commented on KAFKA-12499: Nice catch, but just wo

[jira] [Created] (KAFKA-12505) Should kafka-storage.sh accept a non-UUID for its --cluster-id parameter?

Ron Dagostino created KAFKA-12505: - Summary: Should kafka-storage.sh accept a non-UUID for its --cluster-id parameter? Key: KAFKA-12505 URL: https://issues.apache.org/jira/browse/KAFKA-12505 Project:

[GitHub] [kafka] rondagostino commented on a change in pull request #10357: MINOR: KIP-500 Cluster ID is a String

rondagostino commented on a change in pull request #10357: URL: https://github.com/apache/kafka/pull/10357#discussion_r597326794 ## File path: core/src/main/scala/kafka/tools/StorageTool.scala ## @@ -198,7 +198,7 @@ object StorageTool extends Logging { s"does not appea

[GitHub] [kafka] ableegoldman commented on pull request #10356: KAFKA-12503: cache size set by the thread

ableegoldman commented on pull request #10356: URL: https://github.com/apache/kafka/pull/10356#issuecomment-802411941 One unrelated test failure: `kafka.api.SaslGssapiSslEndToEndAuthorizationTest.testNoConsumeWithDescribeAclViaAssign` -- This is an automated message from the Apache Git S

[jira] [Updated] (KAFKA-12505) Should kafka-storage.sh accept a non-UUID for its --cluster-id parameter?

[ https://issues.apache.org/jira/browse/KAFKA-12505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson updated KAFKA-12505: Labels: kip-500 (was: ) > Should kafka-storage.sh accept a non-UUID for its --cluster-id

[GitHub] [kafka] hachikuji commented on pull request #10357: MINOR: KIP-500 Cluster ID is a String

hachikuji commented on pull request #10357: URL: https://github.com/apache/kafka/pull/10357#issuecomment-802417977 @vvcephei Where are we with the RC? Do we still have time to merge this to 2.8? We can do this after the release if necessary, but we'd prefer not to break the protocol if we

[jira] [Updated] (KAFKA-12506) Expand AdjustStreamThreadCountTest

[ https://issues.apache.org/jira/browse/KAFKA-12506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman updated KAFKA-12506: --- Labels: newbie newbie++ (was: ) > Expand AdjustStreamThreadCountTest >

[jira] [Created] (KAFKA-12506) Expand AdjustStreamThreadCountTest

A. Sophie Blee-Goldman created KAFKA-12506: -- Summary: Expand AdjustStreamThreadCountTest Key: KAFKA-12506 URL: https://issues.apache.org/jira/browse/KAFKA-12506 Project: Kafka Issue

[jira] [Updated] (KAFKA-12506) Expand AdjustStreamThreadCountTest

[ https://issues.apache.org/jira/browse/KAFKA-12506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman updated KAFKA-12506: --- Description: Right now the AdjustStreamThreadCountTest runs a minimal topolo

[jira] [Commented] (KAFKA-12506) Expand AdjustStreamThreadCountTest

[ https://issues.apache.org/jira/browse/KAFKA-12506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17304551#comment-17304551 ] A. Sophie Blee-Goldman commented on KAFKA-12506: cc [~wcarlson5] -- the

[GitHub] [kafka] wcarlson5 commented on pull request #10356: KAFKA-12503: cache size set by the thread

wcarlson5 commented on pull request #10356: URL: https://github.com/apache/kafka/pull/10356#issuecomment-802427967 I will don't know about testing this via an integration test. I worry about introducing just another flaky test, these types of issue get caught mostly in soak. Even with the

[jira] [Commented] (KAFKA-12385) Remove FetchResponse#responseData

[ https://issues.apache.org/jira/browse/KAFKA-12385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17304553#comment-17304553 ] Chia-Ping Tsai commented on KAFKA-12385: This issue is blocked as we are introdu

[jira] [Created] (KAFKA-12507) java.lang.OutOfMemoryError: Direct buffer memory

diehu created KAFKA-12507: - Summary: java.lang.OutOfMemoryError: Direct buffer memory Key: KAFKA-12507 URL: https://issues.apache.org/jira/browse/KAFKA-12507 Project: Kafka Issue Type: Bug

[GitHub] [kafka] chia7712 merged pull request #10332: KAFKA-10697: Remove ProduceResponse.responses

chia7712 merged pull request #10332: URL: https://github.com/apache/kafka/pull/10332 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, pleas

[jira] [Resolved] (KAFKA-10697) Remove ProduceResponse.responses

[ https://issues.apache.org/jira/browse/KAFKA-10697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-10697. Fix Version/s: 3.0.0 Resolution: Fixed > Remove ProduceResponse.responses > ---

[jira] [Comment Edited] (KAFKA-12478) Consumer group may lose data for newly expanded partitions when add partitions for topic if the group is set to consume from the latest

[ https://issues.apache.org/jira/browse/KAFKA-12478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17304285#comment-17304285 ] hudeqi edited comment on KAFKA-12478 at 3/19/21, 2:16 AM: --   Th

[jira] [Issue Comment Deleted] (KAFKA-12478) Consumer group may lose data for newly expanded partitions when add partitions for topic if the group is set to consume from the latest

[ https://issues.apache.org/jira/browse/KAFKA-12478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hudeqi updated KAFKA-12478: --- Comment: was deleted (was:   Thank you very much for your reply! I belong to Kuaishou Message-oriented midd

[jira] [Commented] (KAFKA-12478) Consumer group may lose data for newly expanded partitions when add partitions for topic if the group is set to consume from the latest

[ https://issues.apache.org/jira/browse/KAFKA-12478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17304592#comment-17304592 ] hudeqi commented on KAFKA-12478: Thank you very much for your reply! I belong to Kuaisho

[jira] [Commented] (KAFKA-12507) java.lang.OutOfMemoryError: Direct buffer memory

[ https://issues.apache.org/jira/browse/KAFKA-12507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17304593#comment-17304593 ] Wenbing Shen commented on KAFKA-12507: -- How much do you set about these parameters:

[jira] [Commented] (KAFKA-12507) java.lang.OutOfMemoryError: Direct buffer memory

[ https://issues.apache.org/jira/browse/KAFKA-12507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17304600#comment-17304600 ] diehu commented on KAFKA-12507: --- We did not set the -XX:MaxDirectMemorySize, and its defau

[GitHub] [kafka] ableegoldman commented on pull request #10356: KAFKA-12503: cache size set by the thread

ableegoldman commented on pull request #10356: URL: https://github.com/apache/kafka/pull/10356#issuecomment-802491798 I don't think we should aim to catch things in soak, although I'm not on the side of "we should never catch anything in soak/everything must have an integration test" eithe

[GitHub] [kafka] ableegoldman merged pull request #10356: KAFKA-12503: cache size set by the thread

ableegoldman merged pull request #10356: URL: https://github.com/apache/kafka/pull/10356 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, p

[GitHub] [kafka] ableegoldman commented on pull request #10356: KAFKA-12503: cache size set by the thread

ableegoldman commented on pull request #10356: URL: https://github.com/apache/kafka/pull/10356#issuecomment-802493975 Merged and cherrypicked to 2.8 cc @vvcephei -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

[jira] [Updated] (KAFKA-8608) Broker shows WARN on reassignment partitions on new brokers: Replica LEO, follower position & Cache truncation

[ https://issues.apache.org/jira/browse/KAFKA-8608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhang Jianguo updated KAFKA-8608: - Attachment: image-2021-03-19-10-36-04-328.png > Broker shows WARN on reassignment partitions on n

[jira] [Resolved] (KAFKA-12503) Resizing the thread cache in a non thread safe way can cause records to be redirected throughout the topology

[ https://issues.apache.org/jira/browse/KAFKA-12503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman resolved KAFKA-12503. Resolution: Fixed > Resizing the thread cache in a non thread safe way can

[jira] [Updated] (KAFKA-8608) Broker shows WARN on reassignment partitions on new brokers: Replica LEO, follower position & Cache truncation

[ https://issues.apache.org/jira/browse/KAFKA-8608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhang Jianguo updated KAFKA-8608: - Attachment: image-2021-03-19-10-39-24-728.png > Broker shows WARN on reassignment partitions on n

[jira] [Commented] (KAFKA-8608) Broker shows WARN on reassignment partitions on new brokers: Replica LEO, follower position & Cache truncation

[ https://issues.apache.org/jira/browse/KAFKA-8608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17304601#comment-17304601 ] Zhang Jianguo commented on KAFKA-8608: -- [~LillianY] I meet the same issue. [2021-0

[jira] [Comment Edited] (KAFKA-8608) Broker shows WARN on reassignment partitions on new brokers: Replica LEO, follower position & Cache truncation

[ https://issues.apache.org/jira/browse/KAFKA-8608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17304601#comment-17304601 ] Zhang Jianguo edited comment on KAFKA-8608 at 3/19/21, 2:42 AM: ---

[jira] [Comment Edited] (KAFKA-8608) Broker shows WARN on reassignment partitions on new brokers: Replica LEO, follower position & Cache truncation

[ https://issues.apache.org/jira/browse/KAFKA-8608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17304601#comment-17304601 ] Zhang Jianguo edited comment on KAFKA-8608 at 3/19/21, 2:54 AM: ---

[jira] [Updated] (KAFKA-12495) Unbalanced connectors/tasks distribution will happen in Connect's incremental cooperative assignor

[ https://issues.apache.org/jira/browse/KAFKA-12495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen updated KAFKA-12495: -- Description: In Kafka Connect, we implement incremental cooperative rebalance algorithm based on KIP-

[GitHub] [kafka] ableegoldman edited a comment on pull request #10356: KAFKA-12503: cache size set by the thread

ableegoldman edited a comment on pull request #10356: URL: https://github.com/apache/kafka/pull/10356#issuecomment-802491798 I don't think we should aim to catch things in soak, although I'm not on the side of "we should never catch anything in soak/everything must have an integration test

[jira] [Commented] (KAFKA-12495) Unbalanced connectors/tasks distribution will happen in Connect's incremental cooperative assignor

[ https://issues.apache.org/jira/browse/KAFKA-12495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17304611#comment-17304611 ] Luke Chen commented on KAFKA-12495: --- [~ramkrish1489], I checked your PR in KAFKA-10413

[GitHub] [kafka] chia7712 commented on a change in pull request #10275: KAFKA-12434; Admin support for `DescribeProducers` API

chia7712 commented on a change in pull request #10275: URL: https://github.com/apache/kafka/pull/10275#discussion_r597383123 ## File path: clients/src/main/java/org/apache/kafka/clients/admin/internals/AdminApiHandler.java ## @@ -0,0 +1,127 @@ +/* + * Licensed to the Apache So

[GitHub] [kafka] ableegoldman commented on pull request #10355: KAFKA-12500: fix memory leak in thread cache

ableegoldman commented on pull request #10355: URL: https://github.com/apache/kafka/pull/10355#issuecomment-802535398 Alright I added two integration tests, lmk if this looks good @vvcephei -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [kafka] ableegoldman commented on pull request #10342: KAFKA-12288: remove task-level filesystem locks

ableegoldman commented on pull request #10342: URL: https://github.com/apache/kafka/pull/10342#issuecomment-802542776 @wcarlson5 I ended up putting a minor guard against orphaned task directories into this PR. It's not perfect but basically we just let the cleaner thread verify that any lo

[jira] [Commented] (KAFKA-10563) Make sure task directories don't remain locked by dead threads

[ https://issues.apache.org/jira/browse/KAFKA-10563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17304637#comment-17304637 ] A. Sophie Blee-Goldman commented on KAFKA-10563: I partially addressed t

[GitHub] [kafka] ableegoldman commented on a change in pull request #10342: KAFKA-12288: remove task-level filesystem locks

ableegoldman commented on a change in pull request #10342: URL: https://github.com/apache/kafka/pull/10342#discussion_r596443720 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/ProcessorStateManager.java ## @@ -186,7 +186,7 @@ public ProcessorS

[GitHub] [kafka] cmccabe commented on pull request #10357: MINOR: KIP-500 Cluster ID is a String

cmccabe commented on pull request #10357: URL: https://github.com/apache/kafka/pull/10357#issuecomment-802556947 @vvcephei : I like @hachikuji 's idea of merging this to 2.8. It's worth noting that the protocol that we're changing here is a new one which was introduced by kip-500, and is

[jira] [Created] (KAFKA-12508) Emit-on-change tables may lose updates on error or restart in at_least_once

Nico Habermann created KAFKA-12508: -- Summary: Emit-on-change tables may lose updates on error or restart in at_least_once Key: KAFKA-12508 URL: https://issues.apache.org/jira/browse/KAFKA-12508 Proje

[jira] [Updated] (KAFKA-12508) Emit-on-change tables may lose updates on error or restart in at_least_once

[ https://issues.apache.org/jira/browse/KAFKA-12508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nico Habermann updated KAFKA-12508: --- Description: [KIP-557|https://cwiki.apache.org/confluence/display/KAFKA/KIP-557%3A+Add+emit+

[GitHub] [kafka] rodesai commented on a change in pull request #10342: KAFKA-12288: remove task-level filesystem locks

rodesai commented on a change in pull request #10342: URL: https://github.com/apache/kafka/pull/10342#discussion_r597413180 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StateDirectory.java ## @@ -288,50 +277,31 @@ private String logPrefix()

[GitHub] [kafka] rodesai commented on a change in pull request #10342: KAFKA-12288: remove task-level filesystem locks

rodesai commented on a change in pull request #10342: URL: https://github.com/apache/kafka/pull/10342#discussion_r597413503 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StateDirectory.java ## @@ -288,50 +277,31 @@ private String logPrefix()

[GitHub] [kafka] ableegoldman commented on a change in pull request #10342: KAFKA-12288: remove task-level filesystem locks

ableegoldman commented on a change in pull request #10342: URL: https://github.com/apache/kafka/pull/10342#discussion_r597424785 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StateDirectory.java ## @@ -288,50 +277,31 @@ private String logPref

[jira] [Created] (KAFKA-12509) Tighten up StateDirectory thread locking

A. Sophie Blee-Goldman created KAFKA-12509: -- Summary: Tighten up StateDirectory thread locking Key: KAFKA-12509 URL: https://issues.apache.org/jira/browse/KAFKA-12509 Project: Kafka

[jira] [Updated] (KAFKA-12509) Tighten up StateDirectory thread locking

[ https://issues.apache.org/jira/browse/KAFKA-12509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman updated KAFKA-12509: --- Labels: newbie (was: ) > Tighten up StateDirectory thread locking > ---

[GitHub] [kafka] ableegoldman commented on a change in pull request #10342: KAFKA-12288: remove task-level filesystem locks

ableegoldman commented on a change in pull request #10342: URL: https://github.com/apache/kafka/pull/10342#discussion_r597427365 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StateDirectory.java ## @@ -288,50 +277,31 @@ private String logPref

[jira] [Updated] (KAFKA-12495) Unbalanced connectors/tasks distribution will happen in Connect's incremental cooperative assignor

[ https://issues.apache.org/jira/browse/KAFKA-12495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen updated KAFKA-12495: -- Description: In Kafka Connect, we implement incremental cooperative rebalance algorithm based on KIP-

[jira] [Updated] (KAFKA-12495) Unbalanced connectors/tasks distribution will happen in Connect's incremental cooperative assignor

[ https://issues.apache.org/jira/browse/KAFKA-12495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen updated KAFKA-12495: -- Description: In Kafka Connect, we implement incremental cooperative rebalance algorithm based on KIP-

[jira] [Comment Edited] (KAFKA-12495) Unbalanced connectors/tasks distribution will happen in Connect's incremental cooperative assignor

[ https://issues.apache.org/jira/browse/KAFKA-12495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17304611#comment-17304611 ] Luke Chen edited comment on KAFKA-12495 at 3/19/21, 6:21 AM: -

<    1   2