[GitHub] [kafka] michael-carter-instaclustr opened a new pull request #8844: Kafka 9887 fix failed task or connector count on startup failure

2020-06-09 Thread GitBox
michael-carter-instaclustr opened a new pull request #8844: URL: https://github.com/apache/kafka/pull/8844 Moved the responsibility for recording task and connector startup and failure metrics from the Worker class into the status listener that gets passed into the WorkerTask. The status

[GitHub] [kafka] abbccdda commented on a change in pull request #8843: KAFKA-9991: Fix flaky unit tests

2020-06-09 Thread GitBox
abbccdda commented on a change in pull request #8843: URL: https://github.com/apache/kafka/pull/8843#discussion_r437858956 ## File path: streams/src/test/java/org/apache/kafka/streams/integration/KTableSourceTopicRestartIntegrationTest.java ## @@ -107,7 +109,6 @@ public void

[GitHub] [kafka] vvcephei commented on pull request #8787: KAFKA-10085: correctly compute lag for optimized source changelogs

2020-06-09 Thread GitBox
vvcephei commented on pull request #8787: URL: https://github.com/apache/kafka/pull/8787#issuecomment-641712436 Test this please This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [kafka] vvcephei commented on pull request #8775: KAFKA-10079: improve thread-level stickiness

2020-06-09 Thread GitBox
vvcephei commented on pull request #8775: URL: https://github.com/apache/kafka/pull/8775#issuecomment-641711592 Test this please This is an automated message from the Apache Git Service. To respond to the message, please log

[jira] [Commented] (KAFKA-6666) OffsetOutOfRangeException: Replica Thread Stopped Resulting in Underreplicated Partitions

2020-06-09 Thread zhangzhisheng (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130030#comment-17130030 ] zhangzhisheng commented on KAFKA-: -- on our prd cluster, one topic partition ReplicaFetcherThread

[jira] [Comment Edited] (KAFKA-6666) OffsetOutOfRangeException: Replica Thread Stopped Resulting in Underreplicated Partitions

2020-06-09 Thread zhangzhisheng (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17130030#comment-17130030 ] zhangzhisheng edited comment on KAFKA- at 6/10/20, 3:51 AM: on our

[jira] [Updated] (KAFKA-10135) Extract Task#executeAndMaybeSwallow to be a general utility function into TaskManager

2020-06-09 Thread Boyang Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Boyang Chen updated KAFKA-10135: Component/s: streams > Extract Task#executeAndMaybeSwallow to be a general utility function into

[jira] [Assigned] (KAFKA-10135) Extract Task#executeAndMaybeSwallow to be a general utility function into TaskManager

2020-06-09 Thread feyman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feyman reassigned KAFKA-10135: -- Assignee: feyman > Extract Task#executeAndMaybeSwallow to be a general utility function into >

[jira] [Created] (KAFKA-10135) Extract Task#executeAndMaybeSwallow to be a general utility function into TaskManager

2020-06-09 Thread Boyang Chen (Jira)
Boyang Chen created KAFKA-10135: --- Summary: Extract Task#executeAndMaybeSwallow to be a general utility function into TaskManager Key: KAFKA-10135 URL: https://issues.apache.org/jira/browse/KAFKA-10135

[GitHub] [kafka] feyman2016 commented on a change in pull request #8832: KAFKA-9377: Refactor StreamsPartitionAssignor Repartition Count Logic

2020-06-09 Thread GitBox
feyman2016 commented on a change in pull request #8832: URL: https://github.com/apache/kafka/pull/8832#discussion_r437532739 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamsPartitionAssignor.java ## @@ -513,66 +553,94 @@ private boolean

[GitHub] [kafka] guozhangwang opened a new pull request #8843: KAFKA-9991: Fix flaky unit tests

2020-06-09 Thread GitBox
guozhangwang opened a new pull request #8843: URL: https://github.com/apache/kafka/pull/8843 The latest commit #8254 on this test deleted all topics after each test, but the topic was actually shared among tests before. And after that we are relying on the less-reliable auto-topic

[GitHub] [kafka] guozhangwang commented on pull request #8843: KAFKA-9991: Fix flaky unit tests

2020-06-09 Thread GitBox
guozhangwang commented on pull request #8843: URL: https://github.com/apache/kafka/pull/8843#issuecomment-641688008 @mjsax @ableegoldman This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [kafka] rhauch commented on pull request #8502: KAFKA-9066: Retain metrics for failed tasks

2020-06-09 Thread GitBox
rhauch commented on pull request #8502: URL: https://github.com/apache/kafka/pull/8502#issuecomment-641686831 Retest this please This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [kafka] rhauch commented on pull request #8663: KAFKA-9985: Sink connector may exhaust broker when writing in DLQ

2020-06-09 Thread GitBox
rhauch commented on pull request #8663: URL: https://github.com/apache/kafka/pull/8663#issuecomment-641686021 Retest this please This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [kafka] dielhennr opened a new pull request #8842: Minor: Moving helper methods in one test file to companion object

2020-06-09 Thread GitBox
dielhennr opened a new pull request #8842: URL: https://github.com/apache/kafka/pull/8842 Moving helper methods in test file ReassignPartitionsIntegrationTest into a companion object and removing private keyword from the methods so that they may be used elsewhere.

[GitHub] [kafka] mjsax commented on pull request #8833: KAFKA-9441: remove prepareClose() to simplify task management

2020-06-09 Thread GitBox
mjsax commented on pull request #8833: URL: https://github.com/apache/kafka/pull/8833#issuecomment-641665794 Updated. This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [kafka] mjsax commented on a change in pull request #8833: KAFKA-9441: remove prepareClose() to simplify task management

2020-06-09 Thread GitBox
mjsax commented on a change in pull request #8833: URL: https://github.com/apache/kafka/pull/8833#discussion_r437802329 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java ## @@ -210,9 +214,8 @@ public void handleAssignment(final

[GitHub] [kafka] mjsax commented on a change in pull request #8833: KAFKA-9441: remove prepareClose() to simplify task management

2020-06-09 Thread GitBox
mjsax commented on a change in pull request #8833: URL: https://github.com/apache/kafka/pull/8833#discussion_r437800551 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamTask.java ## @@ -504,88 +438,66 @@ public void update(final Set

[GitHub] [kafka] kkonstantine commented on a change in pull request #8069: KAFKA-9374: Make connector interactions asynchronous

2020-06-09 Thread GitBox
kkonstantine commented on a change in pull request #8069: URL: https://github.com/apache/kafka/pull/8069#discussion_r437229477 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/Worker.java ## @@ -356,40 +390,96 @@ public boolean

[GitHub] [kafka] mjsax commented on a change in pull request #8833: KAFKA-9441: remove prepareClose() to simplify task management

2020-06-09 Thread GitBox
mjsax commented on a change in pull request #8833: URL: https://github.com/apache/kafka/pull/8833#discussion_r437800082 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamTask.java ## @@ -378,42 +320,88 @@ public void prepareCommit() {

[GitHub] [kafka] mjsax commented on a change in pull request #8833: KAFKA-9441: remove prepareClose() to simplify task management

2020-06-09 Thread GitBox
mjsax commented on a change in pull request #8833: URL: https://github.com/apache/kafka/pull/8833#discussion_r437797456 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamTask.java ## @@ -504,88 +438,66 @@ public void update(final Set

[GitHub] [kafka] mjsax commented on a change in pull request #8833: KAFKA-9441: remove prepareClose() to simplify task management

2020-06-09 Thread GitBox
mjsax commented on a change in pull request #8833: URL: https://github.com/apache/kafka/pull/8833#discussion_r437796880 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamTask.java ## @@ -378,42 +320,88 @@ public void prepareCommit() {

[GitHub] [kafka] guozhangwang merged pull request #8816: MINOR: Print all removed dynamic members during join complete

2020-06-09 Thread GitBox
guozhangwang merged pull request #8816: URL: https://github.com/apache/kafka/pull/8816 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [kafka] guozhangwang commented on a change in pull request #8816: MINOR: Print all removed dynamic members during join complete

2020-06-09 Thread GitBox
guozhangwang commented on a change in pull request #8816: URL: https://github.com/apache/kafka/pull/8816#discussion_r437795309 ## File path: core/src/main/scala/kafka/coordinator/group/GroupCoordinator.scala ## @@ -1146,12 +1146,16 @@ class GroupCoordinator(val brokerId: Int,

[jira] [Updated] (KAFKA-9800) [KIP-580] Client Exponential Backoff Implementation

2020-06-09 Thread Cheng Tan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Tan updated KAFKA-9800: - Description: Design: The main idea is to bookkeep the failed attempt. Currently, the retry backoff has

[jira] [Updated] (KAFKA-10134) High CPU issue during rebalance in Kafka consumer after upgrading to 2.5

2020-06-09 Thread Sean Guo (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Guo updated KAFKA-10134: - Description: We want to utilize the new rebalance protocol to mitigate the stop-the-world effect

[jira] [Created] (KAFKA-10134) High CPU issue during rebalance in Kafka consumer after upgrading to 2.5

2020-06-09 Thread Sean Guo (Jira)
Sean Guo created KAFKA-10134: Summary: High CPU issue during rebalance in Kafka consumer after upgrading to 2.5 Key: KAFKA-10134 URL: https://issues.apache.org/jira/browse/KAFKA-10134 Project: Kafka

[GitHub] [kafka] ableegoldman commented on a change in pull request #8787: KAFKA-10085: correctly compute lag for optimized source changelogs

2020-06-09 Thread GitBox
ableegoldman commented on a change in pull request #8787: URL: https://github.com/apache/kafka/pull/8787#discussion_r437788584 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/ClientUtils.java ## @@ -95,19 +99,65 @@ public static String

[jira] [Created] (KAFKA-10133) Cannot compress messages in destination cluster with MM2

2020-06-09 Thread Steve Jacobs (Jira)
Steve Jacobs created KAFKA-10133: Summary: Cannot compress messages in destination cluster with MM2 Key: KAFKA-10133 URL: https://issues.apache.org/jira/browse/KAFKA-10133 Project: Kafka

[GitHub] [kafka] vvcephei commented on a change in pull request #8787: KAFKA-10085: correctly compute lag for optimized source changelogs

2020-06-09 Thread GitBox
vvcephei commented on a change in pull request #8787: URL: https://github.com/apache/kafka/pull/8787#discussion_r436208312 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StoreChangelogReader.java ## @@ -562,23 +564,18 @@ private void

[GitHub] [kafka] vvcephei commented on pull request #8787: KAFKA-10085: correctly compute lag for optimized source changelogs

2020-06-09 Thread GitBox
vvcephei commented on pull request #8787: URL: https://github.com/apache/kafka/pull/8787#issuecomment-641639379 Test this please This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [kafka] vvcephei commented on pull request #8775: KAFKA-10079: improve thread-level stickiness

2020-06-09 Thread GitBox
vvcephei commented on pull request #8775: URL: https://github.com/apache/kafka/pull/8775#issuecomment-641639241 Test this please This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [kafka] vvcephei commented on pull request #8787: KAFKA-10085: correctly compute lag for optimized source changelogs

2020-06-09 Thread GitBox
vvcephei commented on pull request #8787: URL: https://github.com/apache/kafka/pull/8787#issuecomment-641639091 Test this please This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [kafka] vvcephei commented on pull request #8775: KAFKA-10079: improve thread-level stickiness

2020-06-09 Thread GitBox
vvcephei commented on pull request #8775: URL: https://github.com/apache/kafka/pull/8775#issuecomment-641638658 Test this please This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [kafka] vvcephei commented on pull request #8775: KAFKA-10079: improve thread-level stickiness

2020-06-09 Thread GitBox
vvcephei commented on pull request #8775: URL: https://github.com/apache/kafka/pull/8775#issuecomment-641638879 Test this please This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [kafka] vvcephei commented on pull request #8775: KAFKA-10079: improve thread-level stickiness

2020-06-09 Thread GitBox
vvcephei commented on pull request #8775: URL: https://github.com/apache/kafka/pull/8775#issuecomment-641638765 Test this please This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [kafka] vvcephei commented on pull request #8775: KAFKA-10079: improve thread-level stickiness

2020-06-09 Thread GitBox
vvcephei commented on pull request #8775: URL: https://github.com/apache/kafka/pull/8775#issuecomment-641638502 Test this please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [kafka] guozhangwang commented on a change in pull request #8833: KAFKA-9441: remove prepareClose() to simplify task management

2020-06-09 Thread GitBox
guozhangwang commented on a change in pull request #8833: URL: https://github.com/apache/kafka/pull/8833#discussion_r437767910 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamTask.java ## @@ -378,42 +320,88 @@ public void prepareCommit()

[GitHub] [kafka] mjsax commented on a change in pull request #8833: KAFKA-9441: remove prepareClose() to simplify task management

2020-06-09 Thread GitBox
mjsax commented on a change in pull request #8833: URL: https://github.com/apache/kafka/pull/8833#discussion_r437775422 ## File path: streams/src/test/java/org/apache/kafka/streams/processor/internals/StandbyTaskTest.java ## @@ -476,6 +445,26 @@ public void

[GitHub] [kafka] mjsax commented on a change in pull request #8833: KAFKA-9441: remove prepareClose() to simplify task management

2020-06-09 Thread GitBox
mjsax commented on a change in pull request #8833: URL: https://github.com/apache/kafka/pull/8833#discussion_r437775334 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java ## @@ -623,7 +618,11 @@ private long

[GitHub] [kafka] kkonstantine commented on pull request #8630: KAFKA-9969: Exclude ConnectorClientConfigRequest from class loading isolation

2020-06-09 Thread GitBox
kkonstantine commented on pull request #8630: URL: https://github.com/apache/kafka/pull/8630#issuecomment-641612088 retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [kafka] kkonstantine commented on a change in pull request #8630: KAFKA-9969: Exclude ConnectorClientConfigRequest from class loading isolation

2020-06-09 Thread GitBox
kkonstantine commented on a change in pull request #8630: URL: https://github.com/apache/kafka/pull/8630#discussion_r437754411 ## File path: connect/runtime/src/test/java/org/apache/kafka/connect/runtime/isolation/PluginUtilsTest.java ## @@ -114,68 +90,264 @@ public void

[jira] [Updated] (KAFKA-9800) [KIP-580] Client Exponential Backoff Implementation

2020-06-09 Thread Cheng Tan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Tan updated KAFKA-9800: - Description: Design: The main idea is to bookkeep the failed attempt. Currently, the retry backoff has

[jira] [Created] (KAFKA-10132) Kafka Connect JMX MBeans with String values have type double

2020-06-09 Thread Tom Malaher (Jira)
Tom Malaher created KAFKA-10132: --- Summary: Kafka Connect JMX MBeans with String values have type double Key: KAFKA-10132 URL: https://issues.apache.org/jira/browse/KAFKA-10132 Project: Kafka

[GitHub] [kafka] kkonstantine merged pull request #7384: KAFKA-8938: Improve allocations during Struct validation in ConnectSchema

2020-06-09 Thread GitBox
kkonstantine merged pull request #7384: URL: https://github.com/apache/kafka/pull/7384 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[jira] [Created] (KAFKA-10131) Minimize use of --zookeeper flag in ducktape tests

2020-06-09 Thread Vinoth Chandar (Jira)
Vinoth Chandar created KAFKA-10131: -- Summary: Minimize use of --zookeeper flag in ducktape tests Key: KAFKA-10131 URL: https://issues.apache.org/jira/browse/KAFKA-10131 Project: Kafka Issue

[jira] [Updated] (KAFKA-10121) Streams Task Assignment optimization design

2020-06-09 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler updated KAFKA-10121: - Description: Beginning in Kafka 2.6.0, Streams has a new task assignment algorithm that reacts

[GitHub] [kafka] mumrah opened a new pull request #8841: KAFKA-10123 Fix incorrect value for AWAIT_RESET#hasPosition

2020-06-09 Thread GitBox
mumrah opened a new pull request #8841: URL: https://github.com/apache/kafka/pull/8841 Also ensure we are updating the position with the current leader info even when fetching from older brokers. This is an

[jira] [Commented] (KAFKA-10123) Regression resetting offsets in consumer when fetching from old broker

2020-06-09 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17129791#comment-17129791 ] John Roesler commented on KAFKA-10123: -- I've just also added this as a blocker for 2.5.1, because

[GitHub] [kafka] junrao commented on pull request #8680: KAFKA-10027: Implement read path for feature versioning system (KIP-584)

2020-06-09 Thread GitBox
junrao commented on pull request #8680: URL: https://github.com/apache/kafka/pull/8680#issuecomment-641561083 It seems there are some checkstyle failures. ``` > Task :clients:checkstyleTest FAILED 20:31:38 [ant:checkstyle] [ERROR]

[jira] [Updated] (KAFKA-10123) Regression resetting offsets in consumer when fetching from old broker

2020-06-09 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler updated KAFKA-10123: - Fix Version/s: 2.5.1 > Regression resetting offsets in consumer when fetching from old broker >

[GitHub] [kafka] abbccdda commented on a change in pull request #8833: KAFKA-9441: remove prepareClose() to simplify task management

2020-06-09 Thread GitBox
abbccdda commented on a change in pull request #8833: URL: https://github.com/apache/kafka/pull/8833#discussion_r437688409 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamTask.java ## @@ -504,88 +438,66 @@ public void update(final Set

[GitHub] [kafka] junrao commented on pull request #8680: KAFKA-10027: Implement read path for feature versioning system (KIP-584)

2020-06-09 Thread GitBox
junrao commented on pull request #8680: URL: https://github.com/apache/kafka/pull/8680#issuecomment-641549841 ok to test This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [kafka] vvcephei commented on pull request #8816: MINOR: Print all removed dynamic members during join complete

2020-06-09 Thread GitBox
vvcephei commented on pull request #8816: URL: https://github.com/apache/kafka/pull/8816#issuecomment-641543721 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [kafka] kkonstantine merged pull request #8827: KAFKA-9849: Fix issue with worker.unsync.backoff.ms creating zombie workers when incremental cooperative rebalancing is used

2020-06-09 Thread GitBox
kkonstantine merged pull request #8827: URL: https://github.com/apache/kafka/pull/8827 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [kafka] kkonstantine commented on pull request #8827: KAFKA-9849: Fix issue with worker.unsync.backoff.ms creating zombie workers when incremental cooperative rebalancing is used

2020-06-09 Thread GitBox
kkonstantine commented on pull request #8827: URL: https://github.com/apache/kafka/pull/8827#issuecomment-641538952 2/3 builds green. A single test spoiled the fun in JDK11, but it's unrelated to the changes (`kafka.api.PlaintextConsumerTest.testCoordinatorFailover`)

[GitHub] [kafka] hachikuji merged pull request #8664: KAFKA-9716: Clarify meaning of compression rate metrics

2020-06-09 Thread GitBox
hachikuji merged pull request #8664: URL: https://github.com/apache/kafka/pull/8664 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] mjsax commented on pull request #8833: KAFKA-9441: remove prepareClose() to simplify task management

2020-06-09 Thread GitBox
mjsax commented on pull request #8833: URL: https://github.com/apache/kafka/pull/8833#issuecomment-641485789 Updates this PR. This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [kafka] kkonstantine commented on a change in pull request #8453: KAFKA-9841: Connector and Task duplicated when a worker join with old…

2020-06-09 Thread GitBox
kkonstantine commented on a change in pull request #8453: URL: https://github.com/apache/kafka/pull/8453#discussion_r437612063 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/IncrementalCooperativeAssignor.java ## @@ -229,6 +229,10 @@

[GitHub] [kafka] abbccdda commented on a change in pull request #8680: KAFKA-10027: Implement read path for feature versioning system (KIP-584)

2020-06-09 Thread GitBox
abbccdda commented on a change in pull request #8680: URL: https://github.com/apache/kafka/pull/8680#discussion_r437145250 ## File path: clients/src/main/java/org/apache/kafka/common/requests/ApiVersionsResponse.java ## @@ -113,14 +128,45 @@ public static ApiVersionsResponse

[jira] [Updated] (KAFKA-10130) Rewrite ZkData struct with auto-generated protocol

2020-06-09 Thread Boyang Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Boyang Chen updated KAFKA-10130: Component/s: protocol core > Rewrite ZkData struct with auto-generated protocol

[jira] [Created] (KAFKA-10130) Rewrite ZkData struct with auto-generated protocol

2020-06-09 Thread Boyang Chen (Jira)
Boyang Chen created KAFKA-10130: --- Summary: Rewrite ZkData struct with auto-generated protocol Key: KAFKA-10130 URL: https://issues.apache.org/jira/browse/KAFKA-10130 Project: Kafka Issue Type:

[GitHub] [kafka] kkonstantine commented on pull request #8827: KAFKA-9849: Fix issue with worker.unsync.backoff.ms creating zombie workers when incremental cooperative rebalancing is used

2020-06-09 Thread GitBox
kkonstantine commented on pull request #8827: URL: https://github.com/apache/kafka/pull/8827#issuecomment-641456031 Thanks @rhauch ! Indeed, the build on https://github.com/apache/kafka/pull/8805 got stuck on a zookeeper test and expired. I've rebased here to get the changes of

[jira] [Commented] (KAFKA-10126) Remove unused options in ConsumerPerformance

2020-06-09 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17129631#comment-17129631 ] Chia-Ping Tsai commented on KAFKA-10126: After I started to think more about it, removing them

[jira] [Commented] (KAFKA-10126) Remove unused options in ConsumerPerformance

2020-06-09 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17129620#comment-17129620 ] Chia-Ping Tsai commented on KAFKA-10126: > I don't think we could remove options unless in a

[jira] [Commented] (KAFKA-10126) Remove unused options in ConsumerPerformance

2020-06-09 Thread Boyang Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17129615#comment-17129615 ] Boyang Chen commented on KAFKA-10126: - I don't think we could remove options unless in a breaking

[jira] [Updated] (KAFKA-10126) Remove unused options in ConsumerPerformance

2020-06-09 Thread Boyang Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Boyang Chen updated KAFKA-10126: Component/s: tools consumer > Remove unused options in ConsumerPerformance >

[GitHub] [kafka] bellemare commented on a change in pull request #8764: KAFKA-10049: Fixed FKJ bug where wrapped serdes are set incorrectly when using default StreamsConfig serdes

2020-06-09 Thread GitBox
bellemare commented on a change in pull request #8764: URL: https://github.com/apache/kafka/pull/8764#discussion_r437560085 ## File path: streams/src/test/java/org/apache/kafka/streams/kstream/internals/KTableKTableForeignKeyJoinScenarioTest.java ## @@ -243,17 +244,17 @@

[jira] [Updated] (KAFKA-10126) Remove unused options in ConsumerPerformance

2020-06-09 Thread Boyang Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Boyang Chen updated KAFKA-10126: Issue Type: Improvement (was: Bug) > Remove unused options in ConsumerPerformance >

[GitHub] [kafka] chia7712 commented on pull request #8840: KAFKA-10129 Fail QA if there are javadoc warnings

2020-06-09 Thread GitBox
chia7712 commented on pull request #8840: URL: https://github.com/apache/kafka/pull/8840#issuecomment-641393439 the javadoc warnings are traced by #8660 This is an automated message from the Apache Git Service. To respond to

[GitHub] [kafka] chia7712 commented on pull request #8660: MINOR: fix stream javadoc

2020-06-09 Thread GitBox
chia7712 commented on pull request #8660: URL: https://github.com/apache/kafka/pull/8660#issuecomment-641392917 > One other question I had is whether we should consider making doc failures also fail the build? #8840

[GitHub] [kafka] kkonstantine commented on pull request #8805: KAFKA-9848: Avoid triggering scheduled rebalance delay when task assignment fails but Connect workers remain in the group

2020-06-09 Thread GitBox
kkonstantine commented on pull request #8805: URL: https://github.com/apache/kafka/pull/8805#issuecomment-641423090 Thanks for the reviews @rhauch ! 2/3 builds were green here too and the 2 failures on JDK8 were unrelated.

[GitHub] [kafka] feyman2016 edited a comment on pull request #8832: KAFKA-9377: Refactor StreamsPartitionAssignor Repartition Count Logic

2020-06-09 Thread GitBox
feyman2016 edited a comment on pull request #8832: URL: https://github.com/apache/kafka/pull/8832#issuecomment-641392176 @abbccdda Thanks for the comments, I updated the PR with few comments and added algo description. Since the algo is still under discussion, I can add the unit tests

[GitHub] [kafka] chia7712 opened a new pull request #8840: KAFKA-10129 Fail QA if there are javadoc warnings

2020-06-09 Thread GitBox
chia7712 opened a new pull request #8840: URL: https://github.com/apache/kafka/pull/8840 from @hachikuji (https://github.com/apache/kafka/pull/8660#pullrequestreview-425856179) > One other question I had is whether we should consider making doc failures also fail the build?

[GitHub] [kafka] feyman2016 commented on pull request #8832: KAFKA-9377: Refactor StreamsPartitionAssignor Repartition Count Logic

2020-06-09 Thread GitBox
feyman2016 commented on pull request #8832: URL: https://github.com/apache/kafka/pull/8832#issuecomment-641392176 @abbccdda Thanks for the comments, I update the PR with few comments and added algo description. Since the algo is still under discussion, I can add the unit tests later once

[jira] [Commented] (KAFKA-10126) Remove unused options in ConsumerPerformance

2020-06-09 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17129604#comment-17129604 ] Chia-Ping Tsai commented on KAFKA-10126: > What's about numFetchersOpt? Can it be removed? I

[GitHub] [kafka] kkonstantine merged pull request #8805: KAFKA-9848: Avoid triggering scheduled rebalance delay when task assignment fails but Connect workers remain in the group

2020-06-09 Thread GitBox
kkonstantine merged pull request #8805: URL: https://github.com/apache/kafka/pull/8805 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [kafka] mjsax opened a new pull request #8833: KAFKA-9441: remove prepareClose() to simplify task management

2020-06-09 Thread GitBox
mjsax opened a new pull request #8833: URL: https://github.com/apache/kafka/pull/8833 - Removes `prepareCloseClean()` and `prepareCloseDirty()`. - Removes state transition `RUNNING -> CLOSED` (tasks must be suspended before closing now) - Decouples suspending/committing/closing (ie,

[GitHub] [kafka] cmccabe merged pull request #8640: MINOR: equals() should check _unknownTaggedFields

2020-06-09 Thread GitBox
cmccabe merged pull request #8640: URL: https://github.com/apache/kafka/pull/8640 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] jiameixie commented on a change in pull request #8836: KAFKA-10124:Wrong rebalance.time.ms

2020-06-09 Thread GitBox
jiameixie commented on a change in pull request #8836: URL: https://github.com/apache/kafka/pull/8836#discussion_r437180584 ## File path: core/src/main/scala/kafka/tools/ConsumerPerformance.scala ## @@ -105,16 +105,14 @@ object ConsumerPerformance extends LazyLogging {

[GitHub] [kafka] vvcephei commented on a change in pull request #8764: KAFKA-10049: Fixed FKJ bug where wrapped serdes are set incorrectly when using default StreamsConfig serdes

2020-06-09 Thread GitBox
vvcephei commented on a change in pull request #8764: URL: https://github.com/apache/kafka/pull/8764#discussion_r436858939 ## File path: streams/src/test/java/org/apache/kafka/streams/kstream/internals/KTableKTableForeignKeyJoinScenarioTest.java ## @@ -181,60 +182,60 @@

[GitHub] [kafka] viktorsomogyi commented on pull request #4090: [KAFKA-6084] Propagate JSON parsing errors in ReassignPartitionsCommand

2020-06-09 Thread GitBox
viktorsomogyi commented on pull request #4090: URL: https://github.com/apache/kafka/pull/4090#issuecomment-640603847 @ijuma I've rebased this PR. Would you please give it a look? I think it's just the two unresolved comment that we need to settle.

[GitHub] [kafka] junrao merged pull request #8807: KAFKA-10106: log time taken to handle LeaderAndIsr request

2020-06-09 Thread GitBox
junrao merged pull request #8807: URL: https://github.com/apache/kafka/pull/8807 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] ableegoldman commented on pull request #8803: KAFKA-10102: update ProcessorTopology instead of rebuilding it

2020-06-09 Thread GitBox
ableegoldman commented on pull request #8803: URL: https://github.com/apache/kafka/pull/8803#issuecomment-640741696 All tests passed, can we merge @mjsax ? This is an automated message from the Apache Git Service. To respond

[GitHub] [kafka] jiameixie opened a new pull request #8836: KAFKA-10124:Wrong rebalance.time.ms

2020-06-09 Thread GitBox
jiameixie opened a new pull request #8836: URL: https://github.com/apache/kafka/pull/8836 The consumer rebalance protocol is changed. Method onPartitionsRevoked is called after onPartitionsAssigned, so wrong joinTime is got. Change-Id: I561a48a13a870bd3cb03008825b69b804c6a94b4

[GitHub] [kafka] jiameixie commented on pull request #8831: KAFKA-8657:Client-side automatic topic creation on Producer

2020-06-09 Thread GitBox
jiameixie commented on pull request #8831: URL: https://github.com/apache/kafka/pull/8831#issuecomment-640987419 @abbccdda It seems it hasn't been accepted and has not been updated for a long time. This is an automated

[GitHub] [kafka] cmccabe commented on a change in pull request #8640: MINOR: equals() should check _unknownTaggedFields

2020-06-09 Thread GitBox
cmccabe commented on a change in pull request #8640: URL: https://github.com/apache/kafka/pull/8640#discussion_r436860958 ## File path: clients/src/test/java/org/apache/kafka/common/message/MessageTest.java ## @@ -962,4 +962,23 @@ private void verifyWriteSucceeds(short

[GitHub] [kafka] abbccdda commented on a change in pull request #8833: KAFKA-9441: remove prepareClose() to simplify task management

2020-06-09 Thread GitBox
abbccdda commented on a change in pull request #8833: URL: https://github.com/apache/kafka/pull/8833#discussion_r436987081 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamTask.java ## @@ -534,62 +482,38 @@ public void

[GitHub] [kafka] tombentley opened a new pull request #8839: KIP-585: Documentation

2020-06-09 Thread GitBox
tombentley opened a new pull request #8839: URL: https://github.com/apache/kafka/pull/8839 Add documentation for using transformation predicates. Add `PredicateDoc` for generating predicate config docs, following the style of `TransformationDoc`. Fix the header depth mismatch.

[GitHub] [kafka] feyman2016 opened a new pull request #8832: KAFKA-9377: Refactor StreamsPartitionAssignor Repartition Count Logic

2020-06-09 Thread GitBox
feyman2016 opened a new pull request #8832: URL: https://github.com/apache/kafka/pull/8832 ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgrade

[GitHub] [kafka] bob-barrett commented on a change in pull request #8239: KAFKA-9666: Don't increase transactional epoch when trying to fence if the log append fails

2020-06-09 Thread GitBox
bob-barrett commented on a change in pull request #8239: URL: https://github.com/apache/kafka/pull/8239#discussion_r437077452 ## File path: core/src/main/scala/kafka/coordinator/transaction/TransactionCoordinator.scala ## @@ -487,6 +487,33 @@ class

[GitHub] [kafka] guozhangwang opened a new pull request #8834: MINOR: Do not disable heartbeat during Rebalance

2020-06-09 Thread GitBox
guozhangwang opened a new pull request #8834: URL: https://github.com/apache/kafka/pull/8834 1. Allow the heartbeat thread to send hb request during rebalance; in turn when handling responses if the state is not in STABLE ignore the error. 2. Piggy-backing a log4j improvement on the

[GitHub] [kafka] ableegoldman commented on a change in pull request #8676: KAFKA-10005: Decouple RestoreListener from RestoreCallback

2020-06-09 Thread GitBox
ableegoldman commented on a change in pull request #8676: URL: https://github.com/apache/kafka/pull/8676#discussion_r436858916 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/ChangelogRegister.java ## @@ -34,7 +34,6 @@ /** *

[GitHub] [kafka] mjsax commented on pull request #8821: [DO NOT MERGE] Reenable flaky EosBetaUpgradeIntegrationTest

2020-06-09 Thread GitBox
mjsax commented on pull request #8821: URL: https://github.com/apache/kafka/pull/8821#issuecomment-641022736 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [kafka] hachikuji merged pull request #8783: KAFKA-10063 UnsupportedOperation when querying cleaner metrics after …

2020-06-09 Thread GitBox
hachikuji merged pull request #8783: URL: https://github.com/apache/kafka/pull/8783 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] guozhangwang merged pull request #8676: KAFKA-10005: Decouple RestoreListener from RestoreCallback

2020-06-09 Thread GitBox
guozhangwang merged pull request #8676: URL: https://github.com/apache/kafka/pull/8676 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [kafka] kkonstantine commented on a change in pull request #8805: KAFKA-9848: Avoid triggering scheduled rebalance delay when task assignment fails but Connect workers remain in the group

2020-06-09 Thread GitBox
kkonstantine commented on a change in pull request #8805: URL: https://github.com/apache/kafka/pull/8805#discussion_r436950305 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/IncrementalCooperativeAssignor.java ## @@ -159,6 +163,13 @@

[GitHub] [kafka] Lucent-Wong edited a comment on pull request #8453: KAFKA-9841: Connector and Task duplicated when a worker join with old…

2020-06-09 Thread GitBox
Lucent-Wong edited a comment on pull request #8453: URL: https://github.com/apache/kafka/pull/8453#issuecomment-641192263 Looks like the build failure is not related with my changes. > Task :streams:test-utils:integrationTest > Task

[GitHub] [kafka] chia7712 opened a new pull request #8838: MINOR: fix flaky TopicCommandWithAdminClientTest.testDescribeUnderRep…

2020-06-09 Thread GitBox
chia7712 opened a new pull request #8838: URL: https://github.com/apache/kafka/pull/8838 ```TopicCommandWithAdminClientTest.testDescribeUnderReplicatedPartitionsWhenReassignmentIsInProgress``` fails frequently on my local. ### Committer Checklist (excluded from commit message) -

[GitHub] [kafka] omkreddy merged pull request #8800: MINOR: Fix incorrect GC log size with JDK9+

2020-06-09 Thread GitBox
omkreddy merged pull request #8800: URL: https://github.com/apache/kafka/pull/8800 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

  1   2   >