[GitHub] [kafka] rohitrmd commented on pull request #10276: KAFKA-12253: Add tests that cover all of the cases for ReplicatedLog's validateOffsetAndEpoch

2021-03-15 Thread GitBox
rohitrmd commented on pull request #10276: URL: https://github.com/apache/kafka/pull/10276#issuecomment-799971318 @jsancio thanks for the review. Made changes as per review comments. This is an automated message from the

[GitHub] [kafka] wenbingshen commented on pull request #10329: MINOR: remove some specifying types in tool command

2021-03-15 Thread GitBox
wenbingshen commented on pull request #10329: URL: https://github.com/apache/kafka/pull/10329#issuecomment-799968752 :) @dajac Can you help review this minor pr?Thanks. This is an automated message from the Apache Git

[GitHub] [kafka] wenbingshen opened a new pull request #10329: MINOR: remove some specifying types in tool command

2021-03-15 Thread GitBox
wenbingshen opened a new pull request #10329: URL: https://github.com/apache/kafka/pull/10329 As the title.I checked some of the command-line tools and removed some of the specified types. ### Committer Checklist (excluded from commit message) - [ ] Verify design and

[GitHub] [kafka] dengziming commented on a change in pull request #10289: KAFKA-12440: ClusterId validation for Vote, BeginQuorum, EndQuorum and FetchSnapshot

2021-03-15 Thread GitBox
dengziming commented on a change in pull request #10289: URL: https://github.com/apache/kafka/pull/10289#discussion_r594874464 ## File path: raft/src/main/java/org/apache/kafka/raft/KafkaRaftClient.java ## @@ -939,12 +951,27 @@ private FetchResponseData

[jira] [Commented] (KAFKA-12477) Smart rebalancing with dynamic protocol selection

2021-03-15 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17302226#comment-17302226 ] A. Sophie Blee-Goldman commented on KAFKA-12477: [~guozhang] any thoughts? If this makes

[jira] [Commented] (KAFKA-12463) Update default consumer partition assignor for sink tasks

2021-03-15 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17302225#comment-17302225 ] A. Sophie Blee-Goldman commented on KAFKA-12463: Filed

[jira] [Created] (KAFKA-12477) Smart rebalancing with dynamic protocol selection

2021-03-15 Thread A. Sophie Blee-Goldman (Jira)
A. Sophie Blee-Goldman created KAFKA-12477: -- Summary: Smart rebalancing with dynamic protocol selection Key: KAFKA-12477 URL: https://issues.apache.org/jira/browse/KAFKA-12477 Project: Kafka

[GitHub] [kafka] showuon closed pull request #10328: MINOR: update the version number to 2.6.2

2021-03-15 Thread GitBox
showuon closed pull request #10328: URL: https://github.com/apache/kafka/pull/10328 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] showuon commented on a change in pull request #10328: MINOR: update the version number to 2.6.2

2021-03-15 Thread GitBox
showuon commented on a change in pull request #10328: URL: https://github.com/apache/kafka/pull/10328#discussion_r594860616 ## File path: docs/upgrade.html ## @@ -19,7 +19,7 @@

[jira] [Comment Edited] (KAFKA-12463) Update default consumer partition assignor for sink tasks

2021-03-15 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17302205#comment-17302205 ] Chris Egerton edited comment on KAFKA-12463 at 3/16/21, 4:29 AM: - Haha,

[jira] [Commented] (KAFKA-12463) Update default consumer partition assignor for sink tasks

2021-03-15 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17302205#comment-17302205 ] Chris Egerton commented on KAFKA-12463: --- Haha, I should have been clearer–we do use our own type

[GitHub] [kafka] ableegoldman commented on a change in pull request #10328: MINOR: update the version number to 2.6.2

2021-03-15 Thread GitBox
ableegoldman commented on a change in pull request #10328: URL: https://github.com/apache/kafka/pull/10328#discussion_r594848632 ## File path: docs/upgrade.html ## @@ -19,7 +19,7 @@

[GitHub] [kafka] ableegoldman commented on a change in pull request #10328: MINOR: update the version number to 2.6.2

2021-03-15 Thread GitBox
ableegoldman commented on a change in pull request #10328: URL: https://github.com/apache/kafka/pull/10328#discussion_r594848320 ## File path: docs/upgrade.html ## @@ -19,7 +19,7 @@

[GitHub] [kafka] showuon commented on pull request #10328: MINOR: update the version number to 2.6.2

2021-03-15 Thread GitBox
showuon commented on pull request #10328: URL: https://github.com/apache/kafka/pull/10328#issuecomment-799925201 @ableegoldman , please help review. Thanks. This is an automated message from the Apache Git Service. To

[GitHub] [kafka] showuon opened a new pull request #10328: MINOR: update the version number to 2.6.2

2021-03-15 Thread GitBox
showuon opened a new pull request #10328: URL: https://github.com/apache/kafka/pull/10328 update the version number to 2.6.2 ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ]

[GitHub] [kafka] C0urante commented on pull request #10014: KAFKA-12252 and KAFKA-12262: Fix session key rotation when leadership changes

2021-03-15 Thread GitBox
C0urante commented on pull request #10014: URL: https://github.com/apache/kafka/pull/10014#issuecomment-799923052 I've also filed https://issues.apache.org/jira/browse/KAFKA-12474 and https://issues.apache.org/jira/browse/KAFKA-12476 as follow-up items, since they are unaffected by and

[jira] [Commented] (KAFKA-12463) Update default consumer partition assignor for sink tasks

2021-03-15 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17302202#comment-17302202 ] A. Sophie Blee-Goldman commented on KAFKA-12463: Maybe we could improve things on the

[jira] [Comment Edited] (KAFKA-12475) Kafka Streams breaks EOS with remote state stores

2021-03-15 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17302200#comment-17302200 ] A. Sophie Blee-Goldman edited comment on KAFKA-12475 at 3/16/21, 3:37 AM:

[jira] [Comment Edited] (KAFKA-9156) LazyTimeIndex & LazyOffsetIndex may cause niobufferoverflow in concurrent state

2021-03-15 Thread iBlackeyes (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17302197#comment-17302197 ] iBlackeyes edited comment on KAFKA-9156 at 3/16/21, 3:36 AM: - This issue seem

[jira] [Commented] (KAFKA-12475) Kafka Streams breaks EOS with remote state stores

2021-03-15 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17302200#comment-17302200 ] A. Sophie Blee-Goldman commented on KAFKA-12475: One option to fix this would be to add

[jira] [Comment Edited] (KAFKA-9156) LazyTimeIndex & LazyOffsetIndex may cause niobufferoverflow in concurrent state

2021-03-15 Thread iBlackeyes (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17302197#comment-17302197 ] iBlackeyes edited comment on KAFKA-9156 at 3/16/21, 3:36 AM: - This issue seem

[jira] [Comment Edited] (KAFKA-9156) LazyTimeIndex & LazyOffsetIndex may cause niobufferoverflow in concurrent state

2021-03-15 Thread iBlackeyes (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17302197#comment-17302197 ] iBlackeyes edited comment on KAFKA-9156 at 3/16/21, 3:35 AM: - This issue seem

[jira] [Comment Edited] (KAFKA-9156) LazyTimeIndex & LazyOffsetIndex may cause niobufferoverflow in concurrent state

2021-03-15 Thread iBlackeyes (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17302197#comment-17302197 ] iBlackeyes edited comment on KAFKA-9156 at 3/16/21, 3:35 AM: - This issue seem

[jira] [Updated] (KAFKA-12474) Worker can die if unable to write new session key

2021-03-15 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton updated KAFKA-12474: -- Issue Type: Bug (was: Improvement) > Worker can die if unable to write new session key >

[jira] [Created] (KAFKA-12476) Worker can block for longer than scheduled rebalance delay and/or session key TTL

2021-03-15 Thread Chris Egerton (Jira)
Chris Egerton created KAFKA-12476: - Summary: Worker can block for longer than scheduled rebalance delay and/or session key TTL Key: KAFKA-12476 URL: https://issues.apache.org/jira/browse/KAFKA-12476

[jira] [Commented] (KAFKA-9156) LazyTimeIndex & LazyOffsetIndex may cause niobufferoverflow in concurrent state

2021-03-15 Thread iBlackeyes (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17302197#comment-17302197 ] iBlackeyes commented on KAFKA-9156: --- This issue seem not been completed fixed. Because after we patched

[jira] [Updated] (KAFKA-12474) Worker can die if unable to write new session key

2021-03-15 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton updated KAFKA-12474: -- Affects Version/s: 3.0.0 > Worker can die if unable to write new session key >

[jira] [Created] (KAFKA-12475) Kafka Streams breaks EOS with remote state stores

2021-03-15 Thread A. Sophie Blee-Goldman (Jira)
A. Sophie Blee-Goldman created KAFKA-12475: -- Summary: Kafka Streams breaks EOS with remote state stores Key: KAFKA-12475 URL: https://issues.apache.org/jira/browse/KAFKA-12475 Project: Kafka

[jira] [Created] (KAFKA-12474) Worker can die if unable to write new session key

2021-03-15 Thread Chris Egerton (Jira)
Chris Egerton created KAFKA-12474: - Summary: Worker can die if unable to write new session key Key: KAFKA-12474 URL: https://issues.apache.org/jira/browse/KAFKA-12474 Project: Kafka Issue

[GitHub] [kafka] wenbingshen commented on pull request #10327: MINOR: No fetcher for partitions don't need to print remove fetcher log

2021-03-15 Thread GitBox
wenbingshen commented on pull request #10327: URL: https://github.com/apache/kafka/pull/10327#issuecomment-799914157 @chia7712 @dajac @showuon Please help review this minor pr.Thanks. This is an automated message from the

[GitHub] [kafka] wenbingshen opened a new pull request #10327: MINOR: No fetcher for partitions don't need to print remove fetcher log

2021-03-15 Thread GitBox
wenbingshen opened a new pull request #10327: URL: https://github.com/apache/kafka/pull/10327 As the title.In order to properly reflect the cluster state, partitions that are not in the fetcher should not print the remove fecher log. ### Committer Checklist (excluded from commit

[jira] [Commented] (KAFKA-12473) Make the CooperativeStickyAssignor the default assignor

2021-03-15 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17302193#comment-17302193 ] Luke Chen commented on KAFKA-12473: --- Got it! Thank you. > Make the CooperativeStickyAssignor the

[jira] [Commented] (KAFKA-12463) Update default consumer partition assignor for sink tasks

2021-03-15 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17302192#comment-17302192 ] A. Sophie Blee-Goldman commented on KAFKA-12463: I think it still has the

[jira] [Commented] (KAFKA-12463) Update default consumer partition assignor for sink tasks

2021-03-15 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17302191#comment-17302191 ] Guozhang Wang commented on KAFKA-12463: --- Connect does not use ConsumerCoordinator but implements

[jira] [Updated] (KAFKA-12473) Make the CooperativeStickyAssignor the default assignor

2021-03-15 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman updated KAFKA-12473: --- Description: Now that 3.0 is coming up, we can change the default

[GitHub] [kafka] ableegoldman merged pull request #10321: HOTFIX: timeout issue in Remove thread

2021-03-15 Thread GitBox
ableegoldman merged pull request #10321: URL: https://github.com/apache/kafka/pull/10321 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[jira] [Assigned] (KAFKA-12473) Make the CooperativeStickyAssignor the default assignor

2021-03-15 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman reassigned KAFKA-12473: -- Assignee: Luke Chen > Make the CooperativeStickyAssignor the default

[jira] [Commented] (KAFKA-12473) Make the CooperativeStickyAssignor the default assignor

2021-03-15 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17302190#comment-17302190 ] A. Sophie Blee-Goldman commented on KAFKA-12473: Yeah, I believe this will need a KIP.

[jira] [Updated] (KAFKA-12473) Make the CooperativeStickyAssignor the default assignor

2021-03-15 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman updated KAFKA-12473: --- Labels: needs-kip (was: ) > Make the CooperativeStickyAssignor the default

[jira] [Comment Edited] (KAFKA-12463) Update default consumer partition assignor for sink tasks

2021-03-15 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17302179#comment-17302179 ] A. Sophie Blee-Goldman edited comment on KAFKA-12463 at 3/16/21, 3:09 AM:

[jira] [Commented] (KAFKA-12473) Make the CooperativeStickyAssignor the default assignor

2021-03-15 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17302189#comment-17302189 ] Luke Chen commented on KAFKA-12473: --- [~ableegoldman], I'd like to work on this. Does it needs KIP? I

[jira] [Commented] (KAFKA-12472) Add a Consumer / Streams metric to indicate the current rebalance status

2021-03-15 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17302186#comment-17302186 ] Guozhang Wang commented on KAFKA-12472: --- Re. 1/2: Makes sense. I can distinguish them. Re. 3: My

[jira] [Commented] (KAFKA-12283) Flaky Test RebalanceSourceConnectorsIntegrationTest#testMultipleWorkersRejoining

2021-03-15 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17302184#comment-17302184 ] A. Sophie Blee-Goldman commented on KAFKA-12283:

[jira] [Commented] (KAFKA-12463) Update default consumer partition assignor for sink tasks

2021-03-15 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17302183#comment-17302183 ] A. Sophie Blee-Goldman commented on KAFKA-12463: Now I don't know if Connect actually

[GitHub] [kafka] ableegoldman commented on pull request #10321: HOTFIX: timeout issue in Remove thread

2021-03-15 Thread GitBox
ableegoldman commented on pull request #10321: URL: https://github.com/apache/kafka/pull/10321#issuecomment-799907176 One unrelated flaky test failure ([KAFKA-12283](https://issues.apache.org/jira/browse/KAFKA-12283)):

[jira] [Commented] (KAFKA-12463) Update default consumer partition assignor for sink tasks

2021-03-15 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17302179#comment-17302179 ] A. Sophie Blee-Goldman commented on KAFKA-12463: Wait, no, I double checked in the code

[jira] [Commented] (KAFKA-12473) Make the CooperativeStickyAssignor the default assignor

2021-03-15 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17302178#comment-17302178 ] Guozhang Wang commented on KAFKA-12473: --- Thanks for filing this [~ableegoldman]! The plan SGTM. >

[jira] [Comment Edited] (KAFKA-12463) Update default consumer partition assignor for sink tasks

2021-03-15 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17302167#comment-17302167 ] A. Sophie Blee-Goldman edited comment on KAFKA-12463 at 3/16/21, 2:51 AM:

[GitHub] [kafka] lizthegrey opened a new pull request #10326: Add RackAwareInSyncReplicaSelector

2021-03-15 Thread GitBox
lizthegrey opened a new pull request #10326: URL: https://github.com/apache/kafka/pull/10326 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* cross rack

[jira] [Commented] (KAFKA-12463) Update default consumer partition assignor for sink tasks

2021-03-15 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17302167#comment-17302167 ] A. Sophie Blee-Goldman commented on KAFKA-12463: Isn't the upgrade path for Connect

[jira] [Created] (KAFKA-12473) Make the CooperativeStickyAssignor the default assignor

2021-03-15 Thread A. Sophie Blee-Goldman (Jira)
A. Sophie Blee-Goldman created KAFKA-12473: -- Summary: Make the CooperativeStickyAssignor the default assignor Key: KAFKA-12473 URL: https://issues.apache.org/jira/browse/KAFKA-12473 Project:

[GitHub] [kafka] jsancio edited a comment on pull request #10324: MINOR: Add a few more benchmark for the timeline map

2021-03-15 Thread GitBox
jsancio edited a comment on pull request #10324: URL: https://github.com/apache/kafka/pull/10324#issuecomment-799843745 Benchmark results: ``` Benchmark Mode Cnt ScoreError Units

[jira] [Commented] (KAFKA-12472) Add a Consumer / Streams metric to indicate the current rebalance status

2021-03-15 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17302155#comment-17302155 ] A. Sophie Blee-Goldman commented on KAFKA-12472: This sounds like a great (and much

[GitHub] [kafka] jsancio edited a comment on pull request #10324: MINOR: Add a few more benchmark for the timeline map

2021-03-15 Thread GitBox
jsancio edited a comment on pull request #10324: URL: https://github.com/apache/kafka/pull/10324#issuecomment-799843745 Benchmark results: ``` # Run complete. Total time: 00:26:54 Benchmark Mode Cnt ScoreError Units

[GitHub] [kafka] jsancio commented on a change in pull request #10324: MINOR: Add a few more benchmark for the timeline map

2021-03-15 Thread GitBox
jsancio commented on a change in pull request #10324: URL: https://github.com/apache/kafka/pull/10324#discussion_r594806269 ## File path: jmh-benchmarks/src/main/java/org/apache/kafka/jmh/timeline/TimelineHashMapBenchmark.java ## @@ -87,4 +189,129 @@ }

[GitHub] [kafka] jsancio edited a comment on pull request #10324: MINOR: Add a few more benchmark for the timeline map

2021-03-15 Thread GitBox
jsancio edited a comment on pull request #10324: URL: https://github.com/apache/kafka/pull/10324#issuecomment-799843745 Benchmark results: ``` # Run complete. Total time: 00:26:54 Benchmark Mode CntScore Error Units

[GitHub] [kafka] dengziming commented on pull request #10325: MONOR: Remove redudant LocalLogManager

2021-03-15 Thread GitBox
dengziming commented on pull request #10325: URL: https://github.com/apache/kafka/pull/10325#issuecomment-799879463 ping @cmccabe @mumrah to have a look. This is an automated message from the Apache Git Service. To respond

[GitHub] [kafka] dengziming opened a new pull request #10325: MONOR: Remove redudant LocalLogManager

2021-03-15 Thread GitBox
dengziming opened a new pull request #10325: URL: https://github.com/apache/kafka/pull/10325 *More detailed description of your change* We have 2 identical LocalLogManager in `src/main/org.apache.kafka.metalog` and `src/test/org.apache.kafka.metalog`, the only difference is that the

[jira] [Commented] (KAFKA-10518) Consumer fetches could be inefficient when lags are unbalanced

2021-03-15 Thread Tom Lee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17302143#comment-17302143 ] Tom Lee commented on KAFKA-10518: - Attaching a contrived repro case. In case it's not obvious: in the

[GitHub] [kafka] dengziming commented on pull request #10312: MINOR: Fix log statement whose placeholders are inconsistent with arguments

2021-03-15 Thread GitBox
dengziming commented on pull request #10312: URL: https://github.com/apache/kafka/pull/10312#issuecomment-799873639 Similar to #10262, ping @tombentley to have a look. This is an automated message from the Apache Git

[jira] [Updated] (KAFKA-12472) Add a Consumer / Streams metric to indicate the current rebalance status

2021-03-15 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-12472: -- Description: Today to trouble shoot a rebalance issue operators need to do a lot of manual

[jira] [Updated] (KAFKA-12472) Add a Consumer / Streams metric to indicate the current rebalance status

2021-03-15 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-12472: -- Description: Today to trouble shoot a rebalance issue operators need to do a lot of manual

[jira] [Updated] (KAFKA-8127) It may need to import scala.io

2021-03-15 Thread JieFang.He (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JieFang.He updated KAFKA-8127: -- Description: I get an error when compile kafka,which disappear when import scala.io   {code:java}

[jira] [Updated] (KAFKA-12472) Add a Consumer / Streams metric to indicate the current rebalance status

2021-03-15 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-12472: -- Description: Today to trouble shoot a rebalance issue operators need to do a lot of manual

[jira] [Updated] (KAFKA-10518) Consumer fetches could be inefficient when lags are unbalanced

2021-03-15 Thread Tom Lee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom Lee updated KAFKA-10518: Attachment: kafka-slow-consumer-repro.tar.gz > Consumer fetches could be inefficient when lags are

[jira] [Updated] (KAFKA-12472) Add a Consumer / Streams metric to indicate the current rebalance status

2021-03-15 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-12472: -- Description: Today to trouble shoot a rebalance issue operators need to do a lot of manual

[jira] [Updated] (KAFKA-12472) Add a Consumer / Streams metric to indicate the current rebalance status

2021-03-15 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-12472: -- Description: Today to trouble shoot a rebalance issue operators need to do a lot of manual

[jira] [Updated] (KAFKA-12472) Add a Consumer / Streams metric to indicate the current rebalance status

2021-03-15 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-12472: -- Description: Today to trouble shoot a rebalance issue operators need to do a lot of manual

[jira] [Created] (KAFKA-12472) Add a Consumer / Streams metric to indicate the current rebalance status

2021-03-15 Thread Guozhang Wang (Jira)
Guozhang Wang created KAFKA-12472: - Summary: Add a Consumer / Streams metric to indicate the current rebalance status Key: KAFKA-12472 URL: https://issues.apache.org/jira/browse/KAFKA-12472 Project:

[GitHub] [kafka] jsancio commented on a change in pull request #10309: KAFKA-12181; Loosen raft fetch offset validation of remote replicas

2021-03-15 Thread GitBox
jsancio commented on a change in pull request #10309: URL: https://github.com/apache/kafka/pull/10309#discussion_r594787145 ## File path: raft/src/test/java/org/apache/kafka/raft/RaftEventSimulationTest.java ## @@ -401,6 +401,51 @@ private void

[GitHub] [kafka] ikdekker commented on pull request #10313: KAFKA-12456: Log "Listeners are not identical across brokers" message at WARN/INFO instead of ERROR

2021-03-15 Thread GitBox
ikdekker commented on pull request #10313: URL: https://github.com/apache/kafka/pull/10313#issuecomment-799845091 By the way, I sent my JIRA username to the dev mail, not sure if i should have typed some extra lines there. But then I might try again 

[jira] [Updated] (KAFKA-12428) Add a last-heartbeat-seconds-ago metric to Kafka Consumer

2021-03-15 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-12428: -- Labels: newbie++ (was: newbie) > Add a last-heartbeat-seconds-ago metric to Kafka Consumer >

[GitHub] [kafka] jsancio commented on pull request #10324: MINOR: Add a few more benchmark for the timeline map

2021-03-15 Thread GitBox
jsancio commented on pull request #10324: URL: https://github.com/apache/kafka/pull/10324#issuecomment-799843745 Benchmark results: ``` # Run complete. Total time: 00:26:54 Benchmark Mode CntScore Error Units

[GitHub] [kafka] ikdekker commented on pull request #10313: KAFKA-12456: Log "Listeners are not identical across brokers" message at WARN/INFO instead of ERROR

2021-03-15 Thread GitBox
ikdekker commented on pull request #10313: URL: https://github.com/apache/kafka/pull/10313#issuecomment-799843740 Alright, I will check some things out. This PR was created from trunk. I might try to create some tests myself, which I should be able to get output from as a starting point.

[GitHub] [kafka] jsancio opened a new pull request #10324: MINOR: Add a few more benchmark for the timeline map

2021-03-15 Thread GitBox
jsancio opened a new pull request #10324: URL: https://github.com/apache/kafka/pull/10324 Improve the benchmark tests for TimelineHashMap by adding tests for adding entries, removing entries and Scala's immutable hash map. *More detailed description of your change, if necessary.

[GitHub] [kafka] dejan2609 commented on pull request #10056: KAFKA-12293: remove JCenter and Bintray repositories mentions out of build gradle scripts (sunset is announced for those repositories)

2021-03-15 Thread GitBox
dejan2609 commented on pull request #10056: URL: https://github.com/apache/kafka/pull/10056#issuecomment-799842860 @ijuma Done (conflict solved / branch force-pushed). This is an automated message from the Apache Git

[GitHub] [kafka] jolshan edited a comment on pull request #10313: KAFKA-12456: Log "Listeners are not identical across brokers" message at WARN/INFO instead of ERROR

2021-03-15 Thread GitBox
jolshan edited a comment on pull request #10313: URL: https://github.com/apache/kafka/pull/10313#issuecomment-799842000 @ikdekker Hmm. The best advice I can give here is running `./gradlew clean` and trying again. I've seen plenty of green runs with core tests, but yes, it usually takes

[GitHub] [kafka] jolshan commented on pull request #10313: KAFKA-12456: Log "Listeners are not identical across brokers" message at WARN/INFO instead of ERROR

2021-03-15 Thread GitBox
jolshan commented on pull request #10313: URL: https://github.com/apache/kafka/pull/10313#issuecomment-799842000 @ikdekker Hmm. The best advice I can give here is running `./gradlew clean` and trying again. I've seen plenty of green runs with core tests, but yes, it usually takes 20

[GitHub] [kafka] ikdekker edited a comment on pull request #10313: KAFKA-12456: Log "Listeners are not identical across brokers" message at WARN/INFO instead of ERROR

2021-03-15 Thread GitBox
ikdekker edited a comment on pull request #10313: URL: https://github.com/apache/kafka/pull/10313#issuecomment-799830566 Running core:tests takes around 20 minutes and fails at the end. If it is as you say and simply running the gradle, is it expected to have some failures? For example

[GitHub] [kafka] ikdekker commented on pull request #10313: KAFKA-12456: Log "Listeners are not identical across brokers" message at WARN/INFO instead of ERROR

2021-03-15 Thread GitBox
ikdekker commented on pull request #10313: URL: https://github.com/apache/kafka/pull/10313#issuecomment-799830566 Running core:tests takes around 20 minutes and fails at the end. If it is as you say and simply running the gradle, is it expected to have some failures? For example for tests

[GitHub] [kafka] jolshan edited a comment on pull request #10313: KAFKA-12456: Log "Listeners are not identical across brokers" message at WARN/INFO instead of ERROR

2021-03-15 Thread GitBox
jolshan edited a comment on pull request #10313: URL: https://github.com/apache/kafka/pull/10313#issuecomment-799824018 @ikdekker You don't need to follow any of the quickstart steps to run the tests. Simply running `./gradlew test` or any subset of tests should do the trick. As

[GitHub] [kafka] jolshan edited a comment on pull request #10313: KAFKA-12456: Log "Listeners are not identical across brokers" message at WARN/INFO instead of ERROR

2021-03-15 Thread GitBox
jolshan edited a comment on pull request #10313: URL: https://github.com/apache/kafka/pull/10313#issuecomment-799824018 @ikdekker You don't need to follow any of the quickstart steps to run the tests. Simply running `./gradlew test` or any subset of tests should do the trick. As

[GitHub] [kafka] jolshan commented on pull request #10313: KAFKA-12456: Log "Listeners are not identical across brokers" message at WARN/INFO instead of ERROR

2021-03-15 Thread GitBox
jolshan commented on pull request #10313: URL: https://github.com/apache/kafka/pull/10313#issuecomment-799824018 @ikdekker You don't need to follow any of the quickstart steps to run the tests. Simply running `./gradlew test` or any subset of tests should do the trick. As for

[GitHub] [kafka] hachikuji opened a new pull request #10323: KAFKA-12459; Use property testing library for raft event simulation tests

2021-03-15 Thread GitBox
hachikuji opened a new pull request #10323: URL: https://github.com/apache/kafka/pull/10323 This patch changes the raft simulation tests to use jqwik, which is a property testing library. This provides two main benefits: - It simplifies the randomization of test parameters.

[GitHub] [kafka] ikdekker commented on pull request #10313: KAFKA-12456: Log "Listeners are not identical across brokers" message at WARN/INFO instead of ERROR

2021-03-15 Thread GitBox
ikdekker commented on pull request #10313: URL: https://github.com/apache/kafka/pull/10313#issuecomment-799813229 Directly after running (all tests), I see only a clients index.html, streams and core do not generate a report, even when running `./gradlew streams:test` or `./gradlew

[jira] [Commented] (KAFKA-12453) Guidance on whether a topology is eligible for optimisation

2021-03-15 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17302087#comment-17302087 ] A. Sophie Blee-Goldman commented on KAFKA-12453: [~mjsax] I thought we don't enable the

[GitHub] [kafka] abbccdda commented on pull request #10320: MINOR: revert stream logging level back to ERROR

2021-03-15 Thread GitBox
abbccdda commented on pull request #10320: URL: https://github.com/apache/kafka/pull/10320#issuecomment-799811099 Cherry-picked to 2.8 This is an automated message from the Apache Git Service. To respond to the message,

[jira] [Commented] (KAFKA-10518) Consumer fetches could be inefficient when lags are unbalanced

2021-03-15 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17302075#comment-17302075 ] Colin McCabe commented on KAFKA-10518: -- One simple workaround is to increase max.poll.records so

[GitHub] [kafka] jolshan commented on pull request #10313: KAFKA-12456: Log "Listeners are not identical across brokers" message at WARN/INFO instead of ERROR

2021-03-15 Thread GitBox
jolshan commented on pull request #10313: URL: https://github.com/apache/kafka/pull/10313#issuecomment-799800389 @ikdekker Hmm. When you run another test, old files are deleted. Try copy and pasting the file path directly after running a test.

[GitHub] [kafka] ikdekker edited a comment on pull request #10313: KAFKA-12456: Log "Listeners are not identical across brokers" message at WARN/INFO instead of ERROR

2021-03-15 Thread GitBox
ikdekker edited a comment on pull request #10313: URL: https://github.com/apache/kafka/pull/10313#issuecomment-799798100 Hi @jolshan, Thanks for the response! I have looked into my build directories. The ones that my gradle outputs after running tests. These do not contain

[GitHub] [kafka] ikdekker commented on pull request #10313: KAFKA-12456: Log "Listeners are not identical across brokers" message at WARN/INFO instead of ERROR

2021-03-15 Thread GitBox
ikdekker commented on pull request #10313: URL: https://github.com/apache/kafka/pull/10313#issuecomment-799798100 Hi @jolshan, I have looked into my build directories. The ones that my gradle outputs after running tests. These do not contain the reports. For example, both clients

[GitHub] [kafka] hachikuji commented on a change in pull request #10309: KAFKA-12181; Loosen raft fetch offset validation of remote replicas

2021-03-15 Thread GitBox
hachikuji commented on a change in pull request #10309: URL: https://github.com/apache/kafka/pull/10309#discussion_r594725779 ## File path: raft/src/test/java/org/apache/kafka/raft/RaftEventSimulationTest.java ## @@ -401,6 +401,51 @@ private void

[jira] [Assigned] (KAFKA-12471) Implement createPartitions in KIP-500 mode

2021-03-15 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe reassigned KAFKA-12471: Assignee: Colin McCabe Labels: kip-500 (was: ) > Implement createPartitions in

[jira] [Created] (KAFKA-12471) Implement createPartitions in KIP-500 mode

2021-03-15 Thread Colin McCabe (Jira)
Colin McCabe created KAFKA-12471: Summary: Implement createPartitions in KIP-500 mode Key: KAFKA-12471 URL: https://issues.apache.org/jira/browse/KAFKA-12471 Project: Kafka Issue Type: New

[GitHub] [kafka] ijuma commented on pull request #10056: KAFKA-12293: remove JCenter and Bintray repositories mentions out of build gradle scripts (sunset is announced for those repositories)

2021-03-15 Thread GitBox
ijuma commented on pull request #10056: URL: https://github.com/apache/kafka/pull/10056#issuecomment-799785567 @dejan2609 can you please resolve the conflict? This is an automated message from the Apache Git Service. To

[jira] [Updated] (KAFKA-12469) The topic names in the metrics do not retain their format when extracting through JMX.

2021-03-15 Thread Jira
[ https://issues.apache.org/jira/browse/KAFKA-12469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rafał Chmielewski updated KAFKA-12469: -- Component/s: consumer > The topic names in the metrics do not retain their format

[jira] [Updated] (KAFKA-12469) The topic names in the metrics do not retain their format when extracting through JMX.

2021-03-15 Thread Jira
[ https://issues.apache.org/jira/browse/KAFKA-12469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rafał Chmielewski updated KAFKA-12469: -- Description: I have topic names that have a period in the name: product.order

[GitHub] [kafka] rondagostino commented on pull request #10322: KAFKA-12455: OffsetValidationTest.test_broker_rolling_bounce fail: Raft

2021-03-15 Thread GitBox
rondagostino commented on pull request #10322: URL: https://github.com/apache/kafka/pull/10322#issuecomment-799781073 As per an offline conversation, since 2 brokers is a supported cluster size, we would prefer that this system test keep 2 brokers instead of bumping it to 3 -- we have

[GitHub] [kafka] rondagostino commented on pull request #10322: KAFKA-12455: OffsetValidationTest.test_broker_rolling_bounce fail: Raft

2021-03-15 Thread GitBox
rondagostino commented on pull request #10322: URL: https://github.com/apache/kafka/pull/10322#issuecomment-799777289 This patch needs to be cherry-picked to 2.8 This is an automated message from the Apache Git Service. To

[GitHub] [kafka] rondagostino opened a new pull request #10322: KAFKA-12455: OffsetValidationTest.test_broker_rolling_bounce fail: Raft

2021-03-15 Thread GitBox
rondagostino opened a new pull request #10322: URL: https://github.com/apache/kafka/pull/10322 OffsetValidationTest.test_broker_rolling_bounce was failing when used with a Raft-based metadata quorum but succeeding with a ZooKeeper-based quorum. The problem was that in the Raft case the

  1   2   >