[GitHub] [kafka] dajac commented on pull request #10558: KAFKA-12684: Fix noop set is incorrectly replaced with succeeded set from LeaderElectionCommand

2021-04-18 Thread GitBox
dajac commented on pull request #10558: URL: https://github.com/apache/kafka/pull/10558#issuecomment-822213853 @wenbingshen Thanks for the patch. Should we add a unit test to verify this or perhaps extend an existing one? -- This is an automated message from the Apache Git Service. To re

[GitHub] [kafka] wenbingshen commented on pull request #10551: MINOR:Remove nonsense test line from TopicCommandTest

2021-04-18 Thread GitBox
wenbingshen commented on pull request #10551: URL: https://github.com/apache/kafka/pull/10551#issuecomment-822208422 > @wenbingshen thanks for your patch. LGTM > > Could you revise the title of this PR? I'd like to merge it tomorrow. Of course, I will modify it now, thank y

[GitHub] [kafka] wenbingshen commented on a change in pull request #10556: MINOR: Remove redundant code from BrokerApiVersionsCommand

2021-04-18 Thread GitBox
wenbingshen commented on a change in pull request #10556: URL: https://github.com/apache/kafka/pull/10556#discussion_r615572667 ## File path: core/src/main/scala/kafka/admin/BrokerApiVersionsCommand.scala ## @@ -17,32 +17,27 @@ package kafka.admin -import java.io.PrintStre

[GitHub] [kafka] showuon commented on a change in pull request #10552: KAFKA-12675: improve the sticky general assignor scalability and performance

2021-04-18 Thread GitBox
showuon commented on a change in pull request #10552: URL: https://github.com/apache/kafka/pull/10552#discussion_r615228600 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractStickyAssignor.java ## @@ -387,58 +398,121 @@ private boolean all

[GitHub] [kafka] showuon commented on a change in pull request #10552: KAFKA-12675: improve the sticky general assignor scalability and performance

2021-04-18 Thread GitBox
showuon commented on a change in pull request #10552: URL: https://github.com/apache/kafka/pull/10552#discussion_r615227866 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractStickyAssignor.java ## @@ -313,26 +312,24 @@ private boolean allS

[GitHub] [kafka] showuon commented on pull request #10552: KAFKA-12675: improve the sticky general assignor scalability and performance

2021-04-18 Thread GitBox
showuon commented on pull request #10552: URL: https://github.com/apache/kafka/pull/10552#issuecomment-822200511 The performance comparison in jenkins for uniform subscription and non-equal subscription with the setting: ``` topicCount = 500; partitionCount = 2000; consumerCount

[GitHub] [kafka] chia7712 commented on a change in pull request #10556: MINOR: Remove redundant code from BrokerApiVersionsCommand

2021-04-18 Thread GitBox
chia7712 commented on a change in pull request #10556: URL: https://github.com/apache/kafka/pull/10556#discussion_r615554096 ## File path: core/src/main/scala/kafka/admin/BrokerApiVersionsCommand.scala ## @@ -17,32 +17,27 @@ package kafka.admin -import java.io.PrintStream

[GitHub] [kafka] wenbingshen commented on a change in pull request #10551: MINOR:Remove nonsense test line from TopicCommandTest

2021-04-18 Thread GitBox
wenbingshen commented on a change in pull request #10551: URL: https://github.com/apache/kafka/pull/10551#discussion_r615547868 ## File path: core/src/test/scala/unit/kafka/admin/TopicCommandWithAdminClientTest.scala ## @@ -465,7 +465,7 @@ class TopicCommandWithAdminClientTest

[GitHub] [kafka] wenbingshen commented on a change in pull request #10556: MINOR: Remove redundant code from BrokerApiVersionsCommand

2021-04-18 Thread GitBox
wenbingshen commented on a change in pull request #10556: URL: https://github.com/apache/kafka/pull/10556#discussion_r615545404 ## File path: core/src/main/scala/kafka/admin/BrokerApiVersionsCommand.scala ## @@ -17,32 +17,27 @@ package kafka.admin -import java.io.PrintStre

[GitHub] [kafka] showuon commented on pull request #10559: MINOR: diable RaftClusterTest first

2021-04-18 Thread GitBox
showuon commented on pull request #10559: URL: https://github.com/apache/kafka/pull/10559#issuecomment-822144126 @cmccabe , I think the root cause might need some time to get fix. Do you think we should disable the tests first? Thanks. -- This is an automated message from the Apache Git

[GitHub] [kafka] showuon commented on a change in pull request #10559: MINOR: diable RaftClusterTest first

2021-04-18 Thread GitBox
showuon commented on a change in pull request #10559: URL: https://github.com/apache/kafka/pull/10559#discussion_r615524139 ## File path: core/src/test/scala/integration/kafka/server/RaftClusterTest.scala ## @@ -22,14 +22,16 @@ import kafka.utils.TestUtils import org.apache.ka

[GitHub] [kafka] showuon opened a new pull request #10559: MINOR: diable RaftClusterTest first

2021-04-18 Thread GitBox
showuon opened a new pull request #10559: URL: https://github.com/apache/kafka/pull/10559 Disable the flaky RaftClusterTest tests before the root cause KAFKA-12677 got fixed. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Ve

[jira] [Comment Edited] (KAFKA-12675) Improve sticky general assignor scalability and performance

2021-04-18 Thread Travis Bischel (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17324358#comment-17324358 ] Travis Bischel edited comment on KAFKA-12675 at 4/19/21, 3:33 AM:

[jira] [Assigned] (KAFKA-12485) Speed up Consumer#committed by returning cached offsets for owned partitions

2021-04-18 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen reassigned KAFKA-12485: - Assignee: Luke Chen > Speed up Consumer#committed by returning cached offsets for owned partiti

[GitHub] [kafka] chia7712 commented on a change in pull request #10556: MINOR: Remove redundant code from BrokerApiVersionsCommand

2021-04-18 Thread GitBox
chia7712 commented on a change in pull request #10556: URL: https://github.com/apache/kafka/pull/10556#discussion_r615517933 ## File path: core/src/main/scala/kafka/admin/BrokerApiVersionsCommand.scala ## @@ -17,32 +17,27 @@ package kafka.admin -import java.io.PrintStream

[jira] [Assigned] (KAFKA-12684) The valid partition list is incorrectly replaced by the successfully elected partition list

2021-04-18 Thread Wenbing Shen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenbing Shen reassigned KAFKA-12684: Assignee: Wenbing Shen > The valid partition list is incorrectly replaced by the successf

[GitHub] [kafka] wenbingshen commented on pull request #10558: KAFKA-12684: Fix noop set is incorrectly replaced with succeeded set from LeaderElectionCommand

2021-04-18 Thread GitBox
wenbingshen commented on pull request #10558: URL: https://github.com/apache/kafka/pull/10558#issuecomment-822129034 @dajac @chia7712 Can you help review it? Thanks. :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [kafka] wenbingshen opened a new pull request #10558: KAFKA-12684: Fix noop set is incorrectly replaced with succeeded set from LeaderElectionCommand

2021-04-18 Thread GitBox
wenbingshen opened a new pull request #10558: URL: https://github.com/apache/kafka/pull/10558 When using the kafka-election-tool for preferred replica election, if there are partitions in the elected list that are in the preferred replica, the list of partitions already in the preferred re

[jira] [Created] (KAFKA-12684) The valid partition list is incorrectly replaced by the successfully elected partition list

2021-04-18 Thread Wenbing Shen (Jira)
Wenbing Shen created KAFKA-12684: Summary: The valid partition list is incorrectly replaced by the successfully elected partition list Key: KAFKA-12684 URL: https://issues.apache.org/jira/browse/KAFKA-12684

[GitHub] [kafka] highluck commented on pull request #9640: KAFKA-10283; Consolidate client-level and consumer-level assignment within ClientState

2021-04-18 Thread GitBox
highluck commented on pull request #9640: URL: https://github.com/apache/kafka/pull/9640#issuecomment-822113308 @guozhangwang There was an error in the code. Now I have fixed it and the tests are running fine. Thanks for review! -- This is an automated message from the Apache Git

[GitHub] [kafka] guozhangwang commented on pull request #9640: KAFKA-10283; Consolidate client-level and consumer-level assignment within ClientState

2021-04-18 Thread GitBox
guozhangwang commented on pull request #9640: URL: https://github.com/apache/kafka/pull/9640#issuecomment-822036798 @highluck I read through the code, and it looks good to me overall. However the unit test failure seems relevant: ``` org.apache.kafka.streams.processor.internals.

[GitHub] [kafka] guozhangwang commented on pull request #9414: KAFKA-10585: Kafka Streams should clean up the state store directory from cleanup

2021-04-18 Thread GitBox
guozhangwang commented on pull request #9414: URL: https://github.com/apache/kafka/pull/9414#issuecomment-822036454 @dongjinleekr @vvcephei just checking are you still working on it? I saw @vvcephei has approved the PR but it was not yet merged, hence asking. -- This is an automated mess

[GitHub] [kafka] guozhangwang commented on pull request #9640: KAFKA-10283; Consolidate client-level and consumer-level assignment within ClientState

2021-04-18 Thread GitBox
guozhangwang commented on pull request #9640: URL: https://github.com/apache/kafka/pull/9640#issuecomment-822036332 > @guozhangwang @highluck Sorry, I've been swamped with other things lately. If you have a minute to review it yourself Guozhang I would appreciate it! Also ping @cadonna or

[GitHub] [kafka] guozhangwang merged pull request #10537: KAFKA-10847: Delete Time-ordered duplicated records using deleteRange() internally

2021-04-18 Thread GitBox
guozhangwang merged pull request #10537: URL: https://github.com/apache/kafka/pull/10537 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, p

[GitHub] [kafka] guozhangwang commented on pull request #10537: KAFKA-10847: Delete Time-ordered duplicated records using deleteRange() internally

2021-04-18 Thread GitBox
guozhangwang commented on pull request #10537: URL: https://github.com/apache/kafka/pull/10537#issuecomment-822036117 Merged to trunk, thanks @spena ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

[jira] [Commented] (KAFKA-12669) Add deleteRange to WindowStore / KeyValueStore interfaces

2021-04-18 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17324575#comment-17324575 ] Guozhang Wang commented on KAFKA-12669: --- Hi [~dongjin] , thanks for your interests

[jira] [Commented] (KAFKA-9168) Integrate JNI direct buffer support to RocksDBStore

2021-04-18 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17324574#comment-17324574 ] Guozhang Wang commented on KAFKA-9168: -- Hi [~sagarrao] just for my own education cou

[GitHub] [kafka] guozhangwang commented on pull request #9441: KAFKA-10614: Ensure group state (un)load is executed in the submitted order

2021-04-18 Thread GitBox
guozhangwang commented on pull request #9441: URL: https://github.com/apache/kafka/pull/9441#issuecomment-822034184 @tombentley could you see if @hachikuji 's comments can be addressed? This is a pretty tricky bug that I would like to get fixed in 3.0. Thanks! -- This is an automated mes

[GitHub] [kafka] guozhangwang commented on pull request #10557: KAFKA-12683: Remove deprecated UsePreviousTimeOnInvalidTimestamp

2021-04-18 Thread GitBox
guozhangwang commented on pull request #10557: URL: https://github.com/apache/kafka/pull/10557#issuecomment-822032717 ping @abbccdda for reviews. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] guozhangwang opened a new pull request #10557: KAFKA-12683: Remove deprecated UsePreviousTimeOnInvalidTimestamp

2021-04-18 Thread GitBox
guozhangwang opened a new pull request #10557: URL: https://github.com/apache/kafka/pull/10557 ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including

[jira] [Created] (KAFKA-12683) Remove deprecated "UsePreviousTimeOnInvalidTimeStamp"

2021-04-18 Thread Guozhang Wang (Jira)
Guozhang Wang created KAFKA-12683: - Summary: Remove deprecated "UsePreviousTimeOnInvalidTimeStamp" Key: KAFKA-12683 URL: https://issues.apache.org/jira/browse/KAFKA-12683 Project: Kafka Issue

[jira] [Resolved] (KAFKA-12633) Remove deprecated "TopologyTestDriver#pipeInput / readOutput"

2021-04-18 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-12633. --- Fix Version/s: 3.0.0 Resolution: Fixed > Remove deprecated "TopologyTestDriver#pipeIn

[GitHub] [kafka] guozhangwang merged pull request #10508: KAFKA-12633: Remove deprecated APIs in TopologyTestDriver

2021-04-18 Thread GitBox
guozhangwang merged pull request #10508: URL: https://github.com/apache/kafka/pull/10508 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, p

[GitHub] [kafka] guozhangwang commented on pull request #10508: KAFKA-12633: Remove deprecated APIs in TopologyTestDriver

2021-04-18 Thread GitBox
guozhangwang commented on pull request #10508: URL: https://github.com/apache/kafka/pull/10508#issuecomment-822030658 > Looks like there are some related test failures, though. Thanks! I have fixed the unit test, will merge after confirming green builds locally. -- This is an auto

[jira] [Commented] (KAFKA-10800) Validate the snapshot id when the state machine creates a snapshot

2021-04-18 Thread Haoran Xuan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17324536#comment-17324536 ] Haoran Xuan commented on KAFKA-10800: - Hi, [~jagsancio], seems that the dependencies

[jira] [Commented] (KAFKA-12629) Flaky Test RaftClusterTest

2021-04-18 Thread Wenbing Shen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17324517#comment-17324517 ] Wenbing Shen commented on KAFKA-12629: -- Found another error log today:  One of the

[jira] [Updated] (KAFKA-12682) Kraft MetadataPartitionsBuilder _localChanged and _localRemoved out of order

2021-04-18 Thread jacky (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jacky updated KAFKA-12682: -- Description: In version 2.8, MetadataPartitionsBuilder has the field _localChanged and _localRemoved which re

[jira] [Reopened] (KAFKA-12682) Kraft MetadataPartitionsBuilder _localChanged and _localRemoved out of order

2021-04-18 Thread jacky (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jacky reopened KAFKA-12682: --- > Kraft MetadataPartitionsBuilder _localChanged and _localRemoved out of order > --

[jira] [Resolved] (KAFKA-12682) Kraft MetadataPartitionsBuilder _localChanged and _localRemoved out of order

2021-04-18 Thread jacky (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jacky resolved KAFKA-12682. --- Resolution: Not A Problem > Kraft MetadataPartitionsBuilder _localChanged and _localRemoved out of order >

[GitHub] [kafka] wenbingshen commented on pull request #10556: MINOR: Remove redundant code from BrokerApiVersionsCommand

2021-04-18 Thread GitBox
wenbingshen commented on pull request #10556: URL: https://github.com/apache/kafka/pull/10556#issuecomment-821975837 Dear @ijuma @chia7712 Can you take a look at it?Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [kafka] wenbingshen opened a new pull request #10556: MINOR: Remove redundant code from BrokerApiVersionsCommand

2021-04-18 Thread GitBox
wenbingshen opened a new pull request #10556: URL: https://github.com/apache/kafka/pull/10556 Remove some redundant parameters and configurations ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI buil

[GitHub] [kafka] dejan2609 commented on pull request #10466: KAFKA-12417 "streams" module: switch deprecated Gradle configuration (`testRuntime` -->> `testRuntimeClasspath`)

2021-04-18 Thread GitBox
dejan2609 commented on pull request #10466: URL: https://github.com/apache/kafka/pull/10466#issuecomment-821968689 > Could we reintroduce a configuration that behaves the same as `testRuntime` did before it was removed? @ijuma Can you please expand on this ? Are you referring to a `

[jira] [Reopened] (KAFKA-12682) Kraft MetadataPartitionsBuilder _localChanged and _localRemoved out of order

2021-04-18 Thread jacky (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jacky reopened KAFKA-12682: --- > Kraft MetadataPartitionsBuilder _localChanged and _localRemoved out of order > --

[jira] [Resolved] (KAFKA-12682) Kraft MetadataPartitionsBuilder _localChanged and _localRemoved out of order

2021-04-18 Thread jacky (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jacky resolved KAFKA-12682. --- Resolution: Not A Problem > Kraft MetadataPartitionsBuilder _localChanged and _localRemoved out of order >

[GitHub] [kafka] IUSR opened a new pull request #10555: MINOR: adjust HashMap init capacity to avoid unnecessary resizing

2021-04-18 Thread GitBox
IUSR opened a new pull request #10555: URL: https://github.com/apache/kafka/pull/10555 * The initial capacities for some `HashMap` objects are less than expected and result in resizing. * Increase the initial capacity, similar to what `java.util.HashSet` does. -- This is an automa

[GitHub] [kafka] satishd commented on pull request #10271: KAFKA-12429: Added serdes for the default implementation of RLMM based on an internal topic as storage.

2021-04-18 Thread GitBox
satishd commented on pull request #10271: URL: https://github.com/apache/kafka/pull/10271#issuecomment-821960939 Thanks @junrao for your comments. Addressed them with commit [e909c7f](https://github.com/apache/kafka/pull/10271/commits/e909c7fc481945e54b7e364d2f52c9d191f988ec). -- This is

[GitHub] [kafka] satishd commented on a change in pull request #10271: KAFKA-12429: Added serdes for the default implementation of RLMM based on an internal topic as storage.

2021-04-18 Thread GitBox
satishd commented on a change in pull request #10271: URL: https://github.com/apache/kafka/pull/10271#discussion_r615369734 ## File path: storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/serialization/RemoteLogMetadataContextSerdes.java ## @@ -0,0 +1,1

[GitHub] [kafka] satishd commented on a change in pull request #10271: KAFKA-12429: Added serdes for the default implementation of RLMM based on an internal topic as storage.

2021-04-18 Thread GitBox
satishd commented on a change in pull request #10271: URL: https://github.com/apache/kafka/pull/10271#discussion_r615369347 ## File path: storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/RemoteLogMetadataContext.java ## @@ -0,0 +1,83 @@ +/* + * License

[GitHub] [kafka] satishd commented on a change in pull request #10271: KAFKA-12429: Added serdes for the default implementation of RLMM based on an internal topic as storage.

2021-04-18 Thread GitBox
satishd commented on a change in pull request #10271: URL: https://github.com/apache/kafka/pull/10271#discussion_r615368480 ## File path: storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/serialization/RemoteLogMetadataSerdes.java ## @@ -0,0 +1,56 @@ +/

[jira] [Updated] (KAFKA-12682) Kraft MetadataPartitionsBuilder _localChanged and _localRemoved out of order

2021-04-18 Thread jacky (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jacky updated KAFKA-12682: -- Description: In version 2.8, MetadataPartitionsBuilder has the field _localChanged and _localRemoved which re

[jira] [Updated] (KAFKA-12682) Kraft MetadataPartitionsBuilder _localChanged and _localRemoved out of order

2021-04-18 Thread jacky (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jacky updated KAFKA-12682: -- Description: In version 2.8, MetadataPartitionsBuilder has the field _localChanged and _localRemoved which re