[jira] [Commented] (KAFKA-12713) Report "REAL" follower/consumer fetch latency

2021-07-29 Thread Ming Liu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17390068#comment-17390068 ] Ming Liu commented on KAFKA-12713: -- Certainly, [~kaihuang], please take the ownership of the KIP and

[jira] [Commented] (KAFKA-12713) Report "REAL" follower/consumer fetch latency

2021-05-11 Thread Ming Liu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342778#comment-17342778 ] Ming Liu commented on KAFKA-12713: -- Here is PR [https://github.com/apache/kafka/pull/10674/files] for

[jira] [Comment Edited] (KAFKA-12713) Report "REAL" follower/consumer fetch latency

2021-04-26 Thread Ming Liu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17331590#comment-17331590 ] Ming Liu edited comment on KAFKA-12713 at 4/27/21, 3:25 AM: The idea is: #

[jira] [Comment Edited] (KAFKA-12713) Report "REAL" follower/consumer fetch latency

2021-04-26 Thread Ming Liu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17331590#comment-17331590 ] Ming Liu edited comment on KAFKA-12713 at 4/27/21, 12:17 AM: - The idea is:

[jira] [Updated] (KAFKA-12713) Report "REAL" follower/consumer fetch latency

2021-04-26 Thread Ming Liu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Liu updated KAFKA-12713: - Summary: Report "REAL" follower/consumer fetch latency (was: Report "REAL" broker/consumer fetch

[jira] [Comment Edited] (KAFKA-12713) Report "REAL" broker/consumer fetch latency

2021-04-25 Thread Ming Liu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17331590#comment-17331590 ] Ming Liu edited comment on KAFKA-12713 at 4/25/21, 11:16 PM: - The idea is:

[jira] [Comment Edited] (KAFKA-12713) Report "REAL" broker/consumer fetch latency

2021-04-25 Thread Ming Liu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17331590#comment-17331590 ] Ming Liu edited comment on KAFKA-12713 at 4/25/21, 5:33 PM: The idea I am

[jira] [Commented] (KAFKA-12713) Report "REAL" broker/consumer fetch latency

2021-04-25 Thread Ming Liu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17331590#comment-17331590 ] Ming Liu commented on KAFKA-12713: -- The idea is like this,  we can: 1. Add waitTimeMS in FetchResponse.

[jira] [Updated] (KAFKA-12713) Report "REAL" broker/consumer fetch latency

2021-04-24 Thread Ming Liu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Liu updated KAFKA-12713: - Summary: Report "REAL" broker/consumer fetch latency (was: Report broker/consumer fetch latency more

[jira] [Created] (KAFKA-12713) Report broker/consumer fetch latency more accurately

2021-04-24 Thread Ming Liu (Jira)
Ming Liu created KAFKA-12713: Summary: Report broker/consumer fetch latency more accurately Key: KAFKA-12713 URL: https://issues.apache.org/jira/browse/KAFKA-12713 Project: Kafka Issue Type: Bug

[jira] [Commented] (KAFKA-7918) Streams store cleanup: inline byte-store generic parameters

2020-11-30 Thread Ming Liu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17240908#comment-17240908 ] Ming Liu commented on KAFKA-7918: - We also had the code to optimize the changelog supporting it.  We only

[jira] [Commented] (KAFKA-7918) Streams store cleanup: inline byte-store generic parameters

2020-11-23 Thread Ming Liu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17237861#comment-17237861 ] Ming Liu commented on KAFKA-7918: - [~ableegoldman], [~guozhang] [~mjsax],  one good benefit of generics

[jira] [Commented] (KAFKA-8733) Offline partitions occur when leader's disk is slow in reads while responding to follower fetch requests.

2020-10-24 Thread Ming Liu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220139#comment-17220139 ] Ming Liu commented on KAFKA-8733: - One observation is after moving to 2.5 (so 

[jira] [Commented] (KAFKA-10398) Intra-broker disk move failed with onPartitionFenced()

2020-08-13 Thread Ming Liu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177132#comment-17177132 ] Ming Liu commented on KAFKA-10398: -- [~lukech], that is another problem blocking the disk move. I wonder

[jira] [Updated] (KAFKA-10398) Intra-broker disk move failed with onPartitionFenced()

2020-08-13 Thread Ming Liu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Liu updated KAFKA-10398: - Description: When I tried the intra-broker disk move on 2.5.0, it always failed quickly in

[jira] [Created] (KAFKA-10398) Intra-broker disk move failed with onPartitionFenced()

2020-08-13 Thread Ming Liu (Jira)
Ming Liu created KAFKA-10398: Summary: Intra-broker disk move failed with onPartitionFenced() Key: KAFKA-10398 URL: https://issues.apache.org/jira/browse/KAFKA-10398 Project: Kafka Issue Type:

[jira] [Commented] (KAFKA-8362) LogCleaner gets stuck after partition move between log directories

2020-08-13 Thread Ming Liu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177123#comment-17177123 ] Ming Liu commented on KAFKA-8362: - Awesome, what is the patch link? > LogCleaner gets stuck after

[jira] [Commented] (KAFKA-8362) LogCleaner gets stuck after partition move between log directories

2020-08-12 Thread Ming Liu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17176476#comment-17176476 ] Ming Liu commented on KAFKA-8362: - In that case, we need to modify *allCleanerCheckpoints* 

[jira] [Comment Edited] (KAFKA-8733) Offline partitions occur when leader's disk is slow in reads while responding to follower fetch requests.

2020-07-25 Thread Ming Liu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17098264#comment-17098264 ] Ming Liu edited comment on KAFKA-8733 at 7/25/20, 3:03 PM: --- We (at Twitter)

[jira] [Commented] (KAFKA-10242) Adding metrics to track the total count of idempotent producers that Broker need to track

2020-07-07 Thread Ming Liu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17153091#comment-17153091 ] Ming Liu commented on KAFKA-10242: -- Yes, correct, then it should be 2.7.0 instead of 2.6.0  > Adding

[jira] [Created] (KAFKA-10242) Adding metrics to track the total count of idempotent producers that Broker need to track

2020-07-06 Thread Ming Liu (Jira)
Ming Liu created KAFKA-10242: Summary: Adding metrics to track the total count of idempotent producers that Broker need to track Key: KAFKA-10242 URL: https://issues.apache.org/jira/browse/KAFKA-10242

[jira] [Commented] (KAFKA-10242) Adding metrics to track the total count of idempotent producers that Broker need to track

2020-07-06 Thread Ming Liu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17152494#comment-17152494 ] Ming Liu commented on KAFKA-10242: -- The PR is [https://github.com/apache/kafka/pull/8982] > Adding

[jira] [Commented] (KAFKA-8362) LogCleaner gets stuck after partition move between log directories

2020-06-14 Thread Ming Liu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17135400#comment-17135400 ] Ming Liu commented on KAFKA-8362: -  In 2.5 Kafka, this still easily repro for me. With compacted topic,

[jira] [Commented] (KAFKA-8733) Offline partitions occur when leader's disk is slow in reads while responding to follower fetch requests.

2020-05-03 Thread Ming Liu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17098264#comment-17098264 ] Ming Liu commented on KAFKA-8733: - We (at Twitter) also saw this issue almost every month and it is

[jira] [Commented] (KAFKA-8325) Remove from the incomplete set failed. This should be impossible

2019-05-13 Thread Ming Liu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-8325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16838718#comment-16838718 ] Ming Liu commented on KAFKA-8325: - We also run into this problem after we upgrade to Kafka 2.2. > Remove

[jira] [Commented] (KAFKA-7698) Kafka Broker fail to start: ProducerFencedException thrown from producerstatemanager.scala!checkProducerEpoch

2018-12-03 Thread Ming Liu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16707621#comment-16707621 ] Ming Liu commented on KAFKA-7698: - The problem is seems coming from appendEndTxnMarker of

[jira] [Created] (KAFKA-7698) Kafka Broker fail to start: ProducerFencedException thrown from producerstatemanager.scala!checkProducerEpoch

2018-12-03 Thread Ming Liu (JIRA)
Ming Liu created KAFKA-7698: --- Summary: Kafka Broker fail to start: ProducerFencedException thrown from producerstatemanager.scala!checkProducerEpoch Key: KAFKA-7698 URL:

[jira] [Commented] (KAFKA-7693) "IllegalArgumentException: Invalid negative sequence number used" in Kafka Client

2018-12-02 Thread Ming Liu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16706463#comment-16706463 ] Ming Liu commented on KAFKA-7693: - With some further debugging, the bug seems at client 

[jira] [Updated] (KAFKA-7693) "IllegalArgumentException: Invalid negative sequence number used" in Kafka Client

2018-12-02 Thread Ming Liu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Liu updated KAFKA-7693: Description: When operating Kafka, we have seen Kafka client logging this kind of exception: {noformat}

[jira] [Comment Edited] (KAFKA-7692) updateFirstUnstableOffset NPE due to sequenceId overflow in ProducerStateManager.append

2018-12-02 Thread Ming Liu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16706462#comment-16706462 ] Ming Liu edited comment on KAFKA-7692 at 12/2/18 8:33 PM: -- Upon further

[jira] [Created] (KAFKA-7693) "IllegalArgumentException: Invalid negative sequence number used" in Kafka Client

2018-12-02 Thread Ming Liu (JIRA)
Ming Liu created KAFKA-7693: --- Summary: "IllegalArgumentException: Invalid negative sequence number used" in Kafka Client Key: KAFKA-7693 URL: https://issues.apache.org/jira/browse/KAFKA-7693 Project: Kafka

[jira] [Updated] (KAFKA-7692) updateFirstUnstableOffset NPE due to sequenceId overflow in ProducerStateManager.append

2018-12-02 Thread Ming Liu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Liu updated KAFKA-7692: Description: When operating Kafka, we frequently saw this exception on Kafka server log,  {noformat}

[jira] [Commented] (KAFKA-7692) updateFirstUnstableOffset NPE due to sequenceId overflow in ProducerStateManager.append

2018-12-02 Thread Ming Liu (JIRA)
[ https://issues.apache.org/jira/browse/KAFKA-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16706462#comment-16706462 ] Ming Liu commented on KAFKA-7692: - Upon the close analysis, the bug is in ProducerStateManager.append,

[jira] [Created] (KAFKA-7692) updateFirstUnstableOffset NPE due to sequenceId overflow in ProducerStateManager.append

2018-12-02 Thread Ming Liu (JIRA)
Ming Liu created KAFKA-7692: --- Summary: updateFirstUnstableOffset NPE due to sequenceId overflow in ProducerStateManager.append Key: KAFKA-7692 URL: https://issues.apache.org/jira/browse/KAFKA-7692 Project: