[GitHub] [kafka] chia7712 commented on pull request #9590: KAFKA-7556: KafkaConsumer.beginningOffsets does not return actual first offsets

2020-11-19 Thread GitBox
chia7712 commented on pull request #9590: URL: https://github.com/apache/kafka/pull/9590#issuecomment-730231950 (I'm still reading this story so pardon me for asking stupid question.) If using timestamp=0 can bring correct offset, why not following that way to handle ```ListOffsetReq

[jira] [Commented] (KAFKA-10656) NetworkClient.java: print out the feature flags received at DEBUG level, as well as the other version information

2020-11-19 Thread Tom Bentley (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17235305#comment-17235305 ] Tom Bentley commented on KAFKA-10656: - [~cmccabe] I opened a PR for this, please cou

[GitHub] [kafka] dajac commented on a change in pull request #9619: MINOR: Reduce sends created by `SendBuilder`

2020-11-19 Thread GitBox
dajac commented on a change in pull request #9619: URL: https://github.com/apache/kafka/pull/9619#discussion_r526710893 ## File path: clients/src/main/java/org/apache/kafka/common/protocol/SendBuilder.java ## @@ -122,6 +123,14 @@ public void writeVarlong(long i) { Byt

[GitHub] [kafka] dajac commented on a change in pull request #7498: KAFKA-9023: Log request destination when the Producer gets disconnected

2020-11-19 Thread GitBox
dajac commented on a change in pull request #7498: URL: https://github.com/apache/kafka/pull/7498#discussion_r526718951 ## File path: clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java ## @@ -659,6 +655,12 @@ else if (error == Errors.CLUSTER_AUTHORIZ

[GitHub] [kafka] dajac commented on pull request #7498: KAFKA-9023: Log request destination when the Producer gets disconnected

2020-11-19 Thread GitBox
dajac commented on pull request #7498: URL: https://github.com/apache/kafka/pull/7498#issuecomment-730250619 I have triggered another run of the CI. I will merge it afterwards. This is an automated message from the Apache Git

[GitHub] [kafka] dajac commented on a change in pull request #9386: KAFKA-10024: Add dynamic configuration and enforce quota for per-IP connection rate limits (KIP-612, part 2)

2020-11-19 Thread GitBox
dajac commented on a change in pull request #9386: URL: https://github.com/apache/kafka/pull/9386#discussion_r526729269 ## File path: core/src/main/scala/kafka/network/SocketServer.scala ## @@ -1324,7 +1404,60 @@ class ConnectionQuotas(config: KafkaConfig, time: Time, metrics:

[jira] [Commented] (KAFKA-10114) Kafka producer stuck after broker crash

2020-11-19 Thread Tim Fox (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17235343#comment-17235343 ] Tim Fox commented on KAFKA-10114: - > We are still seeing this issue with version 2.6.0.

[GitHub] [kafka] dajac commented on pull request #9547: KAFKA-9630; Replace OffsetsForLeaderEpoch request/response with automated protocol

2020-11-19 Thread GitBox
dajac commented on pull request #9547: URL: https://github.com/apache/kafka/pull/9547#issuecomment-730285178 Failed test seems unrelated to this PR. Merging to trunk. This is an automated message from the Apache Git Service.

[GitHub] [kafka] dajac merged pull request #9547: KAFKA-9630; Replace OffsetsForLeaderEpoch request/response with automated protocol

2020-11-19 Thread GitBox
dajac merged pull request #9547: URL: https://github.com/apache/kafka/pull/9547 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[jira] [Resolved] (KAFKA-9630) Replace OffsetsForLeaderEpoch request/response with automated protocol

2020-11-19 Thread David Jacot (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-9630. Fix Version/s: 2.8.0 Resolution: Fixed > Replace OffsetsForLeaderEpoch request/response with

[GitHub] [kafka] dajac commented on pull request #9601: KAFKA-10729: Bump remaining RPC's to use tagged fields.

2020-11-19 Thread GitBox
dajac commented on pull request #9601: URL: https://github.com/apache/kafka/pull/9601#issuecomment-730287841 @gardnervickers https://github.com/apache/kafka/pull/9401 and https://github.com/apache/kafka/pull/9547 have been merged. You can bring them back in this PR if you like. -

[GitHub] [kafka] dajac commented on pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-11-19 Thread GitBox
dajac commented on pull request #9526: URL: https://github.com/apache/kafka/pull/9526#issuecomment-730292123 @anatasiavela Both PRs have been merged so we can proceed with this one. There is something that we must consider that I was not aware of: https://github.com/apache/kafka/blob

[jira] [Comment Edited] (KAFKA-10114) Kafka producer stuck after broker crash

2020-11-19 Thread Tim Fox (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17235343#comment-17235343 ] Tim Fox edited comment on KAFKA-10114 at 11/19/20, 11:09 AM: -

[jira] [Comment Edited] (KAFKA-10114) Kafka producer stuck after broker crash

2020-11-19 Thread Tim Fox (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17235343#comment-17235343 ] Tim Fox edited comment on KAFKA-10114 at 11/19/20, 11:14 AM: -

[GitHub] [kafka] bristy opened a new pull request #9621: KAFKA-9892: Producer state snapshot needs to be forced to disk

2020-11-19 Thread GitBox
bristy opened a new pull request #9621: URL: https://github.com/apache/kafka/pull/9621 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* FileChannel.close() does n

[jira] [Comment Edited] (KAFKA-10114) Kafka producer stuck after broker crash

2020-11-19 Thread Tim Fox (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17235343#comment-17235343 ] Tim Fox edited comment on KAFKA-10114 at 11/19/20, 11:53 AM: -

[GitHub] [kafka] dengziming opened a new pull request #9622: KAFKA-10547; add topicId in MetadataResp

2020-11-19 Thread GitBox
dengziming opened a new pull request #9622: URL: https://github.com/apache/kafka/pull/9622 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* 1. Bump the versio

[GitHub] [kafka] dengziming commented on pull request #9622: KAFKA-10547; add topicId in MetadataResp

2020-11-19 Thread GitBox
dengziming commented on pull request #9622: URL: https://github.com/apache/kafka/pull/9622#issuecomment-730332596 @rajinisivaram @jolshan Hi, PTAL. This is an automated message from the Apache Git Service. To respond to the m

[GitHub] [kafka] tombentley opened a new pull request #9623: KAFKA-10692: Add delegation.token.secret.key, deprecate ...master.key

2020-11-19 Thread GitBox
tombentley opened a new pull request #9623: URL: https://github.com/apache/kafka/pull/9623 Add delegation.token.secret.key broker config and deprecate delegation.token.master.key as described in [KIP-681](https://cwiki.apache.org/confluence/display/KAFKA/KIP-681%3A+Rename+master+key+in+del

[GitHub] [kafka] tombentley commented on pull request #9623: KAFKA-10692: Add delegation.token.secret.key, deprecate ...master.key

2020-11-19 Thread GitBox
tombentley commented on pull request #9623: URL: https://github.com/apache/kafka/pull/9623#issuecomment-730336828 @omkreddy, @mimaison please could one of you review? This is an automated message from the Apache Git Service.

[jira] [Comment Edited] (KAFKA-10114) Kafka producer stuck after broker crash

2020-11-19 Thread Tim Fox (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17235343#comment-17235343 ] Tim Fox edited comment on KAFKA-10114 at 11/19/20, 1:17 PM:

[GitHub] [kafka] ijuma commented on a change in pull request #9435: KAFKA-10606: Disable auto topic creation for fetch-all-topic-metadata request

2020-11-19 Thread GitBox
ijuma commented on a change in pull request #9435: URL: https://github.com/apache/kafka/pull/9435#discussion_r526911200 ## File path: core/src/main/scala/kafka/server/KafkaApis.scala ## @@ -1333,7 +1343,17 @@ class KafkaApis(val requestChannel: RequestChannel, } }

[GitHub] [kafka] mumrah commented on pull request #9598: KAFKA-10701 : First line of detailed stats from consumer-perf-test.sh incorrect

2020-11-19 Thread GitBox
mumrah commented on pull request #9598: URL: https://github.com/apache/kafka/pull/9598#issuecomment-730402458 @quanuw there are some statements that expect `joinStart` to be a timestamp in order to calculate durations. When it's zero, those durations become huge which leads to subsequent c

[GitHub] [kafka] mumrah merged pull request #9598: KAFKA-10701 : First line of detailed stats from consumer-perf-test.sh incorrect

2020-11-19 Thread GitBox
mumrah merged pull request #9598: URL: https://github.com/apache/kafka/pull/9598 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[jira] [Commented] (KAFKA-9892) Producer state snapshot needs to be forced to disk

2020-11-19 Thread Brajesh Kumar (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17235516#comment-17235516 ] Brajesh Kumar commented on KAFKA-9892: --- [~ijuma]Can you please help me with build

[GitHub] [kafka] cadonna commented on a change in pull request #9614: KAFKA-10500: Add failed-stream-threads metric for adding + removing stream threads

2020-11-19 Thread GitBox
cadonna commented on a change in pull request #9614: URL: https://github.com/apache/kafka/pull/9614#discussion_r526880412 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -1070,7 +1070,9 @@ private Thread shutdownHelper(final boolean error) {

[GitHub] [kafka] mimaison merged pull request #9545: [mm2] Allow Checkpoints for consumers using static partition assignments

2020-11-19 Thread GitBox
mimaison merged pull request #9545: URL: https://github.com/apache/kafka/pull/9545 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [kafka] dajac commented on pull request #9386: KAFKA-10024: Add dynamic configuration and enforce quota for per-IP connection rate limits (KIP-612, part 2)

2020-11-19 Thread GitBox
dajac commented on pull request #9386: URL: https://github.com/apache/kafka/pull/9386#issuecomment-730447091 Builds are green. Merging to trunk. This is an automated message from the Apache Git Service. To respond to the mess

[GitHub] [kafka] ryannedolan commented on pull request #9545: [mm2] Allow Checkpoints for consumers using static partition assignments

2020-11-19 Thread GitBox
ryannedolan commented on pull request #9545: URL: https://github.com/apache/kafka/pull/9545#issuecomment-730448017 Sorry, late to the convo, but lgtm, thanks! This is an automated message from the Apache Git Service. To respo

[GitHub] [kafka] mimaison merged pull request #9623: KAFKA-10692: Add delegation.token.secret.key, deprecate ...master.key

2020-11-19 Thread GitBox
mimaison merged pull request #9623: URL: https://github.com/apache/kafka/pull/9623 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[jira] [Resolved] (KAFKA-10692) Rename broker master key config for KIP 681

2020-11-19 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mickael Maison resolved KAFKA-10692. Fix Version/s: 2.8.0 Resolution: Fixed > Rename broker master key config for KIP 68

[GitHub] [kafka] dajac merged pull request #9386: KAFKA-10024: Add dynamic configuration and enforce quota for per-IP connection rate limits (KIP-612, part 2)

2020-11-19 Thread GitBox
dajac merged pull request #9386: URL: https://github.com/apache/kafka/pull/9386 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[jira] [Resolved] (KAFKA-10024) Add dynamic configuration and enforce quota for per-IP connection rate limits (KIP-612, part 2)

2020-11-19 Thread David Jacot (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-10024. - Fix Version/s: 2.8.0 Resolution: Fixed > Add dynamic configuration and enforce quota for

[jira] [Created] (KAFKA-10746) Consumer poll timeout Expiration should be logged as WARNING not INFO.

2020-11-19 Thread Benedikt Linse (Jira)
Benedikt Linse created KAFKA-10746: -- Summary: Consumer poll timeout Expiration should be logged as WARNING not INFO. Key: KAFKA-10746 URL: https://issues.apache.org/jira/browse/KAFKA-10746 Project:

[jira] [Created] (KAFKA-10747) Implement ClientQuota APIs for altering and describing IP entity quotas

2020-11-19 Thread David Mao (Jira)
David Mao created KAFKA-10747: - Summary: Implement ClientQuota APIs for altering and describing IP entity quotas Key: KAFKA-10747 URL: https://issues.apache.org/jira/browse/KAFKA-10747 Project: Kafka

[jira] [Created] (KAFKA-10748) Add IP connection rate throttling metric

2020-11-19 Thread David Mao (Jira)
David Mao created KAFKA-10748: - Summary: Add IP connection rate throttling metric Key: KAFKA-10748 URL: https://issues.apache.org/jira/browse/KAFKA-10748 Project: Kafka Issue Type: Improvement

[GitHub] [kafka] dajac commented on pull request #7498: KAFKA-9023: Log request destination when the Producer gets disconnected

2020-11-19 Thread GitBox
dajac commented on pull request #7498: URL: https://github.com/apache/kafka/pull/7498#issuecomment-730458347 Failed tests are not related. Merging to trunk. * Build / JDK 11 / org.apache.kafka.streams.integration.EosBetaUpgradeIntegrationTest.shouldUpgradeFromEosAlphaToEosBeta[true] *

[jira] [Updated] (KAFKA-10746) Consumer poll timeout Expiration should be logged as WARNING not INFO.

2020-11-19 Thread Benedikt Linse (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedikt Linse updated KAFKA-10746: --- Description: When a consumer does not poll regularly, and the `max.poll.interval.ms` thresh

[GitHub] [kafka] dajac merged pull request #7498: KAFKA-9023: Log request destination when the Producer gets disconnected

2020-11-19 Thread GitBox
dajac merged pull request #7498: URL: https://github.com/apache/kafka/pull/7498 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[jira] [Resolved] (KAFKA-9023) Producer NETWORK_EXCEPTION response should log more information

2020-11-19 Thread David Jacot (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-9023. Fix Version/s: 2.8.0 Resolution: Fixed > Producer NETWORK_EXCEPTION response should log more

[GitHub] [kafka] mimaison commented on pull request #9224: KAFKA-10304: refactor MM2 integration tests

2020-11-19 Thread GitBox
mimaison commented on pull request #9224: URL: https://github.com/apache/kafka/pull/9224#issuecomment-730467364 > If we have to control the complexity, I would prefer to drop testWithBrokerRestart and keep MirrorConnectorsIntegrationSSLTest, as it makes sense to run simple validation in SS

[jira] [Updated] (KAFKA-10747) Implement ClientQuota APIs for altering and describing IP entity quotas

2020-11-19 Thread David Mao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mao updated KAFKA-10747: -- Component/s: core config > Implement ClientQuota APIs for altering and describing IP

[jira] [Updated] (KAFKA-10748) Add IP connection rate throttling metric

2020-11-19 Thread David Mao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mao updated KAFKA-10748: -- Component/s: network core > Add IP connection rate throttling metric > --

[jira] [Created] (KAFKA-10749) Add throttling of IPs by connection rate

2020-11-19 Thread David Mao (Jira)
David Mao created KAFKA-10749: - Summary: Add throttling of IPs by connection rate Key: KAFKA-10749 URL: https://issues.apache.org/jira/browse/KAFKA-10749 Project: Kafka Issue Type: New Feature

[jira] [Updated] (KAFKA-10747) Implement ClientQuota APIs for altering and describing IP entity quotas

2020-11-19 Thread David Mao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mao updated KAFKA-10747: -- Parent: KAFKA-10749 Issue Type: Sub-task (was: Improvement) > Implement ClientQuota APIs for

[jira] [Updated] (KAFKA-10748) Add IP connection rate throttling metric

2020-11-19 Thread David Mao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mao updated KAFKA-10748: -- Parent: KAFKA-10749 Issue Type: Sub-task (was: Improvement) > Add IP connection rate throttli

[jira] [Updated] (KAFKA-10024) Add dynamic configuration and enforce quota for per-IP connection rate limits (KIP-612, part 2)

2020-11-19 Thread David Mao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mao updated KAFKA-10024: -- Parent: KAFKA-10749 Issue Type: Sub-task (was: Improvement) > Add dynamic configuration and e

[GitHub] [kafka] mimaison commented on a change in pull request #9224: KAFKA-10304: refactor MM2 integration tests

2020-11-19 Thread GitBox
mimaison commented on a change in pull request #9224: URL: https://github.com/apache/kafka/pull/9224#discussion_r527005533 ## File path: connect/mirror/src/test/java/org/apache/kafka/connect/mirror/integration/MirrorConnectorsIntegrationSSLTest.java ## @@ -0,0 +1,98 @@ +/* + *

[GitHub] [kafka] tombentley commented on pull request #9433: KAFKA-10607: Consistent behaviour for response errorCounts()

2020-11-19 Thread GitBox
tombentley commented on pull request #9433: URL: https://github.com/apache/kafka/pull/9433#issuecomment-730476432 @abbccdda I think we're just waiting for your review here. Thanks. This is an automated message from the Apache

[GitHub] [kafka] chia7712 commented on pull request #9433: KAFKA-10607: Consistent behaviour for response errorCounts()

2020-11-19 Thread GitBox
chia7712 commented on pull request #9433: URL: https://github.com/apache/kafka/pull/9433#issuecomment-730478145 @tombentley Could you trigger QA again? This is an automated message from the Apache Git Service. To respond to

[GitHub] [kafka] tombentley commented on pull request #9433: KAFKA-10607: Consistent behaviour for response errorCounts()

2020-11-19 Thread GitBox
tombentley commented on pull request #9433: URL: https://github.com/apache/kafka/pull/9433#issuecomment-730479015 @chia7712 I thought only committers can do that. This is an automated message from the Apache Git Service. To r

[GitHub] [kafka] chia7712 commented on pull request #9433: KAFKA-10607: Consistent behaviour for response errorCounts()

2020-11-19 Thread GitBox
chia7712 commented on pull request #9433: URL: https://github.com/apache/kafka/pull/9433#issuecomment-730480556 IIRC, the retest command does not work currently. Could you rebase code to trigger QA? This is an automated mess

[GitHub] [kafka] jolshan commented on a change in pull request #9590: KAFKA-7556: KafkaConsumer.beginningOffsets does not return actual first offsets

2020-11-19 Thread GitBox
jolshan commented on a change in pull request #9590: URL: https://github.com/apache/kafka/pull/9590#discussion_r527020272 ## File path: core/src/main/scala/kafka/log/Log.scala ## @@ -1749,6 +1752,9 @@ class Log(@volatile private var _dir: File, checkIfMemoryMappedBuf

[GitHub] [kafka] jolshan commented on pull request #9590: KAFKA-7556: KafkaConsumer.beginningOffsets does not return actual first offsets

2020-11-19 Thread GitBox
jolshan commented on pull request #9590: URL: https://github.com/apache/kafka/pull/9590#issuecomment-730487839 @chia7712 The code path you linked is the code path beginningOffsets uses. Are you suggesting removing `ListOffsetRequest.EARLIEST_TIMESTAMP` and replacing it with 0? --

[GitHub] [kafka] chia7712 commented on pull request #9590: KAFKA-7556: KafkaConsumer.beginningOffsets does not return actual first offsets

2020-11-19 Thread GitBox
chia7712 commented on pull request #9590: URL: https://github.com/apache/kafka/pull/9590#issuecomment-730491021 > Are you suggesting removing ListOffsetRequest.EARLIEST_TIMESTAMP and replacing it with 0? yep, it seems like a simple solution without much changes. However, I have not

[GitHub] [kafka] wcarlson5 commented on pull request #9615: KAFKA-10500: Add thread option

2020-11-19 Thread GitBox
wcarlson5 commented on pull request #9615: URL: https://github.com/apache/kafka/pull/9615#issuecomment-730500854 @cadonna @mjsax @vvcephei Part 3 of KIP-663 add threads is ready for review This is an automated message from th

[GitHub] [kafka] jolshan commented on pull request #9590: KAFKA-7556: KafkaConsumer.beginningOffsets does not return actual first offsets

2020-11-19 Thread GitBox
jolshan commented on pull request #9590: URL: https://github.com/apache/kafka/pull/9590#issuecomment-730503145 @chia7712 I did think about this solution initially. I'm wondering if we do want to update the segment offsets and logStartOffsets correctly though. If that isn't as important the

[jira] [Assigned] (KAFKA-10549) Add topic ID support to DeleteTopics,ListOffsets, OffsetForLeaders, StopReplica

2020-11-19 Thread Justine Olshan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justine Olshan reassigned KAFKA-10549: -- Assignee: Justine Olshan > Add topic ID support to DeleteTopics,ListOffsets, OffsetFo

[GitHub] [kafka] hachikuji commented on a change in pull request #9601: KAFKA-10729: Bump remaining RPC's to use tagged fields.

2020-11-19 Thread GitBox
hachikuji commented on a change in pull request #9601: URL: https://github.com/apache/kafka/pull/9601#discussion_r527045689 ## File path: core/src/main/scala/kafka/api/ApiVersion.scala ## @@ -108,7 +108,9 @@ object ApiVersion { // Bup Fetch protocol for Raft protocol (KIP-

[GitHub] [kafka] abbccdda opened a new pull request #9624: KAFKA-10655: wrap and catch exception for appendAsLeader failure

2020-11-19 Thread GitBox
abbccdda opened a new pull request #9624: URL: https://github.com/apache/kafka/pull/9624 When leader append fails, we should trigger the resign process and do a graceful shutdown afterwards. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementat

[GitHub] [kafka] tombentley commented on pull request #9433: KAFKA-10607: Consistent behaviour for response errorCounts()

2020-11-19 Thread GitBox
tombentley commented on pull request #9433: URL: https://github.com/apache/kafka/pull/9433#issuecomment-730508481 @chia7712 done This is an automated message from the Apache Git Service. To respond to the message, please log

[jira] [Assigned] (KAFKA-10548) Implement deletion logic for LeaderAndIsrRequests

2020-11-19 Thread Justine Olshan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justine Olshan reassigned KAFKA-10548: -- Assignee: Justine Olshan > Implement deletion logic for LeaderAndIsrRequests > --

[GitHub] [kafka] chia7712 commented on pull request #9433: KAFKA-10607: Consistent behaviour for response errorCounts()

2020-11-19 Thread GitBox
chia7712 commented on pull request #9433: URL: https://github.com/apache/kafka/pull/9433#issuecomment-730516709 @tombentley thanks! I will merge this PR tomorrow if no objection :) This is an automated message from the Apache

[jira] [Assigned] (KAFKA-10580) Add topic ID support to Fetch request

2020-11-19 Thread Justine Olshan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justine Olshan reassigned KAFKA-10580: -- Assignee: Justine Olshan > Add topic ID support to Fetch request > --

[jira] [Created] (KAFKA-10750) test failure scenarios of MirrorMaker 2

2020-11-19 Thread Ning Zhang (Jira)
Ning Zhang created KAFKA-10750: -- Summary: test failure scenarios of MirrorMaker 2 Key: KAFKA-10750 URL: https://issues.apache.org/jira/browse/KAFKA-10750 Project: Kafka Issue Type: Improvement

[GitHub] [kafka] hachikuji commented on a change in pull request #9619: MINOR: Reduce sends created by `SendBuilder`

2020-11-19 Thread GitBox
hachikuji commented on a change in pull request #9619: URL: https://github.com/apache/kafka/pull/9619#discussion_r527073144 ## File path: clients/src/main/java/org/apache/kafka/common/protocol/SendBuilder.java ## @@ -122,6 +123,14 @@ public void writeVarlong(long i) {

[GitHub] [kafka] vamossagar12 commented on a change in pull request #9539: KAFKA-10634: Adding LeaderId to Voters list in LeaderChangeMessage

2020-11-19 Thread GitBox
vamossagar12 commented on a change in pull request #9539: URL: https://github.com/apache/kafka/pull/9539#discussion_r527073514 ## File path: raft/src/test/java/org/apache/kafka/raft/RaftClientTestContext.java ## @@ -709,6 +709,9 @@ static void verifyLeaderChangeMessage(

[GitHub] [kafka] hachikuji commented on a change in pull request #9619: MINOR: Reduce sends created by `SendBuilder`

2020-11-19 Thread GitBox
hachikuji commented on a change in pull request #9619: URL: https://github.com/apache/kafka/pull/9619#discussion_r527073144 ## File path: clients/src/main/java/org/apache/kafka/common/protocol/SendBuilder.java ## @@ -122,6 +123,14 @@ public void writeVarlong(long i) {

[GitHub] [kafka] mimaison commented on a change in pull request #9224: KAFKA-10304: refactor MM2 integration tests

2020-11-19 Thread GitBox
mimaison commented on a change in pull request #9224: URL: https://github.com/apache/kafka/pull/9224#discussion_r527014720 ## File path: connect/mirror/src/test/java/org/apache/kafka/connect/mirror/integration/MirrorConnectorsIntegrationBaseTest.java ## @@ -0,0 +1,407 @@ +/* +

[GitHub] [kafka] cadonna commented on a change in pull request #9615: KAFKA-10500: Add thread option

2020-11-19 Thread GitBox
cadonna commented on a change in pull request #9615: URL: https://github.com/apache/kafka/pull/9615#discussion_r527055689 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -894,11 +904,57 @@ private KafkaStreams(final InternalTopologyBuilder

[GitHub] [kafka] hachikuji commented on a change in pull request #9617: MINOR: Factor out common response parsing logic

2020-11-19 Thread GitBox
hachikuji commented on a change in pull request #9617: URL: https://github.com/apache/kafka/pull/9617#discussion_r527095050 ## File path: clients/src/main/java/org/apache/kafka/clients/NetworkClient.java ## @@ -974,21 +969,6 @@ private void handleInitiateApiVersionRequests(long

[GitHub] [kafka] hachikuji commented on a change in pull request #9617: MINOR: Factor out common response parsing logic

2020-11-19 Thread GitBox
hachikuji commented on a change in pull request #9617: URL: https://github.com/apache/kafka/pull/9617#discussion_r527095539 ## File path: clients/src/main/java/org/apache/kafka/common/requests/CorrelationIdMismatchException.java ## @@ -0,0 +1,45 @@ +/* + * Licensed to the Apac

[GitHub] [kafka] junrao commented on a change in pull request #8826: KAFKA-10090 Misleading warnings: The configuration was supplied but i…

2020-11-19 Thread GitBox
junrao commented on a change in pull request #8826: URL: https://github.com/apache/kafka/pull/8826#discussion_r527096607 ## File path: clients/src/test/java/org/apache/kafka/clients/producer/KafkaProducerTest.java ## @@ -366,8 +379,9 @@ public void testMetadataFetch() throws I

[GitHub] [kafka] wcarlson5 commented on a change in pull request #9615: KAFKA-10500: Add thread option

2020-11-19 Thread GitBox
wcarlson5 commented on a change in pull request #9615: URL: https://github.com/apache/kafka/pull/9615#discussion_r527099742 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -894,11 +904,57 @@ private KafkaStreams(final InternalTopologyBuilder

[GitHub] [kafka] wcarlson5 commented on a change in pull request #9615: KAFKA-10500: Add thread option

2020-11-19 Thread GitBox
wcarlson5 commented on a change in pull request #9615: URL: https://github.com/apache/kafka/pull/9615#discussion_r527099779 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -894,11 +904,57 @@ private KafkaStreams(final InternalTopologyBuilder

[GitHub] [kafka] hachikuji commented on a change in pull request #9617: MINOR: Factor out common response parsing logic

2020-11-19 Thread GitBox
hachikuji commented on a change in pull request #9617: URL: https://github.com/apache/kafka/pull/9617#discussion_r527101086 ## File path: clients/src/main/java/org/apache/kafka/common/requests/CorrelationIdMismatchException.java ## @@ -0,0 +1,45 @@ +/* + * Licensed to the Apac

[GitHub] [kafka] junrao commented on a change in pull request #9621: KAFKA-9892: Producer state snapshot needs to be forced to disk

2020-11-19 Thread GitBox
junrao commented on a change in pull request #9621: URL: https://github.com/apache/kafka/pull/9621#discussion_r527105252 ## File path: core/src/main/scala/kafka/log/ProducerStateManager.scala ## @@ -437,7 +437,7 @@ object ProducerStateManager { val fileChannel = FileChan

[GitHub] [kafka] lct45 commented on a change in pull request #9614: KAFKA-10500: Add failed-stream-threads metric for adding + removing stream threads

2020-11-19 Thread GitBox
lct45 commented on a change in pull request #9614: URL: https://github.com/apache/kafka/pull/9614#discussion_r527117685 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -221,6 +221,9 @@ State setState(final State newState

[GitHub] [kafka] junrao merged pull request #9596: KAFKA-10723: Fix LogManager shutdown error handling

2020-11-19 Thread GitBox
junrao merged pull request #9596: URL: https://github.com/apache/kafka/pull/9596 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [kafka] cadonna commented on a change in pull request #9614: KAFKA-10500: Add failed-stream-threads metric for adding + removing stream threads

2020-11-19 Thread GitBox
cadonna commented on a change in pull request #9614: URL: https://github.com/apache/kafka/pull/9614#discussion_r527122772 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java ## @@ -221,6 +221,9 @@ State setState(final State newSta

[jira] [Resolved] (KAFKA-10723) LogManager leaks internal thread pool activity during shutdown

2020-11-19 Thread Jun Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-10723. - Fix Version/s: 2.8.0 Resolution: Fixed merged to trunk > LogManager leaks internal thread pool a

[jira] [Created] (KAFKA-10751) Generate log to help estimate messages lost during ULE

2020-11-19 Thread Lucas Wang (Jira)
Lucas Wang created KAFKA-10751: -- Summary: Generate log to help estimate messages lost during ULE Key: KAFKA-10751 URL: https://issues.apache.org/jira/browse/KAFKA-10751 Project: Kafka Issue Type

[jira] [Commented] (KAFKA-10751) Generate log to help estimate messages lost during ULE

2020-11-19 Thread Lucas Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17235702#comment-17235702 ] Lucas Wang commented on KAFKA-10751: PR submitted: https://github.com/apache/kafka/p

[GitHub] [kafka] cadonna commented on a change in pull request #9615: KAFKA-10500: Add thread option

2020-11-19 Thread GitBox
cadonna commented on a change in pull request #9615: URL: https://github.com/apache/kafka/pull/9615#discussion_r527145996 ## File path: streams/src/test/java/org/apache/kafka/streams/KafkaStreamsTest.java ## @@ -588,6 +592,29 @@ public void testCloseIsIdempotent() {

[GitHub] [kafka] hachikuji merged pull request #9619: MINOR: Reduce sends created by `SendBuilder`

2020-11-19 Thread GitBox
hachikuji merged pull request #9619: URL: https://github.com/apache/kafka/pull/9619 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[jira] [Assigned] (KAFKA-8872) Improvements to controller "deleting" state / topic Identifiers

2020-11-19 Thread Justine Olshan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justine Olshan reassigned KAFKA-8872: - Assignee: Justine Olshan > Improvements to controller "deleting" state / topic Identifi

[GitHub] [kafka] lmr3796 commented on a change in pull request #9435: KAFKA-10606: Disable auto topic creation for fetch-all-topic-metadata request

2020-11-19 Thread GitBox
lmr3796 commented on a change in pull request #9435: URL: https://github.com/apache/kafka/pull/9435#discussion_r527192802 ## File path: core/src/main/scala/kafka/server/KafkaApis.scala ## @@ -1333,7 +1343,17 @@ class KafkaApis(val requestChannel: RequestChannel, } }

[GitHub] [kafka] hachikuji merged pull request #9608: MINOR: Enable testLogCleanerStats

2020-11-19 Thread GitBox
hachikuji merged pull request #9608: URL: https://github.com/apache/kafka/pull/9608 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[jira] [Commented] (KAFKA-9458) Kafka crashed in windows environment

2020-11-19 Thread Daniel Dube (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17235769#comment-17235769 ] Daniel Dube commented on KAFKA-9458: Hi everyone. "Windows is not an officially suppo

[jira] [Comment Edited] (KAFKA-9458) Kafka crashed in windows environment

2020-11-19 Thread Daniel Dube (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17235769#comment-17235769 ] Daniel Dube edited comment on KAFKA-9458 at 11/19/20, 10:10 PM: ---

[GitHub] [kafka] Hamza-Slama opened a new pull request #9625: MINOR: remove semicolon

2020-11-19 Thread GitBox
Hamza-Slama opened a new pull request #9625: URL: https://github.com/apache/kafka/pull/9625 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of testi

[GitHub] [kafka] wcarlson5 commented on a change in pull request #9615: KAFKA-10500: Add thread option

2020-11-19 Thread GitBox
wcarlson5 commented on a change in pull request #9615: URL: https://github.com/apache/kafka/pull/9615#discussion_r527277135 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -894,11 +904,72 @@ private KafkaStreams(final InternalTopologyBuilder

[GitHub] [kafka] wcarlson5 commented on a change in pull request #9615: KAFKA-10500: Add thread option

2020-11-19 Thread GitBox
wcarlson5 commented on a change in pull request #9615: URL: https://github.com/apache/kafka/pull/9615#discussion_r527277048 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -894,11 +904,72 @@ private KafkaStreams(final InternalTopologyBuilder

[GitHub] [kafka] jolshan opened a new pull request #9626: KAFKA-10545: Create topic IDs and propagate to brokers

2020-11-19 Thread GitBox
jolshan opened a new pull request #9626: URL: https://github.com/apache/kafka/pull/9626 This change takes the topic IDs created in https://github.com/apache/kafka/pull/9473 and propagates them to brokers using LeaderAndIsr Request. It also removes the topic name from the LeaderAndIsr Resp

[jira] [Commented] (KAFKA-10547) Add topic IDs to MetadataResponse

2020-11-19 Thread Justine Olshan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17235796#comment-17235796 ] Justine Olshan commented on KAFKA-10547: Hi [~dengziming] I'm sorry I didn't see

[GitHub] [kafka] jolshan commented on pull request #9622: KAFKA-10547; add topicId in MetadataResp

2020-11-19 Thread GitBox
jolshan commented on pull request #9622: URL: https://github.com/apache/kafka/pull/9622#issuecomment-730710397 Hi @dengziming! Thanks for the PR! I was hoping to add LeaderAndIsrRequests before UpdateMetadata/Metadata, as ordered in the [JIRA ticket.](https://issues.apache.org/jira/browse/

[GitHub] [kafka] jolshan edited a comment on pull request #9622: KAFKA-10547; add topicId in MetadataResp

2020-11-19 Thread GitBox
jolshan edited a comment on pull request #9622: URL: https://github.com/apache/kafka/pull/9622#issuecomment-730710397 Hi @dengziming! Thanks for the PR! I was hoping to add LeaderAndIsrRequests before UpdateMetadata/Metadata, following the ordering of the [JIRA tickets.](https://issues.ap

[GitHub] [kafka] jolshan edited a comment on pull request #9622: KAFKA-10547; add topicId in MetadataResp

2020-11-19 Thread GitBox
jolshan edited a comment on pull request #9622: URL: https://github.com/apache/kafka/pull/9622#issuecomment-730710397 Hi @dengziming! Thanks for the PR! I was hoping to add LeaderAndIsrRequests before UpdateMetadata/Metadata, following the ordering of the [JIRA tickets.](https://issues.ap

[jira] [Updated] (KAFKA-10745) Please let me know how I check the time which Source connector receive the data from source table.

2020-11-19 Thread NAYUSIK (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] NAYUSIK updated KAFKA-10745: Issue Type: Bug (was: Improvement) > Please let me know how I check the time which Source connector recei

[jira] [Updated] (KAFKA-10745) Please let me know how I check the time which Source connector receive the data from source table.

2020-11-19 Thread NAYUSIK (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] NAYUSIK updated KAFKA-10745: Issue Type: Improvement (was: Bug) > Please let me know how I check the time which Source connector recei

  1   2   >