[GitHub] [kafka] chia7712 commented on pull request #9204: KAFKA-6181 Examining log messages with {{--deep-iteration}} should show superset of fields

2020-11-18 Thread GitBox
chia7712 commented on pull request #9204: URL: https://github.com/apache/kafka/pull/9204#issuecomment-730154806 rebase patch to trigger QA This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [kafka] chia7712 commented on a change in pull request #9435: KAFKA-10606: Disable auto topic creation for fetch-all-topic-metadata request

2020-11-18 Thread GitBox
chia7712 commented on a change in pull request #9435: URL: https://github.com/apache/kafka/pull/9435#discussion_r526616584 ## File path: core/src/main/scala/kafka/server/KafkaApis.scala ## @@ -1333,7 +1343,17 @@ class KafkaApis(val requestChannel: RequestChannel, }

[GitHub] [kafka] chia7712 opened a new pull request #9620: MINOR: Get rid of generic from FetchResponse

2020-11-18 Thread GitBox
chia7712 opened a new pull request #9620: URL: https://github.com/apache/kafka/pull/9620 Except for LazyDownConversionRecords, we don't use the generic. We should handle the specify case directly instead of making generic in whole project. In short, this patch is a kind of cleanup.

[GitHub] [kafka] chia7712 commented on pull request #9619: MINOR: Reduce sends created by `SendBuilder`

2020-11-18 Thread GitBox
chia7712 commented on pull request #9619: URL: https://github.com/apache/kafka/pull/9619#issuecomment-730134289 +1 to last commit This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [kafka] hachikuji commented on a change in pull request #9619: MINOR: Reduce sends created by `SendBuilder`

2020-11-18 Thread GitBox
hachikuji commented on a change in pull request #9619: URL: https://github.com/apache/kafka/pull/9619#discussion_r526584156 ## File path: clients/src/main/java/org/apache/kafka/common/protocol/SendBuilder.java ## @@ -130,13 +139,25 @@ public void writeVarlong(long i) {

[GitHub] [kafka] hachikuji commented on a change in pull request #9619: MINOR: Reduce sends created by `SendBuilder`

2020-11-18 Thread GitBox
hachikuji commented on a change in pull request #9619: URL: https://github.com/apache/kafka/pull/9619#discussion_r526583778 ## File path: clients/src/main/java/org/apache/kafka/common/protocol/SendBuilder.java ## @@ -39,6 +42,7 @@ */ public class SendBuilder implements

[jira] [Created] (KAFKA-10745) Please let me know how I check the time which Source connector receive the data from source table.

2020-11-18 Thread NAYUSIK (Jira)
NAYUSIK created KAFKA-10745: --- Summary: Please let me know how I check the time which Source connector receive the data from source table. Key: KAFKA-10745 URL: https://issues.apache.org/jira/browse/KAFKA-10745

[GitHub] [kafka] chia7712 commented on a change in pull request #9617: MINOR: Factor out common response parsing logic

2020-11-18 Thread GitBox
chia7712 commented on a change in pull request #9617: URL: https://github.com/apache/kafka/pull/9617#discussion_r526577116 ## File path: core/src/main/scala/kafka/server/ForwardingManager.scala ## @@ -17,25 +17,17 @@ package kafka.server -import

[GitHub] [kafka] hachikuji commented on pull request #9619: MINOR: Reduce sends created by `SendBuilder`

2020-11-18 Thread GitBox
hachikuji commented on pull request #9619: URL: https://github.com/apache/kafka/pull/9619#issuecomment-730107392 @chia7712 To be honest, I'm not sure. I think it makes sense in general to try and batch the writes together to reduce number of calls to the kernel in any case, but I am not

[GitHub] [kafka] ijuma commented on a change in pull request #9590: KAFKA-7556: KafkaConsumer.beginningOffsets does not return actual first offsets

2020-11-18 Thread GitBox
ijuma commented on a change in pull request #9590: URL: https://github.com/apache/kafka/pull/9590#discussion_r526571061 ## File path: core/src/main/scala/kafka/log/Log.scala ## @@ -1749,6 +1752,9 @@ class Log(@volatile private var _dir: File,

[GitHub] [kafka] ijuma commented on a change in pull request #9590: KAFKA-7556: KafkaConsumer.beginningOffsets does not return actual first offsets

2020-11-18 Thread GitBox
ijuma commented on a change in pull request #9590: URL: https://github.com/apache/kafka/pull/9590#discussion_r526570690 ## File path: core/src/main/scala/kafka/log/Log.scala ## @@ -1749,6 +1752,9 @@ class Log(@volatile private var _dir: File,

[GitHub] [kafka] chia7712 commented on pull request #9507: KAFKA-10628: remove all the unnecessary parameters from the tests which are using TopologyTestDriver

2020-11-18 Thread GitBox
chia7712 commented on pull request #9507: URL: https://github.com/apache/kafka/pull/9507#issuecomment-730104149 @showuon Thanks for this patch! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [kafka] chia7712 merged pull request #9507: KAFKA-10628: remove all the unnecessary parameters from the tests which are using TopologyTestDriver

2020-11-18 Thread GitBox
chia7712 merged pull request #9507: URL: https://github.com/apache/kafka/pull/9507 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] chia7712 commented on a change in pull request #9619: MINOR: Reduce sends created by `SendBuilder`

2020-11-18 Thread GitBox
chia7712 commented on a change in pull request #9619: URL: https://github.com/apache/kafka/pull/9619#discussion_r526567417 ## File path: clients/src/main/java/org/apache/kafka/common/protocol/SendBuilder.java ## @@ -130,13 +139,25 @@ public void writeVarlong(long i) {

[GitHub] [kafka] chia7712 commented on pull request #9547: KAFKA-9630; Replace OffsetsForLeaderEpoch request/response with automated protocol

2020-11-18 Thread GitBox
chia7712 commented on pull request #9547: URL: https://github.com/apache/kafka/pull/9547#issuecomment-730094552 @dajac +1 to nice updating. sorry that my commit causes conflicting files to you. This is an automated message

[GitHub] [kafka] splett2 commented on a change in pull request #9386: KAFKA-10024: Add dynamic configuration and enforce quota for per-IP connection rate limits (KIP-612, part 2)

2020-11-18 Thread GitBox
splett2 commented on a change in pull request #9386: URL: https://github.com/apache/kafka/pull/9386#discussion_r526525212 ## File path: core/src/main/scala/kafka/network/SocketServer.scala ## @@ -1476,33 +1650,36 @@ class ConnectionQuotas(config: KafkaConfig, time: Time,

[jira] [Created] (KAFKA-10744) Listener and broker connection rate quota tests should use mock time

2020-11-18 Thread David Mao (Jira)
David Mao created KAFKA-10744: - Summary: Listener and broker connection rate quota tests should use mock time Key: KAFKA-10744 URL: https://issues.apache.org/jira/browse/KAFKA-10744 Project: Kafka

[GitHub] [kafka] splett2 commented on a change in pull request #9386: KAFKA-10024: Add dynamic configuration and enforce quota for per-IP connection rate limits (KIP-612, part 2)

2020-11-18 Thread GitBox
splett2 commented on a change in pull request #9386: URL: https://github.com/apache/kafka/pull/9386#discussion_r526528689 ## File path: core/src/test/scala/unit/kafka/network/ConnectionQuotasTest.scala ## @@ -633,18 +837,27 @@ class ConnectionQuotasTest {

[GitHub] [kafka] splett2 commented on a change in pull request #9386: KAFKA-10024: Add dynamic configuration and enforce quota for per-IP connection rate limits (KIP-612, part 2)

2020-11-18 Thread GitBox
splett2 commented on a change in pull request #9386: URL: https://github.com/apache/kafka/pull/9386#discussion_r526527000 ## File path: core/src/main/scala/kafka/network/SocketServer.scala ## @@ -1324,7 +1404,60 @@ class ConnectionQuotas(config: KafkaConfig, time: Time,

[GitHub] [kafka] splett2 commented on a change in pull request #9386: KAFKA-10024: Add dynamic configuration and enforce quota for per-IP connection rate limits (KIP-612, part 2)

2020-11-18 Thread GitBox
splett2 commented on a change in pull request #9386: URL: https://github.com/apache/kafka/pull/9386#discussion_r526527000 ## File path: core/src/main/scala/kafka/network/SocketServer.scala ## @@ -1324,7 +1404,60 @@ class ConnectionQuotas(config: KafkaConfig, time: Time,

[GitHub] [kafka] splett2 commented on a change in pull request #9386: KAFKA-10024: Add dynamic configuration and enforce quota for per-IP connection rate limits (KIP-612, part 2)

2020-11-18 Thread GitBox
splett2 commented on a change in pull request #9386: URL: https://github.com/apache/kafka/pull/9386#discussion_r526527000 ## File path: core/src/main/scala/kafka/network/SocketServer.scala ## @@ -1324,7 +1404,60 @@ class ConnectionQuotas(config: KafkaConfig, time: Time,

[GitHub] [kafka] splett2 commented on a change in pull request #9386: KAFKA-10024: Add dynamic configuration and enforce quota for per-IP connection rate limits (KIP-612, part 2)

2020-11-18 Thread GitBox
splett2 commented on a change in pull request #9386: URL: https://github.com/apache/kafka/pull/9386#discussion_r526525212 ## File path: core/src/main/scala/kafka/network/SocketServer.scala ## @@ -1476,33 +1650,36 @@ class ConnectionQuotas(config: KafkaConfig, time: Time,

[GitHub] [kafka] abbccdda closed pull request #9042: (Back-up draft) redirection with version bump

2020-11-18 Thread GitBox
abbccdda closed pull request #9042: URL: https://github.com/apache/kafka/pull/9042 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] anatasiavela commented on a change in pull request #9526: KAFKA-10525: Emit JSONs with new auto-generated schema

2020-11-18 Thread GitBox
anatasiavela commented on a change in pull request #9526: URL: https://github.com/apache/kafka/pull/9526#discussion_r526511067 ## File path: core/src/main/scala/kafka/network/RequestConvertToJson.scala ## @@ -0,0 +1,362 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] mjsax merged pull request #9570: KAFKA-9274: Handle TimeoutException on commit

2020-11-18 Thread GitBox
mjsax merged pull request #9570: URL: https://github.com/apache/kafka/pull/9570 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [kafka] hachikuji commented on pull request #9547: KAFKA-9630; Replace OffsetsForLeaderEpoch request/response with automated protocol

2020-11-18 Thread GitBox
hachikuji commented on pull request #9547: URL: https://github.com/apache/kafka/pull/9547#issuecomment-730040680 @dajac The updates LGTM. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [kafka] hachikuji commented on a change in pull request #9547: KAFKA-9630; Replace OffsetsForLeaderEpoch request/response with automated protocol

2020-11-18 Thread GitBox
hachikuji commented on a change in pull request #9547: URL: https://github.com/apache/kafka/pull/9547#discussion_r526509285 ## File path: clients/src/main/java/org/apache/kafka/common/requests/OffsetsForLeaderEpochRequest.java ## @@ -51,169 +47,120 @@ */ public

[jira] [Assigned] (KAFKA-9672) Dead brokers in ISR cause isr-expiration to fail with exception

2020-11-18 Thread Jose Armando Garcia Sancio (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jose Armando Garcia Sancio reassigned KAFKA-9672: - Assignee: Jose Armando Garcia Sancio > Dead brokers in ISR

[GitHub] [kafka] rajinisivaram commented on a change in pull request #9485: KAKFA-10619: Idempotent producer will get authorized once it has a WRITE access to at least one topic

2020-11-18 Thread GitBox
rajinisivaram commented on a change in pull request #9485: URL: https://github.com/apache/kafka/pull/9485#discussion_r526479473 ## File path: clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java ## @@ -139,4 +150,129 @@ * @return Iterator for ACL

[GitHub] [kafka] abbccdda commented on a change in pull request #9617: MINOR: Factor out common response parsing logic

2020-11-18 Thread GitBox
abbccdda commented on a change in pull request #9617: URL: https://github.com/apache/kafka/pull/9617#discussion_r526499505 ## File path: clients/src/main/java/org/apache/kafka/clients/NetworkClient.java ## @@ -974,21 +969,6 @@ private void handleInitiateApiVersionRequests(long

[jira] [Resolved] (KAFKA-4748) Need a way to shutdown all workers in a Streams application at the same time

2020-11-18 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-4748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman resolved KAFKA-4748. --- Resolution: Fixed > Need a way to shutdown all workers in a Streams

[jira] [Reopened] (KAFKA-10500) Add API to Start and Stop Stream Threads

2020-11-18 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman reopened KAFKA-10500: > Add API to Start and Stop Stream Threads >

[jira] [Updated] (KAFKA-4748) Need a way to shutdown all workers in a Streams application at the same time

2020-11-18 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-4748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman updated KAFKA-4748: -- Fix Version/s: 2.8.0 > Need a way to shutdown all workers in a Streams

[jira] [Commented] (KAFKA-6943) Have option to shutdown KS cleanly if any threads crashes, or if all threads crash

2020-11-18 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-6943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17235069#comment-17235069 ] A. Sophie Blee-Goldman commented on KAFKA-6943: --- Resolved via 

[jira] [Updated] (KAFKA-6943) Have option to shutdown KS cleanly if any threads crashes, or if all threads crash

2020-11-18 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-6943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman updated KAFKA-6943: -- Fix Version/s: 2.8.0 > Have option to shutdown KS cleanly if any threads

[jira] [Resolved] (KAFKA-6943) Have option to shutdown KS cleanly if any threads crashes, or if all threads crash

2020-11-18 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-6943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman resolved KAFKA-6943. --- Resolution: Fixed > Have option to shutdown KS cleanly if any threads

[jira] [Updated] (KAFKA-10015) React Smartly to Unexpected Errors on Stream Threads

2020-11-18 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-10015: Description: Currently, if an unexpected error occurs on a stream thread, the stream

[jira] [Updated] (KAFKA-10015) React Smartly to Unexpected Errors on Stream Threads

2020-11-18 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-10015: Labels: kip (was: needs-kip) > React Smartly to Unexpected Errors on Stream Threads >

[jira] [Resolved] (KAFKA-10500) Add API to Start and Stop Stream Threads

2020-11-18 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-10500. - Fix Version/s: 2.8.0 Resolution: Fixed > Add API to Start and Stop Stream

[GitHub] [kafka] mjsax commented on pull request #9572: KAFKA-10500: Thread Cache Resizes

2020-11-18 Thread GitBox
mjsax commented on pull request #9572: URL: https://github.com/apache/kafka/pull/9572#issuecomment-730028264 Thanks for the PR @wcarlson5. Merged to `trunk`. This is an automated message from the Apache Git Service. To

[GitHub] [kafka] mjsax merged pull request #9572: KAFKA-10500: Thread Cache Resizes

2020-11-18 Thread GitBox
mjsax merged pull request #9572: URL: https://github.com/apache/kafka/pull/9572 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[jira] [Updated] (KAFKA-10500) Add API to Start and Stop Stream Threads

2020-11-18 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-10500: Description: Currently, there is no possibility in Kafka Streams to increase or decrease

[jira] [Updated] (KAFKA-10500) Add API to Start and Stop Stream Threads

2020-11-18 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-10500: Labels: kip (was: needs-kip) > Add API to Start and Stop Stream Threads >

[jira] [Commented] (KAFKA-10734) Speedup the processing of LeaderAndIsr request

2020-11-18 Thread Lucas Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17235061#comment-17235061 ] Lucas Wang commented on KAFKA-10734: Thanks for your reply [~junrao]. In some clusters, our SREs set

[GitHub] [kafka] ableegoldman commented on a change in pull request #9487: KAFKA-9331: Add a streams specific uncaught exception handler

2020-11-18 Thread GitBox
ableegoldman commented on a change in pull request #9487: URL: https://github.com/apache/kafka/pull/9487#discussion_r526477258 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -932,56 +1028,62 @@ public synchronized boolean close(final long

[GitHub] [kafka] rajinisivaram merged pull request #9473: KAFKA-10545: Create topic IDs in ZooKeeper and Controller

2020-11-18 Thread GitBox
rajinisivaram merged pull request #9473: URL: https://github.com/apache/kafka/pull/9473 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [kafka] rajinisivaram commented on pull request #9473: KAFKA-10545: Create topic IDs in ZooKeeper and Controller

2020-11-18 Thread GitBox
rajinisivaram commented on pull request #9473: URL: https://github.com/apache/kafka/pull/9473#issuecomment-730006469 @jolshan Thanks for the updates, merging to trunk. This is an automated message from the Apache Git

[GitHub] [kafka] hachikuji commented on pull request #9619: MINOR: Reduce sends created by `SendBuilder`

2020-11-18 Thread GitBox
hachikuji commented on pull request #9619: URL: https://github.com/apache/kafka/pull/9619#issuecomment-730001511 cc @chia7712 @dajac This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [kafka] hachikuji opened a new pull request #9619: MINOR: Reduce sends created by `SendBuilder`

2020-11-18 Thread GitBox
hachikuji opened a new pull request #9619: URL: https://github.com/apache/kafka/pull/9619 In #9401, I observed some strange latency behavior when testing locally (on macos). The behavior seemed to be caused by slightly different write behavior as a result of the new `SendBuilder` class.

[GitHub] [kafka] bbejeck commented on pull request #9613: Cherry-pick KAFKA-10687 to 2.7

2020-11-18 Thread GitBox
bbejeck commented on pull request #9613: URL: https://github.com/apache/kafka/pull/9613#issuecomment-729986874 Thanks for the separate PR @abbccdda This is an automated message from the Apache Git Service. To respond to the

[jira] [Resolved] (KAFKA-10687) Produce request should be bumped for new error code PRODUCE_FENCED

2020-11-18 Thread Bill Bejeck (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Bejeck resolved KAFKA-10687. - Resolution: Fixed Fixed for 2.7 via https://github.com/apache/kafka/pull/9613 > Produce

[GitHub] [kafka] bbejeck commented on pull request #9613: Cherry-pick KAFKA-10687 to 2.7

2020-11-18 Thread GitBox
bbejeck commented on pull request #9613: URL: https://github.com/apache/kafka/pull/9613#issuecomment-729985753 merged #9613 into 2.7 This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [kafka] bbejeck merged pull request #9613: Cherry-pick KAFKA-10687 to 2.7

2020-11-18 Thread GitBox
bbejeck merged pull request #9613: URL: https://github.com/apache/kafka/pull/9613 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] bbejeck commented on pull request #9613: Cherry-pick KAFKA-10687 to 2.7

2020-11-18 Thread GitBox
bbejeck commented on pull request #9613: URL: https://github.com/apache/kafka/pull/9613#issuecomment-729985015 Failure in Java 11 is unrelated and a known flaky test. This is an automated message from the Apache Git Service.

[jira] [Resolved] (KAFKA-9628) Replace Produce request/response with automated protocol

2020-11-18 Thread Jason Gustafson (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-9628. Fix Version/s: 2.8.0 Resolution: Fixed > Replace Produce request/response with

[GitHub] [kafka] hachikuji merged pull request #9401: KAFKA-9628 Replace Produce request/response with automated protocol

2020-11-18 Thread GitBox
hachikuji merged pull request #9401: URL: https://github.com/apache/kafka/pull/9401 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] mjsax opened a new pull request #9618: MINOR: change default TX timeout only if EOS is enabled

2020-11-18 Thread GitBox
mjsax opened a new pull request #9618: URL: https://github.com/apache/kafka/pull/9618 Call for review @abbccdda This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[jira] [Created] (KAFKA-10743) Fix NumRequestsForwardingToControllerPerSec metric in ForwardingManager

2020-11-18 Thread Jason Gustafson (Jira)
Jason Gustafson created KAFKA-10743: --- Summary: Fix NumRequestsForwardingToControllerPerSec metric in ForwardingManager Key: KAFKA-10743 URL: https://issues.apache.org/jira/browse/KAFKA-10743

[GitHub] [kafka] hachikuji commented on pull request #9617: MINOR: Factor out common response parsing logic

2020-11-18 Thread GitBox
hachikuji commented on pull request #9617: URL: https://github.com/apache/kafka/pull/9617#issuecomment-729947146 This is a follow-up from #9563. cc @chia7712 @abbccdda While I was working on this patch, I noticed this: https://issues.apache.org/jira/browse/KAFKA-10743. In this

[GitHub] [kafka] mattwong949 commented on a change in pull request #9608: MINOR: Enable testLogCleanerStats

2020-11-18 Thread GitBox
mattwong949 commented on a change in pull request #9608: URL: https://github.com/apache/kafka/pull/9608#discussion_r526412288 ## File path: core/src/test/scala/unit/kafka/log/LogCleanerTest.scala ## @@ -815,9 +815,10 @@ class LogCleanerTest { (0 until leo.toInt

[GitHub] [kafka] hachikuji opened a new pull request #9617: MINOR: Factor out common response parsing logic

2020-11-18 Thread GitBox
hachikuji opened a new pull request #9617: URL: https://github.com/apache/kafka/pull/9617 This patch factors out some common parsing logic from `NetworkClient.parseResponse` and `AbstractResponse.parseResponse`. As a result of this refactor, we are now verifying the correlationId in

[jira] [Commented] (KAFKA-10114) Kafka producer stuck after broker crash

2020-11-18 Thread Tim Fox (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17234997#comment-17234997 ] Tim Fox commented on KAFKA-10114: - Hi Kshitij - can you get a stack dump for when this occurs with

[jira] [Commented] (KAFKA-10741) Kafka ConsumerProperties.addDeserializerToConfig breaks hirarchical property resolution.

2020-11-18 Thread Michael (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17234996#comment-17234996 ] Michael commented on KAFKA-10741: - I think, since in the transition from _ConsumerConfig_ to 

[jira] [Created] (KAFKA-10742) Automated protocol should support java.util.Optional

2020-11-18 Thread David Jacot (Jira)
David Jacot created KAFKA-10742: --- Summary: Automated protocol should support java.util.Optional Key: KAFKA-10742 URL: https://issues.apache.org/jira/browse/KAFKA-10742 Project: Kafka Issue

[jira] [Created] (KAFKA-10741) Kafka ConsumerProperties.addDeserializerToConfig breaks hirarchical property resolution.

2020-11-18 Thread Michael (Jira)
Michael created KAFKA-10741: --- Summary: Kafka ConsumerProperties.addDeserializerToConfig breaks hirarchical property resolution. Key: KAFKA-10741 URL: https://issues.apache.org/jira/browse/KAFKA-10741

[GitHub] [kafka] dajac commented on pull request #9547: KAFKA-9630; Replace OffsetsForLeaderEpoch request/response with automated protocol

2020-11-18 Thread GitBox
dajac commented on pull request #9547: URL: https://github.com/apache/kafka/pull/9547#issuecomment-729926766 @chia7712 @hachikuji Thanks for your comments. I have addressed them. Could you have another quick look? This is

[GitHub] [kafka] dajac commented on a change in pull request #9547: KAFKA-9630; Replace OffsetsForLeaderEpoch request/response with automated protocol

2020-11-18 Thread GitBox
dajac commented on a change in pull request #9547: URL: https://github.com/apache/kafka/pull/9547#discussion_r526383138 ## File path: clients/src/main/java/org/apache/kafka/clients/consumer/internals/OffsetsForLeaderEpochClient.java ## @@ -61,67 +77,76 @@ protected

[jira] [Created] (KAFKA-10740) ReplicaFetcherThread should use OffsetForLeaderPartitionResult directly

2020-11-18 Thread David Jacot (Jira)
David Jacot created KAFKA-10740: --- Summary: ReplicaFetcherThread should use OffsetForLeaderPartitionResult directly Key: KAFKA-10740 URL: https://issues.apache.org/jira/browse/KAFKA-10740 Project: Kafka

[GitHub] [kafka] dajac commented on a change in pull request #9547: KAFKA-9630; Replace OffsetsForLeaderEpoch request/response with automated protocol

2020-11-18 Thread GitBox
dajac commented on a change in pull request #9547: URL: https://github.com/apache/kafka/pull/9547#discussion_r526377753 ## File path: clients/src/main/java/org/apache/kafka/common/requests/OffsetsForLeaderEpochRequest.java ## @@ -51,169 +47,120 @@ */ public static

[GitHub] [kafka] vvcephei commented on a change in pull request #9414: KAFKA-10585: Kafka Streams should clean up the state store directory from cleanup

2020-11-18 Thread GitBox
vvcephei commented on a change in pull request #9414: URL: https://github.com/apache/kafka/pull/9414#discussion_r526377375 ## File path: streams/src/test/java/org/apache/kafka/streams/integration/EOSUncleanShutdownIntegrationTest.java ## @@ -140,9 +140,6 @@ public void

[jira] [Created] (KAFKA-10739) KafkaApis#handleOffsetForLeaderEpochRequest should use OffsetForLeaderPartition directly

2020-11-18 Thread David Jacot (Jira)
David Jacot created KAFKA-10739: --- Summary: KafkaApis#handleOffsetForLeaderEpochRequest should use OffsetForLeaderPartition directly Key: KAFKA-10739 URL: https://issues.apache.org/jira/browse/KAFKA-10739

[GitHub] [kafka] dajac commented on a change in pull request #9547: KAFKA-9630; Replace OffsetsForLeaderEpoch request/response with automated protocol

2020-11-18 Thread GitBox
dajac commented on a change in pull request #9547: URL: https://github.com/apache/kafka/pull/9547#discussion_r526373445 ## File path: clients/src/main/java/org/apache/kafka/common/requests/OffsetsForLeaderEpochResponse.java ## @@ -51,133 +41,82 @@ * - {@link

[GitHub] [kafka] dajac commented on a change in pull request #9547: KAFKA-9630; Replace OffsetsForLeaderEpoch request/response with automated protocol

2020-11-18 Thread GitBox
dajac commented on a change in pull request #9547: URL: https://github.com/apache/kafka/pull/9547#discussion_r526373121 ## File path: clients/src/main/java/org/apache/kafka/common/requests/OffsetsForLeaderEpochRequest.java ## @@ -51,169 +47,120 @@ */ public static

[jira] [Commented] (KAFKA-10091) Improve task idling

2020-11-18 Thread John Roesler (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17234967#comment-17234967 ] John Roesler commented on KAFKA-10091: -- FYI to all watchers, I've begun to draft a KIP:

[GitHub] [kafka] dajac commented on a change in pull request #9401: KAFKA-9628 Replace Produce request/response with automated protocol

2020-11-18 Thread GitBox
dajac commented on a change in pull request #9401: URL: https://github.com/apache/kafka/pull/9401#discussion_r526368983 ## File path: clients/src/main/resources/common/message/ProduceRequest.json ## @@ -33,21 +33,21 @@ "validVersions": "0-8", "flexibleVersions": "none",

[GitHub] [kafka] vvcephei opened a new pull request #9616: POC: fix task idling

2020-11-18 Thread GitBox
vvcephei opened a new pull request #9616: URL: https://github.com/apache/kafka/pull/9616 POC for: https://issues.apache.org/jira/browse/KAFKA-10091 https://cwiki.apache.org/confluence/x/JSXZCQ Please don't bother reviewing this. I did enough of the implementation to

[jira] [Commented] (KAFKA-10114) Kafka producer stuck after broker crash

2020-11-18 Thread Kshitij Wadhwa (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17234954#comment-17234954 ] Kshitij Wadhwa commented on KAFKA-10114: We are still seeing this issue with version 2.6.0. Our

[GitHub] [kafka] apovzner commented on a change in pull request #9386: KAFKA-10024: Add dynamic configuration and enforce quota for per-IP connection rate limits (KIP-612, part 2)

2020-11-18 Thread GitBox
apovzner commented on a change in pull request #9386: URL: https://github.com/apache/kafka/pull/9386#discussion_r526361289 ## File path: core/src/main/scala/kafka/network/SocketServer.scala ## @@ -1324,7 +1401,59 @@ class ConnectionQuotas(config: KafkaConfig, time: Time,

[GitHub] [kafka] lct45 commented on a change in pull request #9614: KAFKA-10500: Add failed-stream-threads metric for adding + removing stream threads

2020-11-18 Thread GitBox
lct45 commented on a change in pull request #9614: URL: https://github.com/apache/kafka/pull/9614#discussion_r526355838 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -1070,6 +1070,10 @@ private Thread shutdownHelper(final boolean error) {

[GitHub] [kafka] wcarlson5 commented on a change in pull request #9614: KAFKA-10500: Add failed-stream-threads metric for adding + removing stream threads

2020-11-18 Thread GitBox
wcarlson5 commented on a change in pull request #9614: URL: https://github.com/apache/kafka/pull/9614#discussion_r526352796 ## File path: streams/src/test/java/org/apache/kafka/streams/integration/MetricsIntegrationTest.java ## @@ -377,7 +378,7 @@ private void

[GitHub] [kafka] wcarlson5 commented on a change in pull request #9614: KAFKA-10500: Add failed-stream-threads metric for adding + removing stream threads

2020-11-18 Thread GitBox
wcarlson5 commented on a change in pull request #9614: URL: https://github.com/apache/kafka/pull/9614#discussion_r526347359 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -1070,6 +1070,10 @@ private Thread shutdownHelper(final boolean

[GitHub] [kafka] gardnervickers edited a comment on pull request #9601: KAFKA-10729: Bump remaining RPC's to use tagged fields.

2020-11-18 Thread GitBox
gardnervickers edited a comment on pull request #9601: URL: https://github.com/apache/kafka/pull/9601#issuecomment-729887376 OffsetForLeaderEpoch and Produce are not yet generated RPCs, but will be once #9401 and #9547 are merged. I've removed the taggedFields bump for these RPC's. We can

[GitHub] [kafka] gardnervickers commented on pull request #9601: KAFKA-10729: Bump remaining RPC's to use tagged fields.

2020-11-18 Thread GitBox
gardnervickers commented on pull request #9601: URL: https://github.com/apache/kafka/pull/9601#issuecomment-729887376 OffsetForLeaderEpoch and Produce are not yet versioned RPCs, but will be once #9401 and #9547 are merged. I've removed the taggedFields bump for these RPC's. We can bump

[GitHub] [kafka] gardnervickers removed a comment on pull request #9601: KAFKA-10729: Bump remaining RPC's to use tagged fields.

2020-11-18 Thread GitBox
gardnervickers removed a comment on pull request #9601: URL: https://github.com/apache/kafka/pull/9601#issuecomment-729796082 I realize now that we have several requests which have not been converted to use the generated RPC's. It appears we need to do that to support tagged fields.

[GitHub] [kafka] wcarlson5 commented on pull request #9581: KAFKA-10500: Add thread

2020-11-18 Thread GitBox
wcarlson5 commented on pull request #9581: URL: https://github.com/apache/kafka/pull/9581#issuecomment-729885913 Moved to https://github.com/apache/kafka/pull/9615 This is an automated message from the Apache Git Service. To

[GitHub] [kafka] wcarlson5 closed pull request #9581: KAFKA-10500: Add thread

2020-11-18 Thread GitBox
wcarlson5 closed pull request #9581: URL: https://github.com/apache/kafka/pull/9581 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] wcarlson5 opened a new pull request #9615: KAFKA-10500: Add thread option

2020-11-18 Thread GitBox
wcarlson5 opened a new pull request #9615: URL: https://github.com/apache/kafka/pull/9615 Can add stream threads now replace https://github.com/apache/kafka/pull/9581 after rebase ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation

[GitHub] [kafka] lct45 commented on a change in pull request #9614: KAFKA-10500: Add failed-stream-threads metric for adding + removing stream threads

2020-11-18 Thread GitBox
lct45 commented on a change in pull request #9614: URL: https://github.com/apache/kafka/pull/9614#discussion_r526331025 ## File path: streams/src/test/java/org/apache/kafka/streams/integration/MetricsIntegrationTest.java ## @@ -377,7 +378,7 @@ private void

[jira] [Updated] (KAFKA-10738) kafka version 2.5.0 on Windows Cygwin could not started when using openjdk or corretto 11

2020-11-18 Thread Duke Pham (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duke Pham updated KAFKA-10738: -- Description: h5. kafka version 2.5.0 on Windows Cygwin could not started when using openjdk or

[jira] [Updated] (KAFKA-10738) kafka version 2.5.0 on Windows Cygwin could not started when using openjdk or corretto 11

2020-11-18 Thread Duke Pham (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duke Pham updated KAFKA-10738: -- Description: h5. kafka version 2.5.0 on Windows Cygwin could not started when using openjdk or

[jira] [Updated] (KAFKA-10738) kafka version 2.5.0 on Windows Cygwin could not started when using openjdk or corretto 11

2020-11-18 Thread Duke Pham (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duke Pham updated KAFKA-10738: -- Description: h5. kafka version 2.5.0 on Windows Cygwin could not started when using openjdk or

[jira] [Created] (KAFKA-10738) kafka version 2.5.0 on Windows Cygwin could not started when using openjdk or corretto 11

2020-11-18 Thread Duke Pham (Jira)
Duke Pham created KAFKA-10738: - Summary: kafka version 2.5.0 on Windows Cygwin could not started when using openjdk or corretto 11 Key: KAFKA-10738 URL: https://issues.apache.org/jira/browse/KAFKA-10738

[GitHub] [kafka] hachikuji commented on pull request #9401: KAFKA-9628 Replace Produce request/response with automated protocol

2020-11-18 Thread GitBox
hachikuji commented on pull request #9401: URL: https://github.com/apache/kafka/pull/9401#issuecomment-729870471 Great work @chia7712 ! With this and #9547, we have converted all of the protocols, which was a huge community effort!

[jira] [Commented] (KAFKA-10734) Speedup the processing of LeaderAndIsr request

2020-11-18 Thread Jun Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17234891#comment-17234891 ] Jun Rao commented on KAFKA-10734: - [~luwang], thanks for the jira. In practice, if n is large, m could

[GitHub] [kafka] lct45 commented on pull request #9614: KAFKA-10500: Add failed-stream-threads metric for adding + removing stream threads

2020-11-18 Thread GitBox
lct45 commented on pull request #9614: URL: https://github.com/apache/kafka/pull/9614#issuecomment-729864151 @wcarlson5 @cadonna for initial review This is an automated message from the Apache Git Service. To respond to the

[GitHub] [kafka] lct45 opened a new pull request #9614: KAFKA-10500: Add failed-stream-threads metric for adding + removing stream threads

2020-11-18 Thread GitBox
lct45 opened a new pull request #9614: URL: https://github.com/apache/kafka/pull/9614 Per [KIP-663](https://cwiki.apache.org/confluence/display/KAFKA/KIP-663%3A+API+to+Start+and+Shut+Down+Stream+Threads), adding a metric to record the failed streams threads over the life of a client.

[GitHub] [kafka] chia7712 commented on a change in pull request #9401: KAFKA-9628 Replace Produce request/response with automated protocol

2020-11-18 Thread GitBox
chia7712 commented on a change in pull request #9401: URL: https://github.com/apache/kafka/pull/9401#discussion_r526314648 ## File path: clients/src/main/resources/common/message/ProduceRequest.json ## @@ -33,21 +33,21 @@ "validVersions": "0-8", "flexibleVersions":

[GitHub] [kafka] hachikuji commented on a change in pull request #9401: KAFKA-9628 Replace Produce request/response with automated protocol

2020-11-18 Thread GitBox
hachikuji commented on a change in pull request #9401: URL: https://github.com/apache/kafka/pull/9401#discussion_r526309208 ## File path: clients/src/main/resources/common/message/ProduceRequest.json ## @@ -33,21 +33,21 @@ "validVersions": "0-8", "flexibleVersions":

[GitHub] [kafka] chia7712 commented on a change in pull request #9547: KAFKA-9630; Replace OffsetsForLeaderEpoch request/response with automated protocol

2020-11-18 Thread GitBox
chia7712 commented on a change in pull request #9547: URL: https://github.com/apache/kafka/pull/9547#discussion_r526301121 ## File path: clients/src/main/java/org/apache/kafka/common/requests/OffsetsForLeaderEpochRequest.java ## @@ -51,169 +47,120 @@ */ public

[GitHub] [kafka] hachikuji commented on a change in pull request #9547: KAFKA-9630; Replace OffsetsForLeaderEpoch request/response with automated protocol

2020-11-18 Thread GitBox
hachikuji commented on a change in pull request #9547: URL: https://github.com/apache/kafka/pull/9547#discussion_r526287129 ## File path: clients/src/main/java/org/apache/kafka/common/requests/OffsetsForLeaderEpochRequest.java ## @@ -51,169 +47,120 @@ */ public

  1   2   >