[jira] [Commented] (KAFKA-12776) Producer sends messages out-of-order inspite of enabling idempotence

2021-05-13 Thread NEERAJ VAIDYA (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17343746#comment-17343746 ] NEERAJ VAIDYA commented on KAFKA-12776: --- Also, why does the Producer.send API call

[jira] [Commented] (KAFKA-12776) Producer sends messages out-of-order inspite of enabling idempotence

2021-05-13 Thread NEERAJ VAIDYA (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17343768#comment-17343768 ] NEERAJ VAIDYA commented on KAFKA-12776: --- Here are the config values that I use in

[GitHub] [kafka] wanwenli opened a new pull request #10686: KAFKA-4083: prevent changing replication factor via partition reassignment

2021-05-13 Thread GitBox
wanwenli opened a new pull request #10686: URL: https://github.com/apache/kafka/pull/10686 ### Summary `ReassignPartitionsCommand` does _not_ check whether the replication factor of a partition is the same as existing replication factor, which may result that different partitions of

[jira] [Updated] (KAFKA-12743) [Kafka Streams] - cluster failover for stateful Kafka Streams applications

2021-05-13 Thread Sergey Zyrianov (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Zyrianov updated KAFKA-12743: Summary: [Kafka Streams] - cluster failover for stateful Kafka Streams applications (was:

[GitHub] [kafka] ViE-17-tuoi opened a new pull request #10687: KAFKA-12380 Executor in Connect's Worker is not shut down when the worker is

2021-05-13 Thread GitBox
ViE-17-tuoi opened a new pull request #10687: URL: https://github.com/apache/kafka/pull/10687 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of tes

[GitHub] [kafka] DuongPTIT commented on pull request #10670: KAFKA-10273 Connect Converters should produce actionable error messages

2021-05-13 Thread GitBox
DuongPTIT commented on pull request #10670: URL: https://github.com/apache/kafka/pull/10670#issuecomment-840464740 @rhauch Please take a look. Thank you. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[GitHub] [kafka] mimaison commented on pull request #10660: MINOR: Updating files with release 2.7.1

2021-05-13 Thread GitBox
mimaison commented on pull request #10660: URL: https://github.com/apache/kafka/pull/10660#issuecomment-840484650 @mjsax @vvcephei can you take a look? Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[GitHub] [kafka] eas5 commented on pull request #10682: MINOR: Refactored test using parameterization

2021-05-13 Thread GitBox
eas5 commented on pull request #10682: URL: https://github.com/apache/kafka/pull/10682#issuecomment-840526680 Dear committers, I believe the build failure is unrelated to this pull request, once I have been able to run the tests locally successfully. If possible, I would like you to

[GitHub] [kafka] cmccabe commented on a change in pull request #10679: KAFKA-12697: Add Global Topic and Partition count metrics to the Quorum Controller

2021-05-13 Thread GitBox
cmccabe commented on a change in pull request #10679: URL: https://github.com/apache/kafka/pull/10679#discussion_r631802796 ## File path: metadata/src/main/java/org/apache/kafka/controller/QuorumControllerMetrics.java ## @@ -30,14 +30,24 @@ "kafka.controller", "Contro

[GitHub] [kafka] cmccabe commented on a change in pull request #10679: KAFKA-12697: Add Global Topic and Partition count metrics to the Quorum Controller

2021-05-13 Thread GitBox
cmccabe commented on a change in pull request #10679: URL: https://github.com/apache/kafka/pull/10679#discussion_r631802943 ## File path: metadata/src/main/java/org/apache/kafka/controller/QuorumControllerMetrics.java ## @@ -30,14 +30,24 @@ "kafka.controller", "Contro

[GitHub] [kafka] cmccabe commented on a change in pull request #10679: KAFKA-12697: Add Global Topic and Partition count metrics to the Quorum Controller

2021-05-13 Thread GitBox
cmccabe commented on a change in pull request #10679: URL: https://github.com/apache/kafka/pull/10679#discussion_r631804785 ## File path: metadata/src/main/java/org/apache/kafka/controller/QuorumControllerMetrics.java ## @@ -30,14 +30,24 @@ "kafka.controller", "Contro

[GitHub] [kafka] cmccabe commented on a change in pull request #10679: KAFKA-12697: Add Global Topic and Partition count metrics to the Quorum Controller

2021-05-13 Thread GitBox
cmccabe commented on a change in pull request #10679: URL: https://github.com/apache/kafka/pull/10679#discussion_r631804888 ## File path: metadata/src/main/java/org/apache/kafka/controller/QuorumControllerMetrics.java ## @@ -30,14 +30,24 @@ "kafka.controller", "Contro

[GitHub] [kafka] cmccabe commented on a change in pull request #10679: KAFKA-12697: Add Global Topic and Partition count metrics to the Quorum Controller

2021-05-13 Thread GitBox
cmccabe commented on a change in pull request #10679: URL: https://github.com/apache/kafka/pull/10679#discussion_r631804785 ## File path: metadata/src/main/java/org/apache/kafka/controller/QuorumControllerMetrics.java ## @@ -30,14 +30,24 @@ "kafka.controller", "Contro

[GitHub] [kafka] cmccabe commented on a change in pull request #10679: KAFKA-12697: Add Global Topic and Partition count metrics to the Quorum Controller

2021-05-13 Thread GitBox
cmccabe commented on a change in pull request #10679: URL: https://github.com/apache/kafka/pull/10679#discussion_r631806532 ## File path: metadata/src/main/java/org/apache/kafka/controller/ReplicationControlManager.java ## @@ -279,6 +279,8 @@ public String toString() { *

[GitHub] [kafka] cmccabe commented on a change in pull request #10679: KAFKA-12697: Add Global Topic and Partition count metrics to the Quorum Controller

2021-05-13 Thread GitBox
cmccabe commented on a change in pull request #10679: URL: https://github.com/apache/kafka/pull/10679#discussion_r631807221 ## File path: metadata/src/main/java/org/apache/kafka/controller/ReplicationControlManager.java ## @@ -343,12 +354,14 @@ public void replay(PartitionReco

[GitHub] [kafka] C0urante commented on a change in pull request #10528: KAFKA-12497: Skip unnecessary source task offset commits

2021-05-13 Thread GitBox
C0urante commented on a change in pull request #10528: URL: https://github.com/apache/kafka/pull/10528#discussion_r631840386 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/SourceTaskOffsetCommitter.java ## @@ -105,6 +105,11 @@ public void remove(C

[GitHub] [kafka] C0urante commented on a change in pull request #10528: KAFKA-12497: Skip unnecessary source task offset commits

2021-05-13 Thread GitBox
C0urante commented on a change in pull request #10528: URL: https://github.com/apache/kafka/pull/10528#discussion_r631842691 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSourceTask.java ## @@ -482,6 +486,22 @@ private synchronized void rec

[GitHub] [kafka] ViE-17-tuoi closed pull request #10687: KAFKA-12380 Executor in Connect's Worker is not shut down when the worker is

2021-05-13 Thread GitBox
ViE-17-tuoi closed pull request #10687: URL: https://github.com/apache/kafka/pull/10687 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, pl

[jira] [Created] (KAFKA-12777) AutoTopicCreationManager does not handle response errors

2021-05-13 Thread David Arthur (Jira)
David Arthur created KAFKA-12777: Summary: AutoTopicCreationManager does not handle response errors Key: KAFKA-12777 URL: https://issues.apache.org/jira/browse/KAFKA-12777 Project: Kafka Issu

[jira] [Updated] (KAFKA-12777) AutoTopicCreationManager does not handle response errors

2021-05-13 Thread David Arthur (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Arthur updated KAFKA-12777: - Affects Version/s: 3.0.0 > AutoTopicCreationManager does not handle response errors > --

[jira] [Updated] (KAFKA-12777) AutoTopicCreationManager does not handle response errors

2021-05-13 Thread David Arthur (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Arthur updated KAFKA-12777: - Description: The request completion callback in AutoTopicCreationManager assumes the response i

[jira] [Updated] (KAFKA-12777) AutoTopicCreationManager does not handle response errors

2021-05-13 Thread David Arthur (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Arthur updated KAFKA-12777: - Description: The request completion callback in AutoTopicCreationManager assumes the response i

[GitHub] [kafka] C0urante commented on a change in pull request #10528: KAFKA-12497: Skip unnecessary source task offset commits

2021-05-13 Thread GitBox
C0urante commented on a change in pull request #10528: URL: https://github.com/apache/kafka/pull/10528#discussion_r631840386 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/SourceTaskOffsetCommitter.java ## @@ -105,6 +105,11 @@ public void remove(C

[jira] [Commented] (KAFKA-12777) AutoTopicCreationManager does not handle response errors

2021-05-13 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17343949#comment-17343949 ] Ismael Juma commented on KAFKA-12777: - Is this really "minor"? > AutoTopicCreationM

[GitHub] [kafka] dejan2609 commented on pull request #10606: KAFKA-12728: version upgrades: gradle (6.8.3 -->> 7.0.1) and gradle shadow plugin (6.1.0 -->> 7.0.0)

2021-05-13 Thread GitBox
dejan2609 commented on pull request #10606: URL: https://github.com/apache/kafka/pull/10606#issuecomment-840647187 Ok, closing the gap here: - Kafka Jenkins could skip JDK update to a version "1.8.0_292" (see previous comments); note: there is a workaround in Gradle for that version, b

[GitHub] [kafka] dejan2609 edited a comment on pull request #10606: KAFKA-12728: version upgrades: gradle (6.8.3 -->> 7.0.1) and gradle shadow plugin (6.1.0 -->> 7.0.0)

2021-05-13 Thread GitBox
dejan2609 edited a comment on pull request #10606: URL: https://github.com/apache/kafka/pull/10606#issuecomment-840647187 Ok, closing the gap here: - Kafka Jenkins could skip JDK update to a version "1.8.0_292" (see previous comments). Note: there is a workaround in Gradle for that ver

[jira] [Updated] (KAFKA-12777) AutoTopicCreationManager does not handle response errors

2021-05-13 Thread David Arthur (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Arthur updated KAFKA-12777: - Priority: Major (was: Minor) > AutoTopicCreationManager does not handle response errors > -

[jira] [Commented] (KAFKA-12777) AutoTopicCreationManager does not handle response errors

2021-05-13 Thread David Arthur (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17343964#comment-17343964 ] David Arthur commented on KAFKA-12777: -- I think this is mostly cosmetic since the o

[GitHub] [kafka] bruto1 commented on pull request #10590: KAFKA-5761: support ByteBuffer as value in ProducerRecord and avoid redundant serialization when it's used

2021-05-13 Thread GitBox
bruto1 commented on pull request #10590: URL: https://github.com/apache/kafka/pull/10590#issuecomment-840670124 @jolshan ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [kafka] ijuma commented on pull request #10606: KAFKA-12728: version upgrades: gradle (6.8.3 -->> 7.0.1) and gradle shadow plugin (6.1.0 -->> 7.0.0)

2021-05-13 Thread GitBox
ijuma commented on pull request #10606: URL: https://github.com/apache/kafka/pull/10606#issuecomment-840682476 @dejan2609 did the exclusion stop working with the upgrade to Gradle 7.0? That seems like the most plausible explanation I've seen so far. -- This is an automated message from t

[GitHub] [kafka] wcarlson5 commented on a change in pull request #10676: KAFKA-12648: Pt. 0 - Add TopologyMetadata.Subtopology class for subtopology metadata

2021-05-13 Thread GitBox
wcarlson5 commented on a change in pull request #10676: URL: https://github.com/apache/kafka/pull/10676#discussion_r631951243 ## File path: streams/src/test/java/org/apache/kafka/streams/TopologyTest.java ## @@ -427,8 +428,8 @@ public void singleSourceShouldHaveSingleSubtopolog

[GitHub] [kafka] wcarlson5 commented on a change in pull request #10666: MINOR: prevent cleanup() from being called while Streams is still shutting down

2021-05-13 Thread GitBox
wcarlson5 commented on a change in pull request #10666: URL: https://github.com/apache/kafka/pull/10666#discussion_r631954181 ## File path: streams/src/test/java/org/apache/kafka/streams/KafkaStreamsTest.java ## @@ -298,7 +300,9 @@ private void prepareStreams() throws Exception

[GitHub] [kafka] dejan2609 commented on pull request #10606: KAFKA-12728: version upgrades: gradle (6.8.3 -->> 7.0.1) and gradle shadow plugin (6.1.0 -->> 7.0.0)

2021-05-13 Thread GitBox
dejan2609 commented on pull request #10606: URL: https://github.com/apache/kafka/pull/10606#issuecomment-840696038 > @dejan2609 did the exclusion stop working with the upgrade to Gradle 7.0? That seems like the most plausible explanation I've seen so far. Thing is that exclusion cove

[GitHub] [kafka] dejan2609 edited a comment on pull request #10606: KAFKA-12728: version upgrades: gradle (6.8.3 -->> 7.0.1) and gradle shadow plugin (6.1.0 -->> 7.0.0)

2021-05-13 Thread GitBox
dejan2609 edited a comment on pull request #10606: URL: https://github.com/apache/kafka/pull/10606#issuecomment-840696038 > @dejan2609 did the exclusion stop working with the upgrade to Gradle 7.0? That seems like the most plausible explanation I've seen so far. Thing is that exclusi

[GitHub] [kafka] wcarlson5 commented on a change in pull request #10609: KAFKA-12648: Pt. 1 - Add NamedTopology to protocol and state directory structure

2021-05-13 Thread GitBox
wcarlson5 commented on a change in pull request #10609: URL: https://github.com/apache/kafka/pull/10609#discussion_r631965080 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/InternalTopologyBuilder.java ## @@ -2035,6 +2035,11 @@ private void up

[GitHub] [kafka] dejan2609 commented on pull request #10606: KAFKA-12728: version upgrades: gradle (6.8.3 -->> 7.0.1) and gradle shadow plugin (6.1.0 -->> 7.0.0)

2021-05-13 Thread GitBox
dejan2609 commented on pull request #10606: URL: https://github.com/apache/kafka/pull/10606#issuecomment-840711267 So, just to sum it up: - similar `UniqueResourceException` issues with `api-ldap-schema-data` module are solved for a `testCompile` level here: https://github.com/apache/

[jira] [Commented] (KAFKA-3539) KafkaProducer.send() may block even though it returns the Future

2021-05-13 Thread Moses Nakamura (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-3539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17344023#comment-17344023 ] Moses Nakamura commented on KAFKA-3539: --- [~ewencp] thanks, I started the KIP proces

[GitHub] [kafka] dejan2609 edited a comment on pull request #10606: KAFKA-12728: version upgrades: gradle (6.8.3 -->> 7.0.1) and gradle shadow plugin (6.1.0 -->> 7.0.0)

2021-05-13 Thread GitBox
dejan2609 edited a comment on pull request #10606: URL: https://github.com/apache/kafka/pull/10606#issuecomment-840711267 So, just to sum it up: - similar `UniqueResourceException` issues with `api-ldap-schema-data` module are solved for a `testCompile` level here: https://github.com/

[jira] [Commented] (KAFKA-10594) Enhance Raft exception handling

2021-05-13 Thread Jose Armando Garcia Sancio (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17344032#comment-17344032 ] Jose Armando Garcia Sancio commented on KAFKA-10594: I agree with th

[jira] [Assigned] (KAFKA-10594) Enhance Raft exception handling

2021-05-13 Thread Jose Armando Garcia Sancio (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jose Armando Garcia Sancio reassigned KAFKA-10594: -- Assignee: (was: Boyang Chen) > Enhance Raft exception han

[GitHub] [kafka] junrao commented on a change in pull request #10494: MINOR: improve KRaft replica placement

2021-05-13 Thread GitBox
junrao commented on a change in pull request #10494: URL: https://github.com/apache/kafka/pull/10494#discussion_r631962390 ## File path: metadata/src/main/java/org/apache/kafka/controller/StripedReplicaPlacer.java ## @@ -0,0 +1,411 @@ +/* + * Licensed to the Apache Software Fo

[jira] [Commented] (KAFKA-12776) Producer sends messages out-of-order inspite of enabling idempotence

2021-05-13 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17344038#comment-17344038 ] A. Sophie Blee-Goldman commented on KAFKA-12776: Hey [~neeraj.vaidya],

[GitHub] [kafka] ableegoldman commented on a change in pull request #10666: MINOR: prevent cleanup() from being called while Streams is still shutting down

2021-05-13 Thread GitBox
ableegoldman commented on a change in pull request #10666: URL: https://github.com/apache/kafka/pull/10666#discussion_r631997442 ## File path: streams/src/test/java/org/apache/kafka/streams/KafkaStreamsTest.java ## @@ -298,7 +300,9 @@ private void prepareStreams() throws Except

[GitHub] [kafka] ijuma commented on a change in pull request #10606: KAFKA-12728: version upgrades: gradle (6.8.3 -->> 7.0.1) and gradle shadow plugin (6.1.0 -->> 7.0.0)

2021-05-13 Thread GitBox
ijuma commented on a change in pull request #10606: URL: https://github.com/apache/kafka/pull/10606#discussion_r63128 ## File path: build.gradle ## @@ -849,7 +849,7 @@ project(':core') { compile.exclude module: 'mail' // To prevent a UniqueResourceException due th

[jira] [Commented] (KAFKA-12776) Producer sends messages out-of-order inspite of enabling idempotence

2021-05-13 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17344042#comment-17344042 ] A. Sophie Blee-Goldman commented on KAFKA-12776: Actually it turns out a

[GitHub] [kafka] ccding commented on pull request #10680: Rework on KAFKA-3968: fsync the parent directory of a segment file when the file is created

2021-05-13 Thread GitBox
ccding commented on pull request #10680: URL: https://github.com/apache/kafka/pull/10680#issuecomment-840742289 I finished performance tests and this PR has minimal impact on performance. It can successfully create a topic with 9000 partitions. @junrao please take a look cc @ijuma

[GitHub] [kafka] ableegoldman commented on a change in pull request #10676: KAFKA-12648: Pt. 0 - Add TopologyMetadata.Subtopology class for subtopology metadata

2021-05-13 Thread GitBox
ableegoldman commented on a change in pull request #10676: URL: https://github.com/apache/kafka/pull/10676#discussion_r632010573 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/TaskId.java ## @@ -59,10 +58,6 @@ public TaskId(final int topicGroupId, final

[GitHub] [kafka] ableegoldman commented on a change in pull request #10676: KAFKA-12648: Pt. 0 - Add TopologyMetadata.Subtopology class for subtopology metadata

2021-05-13 Thread GitBox
ableegoldman commented on a change in pull request #10676: URL: https://github.com/apache/kafka/pull/10676#discussion_r632011373 ## File path: streams/src/test/java/org/apache/kafka/streams/TopologyTest.java ## @@ -427,8 +428,8 @@ public void singleSourceShouldHaveSingleSubtopo

[GitHub] [kafka] ableegoldman commented on pull request #10676: KAFKA-12648: MINOR - Add TopologyMetadata.Subtopology class for subtopology metadata

2021-05-13 Thread GitBox
ableegoldman commented on pull request #10676: URL: https://github.com/apache/kafka/pull/10676#issuecomment-840744536 I'm going to merge this without another committer +1 since it's a minor PR with mostly renaming/wrapping classes. And the PR build is actually completely green for once 🥳

[GitHub] [kafka] ableegoldman merged pull request #10676: KAFKA-12648: MINOR - Add TopologyMetadata.Subtopology class for subtopology metadata

2021-05-13 Thread GitBox
ableegoldman merged pull request #10676: URL: https://github.com/apache/kafka/pull/10676 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, p

[GitHub] [kafka] wcarlson5 commented on a change in pull request #10676: KAFKA-12648: MINOR - Add TopologyMetadata.Subtopology class for subtopology metadata

2021-05-13 Thread GitBox
wcarlson5 commented on a change in pull request #10676: URL: https://github.com/apache/kafka/pull/10676#discussion_r632019986 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/TaskId.java ## @@ -59,10 +58,6 @@ public TaskId(final int topicGroupId, final int

[GitHub] [kafka] wcarlson5 commented on a change in pull request #10666: MINOR: prevent cleanup() from being called while Streams is still shutting down

2021-05-13 Thread GitBox
wcarlson5 commented on a change in pull request #10666: URL: https://github.com/apache/kafka/pull/10666#discussion_r632021643 ## File path: streams/src/test/java/org/apache/kafka/streams/KafkaStreamsTest.java ## @@ -298,7 +300,9 @@ private void prepareStreams() throws Exception

[GitHub] [kafka] ableegoldman commented on a change in pull request #10609: KAFKA-12648: Pt. 1 - Add NamedTopology to protocol and state directory structure

2021-05-13 Thread GitBox
ableegoldman commented on a change in pull request #10609: URL: https://github.com/apache/kafka/pull/10609#discussion_r632022730 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/TaskId.java ## @@ -59,8 +58,8 @@ public TaskId(final int topicGroupId, final i

[GitHub] [kafka] ableegoldman commented on a change in pull request #10609: KAFKA-12648: Pt. 1 - Add NamedTopology to protocol and state directory structure

2021-05-13 Thread GitBox
ableegoldman commented on a change in pull request #10609: URL: https://github.com/apache/kafka/pull/10609#discussion_r632024345 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/InternalTopologyBuilder.java ## @@ -2035,6 +2035,11 @@ private void

[GitHub] [kafka] ableegoldman commented on a change in pull request #10609: KAFKA-12648: Pt. 1 - Add NamedTopology to protocol and state directory structure

2021-05-13 Thread GitBox
ableegoldman commented on a change in pull request #10609: URL: https://github.com/apache/kafka/pull/10609#discussion_r632025872 ## File path: streams/src/test/java/org/apache/kafka/streams/processor/internals/StateDirectoryTest.java ## @@ -372,29 +356,25 @@ public void shoul

[GitHub] [kafka] wcarlson5 commented on a change in pull request #10609: KAFKA-12648: Pt. 1 - Add NamedTopology to protocol and state directory structure

2021-05-13 Thread GitBox
wcarlson5 commented on a change in pull request #10609: URL: https://github.com/apache/kafka/pull/10609#discussion_r632029533 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/TaskId.java ## @@ -59,8 +58,8 @@ public TaskId(final int topicGroupId, final int

[GitHub] [kafka] ableegoldman commented on a change in pull request #10666: MINOR: prevent cleanup() from being called while Streams is still shutting down

2021-05-13 Thread GitBox
ableegoldman commented on a change in pull request #10666: URL: https://github.com/apache/kafka/pull/10666#discussion_r632038675 ## File path: streams/src/test/java/org/apache/kafka/streams/KafkaStreamsTest.java ## @@ -298,7 +300,9 @@ private void prepareStreams() throws Except

[GitHub] [kafka] cmccabe merged pull request #10679: KAFKA-12697: Add Global Topic and Partition count metrics to the Quorum Controller

2021-05-13 Thread GitBox
cmccabe merged pull request #10679: URL: https://github.com/apache/kafka/pull/10679 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [kafka] ableegoldman commented on a change in pull request #10666: MINOR: prevent cleanup() from being called while Streams is still shutting down

2021-05-13 Thread GitBox
ableegoldman commented on a change in pull request #10666: URL: https://github.com/apache/kafka/pull/10666#discussion_r632040863 ## File path: streams/src/test/java/org/apache/kafka/streams/integration/StandbyTaskEOSIntegrationTest.java ## @@ -170,8 +170,8 @@ public void shoul

[jira] [Commented] (KAFKA-10594) Enhance Raft exception handling

2021-05-13 Thread Boyang Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17344088#comment-17344088 ] Boyang Chen commented on KAFKA-10594: - That's correct, thanks! [~jagsancio] > Enhan

[GitHub] [kafka] wcarlson5 commented on a change in pull request #10666: MINOR: prevent cleanup() from being called while Streams is still shutting down

2021-05-13 Thread GitBox
wcarlson5 commented on a change in pull request #10666: URL: https://github.com/apache/kafka/pull/10666#discussion_r632047274 ## File path: streams/src/test/java/org/apache/kafka/streams/integration/StandbyTaskEOSIntegrationTest.java ## @@ -170,8 +170,8 @@ public void shouldSu

[GitHub] [kafka] dejan2609 commented on a change in pull request #10606: KAFKA-12728: version upgrades: gradle (6.8.3 -->> 7.0.1) and gradle shadow plugin (6.1.0 -->> 7.0.0)

2021-05-13 Thread GitBox
dejan2609 commented on a change in pull request #10606: URL: https://github.com/apache/kafka/pull/10606#discussion_r632057098 ## File path: build.gradle ## @@ -849,7 +849,7 @@ project(':core') { compile.exclude module: 'mail' // To prevent a UniqueResourceException du

[jira] [Created] (KAFKA-12778) QuorumController should honor request timeouts

2021-05-13 Thread Colin McCabe (Jira)
Colin McCabe created KAFKA-12778: Summary: QuorumController should honor request timeouts Key: KAFKA-12778 URL: https://issues.apache.org/jira/browse/KAFKA-12778 Project: Kafka Issue Type: Bu

[GitHub] [kafka] cmccabe opened a new pull request #10688: KAFKA-12778: Fix QuorumController request timeouts and electLeaders

2021-05-13 Thread GitBox
cmccabe opened a new pull request #10688: URL: https://github.com/apache/kafka/pull/10688 Not all RPC requests to the quorum controller include a timeout, but we should honor the timeouts that do exist. For electLeaders, attempt to trigger a leader election for all partitions wh

[GitHub] [kafka] ijuma commented on a change in pull request #10606: KAFKA-12728: version upgrades: gradle (6.8.3 -->> 7.0.1) and gradle shadow plugin (6.1.0 -->> 7.0.0)

2021-05-13 Thread GitBox
ijuma commented on a change in pull request #10606: URL: https://github.com/apache/kafka/pull/10606#discussion_r632062402 ## File path: build.gradle ## @@ -849,7 +849,7 @@ project(':core') { compile.exclude module: 'mail' // To prevent a UniqueResourceException due th

[jira] [Comment Edited] (KAFKA-10594) Enhance Raft exception handling

2021-05-13 Thread Jose Armando Garcia Sancio (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17344032#comment-17344032 ] Jose Armando Garcia Sancio edited comment on KAFKA-10594 at 5/13/21, 7:53 PM:

[GitHub] [kafka] ableegoldman commented on a change in pull request #10666: MINOR: prevent cleanup() from being called while Streams is still shutting down

2021-05-13 Thread GitBox
ableegoldman commented on a change in pull request #10666: URL: https://github.com/apache/kafka/pull/10666#discussion_r632065843 ## File path: streams/src/test/java/org/apache/kafka/streams/integration/StandbyTaskEOSIntegrationTest.java ## @@ -170,8 +170,8 @@ public void shoul

[GitHub] [kafka] jsancio commented on a change in pull request #10431: KAFKA-12543: Change RawSnapshotReader ownership model

2021-05-13 Thread GitBox
jsancio commented on a change in pull request #10431: URL: https://github.com/apache/kafka/pull/10431#discussion_r632066530 ## File path: raft/src/main/java/org/apache/kafka/snapshot/Snapshots.java ## @@ -104,18 +105,29 @@ public static Path createTempFile(Path logDir, OffsetA

[GitHub] [kafka] ableegoldman commented on a change in pull request #10666: MINOR: prevent cleanup() from being called while Streams is still shutting down

2021-05-13 Thread GitBox
ableegoldman commented on a change in pull request #10666: URL: https://github.com/apache/kafka/pull/10666#discussion_r632066989 ## File path: streams/src/test/java/org/apache/kafka/streams/integration/StandbyTaskEOSIntegrationTest.java ## @@ -170,8 +170,8 @@ public void shoul

[GitHub] [kafka] dejan2609 commented on a change in pull request #10606: KAFKA-12728: version upgrades: gradle (6.8.3 -->> 7.0.1) and gradle shadow plugin (6.1.0 -->> 7.0.0)

2021-05-13 Thread GitBox
dejan2609 commented on a change in pull request #10606: URL: https://github.com/apache/kafka/pull/10606#discussion_r632083522 ## File path: build.gradle ## @@ -849,7 +849,7 @@ project(':core') { compile.exclude module: 'mail' // To prevent a UniqueResourceException du

[jira] [Updated] (KAFKA-12779) Use TaskId instead of String for the taskId field in TaskMetadata

2021-05-13 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman updated KAFKA-12779: --- Labels: needs-kip (was: ) > Use TaskId instead of String for the taskId fie

[jira] [Assigned] (KAFKA-12779) Use TaskId instead of String for the taskId field in TaskMetadata

2021-05-13 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman reassigned KAFKA-12779: -- Assignee: A. Sophie Blee-Goldman > Use TaskId instead of String for t

[jira] [Updated] (KAFKA-12779) Use TaskId instead of String for the taskId field in TaskMetadata

2021-05-13 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman updated KAFKA-12779: --- Fix Version/s: 3.0.0 > Use TaskId instead of String for the taskId field in

[jira] [Created] (KAFKA-12780) java.lang.IllegalStateException: Tried to lookup lag for unknown task 2_0

2021-05-13 Thread Ireneusz Matysiewicz (Jira)
Ireneusz Matysiewicz created KAFKA-12780: Summary: java.lang.IllegalStateException: Tried to lookup lag for unknown task 2_0 Key: KAFKA-12780 URL: https://issues.apache.org/jira/browse/KAFKA-12780

[jira] [Created] (KAFKA-12779) Use TaskId instead of String for the taskId field in TaskMetadata

2021-05-13 Thread A. Sophie Blee-Goldman (Jira)
A. Sophie Blee-Goldman created KAFKA-12779: -- Summary: Use TaskId instead of String for the taskId field in TaskMetadata Key: KAFKA-12779 URL: https://issues.apache.org/jira/browse/KAFKA-12779

[jira] [Updated] (KAFKA-12779) Use TaskId instead of String for the taskId field in TaskMetadata

2021-05-13 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman updated KAFKA-12779: --- Description: Not sure why this was encoded as a String field instead of usin

[jira] [Updated] (KAFKA-12779) Use TaskId instead of String for the taskId field in TaskMetadata

2021-05-13 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman updated KAFKA-12779: --- Description: Not sure why this was encoded as a String field instead of usin

[GitHub] [kafka] jolshan commented on pull request #10590: KAFKA-5761: support ByteBuffer as value in ProducerRecord and avoid redundant serialization when it's used

2021-05-13 Thread GitBox
jolshan commented on pull request #10590: URL: https://github.com/apache/kafka/pull/10590#issuecomment-840829229 Hi @bruto1 can you take a look at PlaintextConsumerTest.testHeadersSerializerDeserializer()? It seems to be failing and might be related to your change. -- This is an automa

[jira] [Commented] (KAFKA-12776) Producer sends messages out-of-order inspite of enabling idempotence

2021-05-13 Thread NEERAJ VAIDYA (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17344126#comment-17344126 ] NEERAJ VAIDYA commented on KAFKA-12776: --- Thanks Sophie. I assumed that if my messa

[GitHub] [kafka] dejan2609 commented on a change in pull request #10606: KAFKA-12728: version upgrades: gradle (6.8.3 -->> 7.0.1) and gradle shadow plugin (6.1.0 -->> 7.0.0)

2021-05-13 Thread GitBox
dejan2609 commented on a change in pull request #10606: URL: https://github.com/apache/kafka/pull/10606#discussion_r632100385 ## File path: build.gradle ## @@ -849,7 +849,7 @@ project(':core') { compile.exclude module: 'mail' // To prevent a UniqueResourceException du

[GitHub] [kafka] ableegoldman commented on a change in pull request #10676: KAFKA-12648: MINOR - Add TopologyMetadata.Subtopology class for subtopology metadata

2021-05-13 Thread GitBox
ableegoldman commented on a change in pull request #10676: URL: https://github.com/apache/kafka/pull/10676#discussion_r632100527 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/TaskId.java ## @@ -59,10 +58,6 @@ public TaskId(final int topicGroupId, final

[jira] [Commented] (KAFKA-12780) java.lang.IllegalStateException: Tried to lookup lag for unknown task 2_0

2021-05-13 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17344131#comment-17344131 ] A. Sophie Blee-Goldman commented on KAFKA-12780: Hey [~iirekm2] This ex

[GitHub] [kafka] wcarlson5 commented on a change in pull request #10666: MINOR: prevent cleanup() from being called while Streams is still shutting down

2021-05-13 Thread GitBox
wcarlson5 commented on a change in pull request #10666: URL: https://github.com/apache/kafka/pull/10666#discussion_r632106662 ## File path: streams/src/test/java/org/apache/kafka/streams/integration/StandbyTaskEOSIntegrationTest.java ## @@ -170,8 +170,8 @@ public void shouldSu

[jira] [Commented] (KAFKA-12780) java.lang.IllegalStateException: Tried to lookup lag for unknown task 2_0

2021-05-13 Thread Ireneusz Matysiewicz (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17344135#comment-17344135 ] Ireneusz Matysiewicz commented on KAFKA-12780: -- I'll check the topologies t

[jira] [Commented] (KAFKA-12776) Producer sends messages out-of-order inspite of enabling idempotence

2021-05-13 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17344150#comment-17344150 ] A. Sophie Blee-Goldman commented on KAFKA-12776: I don't think there's a

[GitHub] [kafka] yeralin commented on pull request #6592: KAFKA-8326: Introduce List Serde

2021-05-13 Thread GitBox
yeralin commented on pull request #6592: URL: https://github.com/apache/kafka/pull/6592#issuecomment-840856470 @ableegoldman Seems like all checks passed :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[jira] [Closed] (KAFKA-12448) STATE_DIR_CONFIG path cannot be found in windows due to usage of setPosixFilePermissions

2021-05-13 Thread Jira
[ https://issues.apache.org/jira/browse/KAFKA-12448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ahmet Yortanlı closed KAFKA-12448. -- > STATE_DIR_CONFIG path cannot be found in windows due to usage of > setPosixFilePermissions > --

[GitHub] [kafka] guozhangwang commented on a change in pull request #10666: MINOR: prevent cleanup() from being called while Streams is still shutting down

2021-05-13 Thread GitBox
guozhangwang commented on a change in pull request #10666: URL: https://github.com/apache/kafka/pull/10666#discussion_r632117934 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/StateDirectory.java ## @@ -413,43 +413,31 @@ private void cleanRemo

[jira] [Commented] (KAFKA-12776) Producer sends messages out-of-order inspite of enabling idempotence

2021-05-13 Thread NEERAJ VAIDYA (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17344153#comment-17344153 ] NEERAJ VAIDYA commented on KAFKA-12776: --- Thanks Sophie. I did try with max infligh

[GitHub] [kafka] ccding commented on pull request #10680: Rework on KAFKA-3968: fsync the parent directory of a segment file when the file is created

2021-05-13 Thread GitBox
ccding commented on pull request #10680: URL: https://github.com/apache/kafka/pull/10680#issuecomment-840877605 The two failed tests are for Scala 2.13, but not Scala 2.12. Both passed on my local run with ``` ./gradlew -PscalaVersion=2.13 core:test --tests kafka.server.RaftClu

[jira] [Reopened] (KAFKA-9295) KTableKTableForeignKeyInnerJoinMultiIntegrationTest#shouldInnerJoinMultiPartitionQueryable

2021-05-13 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman reopened KAFKA-9295: --- Unfortunately it seems to still be failing :/  [https://ci-builds.apache.org/job

[GitHub] [kafka] ableegoldman commented on pull request #6592: KAFKA-8326: Introduce List Serde

2021-05-13 Thread GitBox
ableegoldman commented on pull request #6592: URL: https://github.com/apache/kafka/pull/6592#issuecomment-840878936 Build has only some unrelated test failures in known flaky `KTableKTableForeignKeyInnerJoinMultiIntegrationTest#shouldInnerJoinMultiPartitionQueryable` and `RaftClusterTe

[GitHub] [kafka] ableegoldman merged pull request #6592: KAFKA-8326: Introduce List Serde

2021-05-13 Thread GitBox
ableegoldman merged pull request #6592: URL: https://github.com/apache/kafka/pull/6592 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, ple

[GitHub] [kafka] ableegoldman commented on pull request #6592: KAFKA-8326: Introduce List Serde

2021-05-13 Thread GitBox
ableegoldman commented on pull request #6592: URL: https://github.com/apache/kafka/pull/6592#issuecomment-840880983 Merged to trunk! Thanks @yeralin for all the work and patience it took to get this PR in, and to all the reviewers who helped get it here along the way. @yeralin, can y

[jira] [Resolved] (KAFKA-8326) Add Serde> support

2021-05-13 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman resolved KAFKA-8326. --- Fix Version/s: 3.0.0 Resolution: Fixed > Add Serde> support >

[GitHub] [kafka] dielhennr opened a new pull request #10689: KAFKA-12697: Add EventQueueSize metric to Quorum controller

2021-05-13 Thread GitBox
dielhennr opened a new pull request #10689: URL: https://github.com/apache/kafka/pull/10689 A counter is kept in the event queue. Tested using events that wait on a CountDownLatch before completing. -- This is an automated message from the Apache Git Service. To respond to

[jira] [Commented] (KAFKA-12776) Producer sends messages out-of-order inspite of enabling idempotence

2021-05-13 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17344184#comment-17344184 ] A. Sophie Blee-Goldman commented on KAFKA-12776: I think the messages sh

[GitHub] [kafka] ableegoldman commented on pull request #10666: MINOR: prevent cleanup() from being called while Streams is still shutting down

2021-05-13 Thread GitBox
ableegoldman commented on pull request #10666: URL: https://github.com/apache/kafka/pull/10666#issuecomment-840888170 Just some unrelated flaky tests: `kafka.network.DynamicConnectionQuotaTest.testDynamicListenerConnectionCreationRateQuota()` `kafka.connect.integration.ConnectorClien

[GitHub] [kafka] ableegoldman merged pull request #10666: MINOR: prevent cleanup() from being called while Streams is still shutting down

2021-05-13 Thread GitBox
ableegoldman merged pull request #10666: URL: https://github.com/apache/kafka/pull/10666 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, p

[jira] [Commented] (KAFKA-12776) Producer sends messages out-of-order inspite of enabling idempotence

2021-05-13 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17344185#comment-17344185 ] Ismael Juma commented on KAFKA-12776: - [~ableegoldman] The max in flight change is o

  1   2   >