[jira] [Updated] (KAFKA-12676) Improve sticky general assignor underlying algorithm for the imbalanced case

2021-04-15 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen updated KAFKA-12676: -- Description: As discussed in KAFKA-12675, we think the general assignor algorithm might be able to

[jira] [Commented] (KAFKA-12675) Improve sticky general assignor scalability and performance

2021-04-15 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17322608#comment-17322608 ] Luke Chen commented on KAFKA-12675: --- [~twmb], I see. I've created another ticket KAFKA-12676 to

[jira] [Created] (KAFKA-12676) Improve sticky general assignor underlying algorithm for the imbalanced case

2021-04-15 Thread Luke Chen (Jira)
Luke Chen created KAFKA-12676: - Summary: Improve sticky general assignor underlying algorithm for the imbalanced case Key: KAFKA-12676 URL: https://issues.apache.org/jira/browse/KAFKA-12676 Project:

[jira] [Comment Edited] (KAFKA-12675) Improve sticky general assignor scalability and performance

2021-04-15 Thread Travis Bischel (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17322603#comment-17322603 ] Travis Bischel edited comment on KAFKA-12675 at 4/16/21, 4:25 AM: -- What

[jira] [Comment Edited] (KAFKA-12675) Improve sticky general assignor scalability and performance

2021-04-15 Thread Travis Bischel (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17322603#comment-17322603 ] Travis Bischel edited comment on KAFKA-12675 at 4/16/21, 4:24 AM: -- What

[jira] [Comment Edited] (KAFKA-12675) Improve sticky general assignor scalability and performance

2021-04-15 Thread Travis Bischel (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17322603#comment-17322603 ] Travis Bischel edited comment on KAFKA-12675 at 4/16/21, 4:10 AM: -- What

[jira] [Commented] (KAFKA-12675) Improve sticky general assignor scalability and performance

2021-04-15 Thread Travis Bischel (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17322603#comment-17322603 ] Travis Bischel commented on KAFKA-12675: What I mean to say is that the logic powering the

[jira] [Comment Edited] (KAFKA-12675) Improve sticky general assignor scalability and performance

2021-04-15 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17322599#comment-17322599 ] Luke Chen edited comment on KAFKA-12675 at 4/16/21, 3:59 AM: - [~twmb],

[jira] [Commented] (KAFKA-12675) Improve sticky general assignor scalability and performance

2021-04-15 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17322599#comment-17322599 ] Luke Chen commented on KAFKA-12675: --- [~twmb], thanks for suggestion. But in this ticket, I'd like to

[GitHub] [kafka] ijuma commented on pull request #10545: KAFKA-12672: Added config for raft testing server

2021-04-15 Thread GitBox
ijuma commented on pull request #10545: URL: https://github.com/apache/kafka/pull/10545#issuecomment-820886172 Merged to trunk and cherry-picked to 2.8. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [kafka] ijuma merged pull request #10545: KAFKA-12672: Added config for raft testing server

2021-04-15 Thread GitBox
ijuma merged pull request #10545: URL: https://github.com/apache/kafka/pull/10545 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [kafka] mjsax commented on a change in pull request #10462: KAFKA-10847: Fix spurious results on left/outer stream-stream joins

2021-04-15 Thread GitBox
mjsax commented on a change in pull request #10462: URL: https://github.com/apache/kafka/pull/10462#discussion_r614538992 ## File path: streams/src/test/java/org/apache/kafka/streams/kstream/internals/KStreamKStreamJoinTest.java ## @@ -486,6 +486,327 @@ public void testJoin()

[GitHub] [kafka] mjsax commented on a change in pull request #10462: KAFKA-10847: Fix spurious results on left/outer stream-stream joins

2021-04-15 Thread GitBox
mjsax commented on a change in pull request #10462: URL: https://github.com/apache/kafka/pull/10462#discussion_r614538557 ## File path: streams/src/test/java/org/apache/kafka/streams/kstream/internals/KStreamKStreamJoinTest.java ## @@ -486,6 +486,327 @@ public void testJoin()

[GitHub] [kafka] mjsax commented on a change in pull request #10462: KAFKA-10847: Fix spurious results on left/outer stream-stream joins

2021-04-15 Thread GitBox
mjsax commented on a change in pull request #10462: URL: https://github.com/apache/kafka/pull/10462#discussion_r614538486 ## File path: streams/src/test/java/org/apache/kafka/streams/kstream/internals/KStreamKStreamJoinTest.java ## @@ -486,6 +486,327 @@ public void testJoin()

[GitHub] [kafka] mjsax commented on a change in pull request #10462: KAFKA-10847: Fix spurious results on left/outer stream-stream joins

2021-04-15 Thread GitBox
mjsax commented on a change in pull request #10462: URL: https://github.com/apache/kafka/pull/10462#discussion_r614538200 ## File path: streams/src/test/java/org/apache/kafka/streams/kstream/internals/KStreamKStreamJoinTest.java ## @@ -486,6 +486,327 @@ public void testJoin()

[jira] [Commented] (KAFKA-12675) Improve sticky general assignor scalability and performance

2021-04-15 Thread Travis Bischel (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17322593#comment-17322593 ] Travis Bischel commented on KAFKA-12675: An option to evaluate is the algorithm I devised in my

[GitHub] [kafka] mjsax commented on a change in pull request #10462: KAFKA-10847: Fix spurious results on left/outer stream-stream joins

2021-04-15 Thread GitBox
mjsax commented on a change in pull request #10462: URL: https://github.com/apache/kafka/pull/10462#discussion_r614537097 ## File path: streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamKStreamJoin.java ## @@ -93,23 +136,118 @@ public void process(final

[GitHub] [kafka] mjsax commented on a change in pull request #10462: KAFKA-10847: Fix spurious results on left/outer stream-stream joins

2021-04-15 Thread GitBox
mjsax commented on a change in pull request #10462: URL: https://github.com/apache/kafka/pull/10462#discussion_r614536735 ## File path: streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamImplJoin.java ## @@ -211,6 +263,66 @@ private void

[GitHub] [kafka] mjsax commented on a change in pull request #10462: KAFKA-10847: Fix spurious results on left/outer stream-stream joins

2021-04-15 Thread GitBox
mjsax commented on a change in pull request #10462: URL: https://github.com/apache/kafka/pull/10462#discussion_r614536011 ## File path: streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamImplJoin.java ## @@ -118,20 +142,47 @@ final

[jira] [Updated] (KAFKA-12675) Improve sticky general assignor scalability and performance

2021-04-15 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen updated KAFKA-12675: -- Description: Currently, we have "general assignor" for non-equal subscription case and "constrained

[jira] [Created] (KAFKA-12675) Improve sticky general assignor scalability and performance

2021-04-15 Thread Luke Chen (Jira)
Luke Chen created KAFKA-12675: - Summary: Improve sticky general assignor scalability and performance Key: KAFKA-12675 URL: https://issues.apache.org/jira/browse/KAFKA-12675 Project: Kafka Issue

[GitHub] [kafka] mjsax commented on a change in pull request #10462: KAFKA-10847: Fix spurious results on left/outer stream-stream joins

2021-04-15 Thread GitBox
mjsax commented on a change in pull request #10462: URL: https://github.com/apache/kafka/pull/10462#discussion_r614534849 ## File path: streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamImplJoin.java ## @@ -118,20 +142,47 @@ final

[jira] [Created] (KAFKA-12674) Client failover takes 2-4 seconds on clean broker shutdown

2021-04-15 Thread Gwen Shapira (Jira)
Gwen Shapira created KAFKA-12674: Summary: Client failover takes 2-4 seconds on clean broker shutdown Key: KAFKA-12674 URL: https://issues.apache.org/jira/browse/KAFKA-12674 Project: Kafka

[GitHub] [kafka] junrao commented on a change in pull request #10271: KAFKA-12429: Added serdes for the default implementation of RLMM based on an internal topic as storage.

2021-04-15 Thread GitBox
junrao commented on a change in pull request #10271: URL: https://github.com/apache/kafka/pull/10271#discussion_r614448708 ## File path: storage/src/main/resources/message/RemotePartitionDeleteMetadata.json ## @@ -0,0 +1,68 @@ +// Licensed to the Apache Software Foundation

[jira] [Comment Edited] (KAFKA-12468) Initial offsets are copied from source to target cluster

2021-04-15 Thread Jitesh Motwani (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17322537#comment-17322537 ] Jitesh Motwani edited comment on KAFKA-12468 at 4/15/21, 11:52 PM: ---

[jira] [Commented] (KAFKA-12468) Initial offsets are copied from source to target cluster

2021-04-15 Thread Jitesh Motwani (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17322537#comment-17322537 ] Jitesh Motwani commented on KAFKA-12468: [~askldjd]  what do you mean by run 10 consecutive

[GitHub] [kafka] guozhangwang commented on pull request #10294: KAFKA-12450: Remove deprecated methods from ReadOnlyWindowStore

2021-04-15 Thread GitBox
guozhangwang commented on pull request #10294: URL: https://github.com/apache/kafka/pull/10294#issuecomment-820756041 @jeqo could you rebase the PR while I review it? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [kafka] wcarlson5 commented on a change in pull request #10544: KAFKA-12648: minimum changes for error handling namedTopologies

2021-04-15 Thread GitBox
wcarlson5 commented on a change in pull request #10544: URL: https://github.com/apache/kafka/pull/10544#discussion_r614398250 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -471,6 +472,10 @@ private void replaceStreamThread(final Throwable

[GitHub] [kafka] wcarlson5 commented on a change in pull request #10544: KAFKA-12648: minimum changes for error handling namedTopologies

2021-04-15 Thread GitBox
wcarlson5 commented on a change in pull request #10544: URL: https://github.com/apache/kafka/pull/10544#discussion_r614398250 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -471,6 +472,10 @@ private void replaceStreamThread(final Throwable

[GitHub] [kafka] cmccabe commented on a change in pull request #10463: KAFKA-12670: support unclean.leader.election.enable in KRaft mode

2021-04-15 Thread GitBox
cmccabe commented on a change in pull request #10463: URL: https://github.com/apache/kafka/pull/10463#discussion_r614404680 ## File path: metadata/src/main/java/org/apache/kafka/controller/ReplicationControlManager.java ## @@ -1119,6 +1148,113 @@ void

[GitHub] [kafka] wcarlson5 commented on a change in pull request #10544: KAFKA-12648: minimum changes for error handling namedTopologies

2021-04-15 Thread GitBox
wcarlson5 commented on a change in pull request #10544: URL: https://github.com/apache/kafka/pull/10544#discussion_r614398757 ## File path: streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java ## @@ -1198,7 +1210,7 @@ int process(final int

[GitHub] [kafka] bbejeck commented on pull request #10545: KAFKA-12672: Added config for raft testing server

2021-04-15 Thread GitBox
bbejeck commented on pull request #10545: URL: https://github.com/apache/kafka/pull/10545#issuecomment-820665466 @ijuma updated README -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[jira] [Commented] (KAFKA-12574) Deprecate eos-alpha

2021-04-15 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17322417#comment-17322417 ] Guozhang Wang commented on KAFKA-12574: --- Hi [~feyman] I'm aware of KAFKA-9689 as well, and also

[GitHub] [kafka] cmccabe commented on pull request #10505: MINOR: fix some bugs in ControllerApis.scala

2021-04-15 Thread GitBox
cmccabe commented on pull request #10505: URL: https://github.com/apache/kafka/pull/10505#issuecomment-820659355 @chia7712: I re-ran the test failures locally and couldn't get them to occur. The failures I saw were mostly streams tests... those have have been a bit flaky lately... I

[GitHub] [kafka] ijuma commented on pull request #10545: KAFKA-12672: Added config for raft testing server

2021-04-15 Thread GitBox
ijuma commented on pull request #10545: URL: https://github.com/apache/kafka/pull/10545#issuecomment-820659248 Thanks for the PR. Don't you need to update the multi-node-quorum section? https://github.com/apache/kafka/tree/trunk/raft#run-multi-node-quorum -- This is an automated message

[GitHub] [kafka] bbejeck opened a new pull request #10545: KAFKA-12672: Added config for raft testing server

2021-04-15 Thread GitBox
bbejeck opened a new pull request #10545: URL: https://github.com/apache/kafka/pull/10545 Adding a property to the `raft/config/kraft.properties` for running the raft test server in development. For testing I ran `./bin/test-kraft-server-start.sh --config config/kraft.properties`

[GitHub] [kafka] spena commented on a change in pull request #10462: KAFKA-10847: Fix spurious results on left/outer stream-stream joins

2021-04-15 Thread GitBox
spena commented on a change in pull request #10462: URL: https://github.com/apache/kafka/pull/10462#discussion_r614313362 ## File path: streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamKStreamJoin.java ## @@ -93,23 +136,118 @@ public void process(final

[GitHub] [kafka] spena commented on a change in pull request #10462: KAFKA-10847: Fix spurious results on left/outer stream-stream joins

2021-04-15 Thread GitBox
spena commented on a change in pull request #10462: URL: https://github.com/apache/kafka/pull/10462#discussion_r614312924 ## File path: streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamImplJoin.java ## @@ -211,6 +263,66 @@ private void

[GitHub] [kafka] spena commented on a change in pull request #10462: KAFKA-10847: Fix spurious results on left/outer stream-stream joins

2021-04-15 Thread GitBox
spena commented on a change in pull request #10462: URL: https://github.com/apache/kafka/pull/10462#discussion_r614312224 ## File path: streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamImplJoin.java ## @@ -211,6 +263,66 @@ private void

[GitHub] [kafka] cmccabe merged pull request #10539: Minor: Move trogdor out of tools and into its own project

2021-04-15 Thread GitBox
cmccabe merged pull request #10539: URL: https://github.com/apache/kafka/pull/10539 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [kafka] spena commented on a change in pull request #10462: KAFKA-10847: Fix spurious results on left/outer stream-stream joins

2021-04-15 Thread GitBox
spena commented on a change in pull request #10462: URL: https://github.com/apache/kafka/pull/10462#discussion_r614304330 ## File path: streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamKStreamJoin.java ## @@ -93,23 +136,118 @@ public void process(final

[GitHub] [kafka] spena commented on a change in pull request #10462: KAFKA-10847: Fix spurious results on left/outer stream-stream joins

2021-04-15 Thread GitBox
spena commented on a change in pull request #10462: URL: https://github.com/apache/kafka/pull/10462#discussion_r614303123 ## File path: streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamKStreamJoin.java ## @@ -93,23 +136,118 @@ public void process(final

[jira] [Updated] (KAFKA-12672) Running test-kraft-server-start results in error

2021-04-15 Thread Bill Bejeck (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Bejeck updated KAFKA-12672: Description: Running the {{test-kraft-server-start}} script in the {{raft}} module results in

[jira] [Created] (KAFKA-12673) Kafka and zookeeper dependency

2021-04-15 Thread suhas (Jira)
suhas created KAFKA-12673: - Summary: Kafka and zookeeper dependency Key: KAFKA-12673 URL: https://issues.apache.org/jira/browse/KAFKA-12673 Project: Kafka Issue Type: Task Reporter:

[GitHub] [kafka] wcarlson5 commented on pull request #10544: KAFKA-12648: minimal changes for error handling namedTopologies

2021-04-15 Thread GitBox
wcarlson5 commented on pull request #10544: URL: https://github.com/apache/kafka/pull/10544#issuecomment-820620451 @ableegoldman This is what I think the error handling should look like. We can try to categorize more but I think we should make this best effort and we can add to it as we

[GitHub] [kafka] wcarlson5 opened a new pull request #10544: KAFKA-12648: minimal changes for error handling namedTopologies

2021-04-15 Thread GitBox
wcarlson5 opened a new pull request #10544: URL: https://github.com/apache/kafka/pull/10544 changed the obvious ones to attribute to a named topology. This might be all we need, we can always add more if they come up. ### Committer Checklist (excluded from commit message) - [ ]

[jira] [Updated] (KAFKA-12671) Out of order processing with a transactional producer can lead to a stuck LastStableOffset

2021-04-15 Thread Travis Bischel (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Travis Bischel updated KAFKA-12671: --- Description: If there is pathological processing of incoming produce requests and EndTxn

[GitHub] [kafka] cmccabe merged pull request #10505: MINOR: fix some bugs in ControllerApis.scala

2021-04-15 Thread GitBox
cmccabe merged pull request #10505: URL: https://github.com/apache/kafka/pull/10505 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [kafka] wcarlson5 commented on pull request #10543: KAFKA-12648: tag uses of StreamsException to add Named to

2021-04-15 Thread GitBox
wcarlson5 commented on pull request #10543: URL: https://github.com/apache/kafka/pull/10543#issuecomment-820610403 @ableegoldman I tagged all the cases of StreamsExceptions that I think can be attributed to a Sub topology. I may have missed some and some might not be attributable. Also

[GitHub] [kafka] wcarlson5 opened a new pull request #10543: KAFKA-12648: tag uses of StreamsException to add Named to

2021-04-15 Thread GitBox
wcarlson5 opened a new pull request #10543: URL: https://github.com/apache/kafka/pull/10543 change each StreamsException so that it has the sub topology name in it ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test

[GitHub] [kafka] chia7712 commented on a change in pull request #10505: MINOR: fix some bugs in ControllerApis.scala

2021-04-15 Thread GitBox
chia7712 commented on a change in pull request #10505: URL: https://github.com/apache/kafka/pull/10505#discussion_r614266078 ## File path: core/src/main/scala/kafka/server/ControllerApis.scala ## @@ -242,87 +253,99 @@ class ControllerApis(val requestChannel: RequestChannel,

[GitHub] [kafka] cmccabe commented on a change in pull request #10505: MINOR: fix some bugs in ControllerApis.scala

2021-04-15 Thread GitBox
cmccabe commented on a change in pull request #10505: URL: https://github.com/apache/kafka/pull/10505#discussion_r614264455 ## File path: core/src/main/scala/kafka/server/ControllerApis.scala ## @@ -242,87 +253,99 @@ class ControllerApis(val requestChannel: RequestChannel,

[jira] [Updated] (KAFKA-12672) Running test-kraft-server-start results in error

2021-04-15 Thread Bill Bejeck (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Bejeck updated KAFKA-12672: Description: Running the {{test-kraft-server-start}} script in the {{raft}} module results in

[GitHub] [kafka] Iskuskov commented on pull request #9541: KAFKA-10675: Add schema name to ConnectSchema.validateValue() error message

2021-04-15 Thread GitBox
Iskuskov commented on pull request #9541: URL: https://github.com/apache/kafka/pull/9541#issuecomment-820596386 @chia7712, @ijuma, could you take a look at this please? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[jira] [Created] (KAFKA-12672) Running test-kraft-server-start results in error

2021-04-15 Thread Bill Bejeck (Jira)
Bill Bejeck created KAFKA-12672: --- Summary: Running test-kraft-server-start results in error Key: KAFKA-12672 URL: https://issues.apache.org/jira/browse/KAFKA-12672 Project: Kafka Issue Type:

[GitHub] [kafka] cmccabe merged pull request #10538: MINOR: shutdown KafkaScheduler at appropriate time

2021-04-15 Thread GitBox
cmccabe merged pull request #10538: URL: https://github.com/apache/kafka/pull/10538 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [kafka] kpatelatwork commented on a change in pull request #10528: KAFKA-12497: Skip unnecessary source task offset commits

2021-04-15 Thread GitBox
kpatelatwork commented on a change in pull request #10528: URL: https://github.com/apache/kafka/pull/10528#discussion_r614230413 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/SourceTaskOffsetCommitter.java ## @@ -105,6 +105,11 @@ public void

[GitHub] [kafka] mumrah commented on a change in pull request #10505: MINOR: fix some bugs in ControllerApis.scala

2021-04-15 Thread GitBox
mumrah commented on a change in pull request #10505: URL: https://github.com/apache/kafka/pull/10505#discussion_r614218074 ## File path: core/src/test/java/kafka/test/MockController.java ## @@ -197,15 +209,73 @@ private MockController(Collection initialTopics) {

[GitHub] [kafka] JoelWee commented on pull request #8923: KAFKA-6435: KIP-623 Add internal topics option to streamResetter

2021-04-15 Thread GitBox
JoelWee commented on pull request #8923: URL: https://github.com/apache/kafka/pull/8923#issuecomment-820560188 Thanks @ableegoldman and @wcarlson5! Will try to get this done over the next week or so and ping you when ready :) -- This is an automated message from the Apache Git Service.

[GitHub] [kafka] C0urante commented on a change in pull request #10528: KAFKA-12497: Skip unnecessary source task offset commits

2021-04-15 Thread GitBox
C0urante commented on a change in pull request #10528: URL: https://github.com/apache/kafka/pull/10528#discussion_r614208924 ## File path: connect/runtime/src/test/java/org/apache/kafka/connect/runtime/WorkerSourceTaskTest.java ## @@ -662,20 +666,94 @@ public void

[GitHub] [kafka] C0urante commented on a change in pull request #10528: KAFKA-12497: Skip unnecessary source task offset commits

2021-04-15 Thread GitBox
C0urante commented on a change in pull request #10528: URL: https://github.com/apache/kafka/pull/10528#discussion_r614208181 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/storage/OffsetStorageWriter.java ## @@ -120,6 +120,13 @@ public synchronized

[GitHub] [kafka] C0urante commented on a change in pull request #10528: KAFKA-12497: Skip unnecessary source task offset commits

2021-04-15 Thread GitBox
C0urante commented on a change in pull request #10528: URL: https://github.com/apache/kafka/pull/10528#discussion_r614207888 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/SourceTaskOffsetCommitter.java ## @@ -105,6 +105,11 @@ public void

[jira] [Commented] (KAFKA-12574) Deprecate eos-alpha

2021-04-15 Thread Haoran Xuan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17322296#comment-17322296 ] Haoran Xuan commented on KAFKA-12574: - Hi, previously I was working on KAFKA-9689, in which

[GitHub] [kafka] wcarlson5 commented on a change in pull request #8923: KAFKA-6435: KIP-623 Add internal topics option to streamResetter

2021-04-15 Thread GitBox
wcarlson5 commented on a change in pull request #8923: URL: https://github.com/apache/kafka/pull/8923#discussion_r614166802 ## File path: core/src/main/scala/kafka/tools/StreamsResetter.java ## @@ -248,7 +257,7 @@ private void parseArguments(final String[] args) {

[GitHub] [kafka] wcarlson5 commented on a change in pull request #8923: KAFKA-6435: KIP-623 Add internal topics option to streamResetter

2021-04-15 Thread GitBox
wcarlson5 commented on a change in pull request #8923: URL: https://github.com/apache/kafka/pull/8923#discussion_r614166802 ## File path: core/src/main/scala/kafka/tools/StreamsResetter.java ## @@ -248,7 +257,7 @@ private void parseArguments(final String[] args) {

[GitHub] [kafka] kpatelatwork edited a comment on pull request #10528: KAFKA-12497: Skip unnecessary source task offset commits

2021-04-15 Thread GitBox
kpatelatwork edited a comment on pull request #10528: URL: https://github.com/apache/kafka/pull/10528#issuecomment-820517908 @C0urante overall looks good and very good job on the tests but as I am new to the code, I tried my best to review but I recommend getting a LGTM from one more

[jira] [Updated] (KAFKA-12671) Out of order processing with a transactional producer can lead to a stuck LastStableOffset

2021-04-15 Thread Travis Bischel (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Travis Bischel updated KAFKA-12671: --- Description: If there is pathological processing of incoming produce requests and EndTxn

[GitHub] [kafka] kpatelatwork commented on pull request #10528: KAFKA-12497: Skip unnecessary source task offset commits

2021-04-15 Thread GitBox
kpatelatwork commented on pull request #10528: URL: https://github.com/apache/kafka/pull/10528#issuecomment-820517908 @C0urante overall looks good but as I am new to the code, I tried my best to review but I recommend getting a LGTM from one more reviewer also who knows this code better

[GitHub] [kafka] kpatelatwork commented on a change in pull request #10528: KAFKA-12497: Skip unnecessary source task offset commits

2021-04-15 Thread GitBox
kpatelatwork commented on a change in pull request #10528: URL: https://github.com/apache/kafka/pull/10528#discussion_r614162085 ## File path: connect/runtime/src/test/java/org/apache/kafka/connect/runtime/WorkerSourceTaskTest.java ## @@ -662,20 +666,94 @@ public void

[GitHub] [kafka] kpatelatwork commented on a change in pull request #10528: KAFKA-12497: Skip unnecessary source task offset commits

2021-04-15 Thread GitBox
kpatelatwork commented on a change in pull request #10528: URL: https://github.com/apache/kafka/pull/10528#discussion_r614161831 ## File path: connect/runtime/src/test/java/org/apache/kafka/connect/runtime/WorkerSourceTaskTest.java ## @@ -662,20 +666,94 @@ public void

[GitHub] [kafka] vamossagar12 opened a new pull request #10542: Kafka 12313: Streamling windowed Deserialiser configs.

2021-04-15 Thread GitBox
vamossagar12 opened a new pull request #10542: URL: https://github.com/apache/kafka/pull/10542 This PR aims to streamline the configurations for WindowedDeserialisers. It deprecates default.windowed.key.serde.inner and default.windowed.value.serde.inner configs in StreamConfig and adds

[GitHub] [kafka] dajac merged pull request #10540: MINOR: update default for field in DeleteTopicsRequest

2021-04-15 Thread GitBox
dajac merged pull request #10540: URL: https://github.com/apache/kafka/pull/10540 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [kafka] dajac commented on pull request #10540: MINOR: update default for field in DeleteTopicsRequest

2021-04-15 Thread GitBox
dajac commented on pull request #10540: URL: https://github.com/apache/kafka/pull/10540#issuecomment-820497042 Test failures are not related. Merging to trunk. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [kafka] kpatelatwork commented on a change in pull request #10528: KAFKA-12497: Skip unnecessary source task offset commits

2021-04-15 Thread GitBox
kpatelatwork commented on a change in pull request #10528: URL: https://github.com/apache/kafka/pull/10528#discussion_r614129645 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/storage/OffsetStorageWriter.java ## @@ -120,6 +120,13 @@ public synchronized

[GitHub] [kafka] kpatelatwork commented on a change in pull request #10528: KAFKA-12497: Skip unnecessary source task offset commits

2021-04-15 Thread GitBox
kpatelatwork commented on a change in pull request #10528: URL: https://github.com/apache/kafka/pull/10528#discussion_r614129645 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/storage/OffsetStorageWriter.java ## @@ -120,6 +120,13 @@ public synchronized

[GitHub] [kafka] kpatelatwork commented on a change in pull request #10528: KAFKA-12497: Skip unnecessary source task offset commits

2021-04-15 Thread GitBox
kpatelatwork commented on a change in pull request #10528: URL: https://github.com/apache/kafka/pull/10528#discussion_r614129645 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/storage/OffsetStorageWriter.java ## @@ -120,6 +120,13 @@ public synchronized

[GitHub] [kafka] kpatelatwork commented on a change in pull request #10528: KAFKA-12497: Skip unnecessary source task offset commits

2021-04-15 Thread GitBox
kpatelatwork commented on a change in pull request #10528: URL: https://github.com/apache/kafka/pull/10528#discussion_r614126665 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/SourceTaskOffsetCommitter.java ## @@ -105,6 +105,11 @@ public void

[GitHub] [kafka] spena commented on a change in pull request #10462: KAFKA-10847: Fix spurious results on left/outer stream-stream joins

2021-04-15 Thread GitBox
spena commented on a change in pull request #10462: URL: https://github.com/apache/kafka/pull/10462#discussion_r614114447 ## File path: streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamKStreamJoin.java ## @@ -60,20 +82,41 @@ } private class

[GitHub] [kafka] spena commented on a change in pull request #10462: KAFKA-10847: Fix spurious results on left/outer stream-stream joins

2021-04-15 Thread GitBox
spena commented on a change in pull request #10462: URL: https://github.com/apache/kafka/pull/10462#discussion_r614105673 ## File path: streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamImplJoin.java ## @@ -118,20 +142,47 @@ final

[GitHub] [kafka] spena commented on a change in pull request #10462: KAFKA-10847: Fix spurious results on left/outer stream-stream joins

2021-04-15 Thread GitBox
spena commented on a change in pull request #10462: URL: https://github.com/apache/kafka/pull/10462#discussion_r614098535 ## File path: streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamImplJoin.java ## @@ -118,20 +142,47 @@ final

[GitHub] [kafka] satishd commented on pull request #10271: KAFKA-12429: Added serdes for the default implementation of RLMM based on an internal topic as storage.

2021-04-15 Thread GitBox
satishd commented on pull request #10271: URL: https://github.com/apache/kafka/pull/10271#issuecomment-820438379 @junrao This PR is rebased with the latest trunk and it is ready for review. -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [kafka] satishd commented on pull request #10271: KAFKA-12429: Added serdes for the default implementation of RLMM based on an internal topic as storage.

2021-04-15 Thread GitBox
satishd commented on pull request #10271: URL: https://github.com/apache/kafka/pull/10271#issuecomment-820437818 Thanks @kowshik for the review. Addressed them with the latest [5250b79](https://github.com/apache/kafka/pull/10271/commits/5250b79cef6d21c73d2b2a12ef7cd95a00e9f034). -- This

[GitHub] [kafka] satishd commented on a change in pull request #10271: KAFKA-12429: Added serdes for the default implementation of RLMM based on an internal topic as storage.

2021-04-15 Thread GitBox
satishd commented on a change in pull request #10271: URL: https://github.com/apache/kafka/pull/10271#discussion_r614083593 ## File path: storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/serde/RemoteLogMetadataContextSerde.java ## @@ -0,0 +1,115 @@

[GitHub] [kafka] satishd commented on a change in pull request #10271: KAFKA-12429: Added serdes for the default implementation of RLMM based on an internal topic as storage.

2021-04-15 Thread GitBox
satishd commented on a change in pull request #10271: URL: https://github.com/apache/kafka/pull/10271#discussion_r614071198 ## File path: storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/serde/RemoteLogSegmentMetadataSerde.java ## @@ -0,0 +1,87 @@ +/*

[GitHub] [kafka] ijuma commented on a change in pull request #10498: MINOR: remove `checksumOrNull` and `isValid` from Record

2021-04-15 Thread GitBox
ijuma commented on a change in pull request #10498: URL: https://github.com/apache/kafka/pull/10498#discussion_r614068123 ## File path: core/src/main/scala/kafka/tools/DumpLogSegments.scala ## @@ -279,8 +279,10 @@ object DumpLogSegments { if (batch.magic >=

[GitHub] [kafka] satishd commented on a change in pull request #10271: KAFKA-12429: Added serdes for the default implementation of RLMM based on an internal topic as storage.

2021-04-15 Thread GitBox
satishd commented on a change in pull request #10271: URL: https://github.com/apache/kafka/pull/10271#discussion_r614066999 ## File path: storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/serde/RemoteLogSegmentMetadataSerde.java ## @@ -0,0 +1,87 @@ +/*

[GitHub] [kafka] satishd commented on a change in pull request #10271: KAFKA-12429: Added serdes for the default implementation of RLMM based on an internal topic as storage.

2021-04-15 Thread GitBox
satishd commented on a change in pull request #10271: URL: https://github.com/apache/kafka/pull/10271#discussion_r614064785 ## File path: storage/src/main/resources/message/RemoteLogSegmentMetadata.json ## @@ -0,0 +1,126 @@ +// Licensed to the Apache Software Foundation (ASF)

[GitHub] [kafka] satishd commented on a change in pull request #10271: KAFKA-12429: Added serdes for the default implementation of RLMM based on an internal topic as storage.

2021-04-15 Thread GitBox
satishd commented on a change in pull request #10271: URL: https://github.com/apache/kafka/pull/10271#discussion_r614031843 ## File path: storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/serde/RemoteLogMetadataContextSerde.java ## @@ -0,0 +1,115 @@

[GitHub] [kafka] satishd commented on a change in pull request #10271: KAFKA-12429: Added serdes for the default implementation of RLMM based on an internal topic as storage.

2021-04-15 Thread GitBox
satishd commented on a change in pull request #10271: URL: https://github.com/apache/kafka/pull/10271#discussion_r614004395 ## File path: storage/src/main/resources/message/RemoteLogSegmentMetadataUpdate.json ## @@ -0,0 +1,83 @@ +// Licensed to the Apache Software Foundation

[GitHub] [kafka] satishd commented on a change in pull request #10271: KAFKA-12429: Added serdes for the default implementation of RLMM based on an internal topic as storage.

2021-04-15 Thread GitBox
satishd commented on a change in pull request #10271: URL: https://github.com/apache/kafka/pull/10271#discussion_r614002154 ## File path: storage/src/main/resources/message/RemoteLogSegmentMetadata.json ## @@ -0,0 +1,126 @@ +// Licensed to the Apache Software Foundation (ASF)

[GitHub] [kafka] showuon commented on pull request #10541: KAFKA-12284: increase request timeout to make tests reliable

2021-04-15 Thread GitBox
showuon commented on pull request #10541: URL: https://github.com/apache/kafka/pull/10541#issuecomment-820346080 Need to rework. Close it first. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] showuon closed pull request #10541: KAFKA-12284: increase request timeout to make tests reliable

2021-04-15 Thread GitBox
showuon closed pull request #10541: URL: https://github.com/apache/kafka/pull/10541 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[jira] [Commented] (KAFKA-12635) Mirrormaker 2 offset sync is incorrect if the target partition is empty

2021-04-15 Thread Alexis Polyzos (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17322092#comment-17322092 ] Alexis Polyzos commented on KAFKA-12635: [~yangguo1220] thanks for the reply! We tried your

[jira] [Assigned] (KAFKA-12662) add unit test for ProducerPerformance

2021-04-15 Thread Chun-Hao Tang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chun-Hao Tang reassigned KAFKA-12662: - Assignee: Chun-Hao Tang > add unit test for ProducerPerformance >

[GitHub] [kafka] mimaison commented on pull request #9549: KIP-145: Add SMTs, HeaderFrom, DropHeaders and InsertHeader

2021-04-15 Thread GitBox
mimaison commented on pull request #9549: URL: https://github.com/apache/kafka/pull/9549#issuecomment-820299395 Not sure why the build had failed, I've rekicked it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [kafka] mimaison commented on a change in pull request #9549: KIP-145: Add SMTs, HeaderFrom, DropHeaders and InsertHeader

2021-04-15 Thread GitBox
mimaison commented on a change in pull request #9549: URL: https://github.com/apache/kafka/pull/9549#discussion_r613931955 ## File path: connect/transforms/src/main/java/org/apache/kafka/connect/transforms/HeaderFrom.java ## @@ -0,0 +1,238 @@ +/* + * Licensed to the Apache

[GitHub] [kafka] tombentley commented on a change in pull request #9549: KIP-145: Add SMTs, HeaderFrom, DropHeaders and InsertHeader

2021-04-15 Thread GitBox
tombentley commented on a change in pull request #9549: URL: https://github.com/apache/kafka/pull/9549#discussion_r613926107 ## File path: connect/transforms/src/main/java/org/apache/kafka/connect/transforms/HeaderFrom.java ## @@ -0,0 +1,238 @@ +/* + * Licensed to the Apache

[GitHub] [kafka] mimaison commented on a change in pull request #9549: KIP-145: Add SMTs, HeaderFrom, DropHeaders and InsertHeader

2021-04-15 Thread GitBox
mimaison commented on a change in pull request #9549: URL: https://github.com/apache/kafka/pull/9549#discussion_r613913901 ## File path: connect/transforms/src/main/java/org/apache/kafka/connect/transforms/HeaderFrom.java ## @@ -0,0 +1,238 @@ +/* + * Licensed to the Apache

[GitHub] [kafka] kowshik commented on a change in pull request #10478: KAFKA-12553: Refactor recovery logic to introduce LogLoader

2021-04-15 Thread GitBox
kowshik commented on a change in pull request #10478: URL: https://github.com/apache/kafka/pull/10478#discussion_r613795734 ## File path: core/src/main/scala/kafka/log/Log.scala ## @@ -255,19 +261,21 @@ case object SnapshotGenerated extends LogStartOffsetIncrementReason {

[GitHub] [kafka] kowshik commented on a change in pull request #10271: KAFKA-12429: Added serdes for the default implementation of RLMM based on an internal topic as storage.

2021-04-15 Thread GitBox
kowshik commented on a change in pull request #10271: URL: https://github.com/apache/kafka/pull/10271#discussion_r613882988 ## File path: storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/serde/RemoteLogMetadataContextSerde.java ## @@ -0,0 +1,115 @@

  1   2   >