[GitHub] [kafka] showuon commented on pull request #9674: KAFKA-10665: close all kafkaStreams before purgeLocalStreamsState

showuon commented on pull request #9674: URL: https://github.com/apache/kafka/pull/9674#issuecomment-737056995 @bbejeck @vvcephei @lkokhreidze , please help review this PR. Thanks. This is an automated message from the

[jira] [Commented] (KAFKA-10636) Bypass log validation for writes to raft log

[ https://issues.apache.org/jira/browse/KAFKA-10636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242122#comment-17242122 ] feyman commented on KAFKA-10636: Hi, [~hachikuji] . I would like to take this task, please let me know

[GitHub] [kafka] showuon commented on a change in pull request #9674: KAFKA-10665: close all kafkaStreams before purgeLocalStreamsState

showuon commented on a change in pull request #9674: URL: https://github.com/apache/kafka/pull/9674#discussion_r533956768 ## File path: streams/src/test/java/org/apache/kafka/streams/integration/StreamTableJoinTopologyOptimizationIntegrationTest.java ## @@ -221,7 +223,7 @@

[GitHub] [kafka] showuon commented on a change in pull request #9674: KAFKA-10665: close all kafkaStreams before purgeLocalStreamsState

showuon commented on a change in pull request #9674: URL: https://github.com/apache/kafka/pull/9674#discussion_r533956676 ## File path: streams/src/test/java/org/apache/kafka/streams/integration/StreamTableJoinTopologyOptimizationIntegrationTest.java ## @@ -149,9 +153,7 @@

[GitHub] [kafka] showuon opened a new pull request #9674: KAFKA-10665: close all kafkaStreams before purgeLocalStreamsState

showuon opened a new pull request #9674: URL: https://github.com/apache/kafka/pull/9674 The flaky tests are because we forgot to close the kafkaStreams before purgeLocalStreamsState, so that sometimes there will be some tmp files be created during streams running/removing (ex:

[jira] [Assigned] (KAFKA-10636) Bypass log validation for writes to raft log

[ https://issues.apache.org/jira/browse/KAFKA-10636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] feyman reassigned KAFKA-10636: -- Assignee: feyman > Bypass log validation for writes to raft log >

[jira] [Updated] (KAFKA-10794) Replica leader election is too slow in the case of too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-10794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] limeng updated KAFKA-10794: --- Description: There is more than 6000 topics and 300 brokers in my kafka cluster, and we  frequently run 

[jira] [Updated] (KAFKA-10794) Replica leader election is too slow in the case of too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-10794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] limeng updated KAFKA-10794: --- Attachment: image-2020-12-02-15-23-43-384.png > Replica leader election is too slow in the case of too many

[jira] [Updated] (KAFKA-10794) Replica leader election is too slow in the case of too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-10794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] limeng updated KAFKA-10794: --- Attachment: 截屏2020-12-01 下午2.36.19.png > Replica leader election is too slow in the case of too many

[jira] [Updated] (KAFKA-10794) Replica leader election is too slow in the case of too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-10794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] limeng updated KAFKA-10794: --- Description: There is more than 6000 topics and 300 brokers in my kafka cluster, and we  frequently run 

[GitHub] [kafka] chia7712 commented on pull request #8826: KAFKA-10090 Misleading warnings: The configuration was supplied but i…

chia7712 commented on pull request #8826: URL: https://github.com/apache/kafka/pull/8826#issuecomment-737028662 > There are 6 failures in the system test run. Are they related to this PR? They are unrelated to this PR and I have opened a PR to fix them (#9673)

[GitHub] [kafka] chia7712 opened a new pull request #9673: KAFKA-10289 fix failed connect_distributed_test.py (ConnectDistribute…

chia7712 opened a new pull request #9673: URL: https://github.com/apache/kafka/pull/9673 issue: https://issues.apache.org/jira/browse/KAFKA-10289 In Python 3, ```filter``` functions return iterators rather than ```list``` so it can traverse only once. Hence, the following loop only

[jira] [Updated] (KAFKA-10289) fix failed connect_distributed_test.py (ConnectDistributedTest.test_bounce)

[ https://issues.apache.org/jira/browse/KAFKA-10289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated KAFKA-10289: --- Summary: fix failed connect_distributed_test.py (ConnectDistributedTest.test_bounce) (was:

[jira] [Updated] (KAFKA-10289) fix failed connect_distributed_test.py.test_bounce

[ https://issues.apache.org/jira/browse/KAFKA-10289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated KAFKA-10289: --- Summary: fix failed connect_distributed_test.py.test_bounce (was: fix flaky

[jira] [Assigned] (KAFKA-10289) fix flaky connect/connect_distributed_test.py

[ https://issues.apache.org/jira/browse/KAFKA-10289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai reassigned KAFKA-10289: -- Assignee: Chia-Ping Tsai > fix flaky connect/connect_distributed_test.py >

[GitHub] [kafka] d8tltanc commented on a change in pull request #9485: KAKFA-10619: Idempotent producer will get authorized once it has a WRITE access to at least one topic

d8tltanc commented on a change in pull request #9485: URL: https://github.com/apache/kafka/pull/9485#discussion_r533922526 ## File path: jmh-benchmarks/src/main/java/org/apache/kafka/jmh/acl/AclAuthorizerBenchmark.java ## @@ -115,45 +122,62 @@ private void

[GitHub] [kafka] d8tltanc commented on a change in pull request #9485: KAKFA-10619: Idempotent producer will get authorized once it has a WRITE access to at least one topic

d8tltanc commented on a change in pull request #9485: URL: https://github.com/apache/kafka/pull/9485#discussion_r533919425 ## File path: core/src/main/scala/kafka/security/authorizer/AclAuthorizer.scala ## @@ -304,6 +308,122 @@ class AclAuthorizer extends Authorizer with

[GitHub] [kafka] d8tltanc commented on a change in pull request #9485: KAKFA-10619: Idempotent producer will get authorized once it has a WRITE access to at least one topic

d8tltanc commented on a change in pull request #9485: URL: https://github.com/apache/kafka/pull/9485#discussion_r533916889 ## File path: clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java ## @@ -139,4 +152,125 @@ * @return Iterator for ACL

[GitHub] [kafka] d8tltanc commented on a change in pull request #9485: KAKFA-10619: Idempotent producer will get authorized once it has a WRITE access to at least one topic

d8tltanc commented on a change in pull request #9485: URL: https://github.com/apache/kafka/pull/9485#discussion_r533915261 ## File path: clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java ## @@ -139,4 +152,125 @@ * @return Iterator for ACL

[GitHub] [kafka] d8tltanc commented on a change in pull request #9485: KAKFA-10619: Idempotent producer will get authorized once it has a WRITE access to at least one topic

d8tltanc commented on a change in pull request #9485: URL: https://github.com/apache/kafka/pull/9485#discussion_r533915142 ## File path: clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java ## @@ -139,4 +152,125 @@ * @return Iterator for ACL

[GitHub] [kafka] d8tltanc commented on a change in pull request #9485: KAKFA-10619: Idempotent producer will get authorized once it has a WRITE access to at least one topic

d8tltanc commented on a change in pull request #9485: URL: https://github.com/apache/kafka/pull/9485#discussion_r533915036 ## File path: clients/src/main/java/org/apache/kafka/server/authorizer/Authorizer.java ## @@ -139,4 +152,125 @@ * @return Iterator for ACL

[jira] [Commented] (KAFKA-10665) Flaky Test StreamTableJoinTopologyOptimizationIntegrationTest.shouldDoStreamTableJoinWithDifferentNumberOfPartitions[Optimization = all]

[ https://issues.apache.org/jira/browse/KAFKA-10665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242062#comment-17242062 ] Luke Chen commented on KAFKA-10665: ---

[jira] [Commented] (KAFKA-10665) Flaky Test StreamTableJoinTopologyOptimizationIntegrationTest.shouldDoStreamTableJoinWithDifferentNumberOfPartitions[Optimization = all]

[ https://issues.apache.org/jira/browse/KAFKA-10665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242063#comment-17242063 ] Luke Chen commented on KAFKA-10665: ---

[jira] [Assigned] (KAFKA-10665) Flaky Test StreamTableJoinTopologyOptimizationIntegrationTest.shouldDoStreamTableJoinWithDifferentNumberOfPartitions[Optimization = all]

[ https://issues.apache.org/jira/browse/KAFKA-10665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen reassigned KAFKA-10665: - Assignee: Luke Chen > Flaky Test >

[GitHub] [kafka] d8tltanc commented on a change in pull request #9485: KAKFA-10619: Idempotent producer will get authorized once it has a WRITE access to at least one topic

d8tltanc commented on a change in pull request #9485: URL: https://github.com/apache/kafka/pull/9485#discussion_r533765909 ## File path: jmh-benchmarks/src/main/java/org/apache/kafka/jmh/acl/AclAuthorizerBenchmark.java ## @@ -69,33 +70,39 @@ @BenchmarkMode(Mode.AverageTime)

[GitHub] [kafka] d8tltanc commented on a change in pull request #9485: KAKFA-10619: Idempotent producer will get authorized once it has a WRITE access to at least one topic

d8tltanc commented on a change in pull request #9485: URL: https://github.com/apache/kafka/pull/9485#discussion_r533765909 ## File path: jmh-benchmarks/src/main/java/org/apache/kafka/jmh/acl/AclAuthorizerBenchmark.java ## @@ -69,33 +70,39 @@ @BenchmarkMode(Mode.AverageTime)

[GitHub] [kafka] junrao commented on pull request #8826: KAFKA-10090 Misleading warnings: The configuration was supplied but i…

junrao commented on pull request #8826: URL: https://github.com/apache/kafka/pull/8826#issuecomment-736983541 @chia7712 : There are 6 failures in the system test run. Are they related to this PR?

[jira] [Commented] (KAFKA-10772) java.lang.IllegalStateException: There are insufficient bytes available to read assignment from the sync-group response (actual byte size 0)

[ https://issues.apache.org/jira/browse/KAFKA-10772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242025#comment-17242025 ] A. Sophie Blee-Goldman commented on KAFKA-10772: Hm, the leadership changes definitely

[GitHub] [kafka] ableegoldman commented on a change in pull request #9615: KAFKA-10500: Add thread option

ableegoldman commented on a change in pull request #9615: URL: https://github.com/apache/kafka/pull/9615#discussion_r533878767 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -870,43 +899,73 @@ private KafkaStreams(final

[jira] [Updated] (KAFKA-10794) Replica leader election is too slow in the case of too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-10794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] limeng updated KAFKA-10794: --- Description: There is more than 6000 topics and 300 brokers in my kafka cluster, and we set the

[jira] [Updated] (KAFKA-10794) Replica leader election is too slow in the case of too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-10794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] limeng updated KAFKA-10794: --- Attachment: clipboard_image_1606806581629.png > Replica leader election is too slow in the case of too many

[jira] [Updated] (KAFKA-10794) Replica leader election is too slow in the case of too many partitions

[ https://issues.apache.org/jira/browse/KAFKA-10794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] limeng updated KAFKA-10794: --- Description: There is more than 6000 topics and 300 brokers in my kafka cluster, and we set the

[GitHub] [kafka] ableegoldman commented on a change in pull request #9609: KAFKA-6687: restrict DSL to allow only Streams from the same source topics

ableegoldman commented on a change in pull request #9609: URL: https://github.com/apache/kafka/pull/9609#discussion_r533876280 ## File path: streams/src/main/java/org/apache/kafka/streams/kstream/internals/graph/SourceGraphNode.java ## @@ -0,0 +1,71 @@ +/* + * Licensed to the

[GitHub] [kafka] ableegoldman commented on pull request #9671: KAFKA-10793: move handling of FindCoordinatorFuture to fix race condition

ableegoldman commented on pull request #9671: URL: https://github.com/apache/kafka/pull/9671#issuecomment-736964914 Waiting to add tests until I get some sanity checks on this proposal This is an automated message from the

[GitHub] [kafka] showuon opened a new pull request #9672: MINOR: work in progress for Eos test(don't review)

showuon opened a new pull request #9672: URL: https://github.com/apache/kafka/pull/9672 work in progress for Eos test(don't review) ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status -

[jira] [Resolved] (KAFKA-9263) The new hw is added to incorrect log when ReplicaAlterLogDirsThread is replacing log (fix PlaintextAdminIntegrationTest.testAlterReplicaLogDirs)

[ https://issues.apache.org/jira/browse/KAFKA-9263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-9263. --- Fix Version/s: 2.8.0 Resolution: Fixed > The new hw is added to incorrect log when

[GitHub] [kafka] chia7712 merged pull request #9423: KAFKA-9263 The new hw is added to incorrect log when ReplicaAlterLogD…

chia7712 merged pull request #9423: URL: https://github.com/apache/kafka/pull/9423 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[jira] [Commented] (KAFKA-9263) Reocurrence: Transient failure in kafka.api.PlaintextAdminIntegrationTest.testLogStartOffsetCheckpoint and kafka.api.PlaintextAdminIntegrationTest.testAlterReplicaLogDi

[ https://issues.apache.org/jira/browse/KAFKA-9263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242016#comment-17242016 ] Chia-Ping Tsai commented on KAFKA-9263: --- PlaintextAdminIntegrationTest.testLogStartOffsetCheckpoint

[jira] [Assigned] (KAFKA-9263) The new hw is added to incorrect log when ReplicaAlterLogDirsThread is replacing log (fix PlaintextAdminIntegrationTest.testAlterReplicaLogDirs)

[ https://issues.apache.org/jira/browse/KAFKA-9263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai reassigned KAFKA-9263: - Assignee: Chia-Ping Tsai > The new hw is added to incorrect log when

[jira] [Updated] (KAFKA-9263) The new hw is added to incorrect log when ReplicaAlterLogDirsThread is replacing log (fix PlaintextAdminIntegrationTest.testAlterReplicaLogDirs)

[ https://issues.apache.org/jira/browse/KAFKA-9263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated KAFKA-9263: -- Summary: The new hw is added to incorrect log when ReplicaAlterLogDirsThread is replacing log

[jira] [Created] (KAFKA-10794) Replica leader election is too slow in the case of too many partitions

limeng created KAFKA-10794: -- Summary: Replica leader election is too slow in the case of too many partitions Key: KAFKA-10794 URL: https://issues.apache.org/jira/browse/KAFKA-10794 Project: Kafka

[GitHub] [kafka] highluck commented on a change in pull request #8965: KAFKA-8147: Add changelog topic configuration to KTable suppress

highluck commented on a change in pull request #8965: URL: https://github.com/apache/kafka/pull/8965#discussion_r533863847 ## File path: docs/streams/developer-guide/dsl-api.html ## @@ -3393,6 +3393,10 @@ KTable-KTable Foreign-Key but

[GitHub] [kafka] ableegoldman opened a new pull request #9671: KAFKA-10793: move handling of FindCoordinatorFuture to fix race condition

ableegoldman opened a new pull request #9671: URL: https://github.com/apache/kafka/pull/9671 A race condition between the consumer and hb thread can lead to a failed but non-null `findCoordinatorFuture`, causing the AbstractCoordinator to wait endlessly on the request which it thinks is

[GitHub] [kafka] chia7712 commented on pull request #9666: MINOR: Remove broken `.travis.yml` with system test hooks

chia7712 commented on pull request #9666: URL: https://github.com/apache/kafka/pull/9666#issuecomment-736948097 > Nice that you got it working! I think there is value running a subset of system tests automatically. It might even encourage us to write more system tests if we can get the

[jira] [Comment Edited] (KAFKA-10793) Race condition in FindCoordinatorFuture permanently severs connection to group coordinator

[ https://issues.apache.org/jira/browse/KAFKA-10793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17241976#comment-17241976 ] A. Sophie Blee-Goldman edited comment on KAFKA-10793 at 12/2/20, 2:30 AM:

[jira] [Comment Edited] (KAFKA-10793) Race condition in FindCoordinatorFuture permanently severs connection to group coordinator

[ https://issues.apache.org/jira/browse/KAFKA-10793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17241976#comment-17241976 ] A. Sophie Blee-Goldman edited comment on KAFKA-10793 at 12/2/20, 2:26 AM:

[jira] [Commented] (KAFKA-10688) Handle accidental truncation of repartition topics as exceptional failure

[ https://issues.apache.org/jira/browse/KAFKA-10688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17241992#comment-17241992 ] Guozhang Wang commented on KAFKA-10688: --- Had some more discussions with [~cadonna] about different

[jira] [Commented] (KAFKA-10793) Race condition in FindCoordinatorFuture permanently severs connection to group coordinator

[ https://issues.apache.org/jira/browse/KAFKA-10793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17241987#comment-17241987 ] A. Sophie Blee-Goldman commented on KAFKA-10793: So that's our best guess. We should

[jira] [Resolved] (KAFKA-10780) Rewrite ControllerZNode struct with auto-generated protocol

[ https://issues.apache.org/jira/browse/KAFKA-10780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dengziming resolved KAFKA-10780. Resolution: Won't Do KIP-500 will replace all this code. > Rewrite ControllerZNode struct with

[jira] [Comment Edited] (KAFKA-10793) Race condition in FindCoordinatorFuture permanently severs connection to group coordinator

[ https://issues.apache.org/jira/browse/KAFKA-10793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17241976#comment-17241976 ] A. Sophie Blee-Goldman edited comment on KAFKA-10793 at 12/2/20, 1:37 AM:

[jira] [Comment Edited] (KAFKA-10793) Race condition in FindCoordinatorFuture permanently severs connection to group coordinator

[ https://issues.apache.org/jira/browse/KAFKA-10793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17241976#comment-17241976 ] A. Sophie Blee-Goldman edited comment on KAFKA-10793 at 12/2/20, 1:37 AM:

[jira] [Comment Edited] (KAFKA-10793) Race condition in FindCoordinatorFuture permanently severs connection to group coordinator

[ https://issues.apache.org/jira/browse/KAFKA-10793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17241976#comment-17241976 ] A. Sophie Blee-Goldman edited comment on KAFKA-10793 at 12/2/20, 1:37 AM:

[jira] [Commented] (KAFKA-10793) Race condition in FindCoordinatorFuture permanently severs connection to group coordinator

[ https://issues.apache.org/jira/browse/KAFKA-10793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17241976#comment-17241976 ] A. Sophie Blee-Goldman commented on KAFKA-10793: At this point we can only guess, but

[GitHub] [kafka] jsancio commented on a change in pull request #9512: KAFKA-10394: generate snapshot

jsancio commented on a change in pull request #9512: URL: https://github.com/apache/kafka/pull/9512#discussion_r533835857 ## File path: raft/src/main/java/org/apache/kafka/raft/ReplicatedLog.java ## @@ -149,6 +152,29 @@ default OptionalLong truncateToEndOffset(OffsetAndEpoch

[GitHub] [kafka] jsancio commented on a change in pull request #9512: KAFKA-10394: generate snapshot

jsancio commented on a change in pull request #9512: URL: https://github.com/apache/kafka/pull/9512#discussion_r533833428 ## File path: raft/src/main/java/org/apache/kafka/snapshot/Snapshots.java ## @@ -0,0 +1,67 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] [kafka] jsancio commented on a change in pull request #9512: KAFKA-10394: generate snapshot

jsancio commented on a change in pull request #9512: URL: https://github.com/apache/kafka/pull/9512#discussion_r533829583 ## File path: raft/src/main/java/org/apache/kafka/snapshot/Snapshots.java ## @@ -0,0 +1,67 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[jira] [Created] (KAFKA-10793) Race condition in FindCoordinatorFuture permanently severs connection to group coordinator

A. Sophie Blee-Goldman created KAFKA-10793: -- Summary: Race condition in FindCoordinatorFuture permanently severs connection to group coordinator Key: KAFKA-10793 URL:

[GitHub] [kafka] jsancio commented on a change in pull request #9512: KAFKA-10394: generate snapshot

jsancio commented on a change in pull request #9512: URL: https://github.com/apache/kafka/pull/9512#discussion_r533828329 ## File path: raft/src/main/java/org/apache/kafka/raft/RaftClient.java ## @@ -100,4 +102,15 @@ default void handleResign() {} */

[GitHub] [kafka] hachikuji commented on pull request #9666: MINOR: Remove broken `.travis.yml` with system test hooks

hachikuji commented on pull request #9666: URL: https://github.com/apache/kafka/pull/9666#issuecomment-736910631 @chia7712 Nice that you got it working! I think there is value running a subset of system tests automatically. It might even encourage us to write more system tests if we can

[GitHub] [kafka] abbccdda commented on a change in pull request #9564: KAFKA-10667: add timeout for forwarding requests

abbccdda commented on a change in pull request #9564: URL: https://github.com/apache/kafka/pull/9564#discussion_r533814137 ## File path: core/src/main/scala/kafka/server/AlterIsrManager.scala ## @@ -88,20 +88,33 @@ class AlterIsrManagerImpl(val controllerChannelManager:

[GitHub] [kafka] chia7712 commented on pull request #9666: MINOR: Remove broken `.travis.yml` with system test hooks

chia7712 commented on pull request #9666: URL: https://github.com/apache/kafka/pull/9666#issuecomment-736902424 > guess you were trying to get this working instead? #9652 Removing travis test is ok to me. This

[GitHub] [kafka] C0urante commented on pull request #9669: KAFKA-10792: Prevent source task shutdown from blocking herder thread

C0urante commented on pull request #9669: URL: https://github.com/apache/kafka/pull/9669#issuecomment-736896160 Thanks, Nigel. @chia7712 @rhauch @kkonstantine would you like to take a look? This is an automated

[jira] [Resolved] (KAFKA-10729) KIP-482: Bump remaining RPC's to use tagged fields

[ https://issues.apache.org/jira/browse/KAFKA-10729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-10729. - Fix Version/s: 2.8.0 Resolution: Fixed > KIP-482: Bump remaining RPC's to use

[GitHub] [kafka] hachikuji merged pull request #9601: KAFKA-10729: Bump remaining RPC's to use tagged fields.

hachikuji merged pull request #9601: URL: https://github.com/apache/kafka/pull/9601 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] ncliang commented on a change in pull request #9669: KAFKA-10792: Prevent source task shutdown from blocking herder thread

ncliang commented on a change in pull request #9669: URL: https://github.com/apache/kafka/pull/9669#discussion_r533788764 ## File path: connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSourceTask.java ## @@ -206,16 +203,13 @@ public void cancel() {

[GitHub] [kafka] hachikuji commented on a change in pull request #9300: KAFKA-10491: Check authorizations first in KafkaApis

hachikuji commented on a change in pull request #9300: URL: https://github.com/apache/kafka/pull/9300#discussion_r533763099 ## File path: core/src/main/scala/kafka/server/KafkaApis.scala ## @@ -1770,45 +1770,47 @@ class KafkaApis(val requestChannel: RequestChannel,

[GitHub] [kafka] d8tltanc commented on a change in pull request #9485: KAKFA-10619: Idempotent producer will get authorized once it has a WRITE access to at least one topic

d8tltanc commented on a change in pull request #9485: URL: https://github.com/apache/kafka/pull/9485#discussion_r533765909 ## File path: jmh-benchmarks/src/main/java/org/apache/kafka/jmh/acl/AclAuthorizerBenchmark.java ## @@ -69,33 +70,39 @@ @BenchmarkMode(Mode.AverageTime)

[GitHub] [kafka] d8tltanc commented on a change in pull request #9485: KAKFA-10619: Idempotent producer will get authorized once it has a WRITE access to at least one topic

d8tltanc commented on a change in pull request #9485: URL: https://github.com/apache/kafka/pull/9485#discussion_r533765909 ## File path: jmh-benchmarks/src/main/java/org/apache/kafka/jmh/acl/AclAuthorizerBenchmark.java ## @@ -69,33 +70,39 @@ @BenchmarkMode(Mode.AverageTime)

[GitHub] [kafka] d8tltanc commented on a change in pull request #9485: KAKFA-10619: Idempotent producer will get authorized once it has a WRITE access to at least one topic

d8tltanc commented on a change in pull request #9485: URL: https://github.com/apache/kafka/pull/9485#discussion_r533765909 ## File path: jmh-benchmarks/src/main/java/org/apache/kafka/jmh/acl/AclAuthorizerBenchmark.java ## @@ -69,33 +70,39 @@ @BenchmarkMode(Mode.AverageTime)

[GitHub] [kafka] d8tltanc commented on a change in pull request #9485: KAKFA-10619: Idempotent producer will get authorized once it has a WRITE access to at least one topic

d8tltanc commented on a change in pull request #9485: URL: https://github.com/apache/kafka/pull/9485#discussion_r533765909 ## File path: jmh-benchmarks/src/main/java/org/apache/kafka/jmh/acl/AclAuthorizerBenchmark.java ## @@ -69,33 +70,39 @@ @BenchmarkMode(Mode.AverageTime)

[GitHub] [kafka] d8tltanc commented on a change in pull request #9485: KAKFA-10619: Idempotent producer will get authorized once it has a WRITE access to at least one topic

d8tltanc commented on a change in pull request #9485: URL: https://github.com/apache/kafka/pull/9485#discussion_r533765909 ## File path: jmh-benchmarks/src/main/java/org/apache/kafka/jmh/acl/AclAuthorizerBenchmark.java ## @@ -69,33 +70,39 @@ @BenchmarkMode(Mode.AverageTime)

[GitHub] [kafka] d8tltanc commented on a change in pull request #9485: KAKFA-10619: Idempotent producer will get authorized once it has a WRITE access to at least one topic

d8tltanc commented on a change in pull request #9485: URL: https://github.com/apache/kafka/pull/9485#discussion_r533765909 ## File path: jmh-benchmarks/src/main/java/org/apache/kafka/jmh/acl/AclAuthorizerBenchmark.java ## @@ -69,33 +70,39 @@ @BenchmarkMode(Mode.AverageTime)

[GitHub] [kafka] d8tltanc commented on a change in pull request #9485: KAKFA-10619: Idempotent producer will get authorized once it has a WRITE access to at least one topic

d8tltanc commented on a change in pull request #9485: URL: https://github.com/apache/kafka/pull/9485#discussion_r533765909 ## File path: jmh-benchmarks/src/main/java/org/apache/kafka/jmh/acl/AclAuthorizerBenchmark.java ## @@ -69,33 +70,39 @@ @BenchmarkMode(Mode.AverageTime)

[GitHub] [kafka] hachikuji commented on a change in pull request #9512: KAFKA-10394: generate snapshot

hachikuji commented on a change in pull request #9512: URL: https://github.com/apache/kafka/pull/9512#discussion_r533732789 ## File path: raft/src/main/java/org/apache/kafka/snapshot/Snapshots.java ## @@ -0,0 +1,67 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] [kafka] d8tltanc commented on a change in pull request #9485: KAKFA-10619: Idempotent producer will get authorized once it has a WRITE access to at least one topic

d8tltanc commented on a change in pull request #9485: URL: https://github.com/apache/kafka/pull/9485#discussion_r533737220 ## File path: core/src/test/scala/unit/kafka/security/authorizer/MockAuthorizer.scala ## @@ -0,0 +1,62 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] rajinisivaram commented on a change in pull request #9485: KAKFA-10619: Idempotent producer will get authorized once it has a WRITE access to at least one topic

rajinisivaram commented on a change in pull request #9485: URL: https://github.com/apache/kafka/pull/9485#discussion_r533510477 ## File path: core/src/main/scala/kafka/security/authorizer/AclAuthorizer.scala ## @@ -304,6 +308,122 @@ class AclAuthorizer extends Authorizer with

[GitHub] [kafka] JimGalasyn opened a new pull request #9670: DOCS-6076: Clarify config names for EOS versions 1 and 2

JimGalasyn opened a new pull request #9670: URL: https://github.com/apache/kafka/pull/9670 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [kafka] hachikuji commented on a change in pull request #9601: KAFKA-10729: Bump remaining RPC's to use tagged fields.

hachikuji commented on a change in pull request #9601: URL: https://github.com/apache/kafka/pull/9601#discussion_r533719708 ## File path: core/src/test/scala/unit/kafka/server/ReplicaFetcherThreadTest.scala ## @@ -437,8 +437,8 @@ class ReplicaFetcherThreadTest {

[jira] [Commented] (KAFKA-10772) java.lang.IllegalStateException: There are insufficient bytes available to read assignment from the sync-group response (actual byte size 0)

[ https://issues.apache.org/jira/browse/KAFKA-10772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17241865#comment-17241865 ] Levani Kokhreidze commented on KAFKA-10772: --- Maybe this helps – we noticed that this problem

[jira] [Commented] (KAFKA-10772) java.lang.IllegalStateException: There are insufficient bytes available to read assignment from the sync-group response (actual byte size 0)

[ https://issues.apache.org/jira/browse/KAFKA-10772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17241863#comment-17241863 ] Levani Kokhreidze commented on KAFKA-10772: --- Hi [~ableegoldman], yes we are using static

[jira] [Comment Edited] (KAFKA-10772) java.lang.IllegalStateException: There are insufficient bytes available to read assignment from the sync-group response (actual byte size 0)

[ https://issues.apache.org/jira/browse/KAFKA-10772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17241863#comment-17241863 ] Levani Kokhreidze edited comment on KAFKA-10772 at 12/1/20, 8:50 PM: -

[jira] [Updated] (KAFKA-10772) java.lang.IllegalStateException: There are insufficient bytes available to read assignment from the sync-group response (actual byte size 0)

[ https://issues.apache.org/jira/browse/KAFKA-10772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Levani Kokhreidze updated KAFKA-10772: -- Attachment: KAFKA-10772.log > java.lang.IllegalStateException: There are insufficient

[GitHub] [kafka] wcarlson5 commented on a change in pull request #9615: KAFKA-10500: Add thread option

wcarlson5 commented on a change in pull request #9615: URL: https://github.com/apache/kafka/pull/9615#discussion_r533707640 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -894,19 +885,88 @@ private KafkaStreams(final

[GitHub] [kafka] hachikuji commented on a change in pull request #9382: KAFKA-10554; Perform follower truncation based on diverging epochs in Fetch response

hachikuji commented on a change in pull request #9382: URL: https://github.com/apache/kafka/pull/9382#discussion_r533648722 ## File path: core/src/main/scala/kafka/server/AbstractFetcherThread.scala ## @@ -426,21 +451,34 @@ abstract class AbstractFetcherThread(name: String,

[GitHub] [kafka] cadonna commented on a change in pull request #9615: KAFKA-10500: Add thread option

cadonna commented on a change in pull request #9615: URL: https://github.com/apache/kafka/pull/9615#discussion_r533687031 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -894,19 +885,88 @@ private KafkaStreams(final InternalTopologyBuilder

[GitHub] [kafka] C0urante commented on pull request #9669: KAFKA-10792: Prevent source task shutdown from blocking herder thread

C0urante commented on pull request #9669: URL: https://github.com/apache/kafka/pull/9669#issuecomment-736784767 @tombentley @gharris1727 @ncliang care to take a look? This is an automated message from the Apache Git Service.

[GitHub] [kafka] C0urante opened a new pull request #9669: KAFKA-10792: Prevent source task shutdown from blocking herder thread

C0urante opened a new pull request #9669: URL: https://github.com/apache/kafka/pull/9669 [Jira](https://issues.apache.org/jira/browse/KAFKA-10792) The functional changes are simple: change the `WorkerSourceTask` class to only call `SourceTask::stop` from one location, during task

[jira] [Created] (KAFKA-10792) Source tasks can block herder thread by hanging during stop

Chris Egerton created KAFKA-10792: - Summary: Source tasks can block herder thread by hanging during stop Key: KAFKA-10792 URL: https://issues.apache.org/jira/browse/KAFKA-10792 Project: Kafka

[GitHub] [kafka] wcarlson5 commented on a change in pull request #9615: KAFKA-10500: Add thread option

wcarlson5 commented on a change in pull request #9615: URL: https://github.com/apache/kafka/pull/9615#discussion_r533676976 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -894,19 +885,88 @@ private KafkaStreams(final

[GitHub] [kafka] cadonna commented on a change in pull request #9615: KAFKA-10500: Add thread option

cadonna commented on a change in pull request #9615: URL: https://github.com/apache/kafka/pull/9615#discussion_r533675429 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -894,19 +885,88 @@ private KafkaStreams(final InternalTopologyBuilder

[GitHub] [kafka] mjsax opened a new pull request #9668: MINOR: add test for repartition/source-topic/changelog optimization

mjsax opened a new pull request #9668: URL: https://github.com/apache/kafka/pull/9668 If topology optimization is enabled, KafkaStreams does not create store changelog topics but re-uses source input topics if possible. However, this optimization should not be applied to internal

[GitHub] [kafka] wcarlson5 commented on a change in pull request #9615: KAFKA-10500: Add thread option

wcarlson5 commented on a change in pull request #9615: URL: https://github.com/apache/kafka/pull/9615#discussion_r533656732 ## File path: streams/src/main/java/org/apache/kafka/streams/KafkaStreams.java ## @@ -894,19 +885,88 @@ private KafkaStreams(final

[GitHub] [kafka] ableegoldman commented on pull request #9609: KAFKA-6687: restrict DSL to allow only Streams from the same source topics

ableegoldman commented on pull request #9609: URL: https://github.com/apache/kafka/pull/9609#issuecomment-736766888 Merged to trunk This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [kafka] ableegoldman merged pull request #9609: KAFKA-6687: restrict DSL to allow only Streams from the same source topics

ableegoldman merged pull request #9609: URL: https://github.com/apache/kafka/pull/9609 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[jira] [Commented] (KAFKA-10772) java.lang.IllegalStateException: There are insufficient bytes available to read assignment from the sync-group response (actual byte size 0)

[ https://issues.apache.org/jira/browse/KAFKA-10772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17241814#comment-17241814 ] A. Sophie Blee-Goldman commented on KAFKA-10772: Hey [~lkokhreidze], thanks for the

[GitHub] [kafka] hachikuji merged pull request #9663: MINOR: Small cleanups in `AlterIsr` handling logic

hachikuji merged pull request #9663: URL: https://github.com/apache/kafka/pull/9663 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] dajac commented on pull request #9666: MINOR: Remove broken `.travis.yml` with system test hooks

dajac commented on pull request #9666: URL: https://github.com/apache/kafka/pull/9666#issuecomment-736754455 I am +1 on removing it. We can always bring it back to trunk if we can make it work later. This is an automated

[GitHub] [kafka] ableegoldman commented on pull request #9654: MINOR: Increase unit test coverage of ProcessorTopology#updateSourceTopics()

ableegoldman commented on pull request #9654: URL: https://github.com/apache/kafka/pull/9654#issuecomment-736752649 Merged to trunk This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [kafka] ableegoldman merged pull request #9654: MINOR: Increase unit test coverage of ProcessorTopology#updateSourceTopics()

ableegoldman merged pull request #9654: URL: https://github.com/apache/kafka/pull/9654 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [kafka] lmr3796 commented on a change in pull request #9435: KAFKA-10606: Disable auto topic creation for fetch-all-topic-metadata request

lmr3796 commented on a change in pull request #9435: URL: https://github.com/apache/kafka/pull/9435#discussion_r533648988 ## File path: core/src/main/scala/kafka/server/KafkaApis.scala ## @@ -1250,13 +1251,23 @@ class KafkaApis(val requestChannel: RequestChannel,

  1   2   >