[GitHub] [kafka] dengziming commented on a change in pull request #10142: KAFKA-12294: forward auto topic request within envelope on behalf of clients

2021-04-02 Thread GitBox
dengziming commented on a change in pull request #10142: URL: https://github.com/apache/kafka/pull/10142#discussion_r606613914 ## File path: core/src/main/scala/kafka/server/AutoTopicCreationManager.scala ## @@ -156,17 +169,44 @@ class DefaultAutoTopicCreationManager(

[GitHub] [kafka] showuon commented on a change in pull request #10457: KAFKA-12588: remove --zookeeper option from topic command

2021-04-02 Thread GitBox
showuon commented on a change in pull request #10457: URL: https://github.com/apache/kafka/pull/10457#discussion_r606609994 ## File path: core/src/test/scala/unit/kafka/admin/TopicCommandWithAdminClientTest.scala ## @@ -65,9 +65,10 @@ class TopicCommandWithAdminClientTest exte

[GitHub] [kafka] ijuma commented on a change in pull request #10437: MINOR: Remove deprecated `host.name`, `advertised.host.name`, `port` and `advertised.port` for 3.0

2021-04-02 Thread GitBox
ijuma commented on a change in pull request #10437: URL: https://github.com/apache/kafka/pull/10437#discussion_r606609310 ## File path: core/src/main/scala/kafka/server/KafkaConfig.scala ## @@ -1804,9 +1773,7 @@ class KafkaConfig(val props: java.util.Map[_, _], doLog: Boolean,

[GitHub] [kafka] ijuma commented on pull request #10405: KAFKA-3968: fsync the parent directory of a segment file when the file is created

2021-04-02 Thread GitBox
ijuma commented on pull request #10405: URL: https://github.com/apache/kafka/pull/10405#issuecomment-812801416 Would the perf impact of this change be more significant with a larger number of partitions? -- This is an automated message from the Apache Git Service. To respond to the messa

[GitHub] [kafka] ijuma commented on a change in pull request #10457: KAFKA-12588: remove --zookeeper option from topic command

2021-04-02 Thread GitBox
ijuma commented on a change in pull request #10457: URL: https://github.com/apache/kafka/pull/10457#discussion_r606608453 ## File path: core/src/test/scala/unit/kafka/admin/TopicCommandWithAdminClientTest.scala ## @@ -65,9 +65,10 @@ class TopicCommandWithAdminClientTest extend

[GitHub] [kafka] showuon commented on a change in pull request #10457: KAFKA-12588: remove --zookeeper option from topic command

2021-04-02 Thread GitBox
showuon commented on a change in pull request #10457: URL: https://github.com/apache/kafka/pull/10457#discussion_r606607578 ## File path: core/src/test/scala/unit/kafka/admin/TopicCommandWithAdminClientTest.scala ## @@ -65,9 +65,10 @@ class TopicCommandWithAdminClientTest exte

[jira] [Commented] (KAFKA-7339) Migrate from JUnit 4 to JUnit 5

2021-04-02 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17314175#comment-17314175 ] Ismael Juma commented on KAFKA-7339: Assigned to [~chia7712] since both subtasks rema

[jira] [Assigned] (KAFKA-7339) Migrate from JUnit 4 to JUnit 5

2021-04-02 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma reassigned KAFKA-7339: -- Assignee: Chia-Ping Tsai (was: Ismael Juma) > Migrate from JUnit 4 to JUnit 5 > -

[jira] [Resolved] (KAFKA-12592) Remove deprecated LogConfig.Compact

2021-04-02 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-12592. - Resolution: Fixed https://github.com/apache/kafka/pull/10451 > Remove deprecated LogConfig.Comp

[jira] [Resolved] (KAFKA-8009) Uppgrade Jenkins job Gradle version from 4.10.2 to 5.x

2021-04-02 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-8009. Resolution: Fixed This was fixed a while back. > Uppgrade Jenkins job Gradle version from 4.10.2 t

[jira] [Resolved] (KAFKA-12600) Remove deprecated config value `default` for client config `client.dns.lookup`

2021-04-02 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-12600. - Resolution: Fixed > Remove deprecated config value `default` for client config `client.dns.looku

[jira] [Resolved] (KAFKA-12599) Remove deprecated --zookeeper in preferredReplicaLeaderElectionCommand

2021-04-02 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-12599. --- Resolution: Duplicate > Remove deprecated --zookeeper in preferredReplicaLeaderElectionCommand > ---

[jira] [Commented] (KAFKA-12463) Update default consumer partition assignor for sink tasks

2021-04-02 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17314151#comment-17314151 ] Chris Egerton commented on KAFKA-12463: --- Looks like this may be accomplished by [

[jira] [Updated] (KAFKA-12610) Resources on classpath break PluginClassLoader isolation

2021-04-02 Thread Nigel Liang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Liang updated KAFKA-12610: Description: The `PluginClassLoader` does not override `getResource` method it inherits from `Cla

[GitHub] [kafka] dengziming commented on pull request #10393: KAFKA-12539: Refactor KafkaRaftCllient handleVoteRequest to reduce cyclomatic complexity

2021-04-02 Thread GitBox
dengziming commented on pull request #10393: URL: https://github.com/apache/kafka/pull/10393#issuecomment-812770982 @hachikuji , Thank you, hava addressed your comments, and will take a look at KAFKA-12607. -- This is an automated message from the Apache Git Service. To respond to the me

[GitHub] [kafka] dengziming edited a comment on pull request #10393: KAFKA-12539: Refactor KafkaRaftCllient handleVoteRequest to reduce cyclomatic complexity

2021-04-02 Thread GitBox
dengziming edited a comment on pull request #10393: URL: https://github.com/apache/kafka/pull/10393#issuecomment-812770982 @hachikuji , Thank you, have addressed your comments, and will take a look at KAFKA-12607. -- This is an automated message from the Apache Git Service. To respond to

[jira] [Created] (KAFKA-12610) Resources on classpath break PluginClassLoader isolation

2021-04-02 Thread Nigel Liang (Jira)
Nigel Liang created KAFKA-12610: --- Summary: Resources on classpath break PluginClassLoader isolation Key: KAFKA-12610 URL: https://issues.apache.org/jira/browse/KAFKA-12610 Project: Kafka Issue

[jira] [Assigned] (KAFKA-12583) Upgrade of netty-codec due to CVE-2021-21295

2021-04-02 Thread Ben Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ben Chen reassigned KAFKA-12583: Assignee: Ben Chen (was: Dongjin Lee) > Upgrade of netty-codec due to CVE-2021-21295 > -

[GitHub] [kafka] dengziming opened a new pull request #10467: KAFKA-12609: Rewrite ListOffsets using new class

2021-04-02 Thread GitBox
dengziming opened a new pull request #10467: URL: https://github.com/apache/kafka/pull/10467 *More detailed description of your change* See here: https://github.com/apache/kafka/pull/10275#issuecomment-806331897 *Summary of testing strategy (including rationale)* 1. add a unit t

[jira] [Created] (KAFKA-12609) Rewrite ListOffsets using AdminApiDriver

2021-04-02 Thread dengziming (Jira)
dengziming created KAFKA-12609: -- Summary: Rewrite ListOffsets using AdminApiDriver Key: KAFKA-12609 URL: https://issues.apache.org/jira/browse/KAFKA-12609 Project: Kafka Issue Type: Improvement

[jira] [Commented] (KAFKA-12283) Flaky Test RebalanceSourceConnectorsIntegrationTest#testMultipleWorkersRejoining

2021-04-02 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17314127#comment-17314127 ] Luke Chen commented on KAFKA-12283: --- Thanks, [~ableegoldman] , I also put a note in KA

[jira] [Commented] (KAFKA-12495) Unbalanced connectors/tasks distribution will happen in Connect's incremental cooperative assignor

2021-04-02 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17314126#comment-17314126 ] Luke Chen commented on KAFKA-12495: --- Remember to enable the _testMultipleWorkersRejoin

[GitHub] [kafka] showuon commented on a change in pull request #10457: KAFKA-12588: remove --zookeeper option from topic command

2021-04-02 Thread GitBox
showuon commented on a change in pull request #10457: URL: https://github.com/apache/kafka/pull/10457#discussion_r606535769 ## File path: core/src/test/scala/unit/kafka/admin/TopicCommandWithAdminClientTest.scala ## @@ -65,9 +65,10 @@ class TopicCommandWithAdminClientTest exte

[jira] [Resolved] (KAFKA-3968) fsync() is not called on parent directory when new FileMessageSet is flushed to disk

2021-04-02 Thread Jun Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-3968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-3968. Fix Version/s: 3.0.0 Assignee: Cong Ding Resolution: Fixed Merged to trunk. Thanks Cong for

[GitHub] [kafka] junrao merged pull request #10405: KAFKA-3968: fsync the parent directory of a segment file when the file is created

2021-04-02 Thread GitBox
junrao merged pull request #10405: URL: https://github.com/apache/kafka/pull/10405 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[jira] [Commented] (KAFKA-9765) Could not add partitions to transaction due to errors

2021-04-02 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17314112#comment-17314112 ] Guozhang Wang commented on KAFKA-9765: -- Thanks for conforming. I'm closing this tick

[jira] [Resolved] (KAFKA-9765) Could not add partitions to transaction due to errors

2021-04-02 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang resolved KAFKA-9765. -- Resolution: Duplicate > Could not add partitions to transaction due to errors > ---

[jira] [Updated] (KAFKA-9765) Could not add partitions to transaction due to errors

2021-04-02 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guozhang Wang updated KAFKA-9765: - Fix Version/s: 2.5.0 2.4.2 > Could not add partitions to transaction due to er

[GitHub] [kafka] guozhangwang commented on pull request #10445: KAFKA-12548; Propagate record error messages to application

2021-04-02 Thread GitBox
guozhangwang commented on pull request #10445: URL: https://github.com/apache/kafka/pull/10445#issuecomment-812753885 LGTM. I'm fine to merge it as is and continue on https://issues.apache.org/jira/browse/KAFKA-12606 -- This is an automated message from the Apache Git Service. To respond

[GitHub] [kafka] ccding edited a comment on pull request #10405: KAFKA-3968: fsync the parent directory of a segment file when the file is created

2021-04-02 Thread GitBox
ccding edited a comment on pull request #10405: URL: https://github.com/apache/kafka/pull/10405#issuecomment-812744839 > It's interesting that the absolute throughput dropped significantly with 10MB segments compared with 1GB segment. I am thinking it is the cost of the extra flush.

[GitHub] [kafka] ccding edited a comment on pull request #10405: KAFKA-3968: fsync the parent directory of a segment file when the file is created

2021-04-02 Thread GitBox
ccding edited a comment on pull request #10405: URL: https://github.com/apache/kafka/pull/10405#issuecomment-812744839 > It's interesting that the absolute throughput dropped significantly with 10MB segments compared with 1GB segment. I am thinking it is the cost of the extra flush.

[GitHub] [kafka] ccding commented on pull request #10405: KAFKA-3968: fsync the parent directory of a segment file when the file is created

2021-04-02 Thread GitBox
ccding commented on pull request #10405: URL: https://github.com/apache/kafka/pull/10405#issuecomment-812744839 > It's interesting that the absolute throughput dropped significantly with 10MB segments compared with 1GB segment. I am thinking it is the cost of the extra flush. We have

[GitHub] [kafka] ccding commented on pull request #10405: KAFKA-3968: fsync the parent directory of a segment file when the file is created

2021-04-02 Thread GitBox
ccding commented on pull request #10405: URL: https://github.com/apache/kafka/pull/10405#issuecomment-812744102 With 100MB segment size: Before this PR: ``` 422206 records sent, 84441.2 records/sec (82.46 MB/sec), 333.0 ms avg latency, 997.0 ms max latency. 1016955 records s

[jira] [Commented] (KAFKA-12594) Various simple examples produce no ouput, and prevent other streams from producing output

2021-04-02 Thread Jamie Brandon (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17314100#comment-17314100 ] Jamie Brandon commented on KAFKA-12594: --- I filed a separate issue for the dropped

[jira] [Updated] (KAFKA-12608) Simple identity pipeline sometimes loses data

2021-04-02 Thread Jamie Brandon (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jamie Brandon updated KAFKA-12608: -- Description: I'm running a very simple streams program that reads records from one topic into

[jira] [Created] (KAFKA-12608) Simple identity pipeline sometimes loses data

2021-04-02 Thread Jamie Brandon (Jira)
Jamie Brandon created KAFKA-12608: - Summary: Simple identity pipeline sometimes loses data Key: KAFKA-12608 URL: https://issues.apache.org/jira/browse/KAFKA-12608 Project: Kafka Issue Type: B

[jira] [Updated] (KAFKA-12283) Flaky Test RebalanceSourceConnectorsIntegrationTest#testMultipleWorkersRejoining

2021-04-02 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman updated KAFKA-12283: --- Fix Version/s: 3.0.0 > Flaky Test > RebalanceSourceConnectorsIntegrationTes

[jira] [Commented] (KAFKA-12283) Flaky Test RebalanceSourceConnectorsIntegrationTest#testMultipleWorkersRejoining

2021-04-02 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17314098#comment-17314098 ] A. Sophie Blee-Goldman commented on KAFKA-12283: Ok I merged the PR to i

[jira] [Updated] (KAFKA-12283) Flaky Test RebalanceSourceConnectorsIntegrationTest#testMultipleWorkersRejoining

2021-04-02 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman updated KAFKA-12283: --- Priority: Blocker (was: Critical) > Flaky Test > RebalanceSourceConnectors

[GitHub] [kafka] ableegoldman merged pull request #10408: KAFKA-12283: disable flaky testMultipleWorkersRejoining to stabilize build

2021-04-02 Thread GitBox
ableegoldman merged pull request #10408: URL: https://github.com/apache/kafka/pull/10408 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, p

[jira] [Updated] (KAFKA-12594) Various simple examples produce no ouput, and prevent other streams from producing output

2021-04-02 Thread Jamie Brandon (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jamie Brandon updated KAFKA-12594: -- Summary: Various simple examples produce no ouput, and prevent other streams from producing ou

[jira] [Updated] (KAFKA-12344) Support SlidingWindows in the Scala API

2021-04-02 Thread Leah Thomas (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leah Thomas updated KAFKA-12344: Labels: newbie scala (was: ) > Support SlidingWindows in the Scala API >

[jira] [Assigned] (KAFKA-12344) Support SlidingWindows in the Scala API

2021-04-02 Thread Leah Thomas (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leah Thomas reassigned KAFKA-12344: --- Assignee: (was: Leah Thomas) > Support SlidingWindows in the Scala API > --

[GitHub] [kafka] junrao commented on pull request #10405: KAFKA-3968: fsync the parent directory of a segment file when the file is created

2021-04-02 Thread GitBox
junrao commented on pull request #10405: URL: https://github.com/apache/kafka/pull/10405#issuecomment-812709188 @ccding : Thanks for the experimental results. It seems there is a 5-10% throughput drop with the new PR for 10MB segment. This may not be a big concern since it's an uncommon se

[jira] [Assigned] (KAFKA-12417) streams module should use `testRuntimeClasspath` instead of `testRuntime` configuration

2021-04-02 Thread Jira
[ https://issues.apache.org/jira/browse/KAFKA-12417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dejan Stojadinović reassigned KAFKA-12417: -- Assignee: Dejan Stojadinović > streams module should use `testRuntimeClasspat

[GitHub] [kafka] ijuma commented on pull request #10466: MINOR: Gradle upgrade: 6.8.3 -->> 7.0-rc-2 [work in progress]

2021-04-02 Thread GitBox
ijuma commented on pull request #10466: URL: https://github.com/apache/kafka/pull/10466#issuecomment-812691270 @dejan2609 yes, please take the ticket. It would be great to solve this so that we can build with Java 16. -- This is an automated message from the Apache Git Service. To respon

[GitHub] [kafka] ijuma commented on a change in pull request #10450: KAFKA-12590: Remove deprecated kafka.security.auth.Authorizer, SimpleAclAuthorizer and related classes in 3.0

2021-04-02 Thread GitBox
ijuma commented on a change in pull request #10450: URL: https://github.com/apache/kafka/pull/10450#discussion_r606398713 ## File path: tests/kafkatest/tests/core/security_rolling_upgrade_test.py ## @@ -93,8 +93,8 @@ def roll_in_sasl_mechanism(self, security_protocol, new_sasl

[GitHub] [kafka] dejan2609 commented on pull request #10466: MINOR: Gradle upgrade: 6.8.3 -->> 7.0-rc-2 [work in progress]

2021-04-02 Thread GitBox
dejan2609 commented on pull request #10466: URL: https://github.com/apache/kafka/pull/10466#issuecomment-812690240 I just pushed `configurations.testRuntimeOnly.setCanBeResolved(true)` (I forgot to include that directive into my first commit). And yes, I do receive same circular dep

[jira] [Resolved] (KAFKA-12312) kafka_2.13:2.6.1 throws NoSuchMethodError when running against scala-sdk-2.13.4

2021-04-02 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-12312. - Resolution: Duplicate Thanks for the report. Marking this as a duplicate of KAFKA-12357 (which h

[GitHub] [kafka] rondagostino commented on a change in pull request #10450: KAFKA-12590: Remove deprecated kafka.security.auth.Authorizer, SimpleAclAuthorizer and related classes in 3.0

2021-04-02 Thread GitBox
rondagostino commented on a change in pull request #10450: URL: https://github.com/apache/kafka/pull/10450#discussion_r606394773 ## File path: tests/kafkatest/tests/core/security_rolling_upgrade_test.py ## @@ -93,8 +93,8 @@ def roll_in_sasl_mechanism(self, security_protocol, n

[GitHub] [kafka] rondagostino commented on a change in pull request #10450: KAFKA-12590: Remove deprecated kafka.security.auth.Authorizer, SimpleAclAuthorizer and related classes in 3.0

2021-04-02 Thread GitBox
rondagostino commented on a change in pull request #10450: URL: https://github.com/apache/kafka/pull/10450#discussion_r606392072 ## File path: docs/upgrade.html ## @@ -27,6 +27,8 @@ Notable changes in 3 or updating the application not to use internal classes. The

[GitHub] [kafka] ijuma commented on a change in pull request #10459: KAFKA-12601: Remove deprecated `delegation.token.master.key` in 3.0

2021-04-02 Thread GitBox
ijuma commented on a change in pull request #10459: URL: https://github.com/apache/kafka/pull/10459#discussion_r606391740 ## File path: core/src/main/scala/kafka/server/KafkaConfig.scala ## @@ -1292,8 +1290,7 @@ object KafkaConfig { .define(SaslLoginRefreshMinPeriodSecon

[GitHub] [kafka] ijuma commented on a change in pull request #10459: KAFKA-12601: Remove deprecated `delegation.token.master.key` in 3.0

2021-04-02 Thread GitBox
ijuma commented on a change in pull request #10459: URL: https://github.com/apache/kafka/pull/10459#discussion_r605722637 ## File path: core/src/main/scala/kafka/server/KafkaConfig.scala ## @@ -1292,8 +1290,7 @@ object KafkaConfig { .define(SaslLoginRefreshMinPeriodSecon

[GitHub] [kafka] ijuma commented on pull request #10450: KAFKA-12590: Remove deprecated kafka.security.auth.Authorizer, SimpleAclAuthorizer and related classes in 3.0

2021-04-02 Thread GitBox
ijuma commented on pull request #10450: URL: https://github.com/apache/kafka/pull/10450#issuecomment-812682607 @omkreddy I updated the PR description to elaborate a bit more on the motivation. I agree we should go ahead with this PR. Can you review it, please? -- This is an automated mes

[GitHub] [kafka] ijuma commented on pull request #10450: KAFKA-12590: Remove deprecated kafka.security.auth.Authorizer, SimpleAclAuthorizer and related classes

2021-04-02 Thread GitBox
ijuma commented on pull request #10450: URL: https://github.com/apache/kafka/pull/10450#issuecomment-812679620 I filed a JIRA for Apache Ranger: https://issues.apache.org/jira/browse/RANGER-3231 -- This is an automated message from the Apache Git Service. To respond to the message, ple

[GitHub] [kafka] ijuma commented on a change in pull request #10457: KAFKA-12588: remove --zookeeper option from topic command

2021-04-02 Thread GitBox
ijuma commented on a change in pull request #10457: URL: https://github.com/apache/kafka/pull/10457#discussion_r606387397 ## File path: core/src/test/scala/unit/kafka/admin/TopicCommandWithAdminClientTest.scala ## @@ -65,9 +65,10 @@ class TopicCommandWithAdminClientTest extend

[jira] [Commented] (KAFKA-12293) Remove JCenter and Bintray repositories mentions out of Gradle build (sunset is announced for those repositories)

2021-04-02 Thread Jira
[ https://issues.apache.org/jira/browse/KAFKA-12293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17314048#comment-17314048 ] Dejan Stojadinović commented on KAFKA-12293: Update: merged into trunk, but

[GitHub] [kafka] ijuma commented on pull request #10466: MINOR: Gradle upgrade: 6.8.3 -->> 7.0-rc-2 [work in progress]

2021-04-02 Thread GitBox
ijuma commented on pull request #10466: URL: https://github.com/apache/kafka/pull/10466#issuecomment-812678436 The change I had attempted was slightly different: https://issues.apache.org/jira/browse/KAFKA-12417 -- This is an automated message from the Apache Git Service. To respond to t

[GitHub] [kafka] ijuma commented on pull request #10466: MINOR: Gradle upgrade: 6.8.3 -->> 7.0-rc-2 [work in progress]

2021-04-02 Thread GitBox
ijuma commented on pull request #10466: URL: https://github.com/apache/kafka/pull/10466#issuecomment-81267 I filed a JIRA for this issue, the build failed with this change before. Did it work for you? -- This is an automated message from the Apache Git Service. To respond to the mess

[GitHub] [kafka] dejan2609 opened a new pull request #10466: MINOR: Gradle upgrade: 6.8.3 -->> 7.0-rc-2 [work in progress]

2021-04-02 Thread GitBox
dejan2609 opened a new pull request #10466: URL: https://github.com/apache/kafka/pull/10466 note: remaining _**'testRuntime'**_ reference changed to _**'testRuntimeOnly'**_ Related link: https://docs.gradle.org/7.0-rc-2/userguide/upgrading_version_6.html#sec:configuration_removal

[jira] [Commented] (KAFKA-9765) Could not add partitions to transaction due to errors

2021-04-02 Thread Mikael Carlstedt (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17314041#comment-17314041 ] Mikael Carlstedt commented on KAFKA-9765: - I ran into the same issue with 2.4.1,

[jira] [Commented] (KAFKA-12605) kafka consumer churns through buffer memory iterating over records

2021-04-02 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17314040#comment-17314040 ] Ismael Juma commented on KAFKA-12605: - Zstd and snappy recycle buffers by themselves

[jira] [Commented] (KAFKA-12605) kafka consumer churns through buffer memory iterating over records

2021-04-02 Thread radai rosenblatt (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17314037#comment-17314037 ] radai rosenblatt commented on KAFKA-12605: -- only compression type that uses thi

[GitHub] [kafka] cmccabe commented on pull request #10366: KAFKA-12467: Add controller-side snapshot generation

2021-04-02 Thread GitBox
cmccabe commented on pull request #10366: URL: https://github.com/apache/kafka/pull/10366#issuecomment-812667473 > Returning an iterator of batches works well when the control managers only have a single iterable thing that needs to be translated to records. Do you think we might have case

[GitHub] [kafka] cmccabe commented on a change in pull request #10366: KAFKA-12467: Add controller-side snapshot generation

2021-04-02 Thread GitBox
cmccabe commented on a change in pull request #10366: URL: https://github.com/apache/kafka/pull/10366#discussion_r606374075 ## File path: metadata/src/main/java/org/apache/kafka/controller/SnapshotGenerator.java ## @@ -0,0 +1,146 @@ +/* + * Licensed to the Apache Software Foun

[GitHub] [kafka] cmccabe commented on a change in pull request #10366: KAFKA-12467: Add controller-side snapshot generation

2021-04-02 Thread GitBox
cmccabe commented on a change in pull request #10366: URL: https://github.com/apache/kafka/pull/10366#discussion_r606373590 ## File path: metadata/src/main/java/org/apache/kafka/controller/QuorumController.java ## @@ -301,6 +328,89 @@ private void appendControlEvent(String nam

[GitHub] [kafka] cmccabe commented on a change in pull request #10366: KAFKA-12467: Add controller-side snapshot generation

2021-04-02 Thread GitBox
cmccabe commented on a change in pull request #10366: URL: https://github.com/apache/kafka/pull/10366#discussion_r606372293 ## File path: metadata/src/main/java/org/apache/kafka/controller/ClientQuotaControlManager.java ## @@ -272,4 +276,44 @@ private ApiError validateEntity(C

[jira] [Commented] (KAFKA-9559) Change the default "default serde" from ByteArraySerde to null

2021-04-02 Thread Leah Thomas (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17314023#comment-17314023 ] Leah Thomas commented on KAFKA-9559: [~mjsax] are you saying refactoring the code whe

[jira] [Commented] (KAFKA-9559) Change the default "default serde" from ByteArraySerde to null

2021-04-02 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-9559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17314022#comment-17314022 ] A. Sophie Blee-Goldman commented on KAFKA-9559: --- Definitely +1 to refactori

[GitHub] [kafka] mumrah commented on a change in pull request #10366: KAFKA-12467: Add controller-side snapshot generation

2021-04-02 Thread GitBox
mumrah commented on a change in pull request #10366: URL: https://github.com/apache/kafka/pull/10366#discussion_r606355983 ## File path: metadata/src/main/java/org/apache/kafka/controller/ClientQuotaControlManager.java ## @@ -272,4 +276,44 @@ private ApiError validateEntity(Cl

[GitHub] [kafka] ccding edited a comment on pull request #10405: KAFKA-3968: fsync the parent directory of a segment file when the file is created

2021-04-02 Thread GitBox
ccding edited a comment on pull request #10405: URL: https://github.com/apache/kafka/pull/10405#issuecomment-812645668 Failed tests are unrelated (connect and streams) and passed on my local run. -- This is an automated message from the Apache Git Service. To respond to the message, pleas

[GitHub] [kafka] ccding commented on pull request #10405: KAFKA-3968: fsync the parent directory of a segment file when the file is created

2021-04-02 Thread GitBox
ccding commented on pull request #10405: URL: https://github.com/apache/kafka/pull/10405#issuecomment-812645668 Failed tests are unrelated and passed on my local run. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

[GitHub] [kafka] ccding commented on pull request #10405: KAFKA-3968: fsync the parent directory of a segment file when the file is created

2021-04-02 Thread GitBox
ccding commented on pull request #10405: URL: https://github.com/apache/kafka/pull/10405#issuecomment-812642793 @junrao Thanks for the reiview. The previous test was run with default settings in `config/server.properties`, where `log.segment.bytes=1073741824` (1GB). I changed it to `log.se

[GitHub] [kafka] cmccabe opened a new pull request #10465: MINOR: add unit test for ControllerApis#createTopics

2021-04-02 Thread GitBox
cmccabe opened a new pull request #10465: URL: https://github.com/apache/kafka/pull/10465 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [kafka] hachikuji commented on a change in pull request #10393: KAFKA-12539: Refactor KafkaRaftCllient handleVoteRequest to reduce cyclomatic complexity

2021-04-02 Thread GitBox
hachikuji commented on a change in pull request #10393: URL: https://github.com/apache/kafka/pull/10393#discussion_r606332415 ## File path: raft/src/main/java/org/apache/kafka/raft/UnattachedState.java ## @@ -88,6 +93,16 @@ public boolean hasElectionTimeoutExpired(long currentT

[jira] [Commented] (KAFKA-12607) Allow votes to be granted in resigned state

2021-04-02 Thread Jason Gustafson (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17313992#comment-17313992 ] Jason Gustafson commented on KAFKA-12607: - [~dengziming] I assigned this to you

[jira] [Created] (KAFKA-12607) Allow votes to be granted in resigned state

2021-04-02 Thread Jason Gustafson (Jira)
Jason Gustafson created KAFKA-12607: --- Summary: Allow votes to be granted in resigned state Key: KAFKA-12607 URL: https://issues.apache.org/jira/browse/KAFKA-12607 Project: Kafka Issue Type:

[GitHub] [kafka] hachikuji commented on pull request #10445: KAFKA-12548; Propagate record error messages to application

2021-04-02 Thread GitBox
hachikuji commented on pull request #10445: URL: https://github.com/apache/kafka/pull/10445#issuecomment-812620642 One of the test failures `testSendWithInvalidCreateTime` is due to this patch. I am thinking how to address. -- This is an automated message from the Apache Git Service. To

[GitHub] [kafka] mjsax commented on pull request #10464: MINOR: remove KTable.to from the docs

2021-04-02 Thread GitBox
mjsax commented on pull request #10464: URL: https://github.com/apache/kafka/pull/10464#issuecomment-812607364 Merged to `trunk` and cherry-picked to `2.8`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[GitHub] [kafka] mjsax merged pull request #10464: MINOR: remove KTable.to from the docs

2021-04-02 Thread GitBox
mjsax merged pull request #10464: URL: https://github.com/apache/kafka/pull/10464 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please c

[GitHub] [kafka] satishd commented on pull request #10218: KAFKA-12368: Added inmemory implementations for RemoteStorageManager and RemoteLogMetadataManager.

2021-04-02 Thread GitBox
satishd commented on pull request #10218: URL: https://github.com/apache/kafka/pull/10218#issuecomment-812572844 @junrao @kowshik thanks for your review and comments. In the latest commit, I refactored the code with better abstractions and addressed the review comments. -- This is an au

[GitHub] [kafka] dengziming commented on a change in pull request #10393: KAFKA-12539: Refactor KafkaRaftCllient handleVoteRequest to reduce cyclomatic complexity

2021-04-02 Thread GitBox
dengziming commented on a change in pull request #10393: URL: https://github.com/apache/kafka/pull/10393#discussion_r606258483 ## File path: raft/src/main/java/org/apache/kafka/raft/KafkaRaftClient.java ## @@ -592,44 +592,17 @@ private VoteResponseData handleVoteRequest(

[GitHub] [kafka] satishd commented on a change in pull request #10218: KAFKA-12368: Added inmemory implementations for RemoteStorageManager and RemoteLogMetadataManager.

2021-04-02 Thread GitBox
satishd commented on a change in pull request #10218: URL: https://github.com/apache/kafka/pull/10218#discussion_r606257469 ## File path: remote-storage/src/test/java/org/apache/kafka/server/log/remote/storage/RemoteLogMetadataCacheTest.java ## @@ -0,0 +1,149 @@ +/* + * Licens

[GitHub] [kafka] satishd commented on a change in pull request #10218: KAFKA-12368: Added inmemory implementations for RemoteStorageManager and RemoteLogMetadataManager.

2021-04-02 Thread GitBox
satishd commented on a change in pull request #10218: URL: https://github.com/apache/kafka/pull/10218#discussion_r606253804 ## File path: remote-storage/src/test/java/org/apache/kafka/server/log/remote/storage/InmemoryRemoteStorageManager.java ## @@ -0,0 +1,171 @@ +/* + * Lice

[GitHub] [kafka] satishd commented on a change in pull request #10218: KAFKA-12368: Added inmemory implementations for RemoteStorageManager and RemoteLogMetadataManager.

2021-04-02 Thread GitBox
satishd commented on a change in pull request #10218: URL: https://github.com/apache/kafka/pull/10218#discussion_r606253220 ## File path: remote-storage/src/test/java/org/apache/kafka/server/log/remote/storage/RemoteLogMetadataCacheTest.java ## @@ -0,0 +1,149 @@ +/* + * Licens

[GitHub] [kafka] satishd commented on a change in pull request #10218: KAFKA-12368: Added inmemory implementations for RemoteStorageManager and RemoteLogMetadataManager.

2021-04-02 Thread GitBox
satishd commented on a change in pull request #10218: URL: https://github.com/apache/kafka/pull/10218#discussion_r606251255 ## File path: remote-storage/src/test/java/org/apache/kafka/server/log/remote/storage/RemoteLogMetadataCacheTest.java ## @@ -0,0 +1,149 @@ +/* + * Licens

[GitHub] [kafka] satishd commented on a change in pull request #10218: KAFKA-12368: Added inmemory implementations for RemoteStorageManager and RemoteLogMetadataManager.

2021-04-02 Thread GitBox
satishd commented on a change in pull request #10218: URL: https://github.com/apache/kafka/pull/10218#discussion_r606250883 ## File path: remote-storage/src/main/java/org/apache/kafka/server/log/remote/storage/RemoteLogMetadataCache.java ## @@ -0,0 +1,172 @@ +/* + * Licensed t

[GitHub] [kafka] satishd commented on a change in pull request #10218: KAFKA-12368: Added inmemory implementations for RemoteStorageManager and RemoteLogMetadataManager.

2021-04-02 Thread GitBox
satishd commented on a change in pull request #10218: URL: https://github.com/apache/kafka/pull/10218#discussion_r606249998 ## File path: remote-storage/src/test/java/org/apache/kafka/server/log/remote/storage/InmemoryRemoteStorageManager.java ## @@ -0,0 +1,171 @@ +/* + * Lice

[GitHub] [kafka] satishd commented on a change in pull request #10218: KAFKA-12368: Added inmemory implementations for RemoteStorageManager and RemoteLogMetadataManager.

2021-04-02 Thread GitBox
satishd commented on a change in pull request #10218: URL: https://github.com/apache/kafka/pull/10218#discussion_r606249948 ## File path: remote-storage/src/test/java/org/apache/kafka/server/log/remote/storage/InmemoryRemoteStorageManager.java ## @@ -0,0 +1,171 @@ +/* + * Lice

[GitHub] [kafka] vamossagar12 commented on pull request #10278: KAFKA-10526: leader fsync deferral on write

2021-04-02 Thread GitBox
vamossagar12 commented on pull request #10278: URL: https://github.com/apache/kafka/pull/10278#issuecomment-812492705 @jsancio could you plz take a look at this PR whenever you get the chance... -- This is an automated message from the Apache Git Service. To respond to the message, pleas

[GitHub] [kafka] vamossagar12 commented on a change in pull request #9756: KAFKA-10652: Adding size based linger semnatics to Raft metadata

2021-04-02 Thread GitBox
vamossagar12 commented on a change in pull request #9756: URL: https://github.com/apache/kafka/pull/9756#discussion_r606198207 ## File path: raft/src/main/java/org/apache/kafka/raft/RaftConfig.java ## @@ -76,6 +76,11 @@ "wait for writes to accumulate before flushing th

[jira] [Commented] (KAFKA-12486) Utilize HighAvailabilityTaskAssignor to avoid downtime on corrupted task

2021-04-02 Thread Sagar Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17313814#comment-17313814 ] Sagar Rao commented on KAFKA-12486: --- hi [~ableegoldman], i would like to take this one

[jira] [Assigned] (KAFKA-12486) Utilize HighAvailabilityTaskAssignor to avoid downtime on corrupted task

2021-04-02 Thread Sagar Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sagar Rao reassigned KAFKA-12486: - Assignee: Sagar Rao > Utilize HighAvailabilityTaskAssignor to avoid downtime on corrupted task

[jira] [Commented] (KAFKA-12468) Initial offsets are copied from source to target cluster

2021-04-02 Thread Duc Minh Le (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17313793#comment-17313793 ] Duc Minh Le commented on KAFKA-12468: - I got the same issue with Mirror Maker 2.7, c

[GitHub] [kafka] showuon edited a comment on pull request #10457: KAFKA-12588: remove --zookeeper option from topic command

2021-04-02 Thread GitBox
showuon edited a comment on pull request #10457: URL: https://github.com/apache/kafka/pull/10457#issuecomment-812456367 @ijuma , thanks for the good suggestion. Yes, we don't need the `TopicService` trait now. Updated. Please help review it again. Thank you. -- This is an automated messa

[GitHub] [kafka] showuon commented on pull request #10457: KAFKA-12588: remove --zookeeper option from topic command

2021-04-02 Thread GitBox
showuon commented on pull request #10457: URL: https://github.com/apache/kafka/pull/10457#issuecomment-812456367 @ijuma , thanks for the good suggestion. Please help review it again. Thank you. -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [kafka] showuon commented on a change in pull request #10457: KAFKA-12588: remove --zookeeper option from topic command

2021-04-02 Thread GitBox
showuon commented on a change in pull request #10457: URL: https://github.com/apache/kafka/pull/10457#discussion_r606161628 ## File path: core/src/test/scala/unit/kafka/admin/TopicCommandWithAdminClientTest.scala ## @@ -98,15 +100,15 @@ class TopicCommandWithAdminClientTest ex

[GitHub] [kafka] showuon commented on a change in pull request #10457: KAFKA-12588: remove --zookeeper option from topic command

2021-04-02 Thread GitBox
showuon commented on a change in pull request #10457: URL: https://github.com/apache/kafka/pull/10457#discussion_r606160749 ## File path: core/src/main/scala/kafka/admin/TopicCommand.scala ## @@ -610,20 +441,18 @@ object TopicCommand extends Logging { } class TopicComma

[GitHub] [kafka] showuon commented on a change in pull request #10457: KAFKA-12588: remove --zookeeper option from topic command

2021-04-02 Thread GitBox
showuon commented on a change in pull request #10457: URL: https://github.com/apache/kafka/pull/10457#discussion_r606160442 ## File path: core/src/main/scala/kafka/admin/TopicCommand.scala ## @@ -204,23 +196,7 @@ object TopicCommand extends Logging { } } - trait Topi

[GitHub] [kafka] showuon commented on a change in pull request #10457: KAFKA-12588: remove --zookeeper option from topic command

2021-04-02 Thread GitBox
showuon commented on a change in pull request #10457: URL: https://github.com/apache/kafka/pull/10457#discussion_r606160332 ## File path: core/src/main/scala/kafka/admin/TopicCommand.scala ## @@ -229,13 +205,21 @@ object TopicCommand extends Logging { Admin.create(comman

  1   2   >