[GitHub] [kafka] ivanyu commented on pull request #13984: KAFKA-15107: Support custom metadata for remote log segment

2023-08-02 Thread via GitHub
ivanyu commented on PR #13984: URL: https://github.com/apache/kafka/pull/13984#issuecomment-1663331558 Thank you @showuon for the review! Please let me know if I need to squash the commits. -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [kafka] lihaosky commented on a diff in pull request #14139: KAFKA-15022: [6/N] use RackAwareTaskAssignor in HAAssignor

2023-08-02 Thread via GitHub
lihaosky commented on code in PR #14139: URL: https://github.com/apache/kafka/pull/14139#discussion_r1282664432 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/assignment/TaskAssignor.java: ## @@ -16,18 +16,29 @@ */ package

[GitHub] [kafka] showuon commented on a diff in pull request #13984: KAFKA-15107: Support custom metadata for remote log segment

2023-08-02 Thread via GitHub
showuon commented on code in PR #13984: URL: https://github.com/apache/kafka/pull/13984#discussion_r1282649360 ## core/src/main/java/kafka/log/remote/CustomMetadataSizeLimitExceededException.java: ## @@ -0,0 +1,20 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] [kafka] ableegoldman commented on a diff in pull request #14139: KAFKA-15022: [6/N] use RackAwareTaskAssignor in HAAssignor

2023-08-02 Thread via GitHub
ableegoldman commented on code in PR #14139: URL: https://github.com/apache/kafka/pull/14139#discussion_r1282604132 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/assignment/RackAwareTaskAssignor.java: ## @@ -43,26 +51,39 @@ import

[GitHub] [kafka] github-actions[bot] commented on pull request #13674: MINOR: Release resources in FetcherTest

2023-08-02 Thread via GitHub
github-actions[bot] commented on PR #13674: URL: https://github.com/apache/kafka/pull/13674#issuecomment-1663242081 This PR is being marked as stale since it has not had any activity in 90 days. If you would like to keep this PR alive, please ask a committer for review. If the PR has

[GitHub] [kafka] ableegoldman commented on a diff in pull request #14139: KAFKA-15022: [6/N] use RackAwareTaskAssignor in HAAssignor

2023-08-02 Thread via GitHub
ableegoldman commented on code in PR #14139: URL: https://github.com/apache/kafka/pull/14139#discussion_r1282600984 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/assignment/TaskAssignor.java: ## @@ -16,18 +16,29 @@ */ package

[GitHub] [kafka] flashmouse commented on pull request #13920: KAFKA-15106 fix AbstractStickyAssignor isBalanced predict

2023-08-02 Thread via GitHub
flashmouse commented on PR #13920: URL: https://github.com/apache/kafka/pull/13920#issuecomment-1663210404 > > I hadn't changed the logic, it is still a slow function. whether execute performReassignments is checked by isBalanced, but it is not effective previously and this pr just fix

[GitHub] [kafka] philipnee commented on pull request #14123: MINOR: Fix committed API in the PrototypeAsyncConsumer timeout

2023-08-02 Thread via GitHub
philipnee commented on PR #14123: URL: https://github.com/apache/kafka/pull/14123#issuecomment-1663201841 There seems to be some weirdness to the build - jdk 17 was timedout or something. Retriggered the build... -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [kafka] lihaosky commented on a diff in pull request #14108: KAFKA-15022: [5/N] compute rack aware assignment for standby tasks

2023-08-02 Thread via GitHub
lihaosky commented on code in PR #14108: URL: https://github.com/apache/kafka/pull/14108#discussion_r1282572143 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/assignment/RackAwareTaskAssignorTest.java: ## @@ -715,23 +1087,49 @@ private Map>>

[GitHub] [kafka] mjsax merged pull request #14108: KAFKA-15022: [5/N] compute rack aware assignment for standby tasks

2023-08-02 Thread via GitHub
mjsax merged PR #14108: URL: https://github.com/apache/kafka/pull/14108 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] mjsax commented on a diff in pull request #14108: KAFKA-15022: [5/N] compute rack aware assignment for standby tasks

2023-08-02 Thread via GitHub
mjsax commented on code in PR #14108: URL: https://github.com/apache/kafka/pull/14108#discussion_r1282569306 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/assignment/RackAwareTaskAssignorTest.java: ## @@ -715,23 +1087,49 @@ private Map>>

[GitHub] [kafka] mjsax commented on a diff in pull request #14108: KAFKA-15022: [5/N] compute rack aware assignment for standby tasks

2023-08-02 Thread via GitHub
mjsax commented on code in PR #14108: URL: https://github.com/apache/kafka/pull/14108#discussion_r1282569306 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/assignment/RackAwareTaskAssignorTest.java: ## @@ -715,23 +1087,49 @@ private Map>>

[GitHub] [kafka] mjsax commented on a diff in pull request #14108: KAFKA-15022: [5/N] compute rack aware assignment for standby tasks

2023-08-02 Thread via GitHub
mjsax commented on code in PR #14108: URL: https://github.com/apache/kafka/pull/14108#discussion_r1282568706 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/assignment/RackAwareTaskAssignorTest.java: ## @@ -715,23 +1087,49 @@ private Map>>

[GitHub] [kafka] mjsax commented on a diff in pull request #14108: KAFKA-15022: [5/N] compute rack aware assignment for standby tasks

2023-08-02 Thread via GitHub
mjsax commented on code in PR #14108: URL: https://github.com/apache/kafka/pull/14108#discussion_r1282562934 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/assignment/RackAwareTaskAssignorTest.java: ## @@ -678,18 +838,230 @@ public void

[GitHub] [kafka] mjsax commented on a diff in pull request #14108: KAFKA-15022: [5/N] compute rack aware assignment for standby tasks

2023-08-02 Thread via GitHub
mjsax commented on code in PR #14108: URL: https://github.com/apache/kafka/pull/14108#discussion_r1282562934 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/assignment/RackAwareTaskAssignorTest.java: ## @@ -678,18 +838,230 @@ public void

[GitHub] [kafka] mjsax commented on a diff in pull request #14108: KAFKA-15022: [5/N] compute rack aware assignment for standby tasks

2023-08-02 Thread via GitHub
mjsax commented on code in PR #14108: URL: https://github.com/apache/kafka/pull/14108#discussion_r1282562934 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/assignment/RackAwareTaskAssignorTest.java: ## @@ -678,18 +838,230 @@ public void

[jira] [Commented] (KAFKA-12829) Remove Deprecated methods under Topology

2023-08-02 Thread Junyao Huang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17750508#comment-17750508 ] Junyao Huang commented on KAFKA-12829: -- Thanks [~mjsax] for clarification! > Remove Deprecated

[jira] [Commented] (KAFKA-15116) Kafka Streams processing blocked during rebalance

2023-08-02 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17750506#comment-17750506 ] Matthias J. Sax commented on KAFKA-15116: - Are you saying you are using a custom store? For this

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-02 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1282518474 ## core/src/main/scala/kafka/server/DelegationTokenManager.scala: ## @@ -165,10 +112,11 @@ object DelegationTokenManager { class DelegationTokenManager(val

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-02 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1282514451 ## metadata/src/main/java/org/apache/kafka/controller/DelegationTokenControlManager.java: ## @@ -0,0 +1,359 @@ +/* + * Licensed to the Apache Software Foundation

[jira] [Commented] (KAFKA-15297) Cache flush order might not be topological order

2023-08-02 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17750504#comment-17750504 ] Matthias J. Sax commented on KAFKA-15297: - The ticket description contains an example to

[jira] [Commented] (KAFKA-12829) Remove Deprecated methods under Topology

2023-08-02 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17750503#comment-17750503 ] Matthias J. Sax commented on KAFKA-12829: - It's actually unclear when 4.0 will come along, so it

[jira] [Updated] (KAFKA-15299) Support left stream-table join on foreign key

2023-08-02 Thread Igor Fomenko (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Fomenko updated KAFKA-15299: - Description: KIP-955:

[GitHub] [kafka] ex172000 commented on pull request #14110: MINOR: Add test for describe topic with ID

2023-08-02 Thread via GitHub
ex172000 commented on PR #14110: URL: https://github.com/apache/kafka/pull/14110#issuecomment-1663074232 Hi @divijvaidya can you take another look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[jira] [Commented] (KAFKA-15297) Cache flush order might not be topological order

2023-08-02 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17750495#comment-17750495 ] A. Sophie Blee-Goldman commented on KAFKA-15297: Were you able to (re)produce this

[jira] [Commented] (KAFKA-12829) Remove Deprecated methods under Topology

2023-08-02 Thread Junyao Huang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17750494#comment-17750494 ] Junyao Huang commented on KAFKA-12829: -- Yes, [~guozhang] , I am willing to raise a PR for this

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-02 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1282479785 ## core/src/main/scala/kafka/server/ControllerApis.scala: ## @@ -842,6 +847,75 @@ class ControllerApis(val requestChannel: RequestChannel, } } + def

[jira] [Commented] (KAFKA-12317) Relax non-null key requirement for left/outer KStream joins

2023-08-02 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17750491#comment-17750491 ] Guozhang Wang commented on KAFKA-12317: --- Thanks [~aki] ! Will take a look soon. > Relax non-null

[jira] [Commented] (KAFKA-12829) Remove Deprecated methods under Topology

2023-08-02 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17750490#comment-17750490 ] Guozhang Wang commented on KAFKA-12829: --- [~pegasas] Your comment seems deleted, are you still

[GitHub] [kafka] guozhangwang commented on pull request #13920: KAFKA-15106 fix AbstractStickyAssignor isBalanced predict

2023-08-02 Thread via GitHub
guozhangwang commented on PR #13920: URL: https://github.com/apache/kafka/pull/13920#issuecomment-1663038140 > I hadn't changed the logic, it is still a slow function. whether execute performReassignments is checked by isBalanced, but it is not effective previously and this pr just fix

[GitHub] [kafka] elkkhan commented on pull request #14077: KAFKA-14112: Expose replication-offset-lag Mirror metric

2023-08-02 Thread via GitHub
elkkhan commented on PR #14077: URL: https://github.com/apache/kafka/pull/14077#issuecomment-1663002964 > @elkkhan I think it was mentioned on the [Jira ticket](https://issues.apache.org/jira/browse/KAFKA-14112) that a KIP is required for new metrics; have you published a KIP for this

[GitHub] [kafka] elkkhan commented on a diff in pull request #14077: KAFKA-14112: Expose replication-offset-lag Mirror metric

2023-08-02 Thread via GitHub
elkkhan commented on code in PR #14077: URL: https://github.com/apache/kafka/pull/14077#discussion_r1282447173 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/AbstractWorkerSourceTask.java: ## @@ -108,7 +108,7 @@ public abstract class AbstractWorkerSourceTask

[GitHub] [kafka] jsancio commented on pull request #14141: KAFKA-15100; KRaft data race with the expiration service

2023-08-02 Thread via GitHub
jsancio commented on PR #14141: URL: https://github.com/apache/kafka/pull/14141#issuecomment-1662978648 The two tests added in the PR fail against trunk. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [kafka] jsancio opened a new pull request, #14141: KAFKA-15100; KRaft data race with the expiration service

2023-08-02 Thread via GitHub
jsancio opened a new pull request, #14141: URL: https://github.com/apache/kafka/pull/14141 The KRaft client uses an expiration service to complete FETCH requests that have timed out. This expiration service uses a different thread from the KRaft polling thread. This means that it is unsafe

[GitHub] [kafka] gharris1727 commented on pull request #14064: KAFKA-15030: Add connect-plugin-path command-line tool.

2023-08-02 Thread via GitHub
gharris1727 commented on PR #14064: URL: https://github.com/apache/kafka/pull/14064#issuecomment-1662952817 Okay I have a potential fix: we can make the URL unique by inserting `.` in the path. It works with the test case you provided, and even this one: ``` cd

[GitHub] [kafka] mumrah opened a new pull request, #14140: KAFKA-14538 Metadata Transactions [2/n]

2023-08-02 Thread via GitHub
mumrah opened a new pull request, #14140: URL: https://github.com/apache/kafka/pull/14140 Allows DeferredEventQueue to complete events for "unstable" offsets. This will be used for metadata transactions where the committed offset is increasing, but the stable offset is not. For

[GitHub] [kafka] omkreddy commented on a diff in pull request #14130: KAFKA-15273: Log common name of expired client certificates

2023-08-02 Thread via GitHub
omkreddy commented on code in PR #14130: URL: https://github.com/apache/kafka/pull/14130#discussion_r1282124322 ## clients/src/main/java/org/apache/kafka/common/security/ssl/DefaultSslEngineFactory.java: ## @@ -16,47 +16,48 @@ */ package org.apache.kafka.common.security.ssl;

[GitHub] [kafka] omkreddy commented on a diff in pull request #14130: KAFKA-15273: Log common name of expired client certificates

2023-08-02 Thread via GitHub
omkreddy commented on code in PR #14130: URL: https://github.com/apache/kafka/pull/14130#discussion_r1282124322 ## clients/src/main/java/org/apache/kafka/common/security/ssl/DefaultSslEngineFactory.java: ## @@ -16,47 +16,48 @@ */ package org.apache.kafka.common.security.ssl;

[GitHub] [kafka] gharris1727 commented on a diff in pull request #14064: KAFKA-15030: Add connect-plugin-path command-line tool.

2023-08-02 Thread via GitHub
gharris1727 commented on code in PR #14064: URL: https://github.com/apache/kafka/pull/14064#discussion_r1282351442 ## tools/src/main/java/org/apache/kafka/tools/ConnectPluginPath.java: ## @@ -0,0 +1,498 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or

[GitHub] [kafka] jeffkbkim commented on a diff in pull request #14124: Kafka-14509; [1/2] Define ConsumerGroupDescribe API request and response schemas and classes.

2023-08-02 Thread via GitHub
jeffkbkim commented on code in PR #14124: URL: https://github.com/apache/kafka/pull/14124#discussion_r1282327275 ## clients/src/main/resources/common/message/ConsumerGroupDescribeResponse.json: ## @@ -0,0 +1,98 @@ +// Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [kafka] jeffkbkim commented on a diff in pull request #14124: Kafka-14509; [1/2] Define ConsumerGroupDescribe API request and response schemas and classes.

2023-08-02 Thread via GitHub
jeffkbkim commented on code in PR #14124: URL: https://github.com/apache/kafka/pull/14124#discussion_r1282326175 ## clients/src/main/resources/common/message/ConsumerGroupDescribeResponse.json: ## @@ -0,0 +1,98 @@ +// Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [kafka] ruslankrivoshein commented on pull request #13562: KAFKA-14581: Moving GetOffsetShell to tools

2023-08-02 Thread via GitHub
ruslankrivoshein commented on PR #13562: URL: https://github.com/apache/kafka/pull/13562#issuecomment-1662850171 @dengziming could you tell me, what is a good approach for rebase here? When I do this, I face conflicts. After resolving I do push, and there are dozens of commits in PR, so I

[GitHub] [kafka] junrao commented on a diff in pull request #14111: KAFKA-9800: Exponential backoff for Kafka clients - KIP-580

2023-08-02 Thread via GitHub
junrao commented on code in PR #14111: URL: https://github.com/apache/kafka/pull/14111#discussion_r1282312134 ## clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordAccumulator.java: ## @@ -73,6 +75,7 @@ public class RecordAccumulator { private final

[jira] [Updated] (KAFKA-15299) Support left stream-table join on foreign key

2023-08-02 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15299: Labels: kip (was: ) > Support left stream-table join on foreign key >

[GitHub] [kafka] gharris1727 commented on pull request #14064: KAFKA-15030: Add connect-plugin-path command-line tool.

2023-08-02 Thread via GitHub
gharris1727 commented on PR #14064: URL: https://github.com/apache/kafka/pull/14064#issuecomment-1662806844 > As far as the epilogue output goes, I wonder if we're getting a little too detailed here... will users get much out of knowing the number of total locations/compatible locations?

[GitHub] [kafka] C0urante commented on a diff in pull request #14064: KAFKA-15030: Add connect-plugin-path command-line tool.

2023-08-02 Thread via GitHub
C0urante commented on code in PR #14064: URL: https://github.com/apache/kafka/pull/14064#discussion_r1281978798 ## tools/src/main/java/org/apache/kafka/tools/ConnectPluginPath.java: ## @@ -0,0 +1,498 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more

[jira] [Updated] (KAFKA-15291) Implement Versioned interfaces in common Connect plugins

2023-08-02 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Harris updated KAFKA-15291: Summary: Implement Versioned interfaces in common Connect plugins (was: Implemented Versioned

[jira] [Resolved] (KAFKA-13562) Mirror Maker 2 Negative Offsets

2023-08-02 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Harris resolved KAFKA-13562. - Fix Version/s: 3.3.3 3.4.1 3.5.0 Resolution: Fixed

[jira] [Resolved] (KAFKA-8597) Give access to the Dead Letter Queue APIs to Kafka Connect Developers

2023-08-02 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Harris resolved KAFKA-8597. Fix Version/s: 2.6.0 Resolution: Fixed > Give access to the Dead Letter Queue APIs to Kafka

[jira] [Resolved] (KAFKA-8754) Connect API: Expose the list of available transformations

2023-08-02 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-8754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Harris resolved KAFKA-8754. Fix Version/s: 3.2.0 Resolution: Fixed > Connect API: Expose the list of available

[jira] [Created] (KAFKA-15300) Include remotelog size in complete log size and also add local log size and remote log size separately in kafka-log-dirs tool.

2023-08-02 Thread Satish Duggana (Jira)
Satish Duggana created KAFKA-15300: -- Summary: Include remotelog size in complete log size and also add local log size and remote log size separately in kafka-log-dirs tool. Key: KAFKA-15300 URL:

[jira] [Comment Edited] (KAFKA-15102) Mirror Maker 2 - KIP690 backward compatibility

2023-08-02 Thread Omnia Ibrahim (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17750371#comment-17750371 ] Omnia Ibrahim edited comment on KAFKA-15102 at 8/2/23 3:06 PM: --- The KIP

[jira] [Comment Edited] (KAFKA-15102) Mirror Maker 2 - KIP690 backward compatibility

2023-08-02 Thread Omnia Ibrahim (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17750371#comment-17750371 ] Omnia Ibrahim edited comment on KAFKA-15102 at 8/2/23 3:06 PM: --- The KIP is

[jira] [Commented] (KAFKA-15102) Mirror Maker 2 - KIP690 backward compatibility

2023-08-02 Thread Omnia Ibrahim (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17750371#comment-17750371 ] Omnia Ibrahim commented on KAFKA-15102: --- The KIP is accepted and the PR is ready > Mirror Maker 2

[jira] [Commented] (KAFKA-15298) Disable DeleteRecords on Tiered Storage topics

2023-08-02 Thread Kamal Chandraprakash (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17750357#comment-17750357 ] Kamal Chandraprakash commented on KAFKA-15298: -- [~christo_lolov] Could you explain why

[jira] [Created] (KAFKA-15299) Support left stream-table join on foreign key

2023-08-02 Thread Igor Fomenko (Jira)
Igor Fomenko created KAFKA-15299: Summary: Support left stream-table join on foreign key Key: KAFKA-15299 URL: https://issues.apache.org/jira/browse/KAFKA-15299 Project: Kafka Issue Type:

[jira] [Commented] (KAFKA-10339) MirrorMaker2 Exactly-once Semantics

2023-08-02 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17750338#comment-17750338 ] Chris Egerton commented on KAFKA-10339: --- Now that KAFKA-6080 has been resolved with

[jira] [Commented] (KAFKA-15116) Kafka Streams processing blocked during rebalance

2023-08-02 Thread David Gammon (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17750311#comment-17750311 ] David Gammon commented on KAFKA-15116: -- # The store is storing everything from all partitions which

[jira] [Created] (KAFKA-15298) Disable DeleteRecords on Tiered Storage topics

2023-08-02 Thread Christo Lolov (Jira)
Christo Lolov created KAFKA-15298: - Summary: Disable DeleteRecords on Tiered Storage topics Key: KAFKA-15298 URL: https://issues.apache.org/jira/browse/KAFKA-15298 Project: Kafka Issue Type:

[jira] [Updated] (KAFKA-15298) Disable DeleteRecords on Tiered Storage topics

2023-08-02 Thread Christo Lolov (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christo Lolov updated KAFKA-15298: -- Labels: tiered-storage (was: ) > Disable DeleteRecords on Tiered Storage topics >

[jira] [Updated] (KAFKA-15297) Cache flush order might not be topological order

2023-08-02 Thread Bruno Cadonna (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Cadonna updated KAFKA-15297: -- Description: The flush order of the state store caches in Kafka Streams might not correspond

[jira] [Updated] (KAFKA-15297) Cache flush order might not be topological order

2023-08-02 Thread Bruno Cadonna (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Cadonna updated KAFKA-15297: -- Description: The flush order of the state store caches in Kafka Streams might not correspond

[jira] [Updated] (KAFKA-15297) Cache flush order might not be topological order

2023-08-02 Thread Bruno Cadonna (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Cadonna updated KAFKA-15297: -- Description: The flush order of the state store caches in Kafka Streams might not correspond

[jira] [Updated] (KAFKA-15297) Cache flush order might not be topological order

2023-08-02 Thread Bruno Cadonna (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Cadonna updated KAFKA-15297: -- Description: The flush order of the state store caches in Kafka Streams might not correspond

[jira] [Created] (KAFKA-15297) Cache flush order might not be topological order

2023-08-02 Thread Bruno Cadonna (Jira)
Bruno Cadonna created KAFKA-15297: - Summary: Cache flush order might not be topological order Key: KAFKA-15297 URL: https://issues.apache.org/jira/browse/KAFKA-15297 Project: Kafka Issue

[jira] [Updated] (KAFKA-15297) Cache flush order might not be topological order

2023-08-02 Thread Bruno Cadonna (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Cadonna updated KAFKA-15297: -- Attachment: minimal_example.png > Cache flush order might not be topological order >

[GitHub] [kafka] satishd commented on a diff in pull request #13561: KAFKA-14888: Added remote log segments retention functionality based on time and size.

2023-08-02 Thread via GitHub
satishd commented on code in PR #13561: URL: https://github.com/apache/kafka/pull/13561#discussion_r1281844159 ## core/src/main/scala/kafka/log/UnifiedLog.scala: ## @@ -945,13 +978,19 @@ class UnifiedLog(@volatile var logStartOffset: Long,

[jira] [Updated] (KAFKA-15154) Potential bug: We don't acquire lock when reading checkQuotas

2023-08-02 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya updated KAFKA-15154: - Component/s: metrics > Potential bug: We don't acquire lock when reading checkQuotas >

[jira] [Updated] (KAFKA-15058) Improve the accuracy of Histogram in client metric

2023-08-02 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya updated KAFKA-15058: - Component/s: metrics > Improve the accuracy of Histogram in client metric >

[GitHub] [kafka] bmscomp commented on pull request #14131: MINOR: Upgrade version of Gradle entreprise to 3.14.1 & custom user data Gradle plugin to 1.11.1

2023-08-02 Thread via GitHub
bmscomp commented on PR #14131: URL: https://github.com/apache/kafka/pull/14131#issuecomment-1662023509 @divijvaidya I just made a rebase, wait to see the build results -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [kafka] ivanyu commented on a diff in pull request #13984: KAFKA-15107: Support custom metadata for remote log segment

2023-08-02 Thread via GitHub
ivanyu commented on code in PR #13984: URL: https://github.com/apache/kafka/pull/13984#discussion_r1281763529 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -621,10 +626,30 @@ private void copyLogSegment(UnifiedLog log, LogSegment segment, long nextSegment

[jira] [Created] (KAFKA-15296) Allow committing offsets for Dropped records via SMTs

2023-08-02 Thread Sagar Rao (Jira)
Sagar Rao created KAFKA-15296: - Summary: Allow committing offsets for Dropped records via SMTs Key: KAFKA-15296 URL: https://issues.apache.org/jira/browse/KAFKA-15296 Project: Kafka Issue Type:

[GitHub] [kafka] divijvaidya commented on pull request #14131: MINOR: Upgrade version of Gradle entreprise to 3.14.1 & custom user data Gradle plugin to 1.11.1

2023-08-02 Thread via GitHub
divijvaidya commented on PR #14131: URL: https://github.com/apache/kafka/pull/14131#issuecomment-1661996961 CI has a compilation error which is fixed by https://github.com/apache/kafka/pull/14126. Please rebase from trunk and we can start the CI run again. -- This is an automated

[GitHub] [kafka] clolov commented on a diff in pull request #14135: KAFKA-14991: Implementation of KIP-937 which improves message timesta…

2023-08-02 Thread via GitHub
clolov commented on code in PR #14135: URL: https://github.com/apache/kafka/pull/14135#discussion_r1281724231 ## core/src/main/scala/kafka/server/KafkaConfig.scala: ## @@ -1862,7 +1884,23 @@ class KafkaConfig private(doLog: Boolean, val props: java.util.Map[_, _], dynami

[jira] [Updated] (KAFKA-15295) Add config validation when remote storage is enabled on a topic

2023-08-02 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya updated KAFKA-15295: - Fix Version/s: 3.6.0 > Add config validation when remote storage is enabled on a topic >

[GitHub] [kafka] divijvaidya commented on a diff in pull request #13947: KAFKA-15130: Delete remote segments when delete a topic

2023-08-02 Thread via GitHub
divijvaidya commented on code in PR #13947: URL: https://github.com/apache/kafka/pull/13947#discussion_r1252999633 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -471,6 +472,9 @@ class ReplicaManager(val config: KafkaConfig, case

[GitHub] [kafka] divijvaidya commented on a diff in pull request #13947: KAFKA-15130: Delete remote segments when delete a topic

2023-08-02 Thread via GitHub
divijvaidya commented on code in PR #13947: URL: https://github.com/apache/kafka/pull/13947#discussion_r1252989860 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -615,6 +661,9 @@ public void run() { try { Optional

[GitHub] [kafka] fvaleri commented on pull request #14121: MINOR: Add comment to onPartitionsLost override

2023-08-02 Thread via GitHub
fvaleri commented on PR #14121: URL: https://github.com/apache/kafka/pull/14121#issuecomment-1661979478 @dajac are you good with these changes? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [kafka] divijvaidya commented on a diff in pull request #13947: KAFKA-15130: Delete remote segments when delete a topic

2023-08-02 Thread via GitHub
divijvaidya commented on code in PR #13947: URL: https://github.com/apache/kafka/pull/13947#discussion_r1252976430 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -556,6 +562,46 @@ public void copyLogSegmentsToRemote(UnifiedLog log) throws

[GitHub] [kafka] kamalcph commented on pull request #14116: KAFKA-15167: Tiered Storage Test Harness Framework

2023-08-02 Thread via GitHub
kamalcph commented on PR #14116: URL: https://github.com/apache/kafka/pull/14116#issuecomment-1661973903 > Hello and thank you for starting on this ! What is your plan when it comes to reviewing this code - will you be splitting it into multiple pull requests for individual

[GitHub] [kafka] kamalcph commented on a diff in pull request #14114: KAFKA-12969: Add broker level config synonyms for topic level tiered storage configs

2023-08-02 Thread via GitHub
kamalcph commented on code in PR #14114: URL: https://github.com/apache/kafka/pull/14114#discussion_r1281723208 ## storage/src/main/java/org/apache/kafka/storage/internals/log/LogConfig.java: ## @@ -102,49 +102,14 @@ public String topicWarningMessage(String topicName) {

[jira] [Updated] (KAFKA-15295) Add config validation when remote storage is enabled on a topic

2023-08-02 Thread Kamal Chandraprakash (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamal Chandraprakash updated KAFKA-15295: - Parent: KAFKA-7739 Issue Type: Sub-task (was: Task) > Add config

[jira] [Created] (KAFKA-15295) Add config validation when remote storage is enabled on a topic

2023-08-02 Thread Kamal Chandraprakash (Jira)
Kamal Chandraprakash created KAFKA-15295: Summary: Add config validation when remote storage is enabled on a topic Key: KAFKA-15295 URL: https://issues.apache.org/jira/browse/KAFKA-15295

[GitHub] [kafka] divijvaidya commented on a diff in pull request #14133: KAFKA-15189: only init remote topic metrics when enabled

2023-08-02 Thread via GitHub
divijvaidya commented on code in PR #14133: URL: https://github.com/apache/kafka/pull/14133#discussion_r1281718684 ## storage/api/src/main/java/org/apache/kafka/server/log/remote/storage/RemoteStorageMetrics.java: ## @@ -0,0 +1,93 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] divijvaidya commented on a diff in pull request #14114: KAFKA-12969: Add broker level config synonyms for topic level tiered storage configs

2023-08-02 Thread via GitHub
divijvaidya commented on code in PR #14114: URL: https://github.com/apache/kafka/pull/14114#discussion_r1281717116 ## storage/src/main/java/org/apache/kafka/storage/internals/log/LogConfig.java: ## @@ -102,49 +102,14 @@ public String topicWarningMessage(String topicName) {

[GitHub] [kafka] clolov commented on pull request #14116: KAFKA-15167: Tiered Storage Test Harness Framework

2023-08-02 Thread via GitHub
clolov commented on PR #14116: URL: https://github.com/apache/kafka/pull/14116#issuecomment-1661952445 Hello and thank you for starting on this ! What is your plan when it comes to reviewing this code - will you be splitting it into multiple pull requests for individual

[GitHub] [kafka] clolov commented on a diff in pull request #14116: KAFKA-15167: Tiered Storage Test Harness Framework

2023-08-02 Thread via GitHub
clolov commented on code in PR #14116: URL: https://github.com/apache/kafka/pull/14116#discussion_r1281706478 ## storage/src/test/java/integration/kafka/tiered/storage/actions/CreateTopicAction.java: ## @@ -0,0 +1,74 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] [kafka] kamalcph commented on a diff in pull request #14114: KAFKA-12969: Add broker level config synonyms for topic level tiered storage configs

2023-08-02 Thread via GitHub
kamalcph commented on code in PR #14114: URL: https://github.com/apache/kafka/pull/14114#discussion_r1281701381 ## storage/src/main/java/org/apache/kafka/storage/internals/log/LogConfig.java: ## @@ -102,49 +102,14 @@ public String topicWarningMessage(String topicName) {

[GitHub] [kafka] ivanyu commented on a diff in pull request #13984: KAFKA-15107: Support custom metadata for remote log segment

2023-08-02 Thread via GitHub
ivanyu commented on code in PR #13984: URL: https://github.com/apache/kafka/pull/13984#discussion_r1281701101 ## core/src/main/java/kafka/log/remote/CustomMetadataSizeLimitExceededException.java: ## @@ -0,0 +1,20 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] [kafka] OmniaGM commented on pull request #13201: KAFKA-14596: Move TopicCommand to tools

2023-08-02 Thread via GitHub
OmniaGM commented on PR #13201: URL: https://github.com/apache/kafka/pull/13201#issuecomment-1661932121 I have updated the PR to use #13158 and #13278. The test checks are failing because of an unrelated class. However the `:tools:compileTestJava` is failing because it doesn't like

[GitHub] [kafka] showuon commented on a diff in pull request #14133: KAFKA-15189: only init remote topic metrics when enabled

2023-08-02 Thread via GitHub
showuon commented on code in PR #14133: URL: https://github.com/apache/kafka/pull/14133#discussion_r1281679405 ## storage/api/src/main/java/org/apache/kafka/server/log/remote/storage/RemoteStorageMetrics.java: ## @@ -0,0 +1,93 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] showuon commented on a diff in pull request #13561: KAFKA-14888: Added remote log segments retention functionality based on time and size.

2023-08-02 Thread via GitHub
showuon commented on code in PR #13561: URL: https://github.com/apache/kafka/pull/13561#discussion_r1281657244 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -667,11 +675,323 @@ public void run() { } } +public void

[GitHub] [kafka] satishd commented on a diff in pull request #13561: KAFKA-14888: Added remote log segments retention functionality based on time and size.

2023-08-02 Thread via GitHub
satishd commented on code in PR #13561: URL: https://github.com/apache/kafka/pull/13561#discussion_r1281672320 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -618,6 +625,230 @@ public void run() { } } +public void

[GitHub] [kafka] ivanyu commented on a diff in pull request #13984: KAFKA-15107: Support custom metadata for remote log segment

2023-08-02 Thread via GitHub
ivanyu commented on code in PR #13984: URL: https://github.com/apache/kafka/pull/13984#discussion_r1281661247 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -621,10 +626,30 @@ private void copyLogSegment(UnifiedLog log, LogSegment segment, long nextSegment

[GitHub] [kafka] divijvaidya commented on a diff in pull request #14133: KAFKA-15189: only init remote topic metrics when enabled

2023-08-02 Thread via GitHub
divijvaidya commented on code in PR #14133: URL: https://github.com/apache/kafka/pull/14133#discussion_r1281642972 ## storage/api/src/main/java/org/apache/kafka/server/log/remote/storage/RemoteStorageMetrics.java: ## @@ -0,0 +1,93 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] ivanyu commented on a diff in pull request #13984: KAFKA-15107: Support custom metadata for remote log segment

2023-08-02 Thread via GitHub
ivanyu commented on code in PR #13984: URL: https://github.com/apache/kafka/pull/13984#discussion_r1281646428 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -572,6 +573,9 @@ public void copyLogSegmentsToRemote(UnifiedLog log) throws InterruptedException

[GitHub] [kafka] divijvaidya commented on a diff in pull request #14114: KAFKA-12969: Add broker level config synonyms for topic level tiered storage configs

2023-08-02 Thread via GitHub
divijvaidya commented on code in PR #14114: URL: https://github.com/apache/kafka/pull/14114#discussion_r1281624661 ## storage/src/main/java/org/apache/kafka/storage/internals/log/LogConfig.java: ## @@ -102,49 +102,14 @@ public String topicWarningMessage(String topicName) {

[GitHub] [kafka] ivanyu commented on a diff in pull request #13984: KAFKA-15107: Support custom metadata for remote log segment

2023-08-02 Thread via GitHub
ivanyu commented on code in PR #13984: URL: https://github.com/apache/kafka/pull/13984#discussion_r1281602105 ## storage/api/src/test/java/org/apache/kafka/server/log/remote/storage/RemoteLogSegmentMetadataTest.java: ## @@ -0,0 +1,67 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] ivanyu commented on a diff in pull request #13984: KAFKA-15107: Support custom metadata for remote log segment

2023-08-02 Thread via GitHub
ivanyu commented on code in PR #13984: URL: https://github.com/apache/kafka/pull/13984#discussion_r1281591602 ## core/src/test/java/kafka/log/remote/RemoteLogManagerTest.java: ## @@ -239,96 +240,186 @@ void testStartup() { assertEquals(logDir,

[jira] [Commented] (KAFKA-15267) Cluster-wide disablement of Tiered Storage

2023-08-02 Thread Christo Lolov (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17750188#comment-17750188 ] Christo Lolov commented on KAFKA-15267: --- Yup, you are correct. 4.1 can be the first deliverable,

  1   2   >