[jira] [Resolved] (KAFKA-16035) add integration test for ExpiresPerSec and RemoteLogSizeComputationTime metrics

2023-12-21 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-16035. --- Fix Version/s: 3.7.0 Resolution: Fixed > add integration test for ExpiresPerSec and

Re: [PR] KAFKA-16035: add tests for remoteLogSizeComputationTime/remoteFetchExpiresPerSec metrics [kafka]

2023-12-21 Thread via GitHub
showuon merged PR #15056: URL: https://github.com/apache/kafka/pull/15056 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-16014: Implement RemoteLogSizeBytes [kafka]

2023-12-21 Thread via GitHub
showuon merged PR #15050: URL: https://github.com/apache/kafka/pull/15050 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-16014: Implement RemoteLogSizeBytes [kafka]

2023-12-21 Thread via GitHub
showuon commented on PR #15050: URL: https://github.com/apache/kafka/pull/15050#issuecomment-1867309312 Had run twice CI and failed tests are unrelated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[jira] [Updated] (KAFKA-16046) Stream Stream Joins fail after restoration with deserialization exceptions

2023-12-21 Thread Almog Gavra (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Almog Gavra updated KAFKA-16046: Labels: streams (was: ) > Stream Stream Joins fail after restoration with deserialization

[jira] [Updated] (KAFKA-16046) Stream Stream Joins fail after restoration with deserialization exceptions

2023-12-21 Thread Almog Gavra (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Almog Gavra updated KAFKA-16046: Component/s: streams > Stream Stream Joins fail after restoration with deserialization exceptions

[jira] [Created] (KAFKA-16046) Stream Stream Joins fail after restoration with deserialization exceptions

2023-12-21 Thread Almog Gavra (Jira)
Almog Gavra created KAFKA-16046: --- Summary: Stream Stream Joins fail after restoration with deserialization exceptions Key: KAFKA-16046 URL: https://issues.apache.org/jira/browse/KAFKA-16046 Project:

[jira] [Commented] (KAFKA-16046) Stream Stream Joins fail after restoration with deserialization exceptions

2023-12-21 Thread Almog Gavra (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17799623#comment-17799623 ] Almog Gavra commented on KAFKA-16046: - https://github.com/apache/kafka/pull/15061 > Stream Stream

[PR] fix stream-stream-join store types [kafka]

2023-12-21 Thread via GitHub
agavra opened a new pull request, #15061: URL: https://github.com/apache/kafka/pull/15061 Before #14648, the `KStreamImplJoin` class would always create non-timestamped persistent windowed stores. After that PR, the default was changed to create timestamped stores. This wasn't compatible

Re: [PR] KAFKA-16040; Rename `Generic` to `Classic` [kafka]

2023-12-21 Thread via GitHub
philipnee commented on code in PR #15059: URL: https://github.com/apache/kafka/pull/15059#discussion_r1434559814 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/LegacyKafkaConsumer.java: ## @@ -96,13 +96,13 @@ import static

Re: [PR] KAFKA-16026: Send Poll event to the background thread [kafka]

2023-12-21 Thread via GitHub
philipnee commented on PR #15035: URL: https://github.com/apache/kafka/pull/15035#issuecomment-1867003094 @dajac - thanks for merging the changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[jira] [Resolved] (KAFKA-16040) Rename `Generic` to `Classic`

2023-12-21 Thread David Jacot (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-16040. - Resolution: Fixed > Rename `Generic` to `Classic` > - > >

Re: [PR] KAFKA-16040; Rename `Generic` to `Classic` [kafka]

2023-12-21 Thread via GitHub
dajac merged PR #15059: URL: https://github.com/apache/kafka/pull/15059 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-16040; Rename `Generic` to `Classic` [kafka]

2023-12-21 Thread via GitHub
dajac commented on PR #15059: URL: https://github.com/apache/kafka/pull/15059#issuecomment-1866963946 As it is really hard to get a clean build due to OOM errors (also in trunk), I have verified the build locally. I am going to merge it to trunk and 3.7. -- This is an automated message

[jira] [Updated] (KAFKA-16045) ZkMigrationIntegrationTest.testMigrateTopicDeletion flaky

2023-12-21 Thread Justine Olshan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justine Olshan updated KAFKA-16045: --- Labels: flaky-test (was: ) > ZkMigrationIntegrationTest.testMigrateTopicDeletion flaky >

[jira] [Created] (KAFKA-16045) ZkMigrationIntegrationTest.testMigrateTopicDeletion flaky

2023-12-21 Thread Justine Olshan (Jira)
Justine Olshan created KAFKA-16045: -- Summary: ZkMigrationIntegrationTest.testMigrateTopicDeletion flaky Key: KAFKA-16045 URL: https://issues.apache.org/jira/browse/KAFKA-16045 Project: Kafka

Re: [PR] KAFKA-16012: Ensure new leader information merged correctly with the current metadata [kafka]

2023-12-21 Thread via GitHub
jolshan commented on PR #15023: URL: https://github.com/apache/kafka/pull/15023#issuecomment-1866936613 jira here: [KAFKA-16045](https://issues.apache.org/jira/browse/KAFKA-16045) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] KAFKA-16012: Ensure new leader information merged correctly with the current metadata [kafka]

2023-12-21 Thread via GitHub
jolshan commented on PR #15023: URL: https://github.com/apache/kafka/pull/15023#issuecomment-1866933588 Yeah. I was looking at builds with @dajac for his PR too. The java 8 one here OOM'd. I think the ZkMigrationTests are causing the thread leak for the replica manager since I see

Re: [PR] KAFKA-16012: Ensure new leader information merged correctly with the current metadata [kafka]

2023-12-21 Thread via GitHub
philipnee commented on PR #15023: URL: https://github.com/apache/kafka/pull/15023#issuecomment-1866886304 jenkins doesn't seem to be in a great state. there are a bunch of zkMigrationIntegrationTest and ReplicaManagerTest complaining timeout and `Found unexpected 1 NonDaemon threads`

[jira] [Updated] (KAFKA-16027) Refactor MetadataTest#testUpdatePartitionLeadership

2023-12-21 Thread Alexander Aghili (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Aghili updated KAFKA-16027: - External issue URL: https://github.com/apache/kafka/pull/15055 > Refactor

[jira] [Updated] (KAFKA-16027) Refactor MetadataTest#testUpdatePartitionLeadership

2023-12-21 Thread Alexander Aghili (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Aghili updated KAFKA-16027: - External issue URL: (was: https://github.com/apache/kafka/pull/15055) > Refactor

[jira] [Assigned] (KAFKA-16027) Refactor MetadataTest#testUpdatePartitionLeadership

2023-12-21 Thread Alexander Aghili (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Aghili reassigned KAFKA-16027: Assignee: Alexander Aghili > Refactor MetadataTest#testUpdatePartitionLeadership

Re: [PR] KAFKA-16012: Ensure new leader information merged correctly with the current metadata [kafka]

2023-12-21 Thread via GitHub
philipnee commented on PR #15023: URL: https://github.com/apache/kafka/pull/15023#issuecomment-1866810047 Thank you! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] KAFKA-16021: Eagerly look up StringSerializer encoding during configure [kafka]

2023-12-21 Thread via GitHub
srdo commented on PR #15024: URL: https://github.com/apache/kafka/pull/15024#issuecomment-1866797498 Thanks. Added the suggested test. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] KAFKA-14505; [3/N] Wire WriteTxnMarkers API [kafka]

2023-12-21 Thread via GitHub
dajac merged PR #14985: URL: https://github.com/apache/kafka/pull/14985 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-14505; [3/N] Wire WriteTxnMarkers API [kafka]

2023-12-21 Thread via GitHub
dajac commented on PR #14985: URL: https://github.com/apache/kafka/pull/14985#issuecomment-1866794392 We've got a decent build [here](https://ci-builds.apache.org/blue/organizations/jenkins/Kafka%2Fkafka-pr/detail/PR-14985/9/tests/). All the failed tests are unrelated. I am going to merge

Re: [PR] KAFKA-15303: Avoid unnecessary re-serialization in FK-join [kafka]

2023-12-21 Thread via GitHub
CharlyRien commented on PR #14157: URL: https://github.com/apache/kafka/pull/14157#issuecomment-1866758963 Hello. Any news on this issue? @mjsax -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[jira] [Commented] (KAFKA-15388) Handle topics that were having compaction as retention earlier are changed to delete only retention policy and onboarded to tiered storage.

2023-12-21 Thread Arpit Goyal (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17799532#comment-17799532 ] Arpit Goyal commented on KAFKA-15388: - [~divijvaidya] [~satish.duggana] [~showuon] Pr for review

Re: [PR] KAFKA-15373: fix exception thrown in Admin#describeTopics for unknown ID [kafka]

2023-12-21 Thread via GitHub
jolshan commented on PR #14599: URL: https://github.com/apache/kafka/pull/14599#issuecomment-1866730662 Sorry for the delay @MikeEdgar. That is a known issue for the test. I will also look at this PR and run the build again. -- This is an automated message from the Apache Git Service. To

Re: [PR] KAFKA-16012: Ensure new leader information merged correctly with the current metadata [kafka]

2023-12-21 Thread via GitHub
jolshan commented on PR #15023: URL: https://github.com/apache/kafka/pull/15023#issuecomment-1866728811 Thanks for checking @philipnee. I will wait for this run to complete and we should be good to go. -- This is an automated message from the Apache Git Service. To respond to the

[PR] KAFKA-15388: Handling remote segment read in case of log compaction [kafka]

2023-12-21 Thread via GitHub
iit2009060 opened a new pull request, #15060: URL: https://github.com/apache/kafka/pull/15060 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of

Re: [PR] MINOR: Use --no-daemon when building with Jenkins [kafka]

2023-12-21 Thread via GitHub
ijuma commented on code in PR #15057: URL: https://github.com/apache/kafka/pull/15057#discussion_r1434339183 ## Jenkinsfile: ## @@ -21,7 +21,7 @@ def doValidation() { // Run all the tasks associated with `check` except for `test` - the latter is executed via `doTest` sh

Re: [PR] KAFKA-16012: Ensure new leader information merged correctly with the current metadata [kafka]

2023-12-21 Thread via GitHub
philipnee commented on PR #15023: URL: https://github.com/apache/kafka/pull/15023#issuecomment-1866678547 Consumer Integration tests also seems fine. ![image](https://github.com/apache/kafka/assets/1930388/df5307c2-0735-4789-9b87-e6d5eb9f1e7d) -- This is an automated message from

[jira] [Commented] (KAFKA-10897) kafka quota optimization

2023-12-21 Thread Afshin Moazami (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17799504#comment-17799504 ] Afshin Moazami commented on KAFKA-10897: For topic-partition quota configuration, I proposed

[jira] [Commented] (KAFKA-16041) Replace Afterburn module with Blackbird

2023-12-21 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17799498#comment-17799498 ] Mickael Maison commented on KAFKA-16041: This is follow up work from

Re: [PR] KAFKA-16012: Ensure new leader information merged correctly with the current metadata [kafka]

2023-12-21 Thread via GitHub
philipnee commented on PR #15023: URL: https://github.com/apache/kafka/pull/15023#issuecomment-1866627736 ![image](https://github.com/apache/kafka/assets/1930388/6d79e6b4-76dc-49cc-8cbb-9b237a5dc7ef) Unit tests seems fine, there is 1 known flaky test under AbstractCoordinator and 1

[jira] [Created] (KAFKA-16044) Throttling using Topic Partition Quota

2023-12-21 Thread Afshin Moazami (Jira)
Afshin Moazami created KAFKA-16044: -- Summary: Throttling using Topic Partition Quota Key: KAFKA-16044 URL: https://issues.apache.org/jira/browse/KAFKA-16044 Project: Kafka Issue Type: New

[jira] [Created] (KAFKA-16043) Add Quota configuration for topics

2023-12-21 Thread Afshin Moazami (Jira)
Afshin Moazami created KAFKA-16043: -- Summary: Add Quota configuration for topics Key: KAFKA-16043 URL: https://issues.apache.org/jira/browse/KAFKA-16043 Project: Kafka Issue Type: New

[jira] [Created] (KAFKA-16042) Quota Metrics based on per topic-partition produce/fetch byte rate

2023-12-21 Thread Afshin Moazami (Jira)
Afshin Moazami created KAFKA-16042: -- Summary: Quota Metrics based on per topic-partition produce/fetch byte rate Key: KAFKA-16042 URL: https://issues.apache.org/jira/browse/KAFKA-16042 Project:

Re: [PR] KAFKA-16012: Ensure new leader information merged correctly with the current metadata [kafka]

2023-12-21 Thread via GitHub
philipnee commented on PR #15023: URL: https://github.com/apache/kafka/pull/15023#issuecomment-1866607465 Hey @jolshan - Thanks. OOM also happens to another PR of mine see #15035 - i thought it was introduced by the AsyncKafkaConsumerTest but apparently there's more to it... Let me verify

Re: [PR] KAFKA-16012: Ensure new leader information merged correctly with the current metadata [kafka]

2023-12-21 Thread via GitHub
jolshan commented on PR #15023: URL: https://github.com/apache/kafka/pull/15023#issuecomment-1866595042 I'm seeing some OOMs and such again. I will run one more time. @philipnee if you don't mind just double checking locally for the tests on your end. I'm not seeing any failing

Re: [PR] KAFKA-16016: Add docker wrapper in core and remove docker utility script [kafka]

2023-12-21 Thread via GitHub
mimaison commented on code in PR #15048: URL: https://github.com/apache/kafka/pull/15048#discussion_r1434227907 ## docker/jvm/launch: ## @@ -28,25 +27,25 @@ fi # the default is to pick the first IP (or network). export KAFKA_JMX_HOSTNAME=${KAFKA_JMX_HOSTNAME:-$(hostname -i |

Re: [PR] KAFKA-14505; [3/N] Wire WriteTxnMarkers API [kafka]

2023-12-21 Thread via GitHub
jolshan commented on code in PR #14985: URL: https://github.com/apache/kafka/pull/14985#discussion_r1434244004 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/runtime/CoordinatorRuntimeTest.java: ## @@ -1073,6 +1151,314 @@ public CoordinatorShardBuilder

Re: [PR] KAFKA-14505; [3/N] Wire WriteTxnMarkers API [kafka]

2023-12-21 Thread via GitHub
jolshan commented on code in PR #14985: URL: https://github.com/apache/kafka/pull/14985#discussion_r1434242991 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/runtime/InMemoryPartitionWriter.java: ## @@ -102,6 +241,31 @@ public long append( }

Re: [PR] KAFKA-14588 ConfigType moved to server-common [kafka]

2023-12-21 Thread via GitHub
nizhikov commented on code in PR #14867: URL: https://github.com/apache/kafka/pull/14867#discussion_r1434240386 ## core/src/main/scala/kafka/zk/ZkData.scala: ## @@ -1103,11 +1105,11 @@ object ZkData { IsrChangeNotificationZNode.path, ProducerIdBlockZNode.path,

Re: [PR] KAFKA-15996: Improve JsonConverter performance [kafka]

2023-12-21 Thread via GitHub
mfvitale commented on PR #14992: URL: https://github.com/apache/kafka/pull/14992#issuecomment-1866538380 > Thank you for this PR. I have left some minor comments. > > Also, please add a JIRA targeting 4.0 (where we will deprecate JDK 8) to replace afterburn with blackbird.

[jira] [Created] (KAFKA-16041) Replace Afterburn module with Blackbird

2023-12-21 Thread Mario Fiore Vitale (Jira)
Mario Fiore Vitale created KAFKA-16041: -- Summary: Replace Afterburn module with Blackbird Key: KAFKA-16041 URL: https://issues.apache.org/jira/browse/KAFKA-16041 Project: Kafka Issue

Re: [PR] KAFKA-15996: Improve JsonConverter performance [kafka]

2023-12-21 Thread via GitHub
mfvitale commented on code in PR #14992: URL: https://github.com/apache/kafka/pull/14992#discussion_r1434225385 ## build.gradle: ## @@ -940,6 +940,7 @@ project(':core') { implementation libs.jacksonModuleScala implementation libs.jacksonDataformatCsv

Re: [PR] KAFKA-14505; [3/N] Wire WriteTxnMarkers API [kafka]

2023-12-21 Thread via GitHub
dajac commented on code in PR #14985: URL: https://github.com/apache/kafka/pull/14985#discussion_r1434220316 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/runtime/InMemoryPartitionWriter.java: ## @@ -102,6 +241,31 @@ public long append( } }

Re: [PR] KAFKA-14505; [3/N] Wire WriteTxnMarkers API [kafka]

2023-12-21 Thread via GitHub
dajac commented on code in PR #14985: URL: https://github.com/apache/kafka/pull/14985#discussion_r1434218393 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/runtime/CoordinatorRuntimeTest.java: ## @@ -1073,6 +1151,314 @@ public CoordinatorShardBuilder

Re: [PR] KAFKA-14505; [3/N] Wire WriteTxnMarkers API [kafka]

2023-12-21 Thread via GitHub
jolshan commented on code in PR #14985: URL: https://github.com/apache/kafka/pull/14985#discussion_r1434215098 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/runtime/InMemoryPartitionWriter.java: ## @@ -102,6 +241,31 @@ public long append( }

Re: [PR] KAFKA-14505; [3/N] Wire WriteTxnMarkers API [kafka]

2023-12-21 Thread via GitHub
jolshan commented on code in PR #14985: URL: https://github.com/apache/kafka/pull/14985#discussion_r1434214195 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/runtime/CoordinatorRuntimeTest.java: ## @@ -1073,6 +1151,314 @@ public CoordinatorShardBuilder

Re: [PR] KAFKA-14588 ConfigType moved to server-common [kafka]

2023-12-21 Thread via GitHub
mimaison commented on code in PR #14867: URL: https://github.com/apache/kafka/pull/14867#discussion_r1434193675 ## core/src/main/scala/kafka/zk/ZkData.scala: ## @@ -1103,11 +1105,11 @@ object ZkData { IsrChangeNotificationZNode.path, ProducerIdBlockZNode.path,

Re: [PR] [KAFKA-16015] Fix custom timeouts overwritten by defaults [kafka]

2023-12-21 Thread via GitHub
pprovenzano commented on PR #15030: URL: https://github.com/apache/kafka/pull/15030#issuecomment-1866442934 > @pprovenzano , I dont have permissions to merge the PR, so I guess somebody else eventually will do it, is that correct? thanks! I don't have them either. I've asked another

[jira] [Resolved] (KAFKA-15456) Add support for OffsetFetch version 9 in consumer

2023-12-21 Thread David Jacot (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-15456. - Resolution: Fixed > Add support for OffsetFetch version 9 in consumer >

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

2023-12-21 Thread via GitHub
dajac merged PR #14557: URL: https://github.com/apache/kafka/pull/14557 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

2023-12-21 Thread via GitHub
dajac commented on PR #14557: URL: https://github.com/apache/kafka/pull/14557#issuecomment-1866401989 We've got a reasonably good build here: https://ci-builds.apache.org/blue/organizations/jenkins/Kafka%2Fkafka-pr/detail/PR-14557/34/tests/. Based on this one, it is safe to merge this PR.

Re: [PR] KAFKA-16040; Rename `Generic` to `Classic` [kafka]

2023-12-21 Thread via GitHub
dajac commented on PR #15059: URL: https://github.com/apache/kafka/pull/15059#issuecomment-1866369216 @lucasbru Thanks. I just fixed them. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] KAFKA-16040; Rename `Generic` to `Classic` [kafka]

2023-12-21 Thread via GitHub
lucasbru commented on PR #15059: URL: https://github.com/apache/kafka/pull/15059#issuecomment-1866354730 Please check * `GroupCoordinatorConfig.GENERIC_GROUP_NEW_MEMBER_JOIN_TIMEOUT_MS` * Some comments in `GroupCoordinator` * Various names in `GroupCoordinatorMetrics` -- This is

[jira] [Resolved] (KAFKA-16030) new group coordinator should check if partition goes offline during load

2023-12-21 Thread David Jacot (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-16030. - Fix Version/s: 3.7.0 Resolution: Fixed > new group coordinator should check if partition

Re: [PR] KAFKA-16030: new group coordinator should check if partition goes offline during load [kafka]

2023-12-21 Thread via GitHub
dajac commented on PR #15043: URL: https://github.com/apache/kafka/pull/15043#issuecomment-1866342682 It is hard to clean a clean build here too but failures are unrelated. I also verified it locally. I will merge it to trunk and 3.7. -- This is an automated message from the Apache Git

Re: [PR] KAFKA-16030: new group coordinator should check if partition goes offline during load [kafka]

2023-12-21 Thread via GitHub
dajac merged PR #15043: URL: https://github.com/apache/kafka/pull/15043 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-15996: Improve JsonConverter performance [kafka]

2023-12-21 Thread via GitHub
divijvaidya commented on code in PR #14992: URL: https://github.com/apache/kafka/pull/14992#discussion_r1434120692 ## build.gradle: ## @@ -940,6 +940,7 @@ project(':core') { implementation libs.jacksonModuleScala implementation libs.jacksonDataformatCsv

Re: [PR] KAFKA-16040; Rename `Generic` to `Classic` [kafka]

2023-12-21 Thread via GitHub
dajac commented on code in PR #15059: URL: https://github.com/apache/kafka/pull/15059#discussion_r1434128032 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/metrics/GroupCoordinatorMetrics.java: ## @@ -70,11 +70,11 @@ public class GroupCoordinatorMetrics

Re: [PR] KAFKA-16040; Rename `Generic` to `Classic` [kafka]

2023-12-21 Thread via GitHub
dajac commented on code in PR #15059: URL: https://github.com/apache/kafka/pull/15059#discussion_r1434112503 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/metrics/GroupCoordinatorMetrics.java: ## @@ -70,11 +70,11 @@ public class GroupCoordinatorMetrics

Re: [PR] KAFKA-16040; Rename `Generic` to `Classic` [kafka]

2023-12-21 Thread via GitHub
divijvaidya commented on code in PR #15059: URL: https://github.com/apache/kafka/pull/15059#discussion_r1434097877 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/metrics/GroupCoordinatorMetrics.java: ## @@ -70,11 +70,11 @@ public class

Re: [PR] KAFKA-16040; Rename `Generic` to `Classic` [kafka]

2023-12-21 Thread via GitHub
dajac commented on code in PR #15059: URL: https://github.com/apache/kafka/pull/15059#discussion_r1434068651 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/metrics/GroupCoordinatorMetrics.java: ## @@ -319,40 +319,40 @@ public Long value() {

Re: [PR] KAFKA-16040; Rename `Generic` to `Classic` [kafka]

2023-12-21 Thread via GitHub
dajac commented on code in PR #15059: URL: https://github.com/apache/kafka/pull/15059#discussion_r1434063518 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/metrics/GroupCoordinatorMetrics.java: ## @@ -319,40 +319,40 @@ public Long value() {

Re: [PR] KAFKA-16040; Rename `Generic` to `Classic` [kafka]

2023-12-21 Thread via GitHub
divijvaidya commented on code in PR #15059: URL: https://github.com/apache/kafka/pull/15059#discussion_r1434047673 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/metrics/GroupCoordinatorMetrics.java: ## @@ -319,40 +319,40 @@ public Long value() {

Re: [PR] KAFKA-16040; Rename `Generic` to `Classic` [kafka]

2023-12-21 Thread via GitHub
divijvaidya commented on code in PR #15059: URL: https://github.com/apache/kafka/pull/15059#discussion_r1434047673 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/metrics/GroupCoordinatorMetrics.java: ## @@ -319,40 +319,40 @@ public Long value() {

[jira] [Comment Edited] (KAFKA-15388) Handle topics that were having compaction as retention earlier are changed to delete only retention policy and onboarded to tiered storage.

2023-12-21 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17799416#comment-17799416 ] Divij Vaidya edited comment on KAFKA-15388 at 12/21/23 12:56 PM: - Hey

[jira] [Updated] (KAFKA-16040) Rename `Generic` to `Classic`

2023-12-21 Thread David Jacot (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot updated KAFKA-16040: Description: People has raised concerned about using {{Generic}} as a name to designate the old

[jira] [Updated] (KAFKA-15388) Handle topics that were having compaction as retention earlier are changed to delete only retention policy and onboarded to tiered storage.

2023-12-21 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya updated KAFKA-15388: - Priority: Major (was: Blocker) > Handle topics that were having compaction as retention

[jira] [Commented] (KAFKA-15388) Handle topics that were having compaction as retention earlier are changed to delete only retention policy and onboarded to tiered storage.

2023-12-21 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17799416#comment-17799416 ] Divij Vaidya commented on KAFKA-15388: -- Hey [~enether]  I will not consider this as a blocker for

Re: [PR] KAFKA-14888: Added remote log segments retention functionality based on time and size. [kafka]

2023-12-21 Thread via GitHub
divijvaidya commented on code in PR #13561: URL: https://github.com/apache/kafka/pull/13561#discussion_r1434040959 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -761,11 +784,385 @@ public void run() { } } +public void

[PR] KAFKA-16040; Rename `Generic` to `Classic` [kafka]

2023-12-21 Thread via GitHub
dajac opened a new pull request, #15059: URL: https://github.com/apache/kafka/pull/15059 People has raised concerned about using `Generic` as a name to designate the old rebalance protocol. We considered using `Legacy` but discarded it because there are still applications, such as Connect,

[jira] [Comment Edited] (KAFKA-15388) Handle topics that were having compaction as retention earlier are changed to delete only retention policy and onboarded to tiered storage.

2023-12-21 Thread Arpit Goyal (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17799399#comment-17799399 ] Arpit Goyal edited comment on KAFKA-15388 at 12/21/23 12:50 PM:

[jira] [Updated] (KAFKA-13950) Resource leak at multiple places in the code

2023-12-21 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya updated KAFKA-13950: - Fix Version/s: 3.8.0 (was: 3.7.0) > Resource leak at multiple places in

[jira] [Created] (KAFKA-16040) Rename `Generic` to `Classic`

2023-12-21 Thread David Jacot (Jira)
David Jacot created KAFKA-16040: --- Summary: Rename `Generic` to `Classic` Key: KAFKA-16040 URL: https://issues.apache.org/jira/browse/KAFKA-16040 Project: Kafka Issue Type: Sub-task

Re: [PR] KAFKA-13950: Fix resource leak in error scenarios [kafka]

2023-12-21 Thread via GitHub
divijvaidya merged PR #12228: URL: https://github.com/apache/kafka/pull/12228 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Resolved] (KAFKA-16036) Add `group.coordinator.rebalance.protocols` and publish all new configs

2023-12-21 Thread David Jacot (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-16036. - Resolution: Fixed > Add `group.coordinator.rebalance.protocols` and publish all new configs >

Re: [PR] KAFKA-16035: add tests for remoteLogSizeComputationTime/remoteFetchExpiresPerSec metrics [kafka]

2023-12-21 Thread via GitHub
showuon commented on PR #15056: URL: https://github.com/apache/kafka/pull/15056#issuecomment-1866185768 @satishd @kamalcph @clolov , call for review. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] KAFKA-16036; Add `group.coordinator.rebalance.protocols` and publish all new configs [kafka]

2023-12-21 Thread via GitHub
dajac merged PR #15053: URL: https://github.com/apache/kafka/pull/15053 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-16036; Add `group.coordinator.rebalance.protocols` and publish all new configs [kafka]

2023-12-21 Thread via GitHub
dajac commented on PR #15053: URL: https://github.com/apache/kafka/pull/15053#issuecomment-1866183110 It is impossible to get a clean build at the moment. There are OOM errors all over the places, including in trunk. Looking at all the builds, it is clear that this PR does not create

Re: [PR] KAFKA-13950: Fix resource leak in error scenarios [kafka]

2023-12-21 Thread via GitHub
divijvaidya commented on PR #12228: URL: https://github.com/apache/kafka/pull/12228#issuecomment-1866183063 Unrelated test failures since they are known to be flaky as per https://ge.apache.org/scans/tests?search.rootProjectNames=kafka=trunk=Europe%2FBerlin ``` [Build / JDK 17 and

[jira] (KAFKA-15388) Handle topics that were having compaction as retention earlier are changed to delete only retention policy and onboarded to tiered storage.

2023-12-21 Thread Arpit Goyal (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15388 ] Arpit Goyal deleted comment on KAFKA-15388: - was (Author: JIRAUSER301926): [~showuon] [~satish.duggana] [~christo_lolov] Any suggestion how to fetch the higher segment for a particular

[jira] [Updated] (KAFKA-15241) Compute tiered offset by keeping the respective epochs in scope.

2023-12-21 Thread Satish Duggana (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Satish Duggana updated KAFKA-15241: --- Fix Version/s: 3.7.0 > Compute tiered offset by keeping the respective epochs in scope. >

[jira] [Commented] (KAFKA-15388) Handle topics that were having compaction as retention earlier are changed to delete only retention policy and onboarded to tiered storage.

2023-12-21 Thread Arpit Goyal (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17799399#comment-17799399 ] Arpit Goyal commented on KAFKA-15388: - [~enether] [~satish.duggana] [~divijvaidya] [~enether] I am

[PR] test [kafka]

2023-12-21 Thread via GitHub
dajac opened a new pull request, #15058: URL: https://github.com/apache/kafka/pull/15058 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of testing

[jira] [Updated] (KAFKA-16017) Checkpointed offset is incorrect when task is revived and restoring

2023-12-21 Thread Bruno Cadonna (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Cadonna updated KAFKA-16017: -- Fix Version/s: 3.7.0 > Checkpointed offset is incorrect when task is revived and restoring >

[jira] [Updated] (KAFKA-15158) Add metrics for RemoteDeleteRequestsPerSec, RemoteDeleteErrorsPerSec, BuildRemoteLogAuxStateRequestsPerSec, BuildRemoteLogAuxStateErrorsPerSec

2023-12-21 Thread Satish Duggana (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Satish Duggana updated KAFKA-15158: --- Labels: tiered-storage (was: ) > Add metrics for RemoteDeleteRequestsPerSec,

Re: [PR] KAFKA-14888: Added remote log segments retention functionality based on time and size. [kafka]

2023-12-21 Thread via GitHub
iit2009060 commented on code in PR #13561: URL: https://github.com/apache/kafka/pull/13561#discussion_r1433944692 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -761,11 +784,385 @@ public void run() { } } +public void

Re: [PR] KAFKA-14505; [3/N] Wire WriteTxnMarkers API [kafka]

2023-12-21 Thread via GitHub
dajac commented on code in PR #14985: URL: https://github.com/apache/kafka/pull/14985#discussion_r1433900022 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupCoordinator.java: ## @@ -266,6 +267,28 @@ CompletableFuture deleteOffsets(

Re: [PR] KAFKA-14505; [3/N] Wire WriteTxnMarkers API [kafka]

2023-12-21 Thread via GitHub
dajac commented on code in PR #14985: URL: https://github.com/apache/kafka/pull/14985#discussion_r1433892655 ## core/src/main/scala/kafka/server/KafkaApis.scala: ## @@ -2432,7 +2477,13 @@ class KafkaApis(val requestChannel: RequestChannel, origin =

Re: [PR] KAFKA-14505; [3/N] Wire WriteTxnMarkers API [kafka]

2023-12-21 Thread via GitHub
dajac commented on code in PR #14985: URL: https://github.com/apache/kafka/pull/14985#discussion_r1433891089 ## core/src/test/scala/unit/kafka/server/KafkaApisTest.scala: ## @@ -3018,6 +3020,223 @@ class KafkaApisTest { any()) } + @Test Review Comment: Yes.

Re: [PR] KAFKA-14505; [3/N] Wire WriteTxnMarkers API [kafka]

2023-12-21 Thread via GitHub
dajac commented on code in PR #14985: URL: https://github.com/apache/kafka/pull/14985#discussion_r1433890364 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/runtime/InMemoryPartitionWriter.java: ## @@ -102,6 +241,31 @@ public long append( } }

Re: [PR] KAFKA-14505; [3/N] Wire WriteTxnMarkers API [kafka]

2023-12-21 Thread via GitHub
dajac commented on code in PR #14985: URL: https://github.com/apache/kafka/pull/14985#discussion_r1433889262 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/runtime/InMemoryPartitionWriter.java: ## @@ -33,10 +37,141 @@ */ public class

Re: [PR] KAFKA-14505; [3/N] Wire WriteTxnMarkers API [kafka]

2023-12-21 Thread via GitHub
dajac commented on code in PR #14985: URL: https://github.com/apache/kafka/pull/14985#discussion_r1433888169 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/runtime/CoordinatorRuntimeTest.java: ## @@ -1073,6 +1151,314 @@ public CoordinatorShardBuilder

Re: [PR] KAFKA-14505; [3/N] Wire WriteTxnMarkers API [kafka]

2023-12-21 Thread via GitHub
dajac commented on code in PR #14985: URL: https://github.com/apache/kafka/pull/14985#discussion_r1433886514 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/runtime/CoordinatorRuntimeTest.java: ## @@ -1073,6 +1151,314 @@ public CoordinatorShardBuilder

Re: [PR] KAFKA-14505; [3/N] Wire WriteTxnMarkers API [kafka]

2023-12-21 Thread via GitHub
dajac commented on code in PR #14985: URL: https://github.com/apache/kafka/pull/14985#discussion_r1433877427 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/runtime/CoordinatorRuntimeTest.java: ## @@ -1073,6 +1151,314 @@ public CoordinatorShardBuilder

  1   2   >