Re: [PR] Update RemoteLogManager configuration in broker server - KAFKA-16790 [kafka]

2024-05-24 Thread via GitHub
muralibasani commented on PR #16005: URL: https://github.com/apache/kafka/pull/16005#issuecomment-2129528563 @nikramakrishnan I have pushed a few changes. Please review. We explicitly throw a KafkaException now if rlm is not configured and remote storage is enabled -- This is an

Re: [PR] KAFKA-16821: Member Subscription Spec Interface [kafka]

2024-05-24 Thread via GitHub
dajac commented on code in PR #16068: URL: https://github.com/apache/kafka/pull/16068#discussion_r1613460654 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/TargetAssignmentBuilder.java: ## @@ -277,13 +277,22 @@ public TargetAssignmentBuilder

Re: [PR] KAFKA-16833: Fixing PartitionInfo and Cluster equals and hashCode [kafka]

2024-05-24 Thread via GitHub
chia7712 commented on code in PR #16062: URL: https://github.com/apache/kafka/pull/16062#discussion_r1613444662 ## clients/src/main/java/org/apache/kafka/common/PartitionInfo.java: ## @@ -88,6 +90,28 @@ public Node[] offlineReplicas() { return offlineReplicas; }

[PR] KAFKA-16371; fix lingering pending commit when handling OFFSET_METADATA_TOO_LARGE [kafka]

2024-05-24 Thread via GitHub
dajac opened a new pull request, #16072: URL: https://github.com/apache/kafka/pull/16072 This patch was initially created in https://github.com/apache/kafka/pull/15536. When there is a commit for multiple topic partitions and some, but not all, exceed the offset metadata limit, the

[PR] [WIP] KAFKA-15265: Integrate RLMQuotaManager for throttling fetches from remote storage [kafka]

2024-05-24 Thread via GitHub
abhijeetk88 opened a new pull request, #16071: URL: https://github.com/apache/kafka/pull/16071 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of

Re: [PR] KAFKA-16815: Handle FencedInstanceId in HB response [kafka]

2024-05-24 Thread via GitHub
dajac merged PR #16047: URL: https://github.com/apache/kafka/pull/16047 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-16822: Abstract consumer group to share functionality with share group (KIP-932) [kafka]

2024-05-24 Thread via GitHub
AndrewJSchofield commented on code in PR #16054: URL: https://github.com/apache/kafka/pull/16054#discussion_r1613325546 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/AbstractGroup.java: ## @@ -0,0 +1,625 @@ +/* + * Licensed to the Apache Software

Re: [PR] KAFKA-16535: Implement AddVoter, RemoveVoter, UpdateVoter RPCs [kafka]

2024-05-24 Thread via GitHub
AndrewJSchofield commented on PR #16058: URL: https://github.com/apache/kafka/pull/16058#issuecomment-2129297575 LGTM apart from a few test failures related to the new RPCs, such as RequestConvertToJsonTest. -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] KAFKA-15541: Add oldest-iterator-open-since-ms metric [kafka]

2024-05-24 Thread via GitHub
nicktelford commented on code in PR #16041: URL: https://github.com/apache/kafka/pull/16041#discussion_r1613293262 ## streams/src/main/java/org/apache/kafka/streams/state/internals/MeteredKeyValueStore.java: ## @@ -169,6 +172,10 @@ private void registerMetrics() {

Re: [PR] MINOR: mention KAFKA-15905 in docs "Notable changes in 3.7.1" [kafka]

2024-05-24 Thread via GitHub
edoardocomar commented on PR #16070: URL: https://github.com/apache/kafka/pull/16070#issuecomment-2129249297 As discussed in https://github.com/apache/kafka/pull/15910#issuecomment-2127588361 -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] KAFKA-15905 Restarts of MirrorCheckpointTask should not permanently i… [kafka]

2024-05-24 Thread via GitHub
edoardocomar commented on PR #15910: URL: https://github.com/apache/kafka/pull/15910#issuecomment-2129247940 @gharris1727 please see https://github.com/apache/kafka/pull/16070 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[PR] MINOR: mention KAFKA-15905 in docs "Notable changes in 3.7.1" [kafka]

2024-05-24 Thread via GitHub
edoardocomar opened a new pull request, #16070: URL: https://github.com/apache/kafka/pull/16070 ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgrade

Re: [PR] Refactor SSL/SASL admin integration tests to not use a custom authorizer [kafka]

2024-05-24 Thread via GitHub
mimaison merged PR #15377: URL: https://github.com/apache/kafka/pull/15377 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-16373: KIP-1028: Adding code to support Apache Kafka Docker Official Images [kafka]

2024-05-24 Thread via GitHub
omkreddy commented on PR #16027: URL: https://github.com/apache/kafka/pull/16027#issuecomment-2129194429 There is an failure due to missing license: ``` [2024-05-23T07:39:35.270Z] Execution failed for task ':rat'. [2024-05-23T07:39:35.270Z] > A failure occurred while

Re: [PR] Update RemoteLogManager configuration in broker server - KAFKA-16790 [kafka]

2024-05-24 Thread via GitHub
nikramakrishnan commented on PR #16005: URL: https://github.com/apache/kafka/pull/16005#issuecomment-2129187298 > Instead we should throw an exception (or stateChangeLogger.error) like mentioned below here, and write a unit test just for this class Do you mean we mock

Re: [PR] KAFKA-15541: Add oldest-iterator-open-since-ms metric [kafka]

2024-05-24 Thread via GitHub
nicktelford commented on code in PR #16041: URL: https://github.com/apache/kafka/pull/16041#discussion_r1613112730 ## streams/src/main/java/org/apache/kafka/streams/state/internals/MeteredKeyValueStore.java: ## @@ -169,6 +172,10 @@ private void registerMetrics() {

Re: [PR] KAFKA-15541: Add oldest-iterator-open-since-ms metric [kafka]

2024-05-24 Thread via GitHub
nicktelford commented on code in PR #16041: URL: https://github.com/apache/kafka/pull/16041#discussion_r1613112730 ## streams/src/main/java/org/apache/kafka/streams/state/internals/MeteredKeyValueStore.java: ## @@ -169,6 +172,10 @@ private void registerMetrics() {

Re: [PR] KAFKA-15541: Add oldest-iterator-open-since-ms metric [kafka]

2024-05-24 Thread via GitHub
nicktelford commented on code in PR #16041: URL: https://github.com/apache/kafka/pull/16041#discussion_r1613112730 ## streams/src/main/java/org/apache/kafka/streams/state/internals/MeteredKeyValueStore.java: ## @@ -169,6 +172,10 @@ private void registerMetrics() {

Re: [PR] KAFKA-15541: Add oldest-iterator-open-since-ms metric [kafka]

2024-05-24 Thread via GitHub
nicktelford commented on code in PR #16041: URL: https://github.com/apache/kafka/pull/16041#discussion_r1613112730 ## streams/src/main/java/org/apache/kafka/streams/state/internals/MeteredKeyValueStore.java: ## @@ -169,6 +172,10 @@ private void registerMetrics() {

[PR] Adding standalone parameter to kafka-storage.sh [KAFKA-16518] [kafka]

2024-05-24 Thread via GitHub
muralibasani opened a new pull request, #16069: URL: https://github.com/apache/kafka/pull/16069 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* -

Re: [PR] KAFKA-16832: LeaveGroup API for upgrading ConsumerGroup [kafka]

2024-05-24 Thread via GitHub
dajac commented on code in PR #16057: URL: https://github.com/apache/kafka/pull/16057#discussion_r1613089003 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -2041,6 +2027,36 @@ private CoordinatorResult

Re: [PR] KAFKA-16639: AsyncKafkaConsumer#close dose not send heartbeat to leave group. [kafka]

2024-05-24 Thread via GitHub
frankvicky commented on PR #16017: URL: https://github.com/apache/kafka/pull/16017#issuecomment-2129009977 Hi @lianetm, I really appreciate your suggestion. These suggestions are very helpful to me. I will give them a try.  -- This is an automated message from the Apache Git Service.

Re: [PR] KAFKA-15541: Add oldest-iterator-open-since-ms metric [kafka]

2024-05-24 Thread via GitHub
nicktelford commented on code in PR #16041: URL: https://github.com/apache/kafka/pull/16041#discussion_r1613112730 ## streams/src/main/java/org/apache/kafka/streams/state/internals/MeteredKeyValueStore.java: ## @@ -169,6 +172,10 @@ private void registerMetrics() {

Re: [PR] KAFKA-15541: Add oldest-iterator-open-since-ms metric [kafka]

2024-05-24 Thread via GitHub
nicktelford commented on code in PR #16041: URL: https://github.com/apache/kafka/pull/16041#discussion_r1613112730 ## streams/src/main/java/org/apache/kafka/streams/state/internals/MeteredKeyValueStore.java: ## @@ -169,6 +172,10 @@ private void registerMetrics() {

Re: [PR] KAFKA-15045: (KIP-924 pt. 9) TaskAssignmentUtils implementation of optimizeRackAwareActiveTasks [kafka]

2024-05-24 Thread via GitHub
ableegoldman commented on code in PR #16033: URL: https://github.com/apache/kafka/pull/16033#discussion_r1613111098 ## streams/src/main/java/org/apache/kafka/streams/processor/assignment/TaskAssignmentUtils.java: ## @@ -74,20 +209,240 @@ public static Map

Re: [PR] KAFKA-15045: (KIP-924 pt. 11) Implemented StickyTaskAssignor [kafka]

2024-05-24 Thread via GitHub
ableegoldman commented on code in PR #16052: URL: https://github.com/apache/kafka/pull/16052#discussion_r1613071356 ## streams/src/main/java/org/apache/kafka/streams/processor/assignment/assignors/StickyTaskAssignor.java: ## @@ -0,0 +1,478 @@ +/* + * Licensed to the Apache

Re: [PR] KAFKA-16713: Define initial set of RPCs for KIP-932 [kafka]

2024-05-24 Thread via GitHub
AndrewJSchofield commented on PR #16022: URL: https://github.com/apache/kafka/pull/16022#issuecomment-2128913832 There are 2 test failures introduced by this PR. I'll get them fixed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] KAFKA-15045: (KIP-924 pt. 11) Implemented StickyTaskAssignor [kafka]

2024-05-24 Thread via GitHub
ableegoldman commented on code in PR #16052: URL: https://github.com/apache/kafka/pull/16052#discussion_r1613071356 ## streams/src/main/java/org/apache/kafka/streams/processor/assignment/assignors/StickyTaskAssignor.java: ## @@ -0,0 +1,478 @@ +/* + * Licensed to the Apache

[PR] KAFKA:16821 - Member subscription spec interface [kafka]

2024-05-24 Thread via GitHub
rreddy-22 opened a new pull request, #16068: URL: https://github.com/apache/kafka/pull/16068 Create a new interface called MemberSubscriptionSpec to hold the members subscription information. The assignment member spec is replaced by the MemberSubscriptionSpecImpl class. Eventually

Re: [PR] KAFKA-16831: CoordinatorRuntime should initialize MemoryRecordsBuilder with max batch size write limit [kafka]

2024-05-24 Thread via GitHub
dajac commented on code in PR #16059: URL: https://github.com/apache/kafka/pull/16059#discussion_r1613028497 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/runtime/CoordinatorRuntime.java: ## @@ -1365,6 +1368,11 @@ public void onHighWatermarkUpdated(

Re: [PR] KAFKA-15265: Add Remote Log Manager quota manager [kafka]

2024-05-24 Thread via GitHub
abhijeetk88 commented on code in PR #15625: URL: https://github.com/apache/kafka/pull/15625#discussion_r1613033249 ## storage/src/main/java/org/apache/kafka/server/log/remote/storage/RemoteLogManagerConfig.java: ## @@ -143,6 +143,38 @@ public final class RemoteLogManagerConfig

Re: [PR] KAFKA-16626: Lazily convert subscribed topic names to topic ids [kafka]

2024-05-24 Thread via GitHub
dajac merged PR #15970: URL: https://github.com/apache/kafka/pull/15970 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-16626: Lazily convert subscribed topic names to topic ids [kafka]

2024-05-24 Thread via GitHub
dajac commented on PR #15970: URL: https://github.com/apache/kafka/pull/15970#issuecomment-2128833459 The failed tests are related to https://github.com/apache/kafka/pull/15972. Merging to trunk. -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] KAFKA-16822: Abstract consumer group to share functionality with share group (KIP-932) [kafka]

2024-05-24 Thread via GitHub
dajac commented on code in PR #16054: URL: https://github.com/apache/kafka/pull/16054#discussion_r1613013960 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/AbstractGroup.java: ## @@ -0,0 +1,625 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] MINOR: migrate ListConsumerGroupTest to use ClusterTestExtensions [kafka]

2024-05-24 Thread via GitHub
chia7712 commented on code in PR #15821: URL: https://github.com/apache/kafka/pull/15821#discussion_r1612965039 ## tools/src/test/java/org/apache/kafka/tools/consumer/group/ConsumerGroupCommandTestUtils.java: ## @@ -68,15 +68,31 @@ static List generator() {

Re: [PR] [MINOR] Code Cleanup (Clients Module) [kafka]

2024-05-24 Thread via GitHub
chia7712 commented on PR #16049: URL: https://github.com/apache/kafka/pull/16049#issuecomment-2128711390 > I rebased the PR! I guess other PRs will encounter same failed tests, so please rebase code for others too, thanks! -- This is an automated message from the Apache Git

Re: [PR] [MINOR] Code Cleanup (Clients Module) [kafka]

2024-05-24 Thread via GitHub
sjhajharia commented on PR #16049: URL: https://github.com/apache/kafka/pull/16049#issuecomment-2128709392 Thanks for pointing out @chia7712 I rebased the PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] [MINOR] Code Cleanup (Clients Module) [kafka]

2024-05-24 Thread via GitHub
chia7712 commented on PR #16049: URL: https://github.com/apache/kafka/pull/16049#issuecomment-2128703246 @sjhajharia Could you please rebase code to include #16044? It fixes a lot of failed tests :) -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] KAFKA-16826: Integrate Native Docker Image with github actions [kafka]

2024-05-24 Thread via GitHub
omkreddy merged PR #16045: URL: https://github.com/apache/kafka/pull/16045 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-16826: Integrate Native Docker Image with github actions [kafka]

2024-05-24 Thread via GitHub
kagarwal06 commented on code in PR #16045: URL: https://github.com/apache/kafka/pull/16045#discussion_r1612932172 ## .github/workflows/docker_promote.yml: ## @@ -19,10 +19,10 @@ on: workflow_dispatch: inputs: rc_docker_image: -description: RC docker image

Re: [PR] KAFKA-15265: Add Remote Log Manager quota manager [kafka]

2024-05-24 Thread via GitHub
abhijeetk88 commented on code in PR #15625: URL: https://github.com/apache/kafka/pull/15625#discussion_r1612931258 ## core/src/main/java/kafka/log/remote/quota/RLMQuotaManager.java: ## @@ -0,0 +1,117 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more

Re: [PR] [MINOR] Code Cleanup [kafka]

2024-05-24 Thread via GitHub
sjhajharia commented on PR #16021: URL: https://github.com/apache/kafka/pull/16021#issuecomment-2128693543 Hey @chia7712 I have created 5 sub-PRs to address the cleanup module-by-module. - https://github.com/apache/kafka/pull/16049: Clients Module -

[PR] [MINOR] : Code Cleanup - Misc modules [kafka]

2024-05-24 Thread via GitHub
sjhajharia opened a new pull request, #16067: URL: https://github.com/apache/kafka/pull/16067 ## What Code Cleanup in Misc Modules ## Changes Some common changes include - Replace the Arrays.asList() with Collections.singletonList() wherever possible - Cleaning up

Re: [PR] KAFKA-16764: New consumer should throw InvalidTopicException on poll when invalid topic in metadata [kafka]

2024-05-24 Thread via GitHub
appchemist commented on PR #16043: URL: https://github.com/apache/kafka/pull/16043#issuecomment-2128669455 > i really want to keep the network client delegate to just interfacing with the network client, so i wonder if it would be a better design to handle the metadata error in a separated

Re: [PR] KAFKA-16764: New consumer should throw InvalidTopicException on poll when invalid topic in metadata [kafka]

2024-05-24 Thread via GitHub
appchemist commented on PR #16043: URL: https://github.com/apache/kafka/pull/16043#issuecomment-2128663846 Hi @philipnee, Thank you for review! > do you think we can put the metadata error in the ConsumerNetworkThread and have an exception handler to relay the error back to the user

Re: [PR] KAFKA-16574: The metrics of LogCleaner disappear after reconfiguration [kafka]

2024-05-24 Thread via GitHub
chia7712 commented on PR #15863: URL: https://github.com/apache/kafka/pull/15863#issuecomment-2128658669 @gaurav-narula this PR adopt your solution (https://github.com/apache/kafka/pull/15863#discussion_r1590313031) now, so it would be great to have your reviews before merging. thanks --

Re: [PR] KAFKA-16574: The metrics of LogCleaner disappear after reconfiguration [kafka]

2024-05-24 Thread via GitHub
chia7712 commented on PR #15863: URL: https://github.com/apache/kafka/pull/15863#issuecomment-2128654632 @chiacyu Could you please rebase PR to run QA with newest code? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] KAFKA-16771 First log directory printed twice when formatting storage [kafka]

2024-05-24 Thread via GitHub
chia7712 commented on code in PR #16010: URL: https://github.com/apache/kafka/pull/16010#discussion_r1612892755 ## core/src/test/scala/unit/kafka/tools/StorageToolTest.scala: ## @@ -206,12 +206,20 @@ Found problem: @Test def

Re: [PR] KAFKA-16308 [1/N]: Create FeatureVersion interface and add `--feature` flag and handling to StorageTool [kafka]

2024-05-24 Thread via GitHub
artemlivshits commented on code in PR #15685: URL: https://github.com/apache/kafka/pull/15685#discussion_r1612863802 ## server-common/src/main/java/org/apache/kafka/server/common/TestFeatureVersion.java: ## @@ -21,16 +21,17 @@ public enum TestFeatureVersion implements

[PR] [MINOR] Code Cleanup - Connect Module [kafka]

2024-05-24 Thread via GitHub
sjhajharia opened a new pull request, #16066: URL: https://github.com/apache/kafka/pull/16066 ## What Code Cleanup in Connect Module ## Changes Some common changes include - Replace the Arrays.asList() with Collections.singletonList() wherever possible - Cleaning up

[PR] [MINOR]: Code Cleanup - Metadata module [kafka]

2024-05-23 Thread via GitHub
sjhajharia opened a new pull request, #16065: URL: https://github.com/apache/kafka/pull/16065 ## What Code Cleanup in metadata Module ## Changes Some common changes include - Replace the Arrays.asList() with Collections.singletonList() wherever possible - Cleaning up

Re: [PR] KAFKA-15045: (KIP-924 pt. 11) Implemented StickyTaskAssignor [kafka]

2024-05-23 Thread via GitHub
ableegoldman commented on code in PR #16052: URL: https://github.com/apache/kafka/pull/16052#discussion_r1612714758 ## streams/src/main/java/org/apache/kafka/streams/processor/assignment/assignors/StickyTaskAssignor.java: ## @@ -0,0 +1,478 @@ +/* + * Licensed to the Apache

Re: [PR] KAFKA-16796: Introduce new org.apache.kafka.tools.api.Decoder to replace kafka.serializer.Decoder [kafka]

2024-05-23 Thread via GitHub
chia7712 commented on code in PR #16064: URL: https://github.com/apache/kafka/pull/16064#discussion_r1612751256 ## core/src/main/scala/kafka/tools/DumpLogSegments.scala: ## @@ -651,4 +651,17 @@ object DumpLogSegments { def checkArgs(): Unit =

[PR] KAFKA-16796: Introduce new org.apache.kafka.tools.api.Decoder to replace kafka.serializer.Decoder [kafka]

2024-05-23 Thread via GitHub
FrankYang0529 opened a new pull request, #16064: URL: https://github.com/apache/kafka/pull/16064 We need a replacement in order to complete https://issues.apache.org/jira/browse/KAFKA-14579 in kafak 4.0 ### Committer Checklist (excluded from commit message) - [ ] Verify design and

[PR] KAFKA-16834: add the reason for the failure of PartitionRegistration#toRecord [kafka]

2024-05-23 Thread via GitHub
funky-eyes opened a new pull request, #16063: URL: https://github.com/apache/kafka/pull/16063 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of

Re: [PR] KAFKA-16583: fix PartitionRegistration#toRecord directory check under metadata version 3_7_IV2 [kafka]

2024-05-23 Thread via GitHub
showuon commented on PR #15751: URL: https://github.com/apache/kafka/pull/15751#issuecomment-2128463581 Thanks for fixing this issue. I'll take a look this week or next week. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] ConfigRegistry [kafka]

2024-05-23 Thread via GitHub
github-actions[bot] commented on PR #15428: URL: https://github.com/apache/kafka/pull/15428#issuecomment-2128459494 This PR is being marked as stale since it has not had any activity in 90 days. If you would like to keep this PR alive, please ask a committer for review. If the PR has

[PR] KAFKA-16833: Fixing PartitionInfo and Cluster equals and hashCode [kafka]

2024-05-23 Thread via GitHub
ahuang98 opened a new pull request, #16062: URL: https://github.com/apache/kafka/pull/16062 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of

Re: [PR] KAFKA-16691: [connect:transform] Support nested field paths on TimestampConverter [kafka]

2024-05-23 Thread via GitHub
jeqo commented on code in PR #15893: URL: https://github.com/apache/kafka/pull/15893#discussion_r1612459168 ## connect/transforms/src/main/java/org/apache/kafka/connect/transforms/field/SingleFieldPath.java: ## @@ -202,6 +204,225 @@ public Object valueFrom(Map map) {

Re: [PR] KAFKA-16691: [connect:transform] Support nested field paths on TimestampConverter [kafka]

2024-05-23 Thread via GitHub
jeqo commented on code in PR #15893: URL: https://github.com/apache/kafka/pull/15893#discussion_r1612458345 ## connect/transforms/src/main/java/org/apache/kafka/connect/transforms/field/SingleFieldPath.java: ## @@ -202,6 +204,225 @@ public Object valueFrom(Map map) {

[PR] return to commit f9db4fa19cce975a6bbaeb09fbe9c91b81846b5a [kafka]

2024-05-23 Thread via GitHub
cmccabe opened a new pull request, #16061: URL: https://github.com/apache/kafka/pull/16061 commit db118aba6fbddfc607c6ef653f9965a5a73c778a (HEAD -> return-to-monkey, cmccabe/return-to-monkey, trunk) Author: Colin P. McCabe Date: Thu May 23 16:11:24 2024 -0700 return to

Re: [PR] MINOR: Disable JDK 11 and 17 tests on PRs [kafka]

2024-05-23 Thread via GitHub
gharris1727 commented on PR #16051: URL: https://github.com/apache/kafka/pull/16051#issuecomment-2128174644 > > Something is wrong with the build of this commit > > There still seems to be some issue with builds 樂 > [2024-05-23T21:45:16.078Z] Could not stop

Re: [PR] KAFKA-16541 Fix potential leader-epoch checkpoint file corruption [kafka]

2024-05-23 Thread via GitHub
junrao commented on code in PR #15993: URL: https://github.com/apache/kafka/pull/15993#discussion_r1612451041 ## storage/src/main/java/org/apache/kafka/storage/internals/epoch/LeaderEpochFileCache.java: ## @@ -55,16 +61,40 @@ public class LeaderEpochFileCache { /** *

Re: [PR] MINOR: Disable JDK 11 and 17 tests on PRs [kafka]

2024-05-23 Thread via GitHub
jolshan commented on PR #16051: URL: https://github.com/apache/kafka/pull/16051#issuecomment-2128160293 Any ideas about ``` [2024-05-23T21:45:16.078Z] > Task :connect:runtime:test [2024-05-23T21:45:16.078Z] Could not stop

Re: [PR] MINOR: Disable JDK 11 and 17 tests on PRs [kafka]

2024-05-23 Thread via GitHub
jolshan commented on PR #16051: URL: https://github.com/apache/kafka/pull/16051#issuecomment-2128158835 > Something is wrong with the build of this commit There still seems to be some issue with builds 樂 -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] KAFKA-16308 [1/N]: Create FeatureVersion interface and add `--feature` flag and handling to StorageTool [kafka]

2024-05-23 Thread via GitHub
jolshan commented on code in PR #15685: URL: https://github.com/apache/kafka/pull/15685#discussion_r1612404062 ## core/src/main/scala/kafka/tools/StorageTool.scala: ## @@ -109,6 +111,52 @@ object StorageTool extends Logging { } } + private def

Re: [PR] KAFKA-15045: (KIP-924 pt. 11) Implemented StickyTaskAssignor [kafka]

2024-05-23 Thread via GitHub
ableegoldman commented on code in PR #16052: URL: https://github.com/apache/kafka/pull/16052#discussion_r1612405084 ## streams/src/main/java/org/apache/kafka/streams/processor/assignment/assignors/StickyTaskAssignor.java: ## @@ -0,0 +1,478 @@ +/* + * Licensed to the Apache

Re: [PR] KAFKA-15045: (KIP-924 pt. 11) Implemented StickyTaskAssignor [kafka]

2024-05-23 Thread via GitHub
ableegoldman commented on code in PR #16052: URL: https://github.com/apache/kafka/pull/16052#discussion_r1612403956 ## streams/src/main/java/org/apache/kafka/streams/processor/assignment/assignors/StickyTaskAssignor.java: ## @@ -0,0 +1,478 @@ +/* + * Licensed to the Apache

Re: [PR] KAFKA-15541: Add oldest-iterator-open-since-ms metric [kafka]

2024-05-23 Thread via GitHub
mjsax commented on code in PR #16041: URL: https://github.com/apache/kafka/pull/16041#discussion_r1612395523 ## streams/src/main/java/org/apache/kafka/streams/state/internals/metrics/StateStoreMetrics.java: ## @@ -451,6 +455,23 @@ public static void

Re: [PR] KAFKA-15045: Fix RackAwareTaskAssignorTest [kafka]

2024-05-23 Thread via GitHub
apourchet closed pull request #16056: KAFKA-15045: Fix RackAwareTaskAssignorTest URL: https://github.com/apache/kafka/pull/16056 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] MINOR: Disable JDK 11 and 17 tests on PRs [kafka]

2024-05-23 Thread via GitHub
gharris1727 commented on code in PR #16051: URL: https://github.com/apache/kafka/pull/16051#discussion_r1612358604 ## Jenkinsfile: ## @@ -36,6 +36,12 @@ def doTest(env, target = "test") { junit '**/build/test-results/**/TEST-*.xml' } +def runTestOnDevBranch(env) { + if

Re: [PR] MINOR: Disable JDK 11 and 17 tests on PRs [kafka]

2024-05-23 Thread via GitHub
gharris1727 commented on PR #16051: URL: https://github.com/apache/kafka/pull/16051#issuecomment-2128068313 Looking at the latest CI run, the 11 build took ~15 minutes, the 17 build took 11 minutes, and the 8 and 21 builds are continuing as normal. In the last 28 days, Kafka has run:

Re: [PR] MINOR: Disable JDK 11 and 17 tests on PRs [kafka]

2024-05-23 Thread via GitHub
ijuma commented on code in PR #16051: URL: https://github.com/apache/kafka/pull/16051#discussion_r1612342902 ## Jenkinsfile: ## @@ -36,6 +36,12 @@ def doTest(env, target = "test") { junit '**/build/test-results/**/TEST-*.xml' } +def runTestOnDevBranch(env) { + if

Re: [PR] MINOR: Disable JDK 11 and 17 tests on PRs [kafka]

2024-05-23 Thread via GitHub
ijuma commented on code in PR #16051: URL: https://github.com/apache/kafka/pull/16051#discussion_r1612341398 ## Jenkinsfile: ## @@ -36,6 +36,12 @@ def doTest(env, target = "test") { junit '**/build/test-results/**/TEST-*.xml' } +def runTestOnDevBranch(env) { + if

Re: [PR] MINOR: Disable JDK 11 and 17 tests on PRs [kafka]

2024-05-23 Thread via GitHub
gharris1727 commented on code in PR #16051: URL: https://github.com/apache/kafka/pull/16051#discussion_r1612327285 ## Jenkinsfile: ## @@ -36,6 +36,12 @@ def doTest(env, target = "test") { junit '**/build/test-results/**/TEST-*.xml' } +def runTestOnDevBranch(env) { + if

Re: [PR] KAFKA-15045: Fix RackAwareTaskAssignorTest [kafka]

2024-05-23 Thread via GitHub
chia7712 commented on PR #16056: URL: https://github.com/apache/kafka/pull/16056#issuecomment-2128047643 @apourchet Thanks for your contribution. The failed test is already fixed by #16044 -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] KAFKA-16828: RackAwareTaskAssignorTest failed [kafka]

2024-05-23 Thread via GitHub
chia7712 merged PR #16044: URL: https://github.com/apache/kafka/pull/16044 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-16308 [1/N]: Create FeatureVersion interface and add `--feature` flag and handling to StorageTool [kafka]

2024-05-23 Thread via GitHub
jolshan commented on code in PR #15685: URL: https://github.com/apache/kafka/pull/15685#discussion_r1612321070 ## core/src/main/scala/kafka/tools/StorageTool.scala: ## @@ -109,6 +111,52 @@ object StorageTool extends Logging { } } + private def

Re: [PR] KAFKA-16308 [1/N]: Create FeatureVersion interface and add `--feature` flag and handling to StorageTool [kafka]

2024-05-23 Thread via GitHub
jolshan commented on code in PR #15685: URL: https://github.com/apache/kafka/pull/15685#discussion_r1612319940 ## core/src/main/scala/kafka/tools/StorageTool.scala: ## @@ -109,6 +111,52 @@ object StorageTool extends Logging { } } + private def

Re: [PR] KAFKA-16308 [1/N]: Create FeatureVersion interface and add `--feature` flag and handling to StorageTool [kafka]

2024-05-23 Thread via GitHub
jolshan commented on code in PR #15685: URL: https://github.com/apache/kafka/pull/15685#discussion_r1612319940 ## core/src/main/scala/kafka/tools/StorageTool.scala: ## @@ -109,6 +111,52 @@ object StorageTool extends Logging { } } + private def

Re: [PR] MINOR: Disable JDK 11 and 17 tests on PRs [kafka]

2024-05-23 Thread via GitHub
gharris1727 commented on PR #16051: URL: https://github.com/apache/kafka/pull/16051#issuecomment-2128037835 > It's rare that there is an issue with only java 11/17 and not 8 or 21. I've most frequently seen issues with scala 2.12 and that's on the java 8 build. I agree, and that

Re: [PR] KAFKA-16308 [1/N]: Create FeatureVersion interface and add `--feature` flag and handling to StorageTool [kafka]

2024-05-23 Thread via GitHub
jolshan commented on code in PR #15685: URL: https://github.com/apache/kafka/pull/15685#discussion_r1612314478 ## core/src/main/scala/kafka/tools/StorageTool.scala: ## @@ -141,6 +189,9 @@ object StorageTool extends Logging { formatParser.addArgument("--release-version",

Re: [PR] KAFKA-16308 [1/N]: Create FeatureVersion interface and add `--feature` flag and handling to StorageTool [kafka]

2024-05-23 Thread via GitHub
jolshan commented on code in PR #15685: URL: https://github.com/apache/kafka/pull/15685#discussion_r1612314269 ## core/src/main/scala/kafka/tools/StorageTool.scala: ## @@ -109,6 +105,51 @@ object StorageTool extends Logging { } } + private def

[PR] MINOR: log coordinator event errors [kafka]

2024-05-23 Thread via GitHub
jeffkbkim opened a new pull request, #16060: URL: https://github.com/apache/kafka/pull/16060 We don't log any error logs when a coordinator event fails to execute to completion. This patch adds some logging ### Committer Checklist (excluded from commit message) - [ ] Verify design

Re: [PR] KAFKA-16308 [1/N]: Create FeatureVersion interface and add `--feature` flag and handling to StorageTool [kafka]

2024-05-23 Thread via GitHub
jolshan commented on code in PR #15685: URL: https://github.com/apache/kafka/pull/15685#discussion_r1612314113 ## core/src/main/scala/kafka/tools/StorageTool.scala: ## @@ -156,16 +200,27 @@ object StorageTool extends Logging { def getMetadataVersion( namespace:

Re: [PR] KAFKA-16308 [1/N]: Create FeatureVersion interface and add `--feature` flag and handling to StorageTool [kafka]

2024-05-23 Thread via GitHub
jolshan commented on code in PR #15685: URL: https://github.com/apache/kafka/pull/15685#discussion_r1612313415 ## core/src/main/scala/kafka/server/BrokerFeatures.scala: ## @@ -75,16 +76,19 @@ object BrokerFeatures extends Logging { } def

Re: [PR] MINOR: Disable JDK 11 and 17 tests on PRs [kafka]

2024-05-23 Thread via GitHub
jolshan commented on PR #16051: URL: https://github.com/apache/kafka/pull/16051#issuecomment-2128028213 > I think it would be worth running the compile steps for these older versions (i.e., "doValidation"). As far as I know, those steps only fail if there's actually a problem. It's

Re: [PR] KAFKA-16308 [1/N]: Create FeatureVersion interface and add `--feature` flag and handling to StorageTool [kafka]

2024-05-23 Thread via GitHub
junrao commented on code in PR #15685: URL: https://github.com/apache/kafka/pull/15685#discussion_r1602305075 ## server-common/src/main/java/org/apache/kafka/server/common/TestFeatureVersion.java: ## @@ -0,0 +1,65 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] MINOR: Disable JDK 11 and 17 tests on PRs [kafka]

2024-05-23 Thread via GitHub
ijuma commented on code in PR #16051: URL: https://github.com/apache/kafka/pull/16051#discussion_r1612302312 ## Jenkinsfile: ## @@ -36,6 +36,12 @@ def doTest(env, target = "test") { junit '**/build/test-results/**/TEST-*.xml' } +def runTestOnDevBranch(env) { + if

Re: [PR] KAFKA-16832: LeaveGroup API for upgrading ConsumerGroup [kafka]

2024-05-23 Thread via GitHub
dongnuo123 commented on code in PR #16057: URL: https://github.com/apache/kafka/pull/16057#discussion_r1612296899 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -4424,14 +4441,113 @@ private ConsumerGroupMember

[PR] KAFKA-16831: CoordinatorRuntime should initialize MemoryRecordsBuilder with max batch size write limit [kafka]

2024-05-23 Thread via GitHub
jeffkbkim opened a new pull request, #16059: URL: https://github.com/apache/kafka/pull/16059 Otherwise, we default the write limit to the min buffer size of 16384 for the write limit. This causes the coordinator to threw RecordTooLargeException even when it's under the 1MB max batch size

[PR] KAFKA-16832: LeaveGroup API for upgrading ConsumerGroup [kafka]

2024-05-23 Thread via GitHub
dongnuo123 opened a new pull request, #16057: URL: https://github.com/apache/kafka/pull/16057 This patch implements the LeaveGroup api to the consumer groups that are in the mixed mode. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation

Re: [PR] Update dependencies [kafka]

2024-05-23 Thread via GitHub
highbolder closed pull request #16039: Update dependencies URL: https://github.com/apache/kafka/pull/16039 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

[PR] KAFKA-15045: Fix RackAwareTaskAssignorTest [kafka]

2024-05-23 Thread via GitHub
apourchet opened a new pull request, #16056: URL: https://github.com/apache/kafka/pull/16056 Some unnecessary changes to RackAwareTaskAssignor broke the tests, this PR reverts those changes. -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] MINOR: Refactor Values class to fix checkstyle, add benchmark, optimize exceptions [kafka]

2024-05-23 Thread via GitHub
gharris1727 merged PR #15469: URL: https://github.com/apache/kafka/pull/15469 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] MINOR: Refactor Values class to fix checkstyle, add benchmark, optimize exceptions [kafka]

2024-05-23 Thread via GitHub
gharris1727 commented on PR #15469: URL: https://github.com/apache/kafka/pull/15469#issuecomment-2127953424 Here's the final performance changes: Benchmark | Before | Before Error | After | After Error | Speedup -- | -- | -- | -- | -- | -- ValuesBenchmark.testConvertToBoolean |

Re: [PR] Revert "KAFKA-15045: (KIP-924 pt. 10) Topic partition rack annotation simplified (#16034)" [kafka]

2024-05-23 Thread via GitHub
ableegoldman closed pull request #16055: Revert "KAFKA-15045: (KIP-924 pt. 10) Topic partition rack annotation simplified (#16034)" URL: https://github.com/apache/kafka/pull/16055 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] Revert "KAFKA-15045: (KIP-924 pt. 10) Topic partition rack annotation simplified (#16034)" [kafka]

2024-05-23 Thread via GitHub
ableegoldman commented on PR #16055: URL: https://github.com/apache/kafka/pull/16055#issuecomment-2127922285 It's actually https://github.com/apache/kafka/pull/15972 that broke this test, closing this -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] KAFKA-15045: (KIP-924 pt. 5) Add rack information to ApplicationState [kafka]

2024-05-23 Thread via GitHub
ableegoldman commented on PR #15972: URL: https://github.com/apache/kafka/pull/15972#issuecomment-2127918475 Yep, just noticed this. Sorry about that. We're taking a look -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] MINOR: Rewrite OptimizedUniformAssignmentBuilder#assignStickyPartitions to improve performance [kafka]

2024-05-23 Thread via GitHub
dajac commented on PR #15883: URL: https://github.com/apache/kafka/pull/15883#issuecomment-2127912971 We will use a different approach. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

  1   2   3   4   5   6   7   8   9   10   >