Re: [PR] KAFKA-15830: Add telemetry API handling (KIP-714) [kafka]

apoorvmittal10 commented on PR #14767: URL: https://github.com/apache/kafka/pull/14767#issuecomment-1837411475 > @apoorvmittal10 : Are the 59 test failures related, especially those ApiVersionsRequestTest? Hi @junrao, I should have realized that integration test in `ApiVersionsReques

Re: [PR] KAFKA-15663, KAFKA-15794: Telemetry reporter and request handling (KIP-714) [kafka]

apoorvmittal10 closed pull request #14724: KAFKA-15663, KAFKA-15794: Telemetry reporter and request handling (KIP-714) URL: https://github.com/apache/kafka/pull/14724 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

Re: [PR] KAFKA-15901: Client changes for registering telemetry and API calls (KIP-714) [kafka]

apoorvmittal10 commented on code in PR #14843: URL: https://github.com/apache/kafka/pull/14843#discussion_r1413029293 ## clients/src/main/java/org/apache/kafka/clients/NetworkClient.java: ## @@ -1222,6 +1249,95 @@ private InProgressData(int requestVersion, boolean isPartialUpda

Re: [PR] KAFKA-15901: Client changes for registering telemetry and API calls (KIP-714) [kafka]

apoorvmittal10 commented on code in PR #14843: URL: https://github.com/apache/kafka/pull/14843#discussion_r1413029723 ## clients/src/main/java/org/apache/kafka/clients/admin/KafkaAdminClient.java: ## @@ -4387,7 +4398,52 @@ public FenceProducersResult fenceProducers(Collection t

Re: [PR] KAFKA-15901: Client changes for registering telemetry and API calls (KIP-714) [kafka]

apoorvmittal10 commented on PR #14843: URL: https://github.com/apache/kafka/pull/14843#issuecomment-1837415101 > Left a couple of comments. Thanks for the PR. Thanks @philipnee for reviewing and your time, I have replied on the comments. Seems the classes follows the pattern to name `

Re: [PR] KAFKA-15663, KAFKA-15794: Telemetry reporter and request handling (KIP-714) [kafka]

apoorvmittal10 commented on PR #14724: URL: https://github.com/apache/kafka/pull/14724#issuecomment-1837416416 I have triggered build multiple times, and it fails un-deterministically to checkout on certain java versions, more recently on JDK 21. Strange to see checkout to run build on CI i

Re: [PR] KAFKA-14516: [2/N] New Group Coordinator IT for Static Member Replacement [kafka]

vamossagar12 commented on PR #14882: URL: https://github.com/apache/kafka/pull/14882#issuecomment-1837429558 Looks like the newly added test `testRejoiningStaticMemberGetsAssignmentsBackWhenNewGroupCoordinatorIsEnabled` did not fail in the latest run. -- This is an automated message from

[jira] [Assigned] (KAFKA-15950) CommunicationEvent should be scheduled with EarliestDeadlineFunction

[ https://issues.apache.org/jira/browse/KAFKA-15950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Soarez reassigned KAFKA-15950: --- Assignee: Igor Soarez > CommunicationEvent should be scheduled with EarliestDeadlineFunctio

[PR] KAFKA-15950: Serialize broker heartbeat requests [kafka]

soarez opened a new pull request, #14903: URL: https://github.com/apache/kafka/pull/14903 In between HeartbeatRequest being sent and the response being handled, i.e. while a HeartbeatRequest is in flight, an extra request may be immediately scheduled if propagateDirectoryFailure, setReadyTo

Re: [PR] MINOR: Comment explaining test approach [kafka]

soarez closed pull request #14874: MINOR: Comment explaining test approach URL: https://github.com/apache/kafka/pull/14874 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

Re: [PR] MINOR: Comment explaining test approach [kafka]

soarez commented on PR #14874: URL: https://github.com/apache/kafka/pull/14874#issuecomment-1837446933 Closing in favor of #14903 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

Re: [PR] KAFKA-15950: Serialize broker heartbeat requests [kafka]

soarez commented on PR #14903: URL: https://github.com/apache/kafka/pull/14903#issuecomment-1837447759 @junrao: please take a look -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [PR] KAFKA-15901: Client changes for registering telemetry and API calls (KIP-714) [kafka]

Lalolokos1 commented on PR #14843: URL: https://github.com/apache/kafka/pull/14843#issuecomment-1837447878 ***@***.*** On Sun, Dec 3, 2023, 1:01 AM Apoorv Mittal ***@***.***> wrote: > Left a couple of comments. Thanks for the PR. > > Thanks @philipnee

[PR] Experimentally turn off consumer integration tests using new consumer [kafka]

AndrewJSchofield opened a new pull request, #14904: URL: https://github.com/apache/kafka/pull/14904 This is part of the investigation into recent build instability. It simply turns off the consumer integration tests that use the new AsyncKafkaConsumer to see whether the build runs smoothly.

Re: [PR] Kafka-14748: Relax non-null FK left-join requirement [kafka]

florin-akermann commented on code in PR #14107: URL: https://github.com/apache/kafka/pull/14107#discussion_r1413117384 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/foreignkeyjoin/SubscriptionSendProcessorSupplier.java: ## @@ -115,102 +116,93 @@ public void

Re: [PR] Kafka-14748: Relax non-null FK left-join requirement [kafka]

florin-akermann commented on code in PR #14107: URL: https://github.com/apache/kafka/pull/14107#discussion_r1413117650 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/foreignkeyjoin/SubscriptionSendProcessorSupplier.java: ## @@ -115,102 +116,93 @@ public void

[jira] [Assigned] (KAFKA-9545) Flaky Test `RegexSourceIntegrationTest.testRegexMatchesTopicsAWhenDeleted`

[ https://issues.apache.org/jira/browse/KAFKA-9545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashwin Pankaj reassigned KAFKA-9545: Assignee: Ashwin Pankaj > Flaky Test `RegexSourceIntegrationTest.testRegexMatchesTopicsAWh

Re: [PR] Kafka-14748: Relax non-null FK left-join requirement [kafka]

florin-akermann commented on PR #14107: URL: https://github.com/apache/kafka/pull/14107#issuecomment-1837513033 > Just a couple of comments Thank you @wcarlson5 for the comments. I pushed according changes. While I have your attention could we revisit the adjutments done to t

[jira] [Assigned] (KAFKA-15936) Flaky tests for class: ConnectorTopicsIntegrationTest

[ https://issues.apache.org/jira/browse/KAFKA-15936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashwin Pankaj reassigned KAFKA-15936: - Assignee: Ashwin Pankaj > Flaky tests for class: ConnectorTopicsIntegrationTest > -

[PR] MINOR: Allow local-log segment deletion when log-start-offset incremented [kafka]

kamalcph opened a new pull request, #14905: URL: https://github.com/apache/kafka/pull/14905 `DELETE_RECORDS` API can move the log-start-offset beyond the highest-copied-remote-offset. In such cases, we should allow deletion of local-log segments since they are not eligible for upload to rem

[jira] [Updated] (KAFKA-15166) Add deletePartition API to the RemoteStorageManager

[ https://issues.apache.org/jira/browse/KAFKA-15166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamal Chandraprakash updated KAFKA-15166: - Labels: kip (was: ) > Add deletePartition API to the RemoteStorageManager > ---

Re: [PR] KAFKA-15830: Add telemetry API handling (KIP-714) [kafka]

apoorvmittal10 commented on PR #14767: URL: https://github.com/apache/kafka/pull/14767#issuecomment-1837534121 @junrao The build passed on all JDK with flaky tests failure. There are some consistent failures as well which are unrelated to the PR changes. -- This is an automated mes

[PR] KAFKA-15527: Support ResultOrder to reverseRange and reverseAll query over kv-store in IQv2 [kafka]

hanyuzheng7 opened a new pull request, #14906: URL: https://github.com/apache/kafka/pull/14906 Support `ResultOrder` to `reverseRange` and `reverseAll` query over kv-store in IQv2.. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation -

[PR] KAFKA-15629: Support ResultOrder to TimestampedRangeQuery. [kafka]

hanyuzheng7 opened a new pull request, #14907: URL: https://github.com/apache/kafka/pull/14907 Support `ResultOrder` to `TimestampedRangeQuery`. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build stat

Re: [PR] KAFKA-15956: MetadataShell must take the log directory lock when reading [kafka]

soarez commented on code in PR #14899: URL: https://github.com/apache/kafka/pull/14899#discussion_r1413167592 ## shell/src/main/java/org/apache/kafka/shell/MetadataShell.java: ## @@ -81,6 +85,55 @@ public MetadataShell build() { } } +/** + * Return the gr

Re: [PR] KAFKA-14412: Better Rocks column family management [kafka]

nicktelford commented on code in PR #14852: URL: https://github.com/apache/kafka/pull/14852#discussion_r1413186918 ## streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBTimestampedStore.java: ## @@ -64,31 +62,14 @@ public RocksDBTimestampedStore(final String n

Re: [PR] KAFKA-14412: Better Rocks column family management [kafka]

nicktelford commented on code in PR #14852: URL: https://github.com/apache/kafka/pull/14852#discussion_r1413187089 ## streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBTimestampedStore.java: ## @@ -64,31 +62,14 @@ public RocksDBTimestampedStore(final String n

[jira] [Created] (KAFKA-15960) Flaky test: testQuotaOverrideDelete(String).quorum=kraft – kafka.api.ClientIdQuotaTest

Apoorv Mittal created KAFKA-15960: - Summary: Flaky test: testQuotaOverrideDelete(String).quorum=kraft – kafka.api.ClientIdQuotaTest Key: KAFKA-15960 URL: https://issues.apache.org/jira/browse/KAFKA-15960

[jira] [Created] (KAFKA-15961) Flaky test: testTopicIdPersistsThroughControllerRestart() – kafka.controller.ControllerIntegrationTest

Apoorv Mittal created KAFKA-15961: - Summary: Flaky test: testTopicIdPersistsThroughControllerRestart() – kafka.controller.ControllerIntegrationTest Key: KAFKA-15961 URL: https://issues.apache.org/jira/browse/KAF

[jira] [Created] (KAFKA-15962) Flaky test: testAlterPartitionVersion2KeepWorkingWhenControllerDowngradeToPre28IBP() – kafka.controller.ControllerIntegrationTest

Apoorv Mittal created KAFKA-15962: - Summary: Flaky test: testAlterPartitionVersion2KeepWorkingWhenControllerDowngradeToPre28IBP() – kafka.controller.ControllerIntegrationTest Key: KAFKA-15962 URL: https://issues.

[jira] [Created] (KAFKA-15963) Flaky test: testBrokerHeartbeatDuringMigration [3] 3.6-IV0 – org.apache.kafka.controller.QuorumControllerTest

Apoorv Mittal created KAFKA-15963: - Summary: Flaky test: testBrokerHeartbeatDuringMigration [3] 3.6-IV0 – org.apache.kafka.controller.QuorumControllerTest Key: KAFKA-15963 URL: https://issues.apache.org/jira/brows

[jira] [Created] (KAFKA-15964) Flaky test: testHighAvailabilityTaskAssignorLargeNumConsumers – org.apache.kafka.streams.processor.internals.StreamsAssignmentScaleTest

Apoorv Mittal created KAFKA-15964: - Summary: Flaky test: testHighAvailabilityTaskAssignorLargeNumConsumers – org.apache.kafka.streams.processor.internals.StreamsAssignmentScaleTest Key: KAFKA-15964 URL: https://i

Re: [PR] KAFKA-15830: Add telemetry API handling (KIP-714) [kafka]

apoorvmittal10 commented on PR #14767: URL: https://github.com/apache/kafka/pull/14767#issuecomment-1837578517 @junrao Below are the status of flaky tests. I have ran locally the flaky tests and can see them passed. I have triaged some of the failing tests, others have existing jira.

Re: [PR] KAFKA-15830: Add telemetry API handling (KIP-714) [kafka]

junrao commented on PR #14767: URL: https://github.com/apache/kafka/pull/14767#issuecomment-1837581305 @apoorvmittal10 : Thanks for triaging the tests. There are a bunch of failures for BrokerRegistrationRequestTest. Is that being tracked? -- This is an automated message from the Apache G

[jira] [Created] (KAFKA-15965) Test failure: org.apache.kafka.common.requests.BrokerRegistrationRequestTest

Apoorv Mittal created KAFKA-15965: - Summary: Test failure: org.apache.kafka.common.requests.BrokerRegistrationRequestTest Key: KAFKA-15965 URL: https://issues.apache.org/jira/browse/KAFKA-15965 Projec

Re: [PR] KAFKA-15830: Add telemetry API handling (KIP-714) [kafka]

apoorvmittal10 commented on PR #14767: URL: https://github.com/apache/kafka/pull/14767#issuecomment-1837583322 > @apoorvmittal10 : Thanks for triaging the tests. There are a bunch of failures for BrokerRegistrationRequestTest. Is that being tracked? I have created jira to track that:

Re: [PR] HOTFIX: Fix BrokerRegistrationRequestTest [kafka]

soarez commented on PR #14892: URL: https://github.com/apache/kafka/pull/14892#issuecomment-1837587002 I think this overlaps with #14887? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

Re: [PR] KAFKA-15922: Add a MetadataVersion for JBOD [kafka]

apoorvmittal10 commented on code in PR #14860: URL: https://github.com/apache/kafka/pull/14860#discussion_r1413198971 ## clients/src/main/java/org/apache/kafka/common/requests/BrokerRegistrationRequest.java: ## @@ -45,10 +45,6 @@ public short oldestAllowedVersion() {

Re: [PR] KAFKA-15922: Add a MetadataVersion for JBOD [kafka]

apoorvmittal10 commented on code in PR #14860: URL: https://github.com/apache/kafka/pull/14860#discussion_r1413198971 ## clients/src/main/java/org/apache/kafka/common/requests/BrokerRegistrationRequest.java: ## @@ -45,10 +45,6 @@ public short oldestAllowedVersion() {

Re: [PR] KAFKA-15922: Add a MetadataVersion for JBOD [kafka]

junrao commented on code in PR #14860: URL: https://github.com/apache/kafka/pull/14860#discussion_r1413208491 ## clients/src/main/java/org/apache/kafka/common/requests/BrokerRegistrationRequest.java: ## @@ -45,10 +45,6 @@ public short oldestAllowedVersion() { @Overrid

Re: [PR] KAFKA-15365: Broker-side replica management changes [kafka]

soarez commented on code in PR #14881: URL: https://github.com/apache/kafka/pull/14881#discussion_r1413201855 ## core/src/main/scala/kafka/cluster/Partition.scala: ## @@ -867,6 +869,25 @@ class Partition(val topicPartition: TopicPartition, } } + private def createLogI

[jira] [Created] (KAFKA-15966) Test failure - ClassCastException in ConsistencyVectorIntegrationTest.shouldHaveSamePositionBoundActiveAndStandBy

Andrew Schofield created KAFKA-15966: Summary: Test failure - ClassCastException in ConsistencyVectorIntegrationTest.shouldHaveSamePositionBoundActiveAndStandBy Key: KAFKA-15966 URL: https://issues.apache.org/

[jira] [Updated] (KAFKA-15966) Test failure - ClassCastException in ConsistencyVectorIntegrationTest.shouldHaveSamePositionBoundActiveAndStandBy

[ https://issues.apache.org/jira/browse/KAFKA-15966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Schofield updated KAFKA-15966: - Labels: test-failure (was: ) > Test failure - ClassCastException in > ConsistencyVecto

Re: [PR] HOTFIX: fix ConsistencyVectorIntegrationTest failure [kafka]

lucasbru merged PR #14895: URL: https://github.com/apache/kafka/pull/14895 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

Re: [PR] KAFKA-15953: Refactor polling delays [kafka]

lucasbru merged PR #14897: URL: https://github.com/apache/kafka/pull/14897 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

[jira] [Resolved] (KAFKA-15953) Refactor polling delays

[ https://issues.apache.org/jira/browse/KAFKA-15953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Schofield resolved KAFKA-15953. -- Resolution: Fixed > Refactor polling delays > --- > >

Re: [PR] KAFKA-14438: Throw if async consumer configured with invalid group ID [kafka]

lucasbru merged PR #14872: URL: https://github.com/apache/kafka/pull/14872 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

[jira] [Resolved] (KAFKA-15966) Test failure - ClassCastException in ConsistencyVectorIntegrationTest.shouldHaveSamePositionBoundActiveAndStandBy

[ https://issues.apache.org/jira/browse/KAFKA-15966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Schofield resolved KAFKA-15966. -- Resolution: Duplicate Just fixed by https://github.com/apache/kafka/pull/14895 > Test

Re: [PR] KAFKA-15795: Support fetch(fromKey, toKey, from, to) to WindowRangeQuery and unify WindowKeyQuery and WindowRangeQuery [kafka]

aliehsaeedii commented on PR #14745: URL: https://github.com/apache/kafka/pull/14745#issuecomment-1837630278 @hanyuzheng7 Thanks for the KIP and implementation. Maybe align the variable names with what you introduced in the KIP (or the other way around)? `newTimeFrom/To` -> `timeFrom/To`

[jira] [Commented] (KAFKA-15328) intra-broker partition reassignment triggers offline log dirs randomly

[ https://issues.apache.org/jira/browse/KAFKA-15328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17792600#comment-17792600 ] Maxime Thébault commented on KAFKA-15328: - I can observe the same behavior with

[jira] [Commented] (KAFKA-15328) intra-broker partition reassignment triggers offline log dirs randomly

[ https://issues.apache.org/jira/browse/KAFKA-15328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17792601#comment-17792601 ] Maxime Thébault commented on KAFKA-15328: - Duplicate of KAFKA-15572, which is fi

Re: [PR] HOTFIX: Fix BrokerRegistrationRequestTest [kafka]

showuon commented on PR #14892: URL: https://github.com/apache/kafka/pull/14892#issuecomment-1837659295 Thanks @soarez ! Closing it! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] HOTFIX: Fix BrokerRegistrationRequestTest [kafka]

showuon closed pull request #14892: HOTFIX: Fix BrokerRegistrationRequestTest URL: https://github.com/apache/kafka/pull/14892 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[jira] [Resolved] (KAFKA-15957) ConsistencyVectorIntegrationTest.shouldHaveSamePositionBoundActiveAndStandBy broken

[ https://issues.apache.org/jira/browse/KAFKA-15957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-15957. - Resolution: Fixed > ConsistencyVectorIntegrationTest.shouldHaveSamePositionBoundActiveAn

Re: [PR] KAFKA-15662: Add support for clientInstanceIds in Kafka Stream [kafka]

mjsax commented on code in PR #14864: URL: https://github.com/apache/kafka/pull/14864#discussion_r1413296164 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/ActiveTaskCreator.java: ## @@ -136,8 +135,8 @@ StreamsProducer streamsProducerForTask(final TaskId t

[PR] KAFKA-15662: Add support for clientInstanceIds in Kafka Stream [kafka]

mjsax opened a new pull request, #14908: URL: https://github.com/apache/kafka/pull/14908 - Part of KIP-714 - Add new configs and public API for Kafka Streams - Implement support to get admin client instance id -- This is an automated message from the Apache Git Service. To respond to

Re: [PR] KAFKA-15662: Add support for clientInstanceIds in Kafka Stream [kafka]

mjsax commented on PR #14864: URL: https://github.com/apache/kafka/pull/14864#issuecomment-1837748113 Split out some parts to make reviewing simpler as this PR grows to large: https://github.com/apache/kafka/pull/14908 -- This is an automated message from the Apache Git Service. To respon

Re: [PR] KAFKA-15445: Add JVM Docker image [kafka]

VedarthConfluent commented on code in PR #14552: URL: https://github.com/apache/kafka/pull/14552#discussion_r1413361010 ## docker/jvm/Dockerfile: ## @@ -0,0 +1,100 @@ +### +# Licensed to the Apache Soft

Re: [PR] KAFKA-15830: Add telemetry API handling (KIP-714) [kafka]

apoorvmittal10 commented on PR #14767: URL: https://github.com/apache/kafka/pull/14767#issuecomment-1837850314 @junrao Please let me know if I need to address anything else or the PR is good to merge. -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] KAFKA-15663, KAFKA-15794: Telemetry reporter and request handling (KIP-714) [kafka]

apoorvmittal10 closed pull request #14724: KAFKA-15663, KAFKA-15794: Telemetry reporter and request handling (KIP-714) URL: https://github.com/apache/kafka/pull/14724 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

[PR] KAFKA-15663, KAFKA-15794: Telemetry reporter and request handling (KIP-714) [kafka]

apoorvmittal10 opened a new pull request, #14909: URL: https://github.com/apache/kafka/pull/14909 ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including up

Re: [PR] KAFKA-15818: ensure leave group on max poll interval [kafka]

philipnee commented on PR #14873: URL: https://github.com/apache/kafka/pull/14873#issuecomment-1837908142 hi @lianetm @lucasbru can I ask for a review for this PR? Modification was made based on our discussion. -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] KAFKA-14412: Better Rocks column family management [kafka]

cadonna commented on code in PR #14852: URL: https://github.com/apache/kafka/pull/14852#discussion_r1413468864 ## streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBTimestampedStore.java: ## @@ -64,31 +62,14 @@ public RocksDBTimestampedStore(final String name,