Re: [PR] KAFKA-15388: Handling remote segment read in case of log compaction [kafka]

2023-12-22 Thread via GitHub
iit2009060 commented on code in PR #15060: URL: https://github.com/apache/kafka/pull/15060#discussion_r1435445309 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -1289,15 +1289,21 @@ public FetchDataInfo read(RemoteStorageFetchInfo remoteStorageFetchInfo)

Re: [PR] KAFKA-15388: Handling remote segment read in case of log compaction [kafka]

2023-12-22 Thread via GitHub
iit2009060 commented on code in PR #15060: URL: https://github.com/apache/kafka/pull/15060#discussion_r1435445309 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -1289,15 +1289,21 @@ public FetchDataInfo read(RemoteStorageFetchInfo remoteStorageFetchInfo)

Re: [PR] KAFKA-15388: Handling remote segment read in case of log compaction [kafka]

2023-12-22 Thread via GitHub
iit2009060 commented on code in PR #15060: URL: https://github.com/apache/kafka/pull/15060#discussion_r1435444785 ## core/src/test/java/kafka/log/remote/RemoteLogManagerTest.java: ## @@ -2161,6 +2167,86 @@ RecordBatch findFirstBatch(RemoteLogInputStream remoteLogInputStream,

Re: [PR] KAFKA-15388: Handling remote segment read in case of log compaction [kafka]

2023-12-22 Thread via GitHub
iit2009060 commented on code in PR #15060: URL: https://github.com/apache/kafka/pull/15060#discussion_r1435445309 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -1289,15 +1289,21 @@ public FetchDataInfo read(RemoteStorageFetchInfo remoteStorageFetchInfo)

Re: [PR] KAFKA-15388: Handling remote segment read in case of log compaction [kafka]

2023-12-22 Thread via GitHub
iit2009060 commented on code in PR #15060: URL: https://github.com/apache/kafka/pull/15060#discussion_r1435446137 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -1412,8 +1422,8 @@ private void collectAbortedTransactionInLocalSegments(long startOffset,

Re: [PR] KAFKA-15388: Handling remote segment read in case of log compaction [kafka]

2023-12-22 Thread via GitHub
iit2009060 commented on code in PR #15060: URL: https://github.com/apache/kafka/pull/15060#discussion_r1435445309 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -1289,15 +1289,21 @@ public FetchDataInfo read(RemoteStorageFetchInfo remoteStorageFetchInfo)

Re: [PR] KAFKA-15388: Handling remote segment read in case of log compaction [kafka]

2023-12-22 Thread via GitHub
iit2009060 commented on code in PR #15060: URL: https://github.com/apache/kafka/pull/15060#discussion_r1435444785 ## core/src/test/java/kafka/log/remote/RemoteLogManagerTest.java: ## @@ -2161,6 +2167,86 @@ RecordBatch findFirstBatch(RemoteLogInputStream remoteLogInputStream,

[jira] [Comment Edited] (KAFKA-16027) Refactor MetadataTest#testUpdatePartitionLeadership

2023-12-22 Thread Alexander Aghili (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17799209#comment-17799209 ] Alexander Aghili edited comment on KAFKA-16027 at 12/23/23 2:09 AM:

Re: [PR] MINOR: Close RemoteLogManager in RemoteLogManagerTest [kafka]

2023-12-22 Thread via GitHub
satishd commented on PR #15063: URL: https://github.com/apache/kafka/pull/15063#issuecomment-1868160079 Nice catch @dajac, thanks for resolving it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[jira] [Resolved] (KAFKA-16012) Incomplete range assignment in consumer

2023-12-22 Thread Justine Olshan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justine Olshan resolved KAFKA-16012. Resolution: Fixed > Incomplete range assignment in consumer >

Re: [PR] KAFKA-16012: Ensure new leader information merged correctly with the current metadata [kafka]

2023-12-22 Thread via GitHub
jolshan merged PR #15023: URL: https://github.com/apache/kafka/pull/15023 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-16012: Ensure new leader information merged correctly with the current metadata [kafka]

2023-12-22 Thread via GitHub
jolshan commented on PR #15023: URL: https://github.com/apache/kafka/pull/15023#issuecomment-1868141386 Ok -- looks good. I will merge and backport to 3.7. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[jira] [Updated] (KAFKA-16047) Source connector with EOS enabled have some InitProducerId requests timing out, effectively failing all the tasks & the whole connector

2023-12-22 Thread Angelos Kaltsikis (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Angelos Kaltsikis updated KAFKA-16047: -- Description: Source Connectors with 'exactly.once.support = required' may have some

[jira] [Updated] (KAFKA-16047) Source connector with EOS enabled have some InitProducerId requests timing out, effectively failing all the tasks & the whole connector

2023-12-22 Thread Angelos Kaltsikis (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Angelos Kaltsikis updated KAFKA-16047: -- Summary: Source connector with EOS enabled have some InitProducerId requests timing

[jira] [Commented] (KAFKA-16047) Source connector with EOS enabled have some InitProducerId requests timing out, effectively failing the whole connector

2023-12-22 Thread Angelos Kaltsikis (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17799953#comment-17799953 ] Angelos Kaltsikis commented on KAFKA-16047: --- cc. [~gregharris73] > Source connector with EOS

[jira] [Updated] (KAFKA-16047) Source connector with EOS enabled have some InitProducerId requests timing out, effectively failing the whole connector

2023-12-22 Thread Angelos Kaltsikis (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Angelos Kaltsikis updated KAFKA-16047: -- Summary: Source connector with EOS enabled have some InitProducerId requests timing

[jira] [Created] (KAFKA-16047) Source connector with EOS enabled have some InitProducerId requests timing out

2023-12-22 Thread Angelos Kaltsikis (Jira)
Angelos Kaltsikis created KAFKA-16047: - Summary: Source connector with EOS enabled have some InitProducerId requests timing out Key: KAFKA-16047 URL: https://issues.apache.org/jira/browse/KAFKA-16047

[PR] Duplicate method; The QuotaUtils one is used. [kafka]

2023-12-22 Thread via GitHub
afshing opened a new pull request, #15066: URL: https://github.com/apache/kafka/pull/15066 It seems like this PR duplicated the implementation to QuotaUtils, but didn't remove this implementation *More detailed description of your change, if necessary. The PR title and PR message

Re: [PR] KAFKA-15388: Handling remote segment read in case of log compaction [kafka]

2023-12-22 Thread via GitHub
clolov commented on code in PR #15060: URL: https://github.com/apache/kafka/pull/15060#discussion_r1435282790 ## core/src/test/java/kafka/log/remote/RemoteLogManagerTest.java: ## @@ -2161,6 +2167,86 @@ RecordBatch findFirstBatch(RemoteLogInputStream remoteLogInputStream, long

Re: [PR] KAFKA-16012: Ensure new leader information merged correctly with the current metadata [kafka]

2023-12-22 Thread via GitHub
philipnee commented on PR #15023: URL: https://github.com/apache/kafka/pull/15023#issuecomment-1867985789 Hey @jolshan seems like the build finished without oom, what do you think? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] KAFKA-14588 ConfigType moved to server-common [kafka]

2023-12-22 Thread via GitHub
mimaison merged PR #14867: URL: https://github.com/apache/kafka/pull/14867 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-14588 ConfigType moved to server-common [kafka]

2023-12-22 Thread via GitHub
mimaison commented on PR #14867: URL: https://github.com/apache/kafka/pull/14867#issuecomment-1867895289 I'm waiting for the CI to finish. I don't want to risk breaking trunk just before logging off for PTO. -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] KAFKA-15720: KRaft support in DeleteTopicTest [kafka]

2023-12-22 Thread via GitHub
tinaselenge commented on PR #14846: URL: https://github.com/apache/kafka/pull/14846#issuecomment-1867865063 @dengziming I have triggered the build 3 more times and DeleteTopicTest passed. I have also enabled KRaft for testIncreasePartitionCountDuringDeleteTopic (they didn't have much

Re: [PR] KAFKA-16021: Eagerly look up StringSerializer encoding during configure [kafka]

2023-12-22 Thread via GitHub
srdo commented on PR #15024: URL: https://github.com/apache/kafka/pull/15024#issuecomment-1867860145 Sure, done. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] MINOR: Refactor to only require one stopPartitions helper [kafka]

2023-12-22 Thread via GitHub
divijvaidya merged PR #14662: URL: https://github.com/apache/kafka/pull/14662 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] MINOR: Refactor to only require one stopPartitions helper [kafka]

2023-12-22 Thread via GitHub
divijvaidya commented on PR #14662: URL: https://github.com/apache/kafka/pull/14662#issuecomment-1867857397 Unrelated test failures ``` [Build / JDK 8 and Scala 2.12 /

Re: [PR] KAFKA-16021: Eagerly look up StringSerializer encoding during configure [kafka]

2023-12-22 Thread via GitHub
divijvaidya commented on PR #15024: URL: https://github.com/apache/kafka/pull/15024#issuecomment-1867854246 A fix to make CI stable was pushed today. @srdo Can you please rebase from trunk one time? It will probably help us get a better picture of CI failures/success. -- This is an

Re: [PR] KAFKA-14588 ConfigType moved to server-common [kafka]

2023-12-22 Thread via GitHub
nizhikov commented on PR #14867: URL: https://github.com/apache/kafka/pull/14867#issuecomment-1867850331 @mimaison Are we want to merge this now? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[jira] [Resolved] (KAFKA-16046) Stream Stream Joins fail after restoration with deserialization exceptions

2023-12-22 Thread Almog Gavra (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Almog Gavra resolved KAFKA-16046. - Resolution: Fixed > Stream Stream Joins fail after restoration with deserialization exceptions

Re: [PR] DNM: test [kafka]

2023-12-22 Thread via GitHub
lucasbru closed pull request #15064: DNM: test URL: https://github.com/apache/kafka/pull/15064 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Comment Edited] (KAFKA-10875) offsetsForTimes returns null for some partitions when it shouldn't?

2023-12-22 Thread Hugo Abreu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17799859#comment-17799859 ] Hugo Abreu edited comment on KAFKA-10875 at 12/22/23 3:22 PM: -- We are

[jira] [Comment Edited] (KAFKA-10875) offsetsForTimes returns null for some partitions when it shouldn't?

2023-12-22 Thread Hugo Abreu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17799859#comment-17799859 ] Hugo Abreu edited comment on KAFKA-10875 at 12/22/23 3:21 PM: -- We are

[jira] [Commented] (KAFKA-10875) offsetsForTimes returns null for some partitions when it shouldn't?

2023-12-22 Thread Hugo Abreu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-10875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17799859#comment-17799859 ] Hugo Abreu commented on KAFKA-10875: We are seeing this behaviour with kafka-clients 3.6.1 and

Re: [PR] KAFKA-15373: fix exception thrown in Admin#describeTopics for unknown ID [kafka]

2023-12-22 Thread via GitHub
jolshan commented on PR #14599: URL: https://github.com/apache/kafka/pull/14599#issuecomment-1867795357 Hey @MikeEdgar I'm seeing KafkaAdminClientTest.testDescribeTopicByIds failing due to it expecting the old error. Can we update that test? -- This is an automated message from the

Re: [PR] KAFKA-16046: fix stream-stream-join store types [kafka]

2023-12-22 Thread via GitHub
lucasbru merged PR #15061: URL: https://github.com/apache/kafka/pull/15061 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-16046: fix stream-stream-join store types [kafka]

2023-12-22 Thread via GitHub
lucasbru commented on PR #15061: URL: https://github.com/apache/kafka/pull/15061#issuecomment-1867744948 Cause of OOM on trunk identified, so merging this -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] KAFKA-16026: Send Poll event to the background thread [kafka]

2023-12-22 Thread via GitHub
lucasbru merged PR #15035: URL: https://github.com/apache/kafka/pull/15035 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] MINOR: Exit catcher should be reset after the cluster is shutdown [kafka]

2023-12-22 Thread via GitHub
dajac merged PR #15062: URL: https://github.com/apache/kafka/pull/15062 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] MINOR: Close RemoteLogManager in RemoteLogManagerTest [kafka]

2023-12-22 Thread via GitHub
dajac merged PR #15063: URL: https://github.com/apache/kafka/pull/15063 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] MINOR: set test global timeout as 10 mins [kafka]

2023-12-22 Thread via GitHub
showuon commented on PR #15065: URL: https://github.com/apache/kafka/pull/15065#issuecomment-1867639044 @dajac @ijuma @jolshan @stanislavkozlovski , call for review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[PR] MINOR: set test global timeout as 10 mins [kafka]

2023-12-22 Thread via GitHub
showuon opened a new pull request, #15065: URL: https://github.com/apache/kafka/pull/15065 As @stanislavkozlovski found [here](https://gist.github.com/stanislavkozlovski/8959f7ee59434f774841f4ae2f5228c2), there are some tests ran more than 1 ~ 2 hours, which make our CI build meet 8 hours

Re: [PR] KAFKA-16046: fix stream-stream-join store types [kafka]

2023-12-22 Thread via GitHub
lucasbru commented on PR #15061: URL: https://github.com/apache/kafka/pull/15061#issuecomment-1867506949 Restarting the CI since we had OOM before - however, OOM affects trunk so if we don't see anything in the next run, I think we should merge this. -- This is an automated message from

[PR] DNM: test [kafka]

2023-12-22 Thread via GitHub
lucasbru opened a new pull request, #15064: URL: https://github.com/apache/kafka/pull/15064 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of

Re: [PR] KAFKA-15996: Improve JsonConverter performance [kafka]

2023-12-22 Thread via GitHub
mfvitale commented on PR #14992: URL: https://github.com/apache/kafka/pull/14992#issuecomment-1867484294 > haha! I like your commit message "code beauty" :)  > I see that this is your first change to Apache Kafka. Congratulations and welcome! Yes! Thanks! > As

Re: [PR] [KAFKA-16015] Fix custom timeouts overwritten by defaults [kafka]

2023-12-22 Thread via GitHub
sciclon2 commented on code in PR #15030: URL: https://github.com/apache/kafka/pull/15030#discussion_r1434907926 ## tools/src/main/java/org/apache/kafka/tools/LeaderElectionCommand.java: ## @@ -99,8 +99,12 @@ static void run(Duration timeoutMs, String... args) throws Exception

Re: [PR] [KAFKA-16015] Fix custom timeouts overwritten by defaults [kafka]

2023-12-22 Thread via GitHub
divijvaidya commented on code in PR #15030: URL: https://github.com/apache/kafka/pull/15030#discussion_r1434893044 ## tools/src/main/java/org/apache/kafka/tools/LeaderElectionCommand.java: ## @@ -99,8 +99,12 @@ static void run(Duration timeoutMs, String... args) throws

[PR] MINOR: Close RemoteLogManager in RemoteLogManagerTest [kafka]

2023-12-22 Thread via GitHub
dajac opened a new pull request, #15063: URL: https://github.com/apache/kafka/pull/15063 I was inspecting heap dumps to chase the OOM errors that we see in builds and I have noticed that `remote-log-index-cleaner` threads were still running. This may be one of the sources. We should

[jira] [Commented] (KAFKA-15301) [Tiered Storage] Historically compacted topics send request to remote for active segment during consume

2023-12-22 Thread Arpit Goyal (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17799742#comment-17799742 ] Arpit Goyal commented on KAFKA-15301: - [~mital.awachat] does this ticket addresses your concern.

Re: [PR] KAFKA-15996: Improve JsonConverter performance [kafka]

2023-12-22 Thread via GitHub
mfvitale commented on PR #14992: URL: https://github.com/apache/kafka/pull/14992#issuecomment-1867448606 @divijvaidya Fixed! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] KAFKA-15388: Handling remote segment read in case of log compaction [kafka]

2023-12-22 Thread via GitHub
clolov commented on PR #15060: URL: https://github.com/apache/kafka/pull/15060#issuecomment-1867443497 Thank you for the contribution! I will aim to provide a review throughout the day! -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] KAFKA-15996: Improve JsonConverter performance [kafka]

2023-12-22 Thread via GitHub
divijvaidya commented on PR #14992: URL: https://github.com/apache/kafka/pull/14992#issuecomment-1867441468 @mfvitale one last comment about fixing indentation, otherwise we should be good to ship! -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] KAFKA-15996: Improve JsonConverter performance [kafka]

2023-12-22 Thread via GitHub
divijvaidya commented on code in PR #14992: URL: https://github.com/apache/kafka/pull/14992#discussion_r1434876077 ## connect/json/src/main/java/org/apache/kafka/connect/json/JsonConverter.java: ## @@ -235,18 +235,31 @@ public Object toConnect(final Schema schema, final

Re: [PR] KAFKA-16021: Eagerly look up StringSerializer encoding during configure [kafka]

2023-12-22 Thread via GitHub
divijvaidya commented on PR #15024: URL: https://github.com/apache/kafka/pull/15024#issuecomment-1867439333 I have restarted the CI to get a better build status since last one failed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[PR] MINOR: Exit catcher should be reset after the cluster is shutdown [kafka]

2023-12-22 Thread via GitHub
dajac opened a new pull request, #15062: URL: https://github.com/apache/kafka/pull/15062 I was investigating a build which failed with "exit 1". In the logs of the broker, I was that the first call to exist was caught. However, a second one was not. See the logs below. The issue seems to

Re: [PR] KAFKA-14588 ConfigType moved to server-common [kafka]

2023-12-22 Thread via GitHub
nizhikov commented on PR #14867: URL: https://github.com/apache/kafka/pull/14867#issuecomment-1867374109 @mimaison > This commit cannot be built Build status is scary, but looks like some tests just failed. And they not related to my changes. -- This is an automated