[GitHub] [kafka] showuon commented on pull request #14434: KAFKA-15498: bump snappy-java version to 1.1.10.4

2023-09-26 Thread via GitHub
showuon commented on PR #14434: URL: https://github.com/apache/kafka/pull/14434#issuecomment-1736782137 @jlprat , thanks for the comments. PR updated in https://github.com/apache/kafka/pull/14434/commits/c21e79e5ecf46f34b38fa6c1547c615c4d11fe1d . -- This is an automated message from the

[GitHub] [kafka] showuon commented on pull request #14434: KAFKA-15498: bump snappy-java version to 1.1.10.4

2023-09-26 Thread via GitHub
showuon commented on PR #14434: URL: https://github.com/apache/kafka/pull/14434#issuecomment-1736778285 PR updated to add more tests to verify the compressed data can be correctly decompressed and read. Also update the PR description. -- This is an automated message from the Apache Git Se

[GitHub] [kafka] jlprat commented on a diff in pull request #14434: KAFKA-15498: bump snappy-java version to 1.1.10.4

2023-09-26 Thread via GitHub
jlprat commented on code in PR #14434: URL: https://github.com/apache/kafka/pull/14434#discussion_r1338092163 ## core/src/test/scala/integration/kafka/api/ProducerCompressionTest.scala: ## @@ -103,12 +123,37 @@ class ProducerCompressionTest extends QuorumTestHarness { //

[jira] [Updated] (KAFKA-15510) Follower's lastFetchedEpoch wrongly set when fetch response has no record

2023-09-26 Thread Chern Yih Cheah (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chern Yih Cheah updated KAFKA-15510: Affects Version/s: 3.5.1 3.6.0 > Follower's lastFetchedEpoch wrongl

[GitHub] [kafka] chernyih opened a new pull request, #14457: KAFKA-15510: Fix follower's lastFetchedEpoch when fetch response has …

2023-09-26 Thread via GitHub
chernyih opened a new pull request, #14457: URL: https://github.com/apache/kafka/pull/14457 …no record A regression is introduced by https://github.com/apache/kafka/pull/13843. When a fetch response has no record for a partition, validBytes is 0. We shouldn't set the last fetched epo

[jira] [Comment Edited] (KAFKA-15169) Add tests for RemoteIndexCache

2023-09-26 Thread Arpit Goyal (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17769425#comment-17769425 ] Arpit Goyal edited comment on KAFKA-15169 at 9/27/23 5:37 AM:

[jira] [Commented] (KAFKA-15169) Add tests for RemoteIndexCache

2023-09-26 Thread Arpit Goyal (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17769425#comment-17769425 ] Arpit Goyal commented on KAFKA-15169: - [~divijvaidya] As per the code RemoteIndexCac

[jira] [Created] (KAFKA-15510) Follower's lastFetchedEpoch wrongly set when fetch response has no record

2023-09-26 Thread Chern Yih Cheah (Jira)
Chern Yih Cheah created KAFKA-15510: --- Summary: Follower's lastFetchedEpoch wrongly set when fetch response has no record Key: KAFKA-15510 URL: https://issues.apache.org/jira/browse/KAFKA-15510 Proje

[jira] [Commented] (KAFKA-15493) Ensure system tests work with Java 21

2023-09-26 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17769418#comment-17769418 ] Ismael Juma commented on KAFKA-15493: - And you used Java 21 for the `ducker-ak up` s

[GitHub] [kafka] kirktrue commented on a diff in pull request #14406: KAFKA-14274 [6, 7]: Introduction of fetch request manager

2023-09-26 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1337882369 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/DefaultBackgroundThread.java: ## @@ -259,17 +144,41 @@ long handlePollResult(NetworkClientDelegate

[GitHub] [kafka] kirktrue commented on a diff in pull request #14406: KAFKA-14274 [6, 7]: Introduction of fetch request manager

2023-09-26 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1337881557 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/DefaultBackgroundThread.java: ## @@ -217,29 +98,33 @@ public void run() { }

[GitHub] [kafka] kirktrue commented on a diff in pull request #14406: KAFKA-14274 [6, 7]: Introduction of fetch request manager

2023-09-26 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1337881130 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/events/BackgroundEventProcessor.java: ## @@ -0,0 +1,103 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] kirktrue commented on a diff in pull request #14406: KAFKA-14274 [6, 7]: Introduction of fetch request manager

2023-09-26 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1337849711 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/FetchRequestManager.java: ## @@ -0,0 +1,136 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] showuon commented on pull request #14434: KAFKA-15498: bump snappy-java version to 1.1.10.4

2023-09-26 Thread via GitHub
showuon commented on PR #14434: URL: https://github.com/apache/kafka/pull/14434#issuecomment-1736393174 Thanks all. (Sorry, pressing the wrong button on phone). Will verify the decompressed results are able to be correctly read today. -- This is an automated message from the Apache Git S

[GitHub] [kafka] showuon closed pull request #14434: KAFKA-15498: bump snappy-java version to 1.1.10.4

2023-09-26 Thread via GitHub
showuon closed pull request #14434: KAFKA-15498: bump snappy-java version to 1.1.10.4 URL: https://github.com/apache/kafka/pull/14434 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

[GitHub] [kafka] showuon commented on pull request #14434: KAFKA-15498: bump snappy-java version to 1.1.10.4

2023-09-26 Thread via GitHub
showuon commented on PR #14434: URL: https://github.com/apache/kafka/pull/14434#issuecomment-1736386116 One question I'd like to get your thoughts. This test is doing: 1. Generate a 2 byte key, and 128 byte value. 2. Create a record using snappy codec compressed. 3. Get the size of t

[jira] [Commented] (KAFKA-15493) Ensure system tests work with Java 21

2023-09-26 Thread Said BOUDJELDA (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17769355#comment-17769355 ] Said BOUDJELDA commented on KAFKA-15493: bash tests/docker/run_tests.sh passes w

[GitHub] [kafka] cmccabe merged pull request #14399: KAFKA-15466: Add KIP-919 support for some admin APIs

2023-09-26 Thread via GitHub
cmccabe merged PR #14399: URL: https://github.com/apache/kafka/pull/14399 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

[jira] [Commented] (KAFKA-15493) Ensure system tests work with Java 21

2023-09-26 Thread Said BOUDJELDA (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17769348#comment-17769348 ] Said BOUDJELDA commented on KAFKA-15493: ./gradlew clean systemTestLibs is passi

[GitHub] [kafka] divijvaidya commented on pull request #14434: KAFKA-15498: bump snappy-java version to 1.1.10.4

2023-09-26 Thread via GitHub
divijvaidya commented on PR #14434: URL: https://github.com/apache/kafka/pull/14434#issuecomment-1736256796 I have the run the test ~200 times locally using the command below and they all succeeded. Also, the CI build on jenkins has succeeded twice [1] and [2]. IMO, we can safely merge this

[GitHub] [kafka] bmscomp commented on pull request #14449: MINOR: Upgrade version of zstd-jni to the latest stable version 1.5.5-5

2023-09-26 Thread via GitHub
bmscomp commented on PR #14449: URL: https://github.com/apache/kafka/pull/14449#issuecomment-1736244500 @divijvaidya Done -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

[GitHub] [kafka] lucasbru commented on pull request #14281: KAFKA-15326: [9/N] Start and stop executors and cornercases

2023-09-26 Thread via GitHub
lucasbru commented on PR #14281: URL: https://github.com/apache/kafka/pull/14281#issuecomment-1736221407 Test failures are unrelated -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[GitHub] [kafka] nizhikov opened a new pull request, #14456: KAFKA-14595 ReassignPartitionsIntegrationTest rewritten in java

2023-09-26 Thread via GitHub
nizhikov opened a new pull request, #14456: URL: https://github.com/apache/kafka/pull/14456 This PR is part of #13247 It contains `ReassignPartitionsIntegrationTest` rewritten in java. Goal of PR is reduce changes size in main PR. ### Committer Checklist (excluded from commit me

[GitHub] [kafka] kirktrue commented on a diff in pull request #14406: KAFKA-14274 [6, 7]: Introduction of fetch request manager

2023-09-26 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1337661714 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/PrototypeAsyncConsumer.java: ## @@ -636,42 +857,148 @@ public void assign(Collection partitions)

[GitHub] [kafka] kirktrue commented on a diff in pull request #14406: KAFKA-14274 [6, 7]: Introduction of fetch request manager

2023-09-26 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1337660657 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/PrototypeAsyncConsumer.java: ## @@ -636,42 +857,148 @@ public void assign(Collection partitions)

[GitHub] [kafka] ahuang98 commented on a diff in pull request #14428: KAFKA-15489: resign leadership when no fetch from majority voters

2023-09-26 Thread via GitHub
ahuang98 commented on code in PR #14428: URL: https://github.com/apache/kafka/pull/14428#discussion_r1337653712 ## raft/src/main/java/org/apache/kafka/raft/LeaderState.java: ## @@ -76,9 +85,37 @@ protected LeaderState( boolean hasAcknowledgedLeader = voterId == loca

[GitHub] [kafka] lianetm commented on a diff in pull request #14364: KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests

2023-09-26 Thread via GitHub
lianetm commented on code in PR #14364: URL: https://github.com/apache/kafka/pull/14364#discussion_r1337639317 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -0,0 +1,294 @@ +/* + * Licensed to the Apache Software Foundati

[GitHub] [kafka] zhengyd2014 commented on a diff in pull request #14444: KIP-951: Server side and protocol changes for KIP-951

2023-09-26 Thread via GitHub
zhengyd2014 commented on code in PR #1: URL: https://github.com/apache/kafka/pull/1#discussion_r1337630516 ## clients/src/main/java/org/apache/kafka/common/requests/FetchResponse.java: ## @@ -276,11 +289,16 @@ private static FetchResponseData toMessage(Errors error,

[GitHub] [kafka] nizhikov commented on pull request #14064: KAFKA-15030: Add connect-plugin-path command-line tool.

2023-09-26 Thread via GitHub
nizhikov commented on PR #14064: URL: https://github.com/apache/kafka/pull/14064#issuecomment-1736054928 In case someone interested, adding `:connect:api` dependency to `tools` somehow make `:storage:api` classes invisible. I fix it adding explicit dependency on `:storage:api` project.

[jira] [Created] (KAFKA-15509) KRaft Controller doesn't consider brokers that are shutting down during topic creating

2023-09-26 Thread Jira
José Armando García Sancio created KAFKA-15509: -- Summary: KRaft Controller doesn't consider brokers that are shutting down during topic creating Key: KAFKA-15509 URL: https://issues.apache.org/jira/br

[GitHub] [kafka] lucasbru merged pull request #14226: KAFKA-15326: [8/N] Move consumer interaction out of processing methods

2023-09-26 Thread via GitHub
lucasbru merged PR #14226: URL: https://github.com/apache/kafka/pull/14226 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

[GitHub] [kafka] lucasbru commented on pull request #14226: KAFKA-15326: [8/N] Move consumer interaction out of processing methods

2023-09-26 Thread via GitHub
lucasbru commented on PR #14226: URL: https://github.com/apache/kafka/pull/14226#issuecomment-1735852839 Failures are unrelated -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

[GitHub] [kafka] kirktrue commented on a diff in pull request #14406: KAFKA-14274 [6, 7]: Introduction of fetch request manager

2023-09-26 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1337444877 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/PrototypeAsyncConsumer.java: ## @@ -526,13 +677,57 @@ public void close() { close(Duratio

[GitHub] [kafka] kirktrue commented on a diff in pull request #14406: KAFKA-14274 [6, 7]: Introduction of fetch request manager

2023-09-26 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r133756 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/PrototypeAsyncConsumer.java: ## @@ -526,13 +677,57 @@ public void close() { close(Duratio

[GitHub] [kafka] kirktrue commented on a diff in pull request #14406: KAFKA-14274 [6, 7]: Introduction of fetch request manager

2023-09-26 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1337444033 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/PrototypeAsyncConsumer.java: ## @@ -526,13 +677,57 @@ public void close() { close(Duratio

[GitHub] [kafka] kirktrue commented on a diff in pull request #14406: KAFKA-14274 [6, 7]: Introduction of fetch request manager

2023-09-26 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1337443244 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/PrototypeAsyncConsumer.java: ## @@ -684,7 +1011,7 @@ private boolean isCommittedOffsetsManagementE

[GitHub] [kafka] lianetm commented on a diff in pull request #14364: KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests

2023-09-26 Thread via GitHub
lianetm commented on code in PR #14364: URL: https://github.com/apache/kafka/pull/14364#discussion_r1337428627 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -0,0 +1,324 @@ +/* + * Licensed to the Apache Software Foundati

[GitHub] [kafka] philipnee commented on a diff in pull request #14364: KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests

2023-09-26 Thread via GitHub
philipnee commented on code in PR #14364: URL: https://github.com/apache/kafka/pull/14364#discussion_r1337423659 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManagerTest.java: ## @@ -0,0 +1,308 @@ +/* + * Licensed to the Apache Software Fo

[GitHub] [kafka] philipnee commented on a diff in pull request #14364: KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests

2023-09-26 Thread via GitHub
philipnee commented on code in PR #14364: URL: https://github.com/apache/kafka/pull/14364#discussion_r1337420489 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -0,0 +1,324 @@ +/* + * Licensed to the Apache Software Founda

[GitHub] [kafka] philipnee commented on a diff in pull request #14364: KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests

2023-09-26 Thread via GitHub
philipnee commented on code in PR #14364: URL: https://github.com/apache/kafka/pull/14364#discussion_r1337418980 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -0,0 +1,324 @@ +/* + * Licensed to the Apache Software Founda

[GitHub] [kafka] lianetm commented on pull request #14364: KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests

2023-09-26 Thread via GitHub
lianetm commented on PR #14364: URL: https://github.com/apache/kafka/pull/14364#issuecomment-1735694640 Thanks for the changes @philipnee, left a few other minor comments and questions. LGTM. As I see it, the main areas requiring follow-up in other PRs would be: - fully integr

[GitHub] [kafka] divijvaidya commented on pull request #14434: KAFKA-15498: bump snappy-java version to 1.1.10.4

2023-09-26 Thread via GitHub
divijvaidya commented on PR #14434: URL: https://github.com/apache/kafka/pull/14434#issuecomment-1735667819 Since @showuon is asleep, I took the liberty of adding an extra byte to the assumed size of compressed message produced by snappy. Let's see if the test passes now. Will run it multip

[GitHub] [kafka] lianetm commented on a diff in pull request #14364: KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests

2023-09-26 Thread via GitHub
lianetm commented on code in PR #14364: URL: https://github.com/apache/kafka/pull/14364#discussion_r1337309504 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManagerTest.java: ## @@ -58,7 +59,7 @@ public class HeartbeatRequestManagerTest

[GitHub] [kafka] lianetm commented on a diff in pull request #14364: KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests

2023-09-26 Thread via GitHub
lianetm commented on code in PR #14364: URL: https://github.com/apache/kafka/pull/14364#discussion_r1337295428 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManagerTest.java: ## @@ -173,24 +194,22 @@ public void testHeartbeatResponseOnError

[GitHub] [kafka] divijvaidya commented on pull request #14434: KAFKA-15498: bump snappy-java version to 1.1.10.4

2023-09-26 Thread via GitHub
divijvaidya commented on PR #14434: URL: https://github.com/apache/kafka/pull/14434#issuecomment-1735632838 At https://github.com/apache/kafka/blob/65efb981347d6f81fb2713cd27cdfdfa9d8781b9/core/src/test/scala/unit/kafka/log/LogCleanerParameterizedIntegrationTest.scala#L149 we assume that af

[GitHub] [kafka] lianetm commented on a diff in pull request #14364: KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests

2023-09-26 Thread via GitHub
lianetm commented on code in PR #14364: URL: https://github.com/apache/kafka/pull/14364#discussion_r1337282860 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManagerTest.java: ## @@ -0,0 +1,308 @@ +/* + * Licensed to the Apache Software Foun

[GitHub] [kafka] lianetm commented on a diff in pull request #14364: KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests

2023-09-26 Thread via GitHub
lianetm commented on code in PR #14364: URL: https://github.com/apache/kafka/pull/14364#discussion_r1337282271 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManagerTest.java: ## @@ -0,0 +1,308 @@ +/* + * Licensed to the Apache Software Foun

[GitHub] [kafka] tinaselenge opened a new pull request, #14455: KAFKA-15507: Make AdminClient throw non-retriable exception for a new call while closing

2023-09-26 Thread via GitHub
tinaselenge opened a new pull request, #14455: URL: https://github.com/apache/kafka/pull/14455 AdminClient will throw IllegalStateException instead of TimeoutException if it receives new calls while closing down. This is more consistent with how Consumer and Producer clients handle new call

[GitHub] [kafka] lianetm commented on a diff in pull request #14364: KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests

2023-09-26 Thread via GitHub
lianetm commented on code in PR #14364: URL: https://github.com/apache/kafka/pull/14364#discussion_r1337278130 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManager.java: ## @@ -59,4 +59,20 @@ public interface MembershipManager { * current

[GitHub] [kafka] ijuma commented on pull request #14434: KAFKA-15498: bump snappy-java version to 1.1.10.4

2023-09-26 Thread via GitHub
ijuma commented on PR #14434: URL: https://github.com/apache/kafka/pull/14434#issuecomment-1735623002 We don't know, but it's the nature of compression libraries - there can be some variations as the algorithms are tweaked for performance. -- This is an automated message from the Apache

[GitHub] [kafka] lianetm commented on a diff in pull request #14364: KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests

2023-09-26 Thread via GitHub
lianetm commented on code in PR #14364: URL: https://github.com/apache/kafka/pull/14364#discussion_r1337275381 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -0,0 +1,324 @@ +/* + * Licensed to the Apache Software Foundati

[GitHub] [kafka] lianetm commented on a diff in pull request #14364: KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests

2023-09-26 Thread via GitHub
lianetm commented on code in PR #14364: URL: https://github.com/apache/kafka/pull/14364#discussion_r1337275381 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -0,0 +1,324 @@ +/* + * Licensed to the Apache Software Foundati

[GitHub] [kafka] jlprat commented on pull request #14434: KAFKA-15498: bump snappy-java version to 1.1.10.4

2023-09-26 Thread via GitHub
jlprat commented on PR #14434: URL: https://github.com/apache/kafka/pull/14434#issuecomment-1735612270 Is it just 1 extra byte? Or are there cases where some more extra bytes are added? -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

[GitHub] [kafka] lianetm commented on a diff in pull request #14364: KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests

2023-09-26 Thread via GitHub
lianetm commented on code in PR #14364: URL: https://github.com/apache/kafka/pull/14364#discussion_r1337269007 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -0,0 +1,324 @@ +/* + * Licensed to the Apache Software Foundati

[GitHub] [kafka] ijuma commented on pull request #14434: KAFKA-15498: bump snappy-java version to 1.1.10.4

2023-09-26 Thread via GitHub
ijuma commented on PR #14434: URL: https://github.com/apache/kafka/pull/14434#issuecomment-1735607326 The extra byte may be ok since the underlying snappy library was also upgraded in this new version https://github.com/xerial/snappy-java/commit/f2e97f27be0dc6c691369040ba8a673bface484c

[GitHub] [kafka] lianetm commented on a diff in pull request #14364: KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests

2023-09-26 Thread via GitHub
lianetm commented on code in PR #14364: URL: https://github.com/apache/kafka/pull/14364#discussion_r1337265790 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -0,0 +1,324 @@ +/* + * Licensed to the Apache Software Foundati

[GitHub] [kafka] lianetm commented on a diff in pull request #14364: KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests

2023-09-26 Thread via GitHub
lianetm commented on code in PR #14364: URL: https://github.com/apache/kafka/pull/14364#discussion_r1337263803 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -0,0 +1,324 @@ +/* + * Licensed to the Apache Software Foundati

[GitHub] [kafka] jlprat commented on pull request #14434: KAFKA-15498: bump snappy-java version to 1.1.10.4

2023-09-26 Thread via GitHub
jlprat commented on PR #14434: URL: https://github.com/apache/kafka/pull/14434#issuecomment-1735603472 I think the problem is in the details, why this extra byte? Also, 1 extra byte might not be relevant, but at some point the overhead of extra bytes might cause performance regressions (ove

[GitHub] [kafka] divijvaidya commented on pull request #14434: KAFKA-15498: bump snappy-java version to 1.1.10.4

2023-09-26 Thread via GitHub
divijvaidya commented on PR #14434: URL: https://github.com/apache/kafka/pull/14434#issuecomment-1735602069 Hey @showuon > Add hardcoded 5 for decompression, so, set the max message size to 202. After decompressed, the record size is 203, throw exception, while before bumping snap

[GitHub] [kafka] lianetm commented on a diff in pull request #14364: KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests

2023-09-26 Thread via GitHub
lianetm commented on code in PR #14364: URL: https://github.com/apache/kafka/pull/14364#discussion_r1337258265 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -0,0 +1,324 @@ +/* + * Licensed to the Apache Software Foundati

[GitHub] [kafka] lianetm commented on a diff in pull request #14364: KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests

2023-09-26 Thread via GitHub
lianetm commented on code in PR #14364: URL: https://github.com/apache/kafka/pull/14364#discussion_r1337239851 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -0,0 +1,324 @@ +/* + * Licensed to the Apache Software Foundati

[GitHub] [kafka] lianetm commented on a diff in pull request #14364: KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests

2023-09-26 Thread via GitHub
lianetm commented on code in PR #14364: URL: https://github.com/apache/kafka/pull/14364#discussion_r1337238424 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -0,0 +1,324 @@ +/* + * Licensed to the Apache Software Foundati

[GitHub] [kafka] lianetm commented on a diff in pull request #14364: KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests

2023-09-26 Thread via GitHub
lianetm commented on code in PR #14364: URL: https://github.com/apache/kafka/pull/14364#discussion_r1337235694 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -0,0 +1,324 @@ +/* + * Licensed to the Apache Software Foundati

[GitHub] [kafka] lianetm commented on a diff in pull request #14364: KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests

2023-09-26 Thread via GitHub
lianetm commented on code in PR #14364: URL: https://github.com/apache/kafka/pull/14364#discussion_r1337234985 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -0,0 +1,324 @@ +/* + * Licensed to the Apache Software Foundati

[GitHub] [kafka] cadonna commented on pull request #14265: KAFKA-10199: Do not process when in PARTITIONS_REVOKED

2023-09-26 Thread via GitHub
cadonna commented on PR #14265: URL: https://github.com/apache/kafka/pull/14265#issuecomment-173554 Build failures are unrelated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [kafka] cadonna merged pull request #14265: KAFKA-10199: Do not process when in PARTITIONS_REVOKED

2023-09-26 Thread via GitHub
cadonna merged PR #14265: URL: https://github.com/apache/kafka/pull/14265 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

[GitHub] [kafka] kamalcph commented on pull request #14439: KAFKA-15499: Fix the flaky DeleteSegmentsDueToLogStartOffsetBreach test.

2023-09-26 Thread via GitHub
kamalcph commented on PR #14439: URL: https://github.com/apache/kafka/pull/14439#issuecomment-1735500717 The test failures are unrelated. @showuon @divijvaidya Call for review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

[GitHub] [kafka] showuon commented on pull request #14434: KAFKA-15498: bump snappy-java version to 1.1.10.4

2023-09-26 Thread via GitHub
showuon commented on PR #14434: URL: https://github.com/apache/kafka/pull/14434#issuecomment-1735496347 One question I'd like to get your thoughts. This test is doing: 1. Generate a 2 byte key, and 128 byte value. 2. Create a record using snappy codec compressed. 3. Get the size of t

[GitHub] [kafka] lucasbru commented on a diff in pull request #14454: KAFKA-15344: Commit leader epoch where possible

2023-09-26 Thread via GitHub
lucasbru commented on code in PR #14454: URL: https://github.com/apache/kafka/pull/14454#discussion_r1337139205 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamTask.java: ## @@ -431,11 +433,20 @@ public Map prepareCommit() { } } -

[GitHub] [kafka] showuon commented on pull request #14434: KAFKA-15498: bump snappy-java version to 1.1.10.4

2023-09-26 Thread via GitHub
showuon commented on PR #14434: URL: https://github.com/apache/kafka/pull/14434#issuecomment-1735456298 I'll try to investigate more tomorrow, and try to write a simple test case in snappy repo to simulate our test case. But again, if somebody on other timezones wants to take a stab at it,

[GitHub] [kafka] lucasbru opened a new pull request, #14454: KAFKA-15344: Commit leader epoch where possible

2023-09-26 Thread via GitHub
lucasbru opened a new pull request, #14454: URL: https://github.com/apache/kafka/pull/14454 Kafka Streams needs to include the leader epoch when committing offsets. Leader epoch is required to detect situations where a consumer with outdated metadata is trying to fetch the committed o

[jira] [Commented] (KAFKA-13882) Dockerfile for previewing website

2023-09-26 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17769160#comment-17769160 ] ASF GitHub Bot commented on KAFKA-13882: divijvaidya commented on PR #410: URL:

[GitHub] [kafka] cadonna commented on pull request #14437: KAFKA-10199: Fix restoration behavior for paused tasks

2023-09-26 Thread via GitHub
cadonna commented on PR #14437: URL: https://github.com/apache/kafka/pull/14437#issuecomment-1735413305 Build failures are unrelated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [kafka] cadonna merged pull request #14437: KAFKA-10199: Fix restoration behavior for paused tasks

2023-09-26 Thread via GitHub
cadonna merged PR #14437: URL: https://github.com/apache/kafka/pull/14437 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

[jira] [Created] (KAFKA-15508) Method always return the same value ApplicationEventProcessor.java

2023-09-26 Thread Svyatoslav (Jira)
Svyatoslav created KAFKA-15508: -- Summary: Method always return the same value ApplicationEventProcessor.java Key: KAFKA-15508 URL: https://issues.apache.org/jira/browse/KAFKA-15508 Project: Kafka

[GitHub] [kafka] divijvaidya commented on a diff in pull request #14453: MINOR: Close UnifiedLog created in tests to avoid resource leak

2023-09-26 Thread via GitHub
divijvaidya commented on code in PR #14453: URL: https://github.com/apache/kafka/pull/14453#discussion_r1337054159 ## core/src/test/scala/unit/kafka/log/LogLoaderTest.scala: ## @@ -1800,5 +1822,6 @@ class LogLoaderTest { isRemoteLogEnabled = isRemoteLogEnabled ).load

[jira] [Commented] (KAFKA-15169) Add tests for RemoteIndexCache

2023-09-26 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17769119#comment-17769119 ] Divij Vaidya commented on KAFKA-15169: -- First some correction on what I said above.

[jira] [Assigned] (KAFKA-15507) adminClient should not throw retriable exception when closing instance

2023-09-26 Thread Gantigmaa Selenge (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gantigmaa Selenge reassigned KAFKA-15507: - Assignee: Gantigmaa Selenge > adminClient should not throw retriable exception

[GitHub] [kafka] jlprat commented on pull request #14434: KAFKA-15498: bump snappy-java version to 1.1.10.4

2023-09-26 Thread via GitHub
jlprat commented on PR #14434: URL: https://github.com/apache/kafka/pull/14434#issuecomment-1735292502 I spent some little time, but I couldn't find the root cause of why this is failing. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[jira] [Comment Edited] (KAFKA-15169) Add tests for RemoteIndexCache

2023-09-26 Thread Arpit Goyal (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17769103#comment-17769103 ] Arpit Goyal edited comment on KAFKA-15169 at 9/26/23 10:42 AM: ---

[jira] [Commented] (KAFKA-15169) Add tests for RemoteIndexCache

2023-09-26 Thread Arpit Goyal (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17769103#comment-17769103 ] Arpit Goyal commented on KAFKA-15169: - [~divijvaidya] Just to confirm what I underst

[GitHub] [kafka] tinaselenge opened a new pull request, #14453: MINOR: Close UnifiedLog created in tests to avoid resource leak

2023-09-26 Thread via GitHub
tinaselenge opened a new pull request, #14453: URL: https://github.com/apache/kafka/pull/14453 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of tes

[GitHub] [kafka] divijvaidya commented on a diff in pull request #14425: KAFKA-15401: Segment with corrupted index should not be uploaded to r…

2023-09-26 Thread via GitHub
divijvaidya commented on code in PR #14425: URL: https://github.com/apache/kafka/pull/14425#discussion_r1336965148 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -697,12 +711,22 @@ public void copyLogSegmentsToRemote(UnifiedLog log) throws InterruptedExcept

[GitHub] [kafka] divijvaidya commented on a diff in pull request #14425: KAFKA-15401: Segment with corrupted index should not be uploaded to r…

2023-09-26 Thread via GitHub
divijvaidya commented on code in PR #14425: URL: https://github.com/apache/kafka/pull/14425#discussion_r1336950457 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -697,12 +711,22 @@ public void copyLogSegmentsToRemote(UnifiedLog log) throws InterruptedExcept

[jira] [Commented] (KAFKA-15507) adminClient should not throw retriable exception when closing instance

2023-09-26 Thread Keith Wall (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17769087#comment-17769087 ] Keith Wall commented on KAFKA-15507: Thank you for raising [~showuon]  When I was l

[jira] [Commented] (KAFKA-15169) Add tests for RemoteIndexCache

2023-09-26 Thread Arpit Goyal (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17769086#comment-17769086 ] Arpit Goyal commented on KAFKA-15169: - Thanks [~divijvaidya] I have two questions

[jira] [Commented] (KAFKA-15169) Add tests for RemoteIndexCache

2023-09-26 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17769072#comment-17769072 ] Divij Vaidya commented on KAFKA-15169: -- Separately, while you are writing tests for

[jira] [Commented] (KAFKA-15169) Add tests for RemoteIndexCache

2023-09-26 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17769071#comment-17769071 ] Divij Vaidya commented on KAFKA-15169: -- Hey Arpit Asserting the sanity of the inde

[jira] [Resolved] (KAFKA-15487) CVE-2023-40167, CVE-2023-36479 - Upgrade jetty to 9.4.52, 10.0.16, 11.0.16, 12.0.1

2023-09-26 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya resolved KAFKA-15487. -- Resolution: Fixed > CVE-2023-40167, CVE-2023-36479 - Upgrade jetty to 9.4.52, 10.0.16, 11.0.16

[jira] [Commented] (KAFKA-15487) CVE-2023-40167, CVE-2023-36479 - Upgrade jetty to 9.4.52, 10.0.16, 11.0.16, 12.0.1

2023-09-26 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17769066#comment-17769066 ] Divij Vaidya commented on KAFKA-15487: -- We have backported this to all community su

[GitHub] [kafka] lucasbru merged pull request #14403: KAFKA-10199: Add missing catch for lock exception

2023-09-26 Thread via GitHub
lucasbru merged PR #14403: URL: https://github.com/apache/kafka/pull/14403 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

[jira] [Updated] (KAFKA-15487) CVE-2023-40167, CVE-2023-36479 - Upgrade jetty to 9.4.52, 10.0.16, 11.0.16, 12.0.1

2023-09-26 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya updated KAFKA-15487: - Fix Version/s: 3.4.2 > CVE-2023-40167, CVE-2023-36479 - Upgrade jetty to 9.4.52, 10.0.16, 11.0.1

[GitHub] [kafka] lucasbru commented on pull request #14403: KAFKA-10199: Add missing catch for lock exception

2023-09-26 Thread via GitHub
lucasbru commented on PR #14403: URL: https://github.com/apache/kafka/pull/14403#issuecomment-1735115556 build failures are unrelated -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [kafka] divijvaidya commented on pull request #14438: KAFKA-15487: Upgrade Jetty to 9.4.52.v20230823

2023-09-26 Thread via GitHub
divijvaidya commented on PR #14438: URL: https://github.com/apache/kafka/pull/14438#issuecomment-1735115448 Since this PR addresses a CVE, I have backported this fix to all community supported version branches i.e. 3.4 and 3.5 (in addition to 3.6 backported by Satish) -- This is an autom

[GitHub] [kafka] lucasbru merged pull request #14436: MINOR: Revert log level changes in LogCaptureAppender

2023-09-26 Thread via GitHub
lucasbru merged PR #14436: URL: https://github.com/apache/kafka/pull/14436 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

[GitHub] [kafka] lucasbru commented on pull request #14436: MINOR: Revert log level changes in LogCaptureAppender

2023-09-26 Thread via GitHub
lucasbru commented on PR #14436: URL: https://github.com/apache/kafka/pull/14436#issuecomment-1735100693 Build failures are unrelated -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[jira] [Updated] (KAFKA-15487) CVE-2023-40167, CVE-2023-36479 - Upgrade jetty to 9.4.52, 10.0.16, 11.0.16, 12.0.1

2023-09-26 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya updated KAFKA-15487: - Fix Version/s: 3.6.0 3.5.2 (was: 3.0.0)

[jira] [Resolved] (KAFKA-15504) Upgrade snappy java to version 1.1.10.4

2023-09-26 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya resolved KAFKA-15504. -- Resolution: Duplicate Closing as duplicate of prior ticket https://issues.apache.org/jira/bro

[GitHub] [kafka] divijvaidya closed pull request #14450: KAFKA-15504: Upgrade snappy java to version 1.1.10.4

2023-09-26 Thread via GitHub
divijvaidya closed pull request #14450: KAFKA-15504: Upgrade snappy java to version 1.1.10.4 URL: https://github.com/apache/kafka/pull/14450 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

[GitHub] [kafka] divijvaidya commented on pull request #14450: KAFKA-15504: Upgrade snappy java to version 1.1.10.4

2023-09-26 Thread via GitHub
divijvaidya commented on PR #14450: URL: https://github.com/apache/kafka/pull/14450#issuecomment-1735075163 Hey @bmscomp We already have a prior PR and JIRA (https://issues.apache.org/jira/browse/KAFKA-15498) for this change https://github.com/apache/kafka/pull/14434 I am goi

  1   2   >