Re: [PR] KAFKA-15571: `StateRestoreListener#onRestoreSuspended` is never called because `DelegatingStateRestoreListener` doesn't implement `onRestoreSuspended` [kafka]

2023-10-12 Thread via GitHub
ableegoldman commented on PR #14519: URL: https://github.com/apache/kafka/pull/14519#issuecomment-1759052065 I would personally love to see a 3.5.2 release, but having managed both a bugfix release myself in the past, I can attest that it's not a trivial amount of work. Usually we don't do

Re: [PR] KAFKA-15062: Adding ppc64le build stage [kafka]

2023-10-12 Thread via GitHub
Vaibhav-Nazare commented on PR #13817: URL: https://github.com/apache/kafka/pull/13817#issuecomment-1759053190 Hi @mimaison I have updated the stage for PowerPC as of now, for the Java Version upgrade I need to figure out how to do it, may be we can have it in another PR? -- This is an

Re: [PR] KAFKA-15571: `StateRestoreListener#onRestoreSuspended` is never called because `DelegatingStateRestoreListener` doesn't implement `onRestoreSuspended` [kafka]

2023-10-12 Thread via GitHub
lkokhreidze commented on PR #14519: URL: https://github.com/apache/kafka/pull/14519#issuecomment-1759069004 Thanks Sophie. I will do that. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

Re: [PR] KAFKA-14519; [1/N] Implement coordinator runtime metrics [kafka]

2023-10-12 Thread via GitHub
dajac commented on code in PR #14417: URL: https://github.com/apache/kafka/pull/14417#discussion_r1356448238 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/runtime/CoordinatorLoader.java: ## @@ -46,6 +46,50 @@ public short unknownType() { } }

Re: [PR] KAFKA-15578: System Tests for running old protocol with new coordinator [kafka]

2023-10-12 Thread via GitHub
dajac commented on code in PR #14524: URL: https://github.com/apache/kafka/pull/14524#discussion_r1356456410 ## tests/kafkatest/tests/core/consume_bench_test.py: ## @@ -44,7 +44,6 @@ def setUp(self): self.trogdor.start() if self.zk: self.zk.start()

Re: [PR] KAFKA-15578: System Tests for running old protocol with new coordinator [kafka]

2023-10-12 Thread via GitHub
dajac commented on code in PR #14524: URL: https://github.com/apache/kafka/pull/14524#discussion_r1356464517 ## tests/kafkatest/tests/core/consume_bench_test.py: ## @@ -68,12 +67,21 @@ def produce_messages(self, topics, max_messages=1): self.logger.debug("Produce wo

Re: [PR] KAFKA-15578: System Tests for running old protocol with new coordinator [kafka]

2023-10-12 Thread via GitHub
dajac commented on code in PR #14524: URL: https://github.com/apache/kafka/pull/14524#discussion_r1356465229 ## tests/kafkatest/services/kafka/config_property.py: ## @@ -199,6 +200,9 @@ val SSLEndpointIdentificationAlgorithmProp = SSLConfigs.SSL_ENDPOINT_IDENTIFICATION_ALGOR

Re: [PR] KAFKA-15578: System Tests for running old protocol with new coordinator [kafka]

2023-10-12 Thread via GitHub
dajac commented on code in PR #14524: URL: https://github.com/apache/kafka/pull/14524#discussion_r1356466186 ## tests/kafkatest/tests/core/consume_bench_test.py: ## @@ -23,14 +23,14 @@ from kafkatest.services.trogdor.task_spec import TaskSpec from kafkatest.services.trogdor.tr

Re: [PR] KAFKA-15578: System Tests for running old protocol with new coordinator [kafka]

2023-10-12 Thread via GitHub
dajac commented on code in PR #14524: URL: https://github.com/apache/kafka/pull/14524#discussion_r1356467053 ## tests/kafkatest/tests/core/consume_bench_test.py: ## @@ -68,12 +67,28 @@ def produce_messages(self, topics, max_messages=1): self.logger.debug("Produce wo

Re: [PR] KAFKA-15578: System Tests for running old protocol with new coordinator [kafka]

2023-10-12 Thread via GitHub
dajac commented on code in PR #14524: URL: https://github.com/apache/kafka/pull/14524#discussion_r1356465862 ## tests/kafkatest/services/kafka/util.py: ## @@ -12,9 +12,11 @@ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the sp

Re: [PR] KAFKA-15492: Enable spotbugs when building with Java 21 [kafka]

2023-10-12 Thread via GitHub
ijuma commented on PR #14533: URL: https://github.com/apache/kafka/pull/14533#issuecomment-1759177306 Thanks for the review. In general, we have assumed UTF-8 since that's the default for Linux and MacOS. It has become the default for all platforms since Java 18: https://openjdk.org/jeps/40

Re: [PR] KAFKA-15355: Message schema changes [kafka]

2023-10-12 Thread via GitHub
clolov commented on code in PR #14290: URL: https://github.com/apache/kafka/pull/14290#discussion_r1356485990 ## generator/src/main/java/org/apache/kafka/message/FieldSpec.java: ## @@ -157,11 +157,6 @@ private void checkTagInvariants() { this.taggedVersions

Re: [PR] KAFKA-15570: Add unit tests for MemoryConfigBackingStore [kafka]

2023-10-12 Thread via GitHub
yashmayya commented on PR #14518: URL: https://github.com/apache/kafka/pull/14518#issuecomment-1759205904 Test failures are unrelated, merging to `trunk`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

Re: [PR] KAFKA-15570: Add unit tests for MemoryConfigBackingStore [kafka]

2023-10-12 Thread via GitHub
yashmayya merged PR #14518: URL: https://github.com/apache/kafka/pull/14518 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apach

Re: [PR] MINOR: Only commit running active and standby tasks when tasks corrupted [kafka]

2023-10-12 Thread via GitHub
lucasbru commented on code in PR #14508: URL: https://github.com/apache/kafka/pull/14508#discussion_r1356569841 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java: ## @@ -223,10 +223,7 @@ boolean handleCorruption(final Set corruptedTasks) {

Re: [PR] KAFKA-15489: resign leadership when no fetch from majority voters [kafka]

2023-10-12 Thread via GitHub
showuon commented on PR #14428: URL: https://github.com/apache/kafka/pull/14428#issuecomment-1759306603 > I'm wondering if we may start causing leaders to resign when followers are slow/backlogged and make the situation worse? E.g. if we have multiple followers that need to catch up via a l

Re: [PR] MINOR: add quick start for tiered storage feature [kafka]

2023-10-12 Thread via GitHub
showuon commented on PR #14528: URL: https://github.com/apache/kafka/pull/14528#issuecomment-1759309460 @divijvaidya , they are all good suggestions. Will update the PR later. Thanks all for the comments. -- This is an automated message from the Apache Git Service. To respond to the messa

Re: [PR] KAFKA-15498: bump snappy-java version to 1.1.10.5 [kafka]

2023-10-12 Thread via GitHub
mimaison commented on PR #14458: URL: https://github.com/apache/kafka/pull/14458#issuecomment-1759316582 Backported to 3.5 too -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[jira] [Updated] (KAFKA-15498) Upgrade Snappy-Java to 1.1.10.4

2023-10-12 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mickael Maison updated KAFKA-15498: --- Fix Version/s: 3.5.2 > Upgrade Snappy-Java to 1.1.10.4 > --- > >

[jira] [Created] (KAFKA-15595) Session window aggregate drops records headers

2023-10-12 Thread Abdullah alkhawatrah (Jira)
Abdullah alkhawatrah created KAFKA-15595: Summary: Session window aggregate drops records headers Key: KAFKA-15595 URL: https://issues.apache.org/jira/browse/KAFKA-15595 Project: Kafka

[jira] [Created] (KAFKA-15596) Upgrade ZooKeeper to 3.8.3

2023-10-12 Thread Mickael Maison (Jira)
Mickael Maison created KAFKA-15596: -- Summary: Upgrade ZooKeeper to 3.8.3 Key: KAFKA-15596 URL: https://issues.apache.org/jira/browse/KAFKA-15596 Project: Kafka Issue Type: Improvement

[jira] [Created] (KAFKA-15597) Allow Connect DropHeaders SMT remove headers on a wildcard-basis

2023-10-12 Thread Roman Schmitz (Jira)
Roman Schmitz created KAFKA-15597: - Summary: Allow Connect DropHeaders SMT remove headers on a wildcard-basis Key: KAFKA-15597 URL: https://issues.apache.org/jira/browse/KAFKA-15597 Project: Kafka

[jira] [Updated] (KAFKA-15597) Allow Connect DropHeaders SMT remove headers on a wildcard/regexp-basis

2023-10-12 Thread Roman Schmitz (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Schmitz updated KAFKA-15597: -- Summary: Allow Connect DropHeaders SMT remove headers on a wildcard/regexp-basis (was: Allow

Re: [PR] MINOR - KAFKA-15550: Validation for negative target times in offsetsForTimes [kafka]

2023-10-12 Thread via GitHub
lucasbru commented on code in PR #14503: URL: https://github.com/apache/kafka/pull/14503#discussion_r1356610621 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/PrototypeAsyncConsumer.java: ## @@ -252,8 +255,10 @@ private boolean updateFetchPositionsIfNeeded(

[jira] [Updated] (KAFKA-15595) Session window aggregate drops records headers

2023-10-12 Thread Abdullah alkhawatrah (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abdullah alkhawatrah updated KAFKA-15595: - Description: Hey, While upgrading to 3.5.1 from 3.2.X I noticed a change in Ses

[jira] [Updated] (KAFKA-15595) Session window aggregate drops records headers

2023-10-12 Thread Abdullah alkhawatrah (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abdullah alkhawatrah updated KAFKA-15595: - Description: Hey, While upgrading to 3.5.1 from 3.2.X I noticed a change in Ses

Re: [PR] KAFKA-14481: Move LogSegment/LogSegments to storage module [kafka]

2023-10-12 Thread via GitHub
divijvaidya commented on code in PR #14529: URL: https://github.com/apache/kafka/pull/14529#discussion_r1356629072 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -733,8 +733,8 @@ private void copyLogSegment(UnifiedLog log, LogSegment segment, long nextSegme

[PR] KAFKA-15596: Upgrade ZooKeeper to 3.8.3 [kafka]

2023-10-12 Thread via GitHub
mimaison opened a new pull request, #14535: URL: https://github.com/apache/kafka/pull/14535 ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgr

Re: [PR] MINOR - KAFKA-15550: Validation for negative target times in offsetsForTimes [kafka]

2023-10-12 Thread via GitHub
lucasbru commented on code in PR #14503: URL: https://github.com/apache/kafka/pull/14503#discussion_r1356663540 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/PrototypeAsyncConsumerTest.java: ## @@ -322,6 +323,25 @@ public void testOffsetsForTimesOnNullPart

Re: [PR] MINOR: Only commit running active and standby tasks when tasks corrupted [kafka]

2023-10-12 Thread via GitHub
cadonna commented on code in PR #14508: URL: https://github.com/apache/kafka/pull/14508#discussion_r1356667200 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java: ## @@ -223,10 +223,7 @@ boolean handleCorruption(final Set corruptedTasks) {

Re: [PR] MINOR: Only commit running active and standby tasks when tasks corrupted [kafka]

2023-10-12 Thread via GitHub
cadonna merged PR #14508: URL: https://github.com/apache/kafka/pull/14508 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

[PR] KAFKA-15597: Allow Connect DropHeaders SMT remove headers on a wildcard/regexp-basis [kafka]

2023-10-12 Thread via GitHub
Schm1tz1 opened a new pull request, #14536: URL: https://github.com/apache/kafka/pull/14536 In many use cases you might not only want to drop a few specific Kafka headers but a set of headers whose names can also dynamically change (e.g. when used with some end-to-end-encryption libraries).

[jira] [Updated] (KAFKA-15597) Allow Connect DropHeaders SMT remove headers on a wildcard/regexp-basis

2023-10-12 Thread Roman Schmitz (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Schmitz updated KAFKA-15597: -- Description: In many use cases you might not only want to drop a few specific Kafka headers b

[jira] [Updated] (KAFKA-15597) Allow Connect DropHeaders SMT remove headers on a wildcard/regexp-basis

2023-10-12 Thread Roman Schmitz (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Schmitz updated KAFKA-15597: -- Labels: connect-transformation (was: ) > Allow Connect DropHeaders SMT remove headers on a wi

Re: [PR] KAFKA-15492: Enable spotbugs when building with Java 21 [kafka]

2023-10-12 Thread via GitHub
divijvaidya commented on PR #14533: URL: https://github.com/apache/kafka/pull/14533#issuecomment-1759475621 Build is successful. Unrelated test failures: ``` [Build / JDK 21 and Scala 2.13 / org.apache.kafka.streams.integration.NamedTopologyIntegrationTest.shouldAddNamedTopologyToRunni

Re: [PR] KAFKA-15492: Enable spotbugs when building with Java 21 [kafka]

2023-10-12 Thread via GitHub
divijvaidya merged PR #14533: URL: https://github.com/apache/kafka/pull/14533 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apa

[jira] [Resolved] (KAFKA-15492) Enable spotbugs when building with Java 21

2023-10-12 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya resolved KAFKA-15492. -- Resolution: Fixed > Enable spotbugs when building with Java 21 > -

[jira] [Assigned] (KAFKA-15597) Allow Connect DropHeaders SMT remove headers on a wildcard/regexp-basis

2023-10-12 Thread Roman Schmitz (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Schmitz reassigned KAFKA-15597: - Assignee: Roman Schmitz > Allow Connect DropHeaders SMT remove headers on a wildcard/re

[jira] [Updated] (KAFKA-15597) Enable Connect DropHeaders SMT to drop headers on a wildcard/regexp-basis

2023-10-12 Thread Roman Schmitz (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Schmitz updated KAFKA-15597: -- Description: In many use cases you might not only want to drop a few specific Kafka headers b

[jira] [Commented] (KAFKA-15330) Migration from ZK to KRaft works with 3.4 but fails from 3.5 upwards

2023-10-12 Thread Roland Sommer (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774491#comment-17774491 ] Roland Sommer commented on KAFKA-15330: --- I tried the migration in our staging envi

[jira] [Updated] (KAFKA-15597) Enable Connect DropHeaders SMT to drop headers on a wildcard/regexp-basis

2023-10-12 Thread Roman Schmitz (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Schmitz updated KAFKA-15597: -- Summary: Enable Connect DropHeaders SMT to drop headers on a wildcard/regexp-basis (was: Allo

[jira] [Updated] (KAFKA-15597) Enable Connect DropHeaders SMT to drop headers on a wildcard/regexp-basis

2023-10-12 Thread Roman Schmitz (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Schmitz updated KAFKA-15597: -- Description: In many use cases you might not only want to drop a few specific Kafka headers b

Re: [PR] KAFKA-15481: Fix concurrency bug in RemoteIndexCache [kafka]

2023-10-12 Thread via GitHub
jeel2420 commented on code in PR #14483: URL: https://github.com/apache/kafka/pull/14483#discussion_r1356780374 ## core/src/test/scala/unit/kafka/log/remote/RemoteIndexCacheTest.scala: ## @@ -132,7 +133,7 @@ class RemoteIndexCacheTest { // this call should have invoked fetc

Re: [PR] KAFKA-15481: Fix concurrency bug in RemoteIndexCache [kafka]

2023-10-12 Thread via GitHub
jeel2420 commented on code in PR #14483: URL: https://github.com/apache/kafka/pull/14483#discussion_r1348624218 ## core/src/test/scala/unit/kafka/log/remote/RemoteIndexCacheTest.scala: ## @@ -568,27 +621,26 @@ class RemoteIndexCacheTest { } private def verifyFetchIndexIn

[jira] [Assigned] (KAFKA-15597) Enable Connect DropHeaders SMT to drop headers on a wildcard/regexp-basis

2023-10-12 Thread Roman Schmitz (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Schmitz reassigned KAFKA-15597: - Assignee: (was: Roman Schmitz) > Enable Connect DropHeaders SMT to drop headers on

Re: [PR] KAFKA-15536:Dynamically resize remoteIndexCache [kafka]

2023-10-12 Thread via GitHub
divijvaidya commented on code in PR #14511: URL: https://github.com/apache/kafka/pull/14511#discussion_r1356830611 ## storage/src/main/java/org/apache/kafka/storage/internals/log/RemoteIndexCache.java: ## @@ -137,15 +138,8 @@ public RemoteIndexCache(long maxSize, RemoteStorageMa

Re: [PR] MINOR - KAFKA-15550: Validation for negative target times in offsetsForTimes [kafka]

2023-10-12 Thread via GitHub
lianetm commented on code in PR #14503: URL: https://github.com/apache/kafka/pull/14503#discussion_r1356848021 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/PrototypeAsyncConsumer.java: ## @@ -235,15 +235,18 @@ public ConsumerRecords poll(final Duration ti

[jira] [Created] (KAFKA-15598) Add integration tests for DescribeGroups API, DeleteGroups API and OffsetDelete API

2023-10-12 Thread Dongnuo Lyu (Jira)
Dongnuo Lyu created KAFKA-15598: --- Summary: Add integration tests for DescribeGroups API, DeleteGroups API and OffsetDelete API Key: KAFKA-15598 URL: https://issues.apache.org/jira/browse/KAFKA-15598 Pro

[PR] KAFKA-15598: Add integration tests for DescribeGroups API, DeleteGroups API and OffsetDelete API [kafka]

2023-10-12 Thread via GitHub
dongnuo123 opened a new pull request, #14537: URL: https://github.com/apache/kafka/pull/14537 This pr includes integration tests for three group coordinator APIs. - DescribeGroups - DeleteGroups - OffsetDelete ### Jira https://issues.apache.org/jira/browse/KAFKA-15598

Re: [PR] MINOR - KAFKA-15550: Validation for negative target times in offsetsForTimes [kafka]

2023-10-12 Thread via GitHub
lianetm commented on code in PR #14503: URL: https://github.com/apache/kafka/pull/14503#discussion_r1356866475 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/PrototypeAsyncConsumerTest.java: ## @@ -322,6 +323,25 @@ public void testOffsetsForTimesOnNullParti

Re: [PR] MINOR - KAFKA-15550: Validation for negative target times in offsetsForTimes [kafka]

2023-10-12 Thread via GitHub
lianetm commented on PR #14503: URL: https://github.com/apache/kafka/pull/14503#issuecomment-1759658605 hey @lucasbru, thanks a lot for all the comments, all addressed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

Re: [PR] KAFKA-15581: Introduce ELR [kafka]

2023-10-12 Thread via GitHub
mumrah commented on code in PR #14312: URL: https://github.com/apache/kafka/pull/14312#discussion_r1349018067 ## metadata/src/main/java/org/apache/kafka/controller/QuorumController.java: ## @@ -203,6 +203,7 @@ static public class Builder { private QuorumFeatures quorumF

[jira] [Resolved] (KAFKA-14506) Implement DeleteGroups API and OffsetDelete API

2023-10-12 Thread David Jacot (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-14506. - Reviewer: David Jacot Resolution: Fixed > Implement DeleteGroups API and OffsetDelete API

Re: [PR] KAFKA-14481: Move LogSegment/LogSegments to storage module [kafka]

2023-10-12 Thread via GitHub
ijuma commented on code in PR #14529: URL: https://github.com/apache/kafka/pull/14529#discussion_r1356880611 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -733,8 +733,8 @@ private void copyLogSegment(UnifiedLog log, LogSegment segment, long nextSegment

[jira] [Resolved] (KAFKA-14504) Implement DescribeGroups API

2023-10-12 Thread David Jacot (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-14504. - Reviewer: David Jacot Resolution: Fixed > Implement DescribeGroups API > -

Re: [PR] KAFKA-14481: Move LogSegment/LogSegments to storage module [kafka]

2023-10-12 Thread via GitHub
ijuma commented on code in PR #14529: URL: https://github.com/apache/kafka/pull/14529#discussion_r1356881661 ## storage/src/main/java/org/apache/kafka/storage/internals/log/LazyIndex.java: ## @@ -41,7 +41,7 @@ * Methods of this class are thread safe. Make sure to check `Abstra

Re: [PR] KAFKA-14481: Move LogSegment/LogSegments to storage module [kafka]

2023-10-12 Thread via GitHub
ijuma commented on code in PR #14529: URL: https://github.com/apache/kafka/pull/14529#discussion_r1356881661 ## storage/src/main/java/org/apache/kafka/storage/internals/log/LazyIndex.java: ## @@ -41,7 +41,7 @@ * Methods of this class are thread safe. Make sure to check `Abstra

Re: [PR] KAFKA-14481: Move LogSegment/LogSegments to storage module [kafka]

2023-10-12 Thread via GitHub
ijuma commented on code in PR #14529: URL: https://github.com/apache/kafka/pull/14529#discussion_r1356883718 ## storage/src/main/java/org/apache/kafka/storage/internals/log/LogSegment.java: ## @@ -0,0 +1,873 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one o

Re: [PR] KAFKA-14481: Move LogSegment/LogSegments to storage module [kafka]

2023-10-12 Thread via GitHub
ijuma commented on code in PR #14529: URL: https://github.com/apache/kafka/pull/14529#discussion_r1356883718 ## storage/src/main/java/org/apache/kafka/storage/internals/log/LogSegment.java: ## @@ -0,0 +1,873 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one o

Re: [PR] KAFKA-14481: Move LogSegment/LogSegments to storage module [kafka]

2023-10-12 Thread via GitHub
ijuma commented on code in PR #14529: URL: https://github.com/apache/kafka/pull/14529#discussion_r1356884927 ## storage/src/main/java/org/apache/kafka/storage/internals/log/LogSegment.java: ## @@ -0,0 +1,873 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one o

Re: [PR] KAFKA-14481: Move LogSegment/LogSegments to storage module [kafka]

2023-10-12 Thread via GitHub
ijuma commented on code in PR #14529: URL: https://github.com/apache/kafka/pull/14529#discussion_r1356886567 ## storage/src/main/java/org/apache/kafka/storage/internals/log/LogSegment.java: ## @@ -0,0 +1,873 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one o

Re: [PR] KAFKA-14481: Move LogSegment/LogSegments to storage module [kafka]

2023-10-12 Thread via GitHub
ijuma commented on code in PR #14529: URL: https://github.com/apache/kafka/pull/14529#discussion_r1356889538 ## storage/src/main/java/org/apache/kafka/storage/internals/log/LogSegment.java: ## @@ -0,0 +1,873 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one o

Re: [PR] KAFKA-14481: Move LogSegment/LogSegments to storage module [kafka]

2023-10-12 Thread via GitHub
ijuma commented on code in PR #14529: URL: https://github.com/apache/kafka/pull/14529#discussion_r1356893064 ## storage/src/main/java/org/apache/kafka/storage/internals/log/LogSegment.java: ## @@ -0,0 +1,873 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one o

Re: [PR] KAFKA-14481: Move LogSegment/LogSegments to storage module [kafka]

2023-10-12 Thread via GitHub
ijuma commented on code in PR #14529: URL: https://github.com/apache/kafka/pull/14529#discussion_r1356894691 ## clients/src/main/java/org/apache/kafka/common/record/Records.java: ## @@ -70,6 +71,13 @@ public interface Records extends TransferableRecords { */ AbstractI

Re: [PR] KAFKA-14481: Move LogSegment/LogSegments to storage module [kafka]

2023-10-12 Thread via GitHub
ijuma commented on code in PR #14529: URL: https://github.com/apache/kafka/pull/14529#discussion_r1356896786 ## clients/src/main/java/org/apache/kafka/common/record/AbstractRecords.java: ## @@ -44,6 +45,17 @@ public RecordBatch firstBatch() { return iterator.next();

Re: [PR] KAFKA-14481: Move LogSegment/LogSegments to storage module [kafka]

2023-10-12 Thread via GitHub
ijuma commented on PR #14529: URL: https://github.com/apache/kafka/pull/14529#issuecomment-1759698972 Thanks for the prompt review @divijvaidya! Please note that this is still in draft because there are still failing tests. Since it was still in draft mode, I have been force pushing and so

Re: [PR] KAFKA-14481: Move LogSegment/LogSegments to storage module [kafka]

2023-10-12 Thread via GitHub
divijvaidya commented on code in PR #14529: URL: https://github.com/apache/kafka/pull/14529#discussion_r1356912887 ## storage/src/main/java/org/apache/kafka/storage/internals/log/LazyIndex.java: ## @@ -41,7 +41,7 @@ * Methods of this class are thread safe. Make sure to check `

[jira] [Commented] (KAFKA-15143) MockFixedKeyProcessorContext is missing from test-utils

2023-10-12 Thread Shashwat Pandey (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774558#comment-17774558 ] Shashwat Pandey commented on KAFKA-15143: - I am also having the same problem (an

Re: [PR] KAFKA-14481: Move LogSegment/LogSegments to storage module [kafka]

2023-10-12 Thread via GitHub
ijuma commented on code in PR #14529: URL: https://github.com/apache/kafka/pull/14529#discussion_r1356961289 ## core/src/test/scala/unit/kafka/log/LogCleanerParameterizedIntegrationTest.scala: ## @@ -120,12 +120,13 @@ class LogCleanerParameterizedIntegrationTest extends Abstrac

Re: [PR] KAFKA-14481: Move LogSegment/LogSegments to storage module [kafka]

2023-10-12 Thread via GitHub
ijuma commented on code in PR #14529: URL: https://github.com/apache/kafka/pull/14529#discussion_r1356966266 ## storage/src/main/java/org/apache/kafka/storage/internals/log/LogSegment.java: ## @@ -0,0 +1,873 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one o

Re: [PR] KAFKA-14481: Move LogSegment/LogSegments to storage module [kafka]

2023-10-12 Thread via GitHub
ijuma commented on code in PR #14529: URL: https://github.com/apache/kafka/pull/14529#discussion_r1356970370 ## storage/src/main/java/org/apache/kafka/storage/internals/log/LogSegment.java: ## @@ -0,0 +1,887 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one o

Re: [PR] KAFKA-14481: Move LogSegment/LogSegments to storage module [kafka]

2023-10-12 Thread via GitHub
ijuma commented on PR #14529: URL: https://github.com/apache/kafka/pull/14529#issuecomment-1759795074 I believe the tests should pass this time, let's see. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] KAFKA-14481: Move LogSegment/LogSegments to storage module [kafka]

2023-10-12 Thread via GitHub
ijuma commented on code in PR #14529: URL: https://github.com/apache/kafka/pull/14529#discussion_r1356974305 ## storage/src/main/java/org/apache/kafka/storage/internals/log/LazyIndex.java: ## @@ -41,7 +41,7 @@ * Methods of this class are thread safe. Make sure to check `Abstra

Re: [PR] KAFKA-15597: Allow Connect DropHeaders SMT remove headers on a wildcard/regexp-basis [kafka]

2023-10-12 Thread via GitHub
mimaison commented on PR #14536: URL: https://github.com/apache/kafka/pull/14536#issuecomment-1759819074 Hi @Schm1tz1, thanks for the PR! I think we should have a small KIP before accepting this change. Today you could set `headers` to `header.*` and it would only drop records with th

[jira] [Commented] (KAFKA-15591) Trogdor produce workload reports errors in KRaft mode

2023-10-12 Thread Ron Dagostino (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774570#comment-17774570 ] Ron Dagostino commented on KAFKA-15591: --- > Is this caused by that in KRaft protoca

Re: [PR] KAFKA-15596: Upgrade ZooKeeper to 3.8.3 [kafka]

2023-10-12 Thread via GitHub
mimaison merged PR #14535: URL: https://github.com/apache/kafka/pull/14535 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

Re: [PR] KAFKA-15596: Upgrade ZooKeeper to 3.8.3 [kafka]

2023-10-12 Thread via GitHub
mimaison commented on PR #14535: URL: https://github.com/apache/kafka/pull/14535#issuecomment-1759883871 Backported to 3.6 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

[jira] [Resolved] (KAFKA-15596) Upgrade ZooKeeper to 3.8.3

2023-10-12 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mickael Maison resolved KAFKA-15596. Fix Version/s: 3.7.0 3.6.1 Resolution: Fixed > Upgrade ZooKeeper

Re: [PR] KAFKA-15276: Implement partition assignment reconciliation [kafka]

2023-10-12 Thread via GitHub
dajac commented on PR #14357: URL: https://github.com/apache/kafka/pull/14357#issuecomment-1759923735 @kirktrue Could you briefly explain how the reconciler will be integrated into the rest? I would like to better understand from where `maybeAssign`/`maybeRevoke` will be called. Thanks. -

Re: [PR] MINOR - KAFKA-15550: Validation for negative target times in offsetsForTimes [kafka]

2023-10-12 Thread via GitHub
lucasbru commented on code in PR #14503: URL: https://github.com/apache/kafka/pull/14503#discussion_r1357057172 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/PrototypeAsyncConsumerTest.java: ## @@ -322,6 +323,25 @@ public void testOffsetsForTimesOnNullPart

Re: [PR] KAFKA-15276: Implement partition assignment reconciliation [kafka]

2023-10-12 Thread via GitHub
dajac commented on code in PR #14357: URL: https://github.com/apache/kafka/pull/14357#discussion_r1357062078 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AssignmentReconciler.java: ## @@ -0,0 +1,328 @@ +/* + * Licensed to the Apache Software Foundation (A

Re: [PR] KAFKA-15276: Implement partition assignment reconciliation [kafka]

2023-10-12 Thread via GitHub
dajac commented on code in PR #14357: URL: https://github.com/apache/kafka/pull/14357#discussion_r1357062078 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AssignmentReconciler.java: ## @@ -0,0 +1,328 @@ +/* + * Licensed to the Apache Software Foundation (A

Re: [PR] KAFKA-14481: Move LogSegment/LogSegments to storage module [kafka]

2023-10-12 Thread via GitHub
ijuma commented on code in PR #14529: URL: https://github.com/apache/kafka/pull/14529#discussion_r1356970370 ## storage/src/main/java/org/apache/kafka/storage/internals/log/LogSegment.java: ## @@ -0,0 +1,887 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one o

Re: [PR] KAFKA-14481: Move LogSegment/LogSegments to storage module [kafka]

2023-10-12 Thread via GitHub
ijuma commented on PR #14529: URL: https://github.com/apache/kafka/pull/14529#issuecomment-1759952077 The Java 11 build has 3 unrelated failures: > Build / JDK 11 and Scala 2.13 / org.apache.kafka.common.network.SslTransportLayerTest.[3] tlsProtocol=TLSv1.3, useInlinePem=false Bui

[jira] [Updated] (KAFKA-15428) Cluster-wide dynamic log adjustments for Connect

2023-10-12 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton updated KAFKA-15428: -- Labels: kip (was: needs-kip) > Cluster-wide dynamic log adjustments for Connect > ---

[jira] [Commented] (KAFKA-14482) Move LogLoader to storage module

2023-10-12 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774605#comment-17774605 ] Ismael Juma commented on KAFKA-14482: - [~adupriez] We now have a PR for KAFKA-14481,

Re: [PR] KAFKA-14483 Move/Rewrite of LocalLog to storage module. [kafka]

2023-10-12 Thread via GitHub
ijuma commented on PR #14034: URL: https://github.com/apache/kafka/pull/14034#issuecomment-1759964557 @satishd Since we now have #14529, do you think you'll be able to pick this up again soon? -- This is an automated message from the Apache Git Service. To respond to the message, please l

Re: [PR] KAFKA-15158: Add metrics for RemoteRequestsPerSec [kafka]

2023-10-12 Thread via GitHub
nikramakrishnan commented on code in PR #14375: URL: https://github.com/apache/kafka/pull/14375#discussion_r1357082765 ## core/src/main/java/kafka/server/ReplicaFetcherTierStateMachine.java: ## @@ -68,14 +68,16 @@ */ public class ReplicaFetcherTierStateMachine implements Tier

Re: [PR] KAFKA-15158: Add metrics for RemoteRequestsPerSec [kafka]

2023-10-12 Thread via GitHub
nikramakrishnan commented on code in PR #14375: URL: https://github.com/apache/kafka/pull/14375#discussion_r1357082765 ## core/src/main/java/kafka/server/ReplicaFetcherTierStateMachine.java: ## @@ -68,14 +68,16 @@ */ public class ReplicaFetcherTierStateMachine implements Tier

[PR] KAFKA-15428: Cluster-wide dynamic log adjustments for Connect [kafka]

2023-10-12 Thread via GitHub
C0urante opened a new pull request, #14538: URL: https://github.com/apache/kafka/pull/14538 [Jira](https://issues.apache.org/jira/browse/KAFKA-15428), [KIP-976](https://cwiki.apache.org/confluence/display/KAFKA/KIP-976%3A+Cluster-wide+dynamic+log+adjustment+for+Kafka+Connect) Adds a n

[jira] [Created] (KAFKA-15599) Move KafkaMetadataLog and RaftManager to metadata module

2023-10-12 Thread Ismael Juma (Jira)
Ismael Juma created KAFKA-15599: --- Summary: Move KafkaMetadataLog and RaftManager to metadata module Key: KAFKA-15599 URL: https://issues.apache.org/jira/browse/KAFKA-15599 Project: Kafka Issue

Re: [PR] KAFKA-15428: Cluster-wide dynamic log adjustments for Connect [kafka]

2023-10-12 Thread via GitHub
C0urante commented on PR #14538: URL: https://github.com/apache/kafka/pull/14538#issuecomment-1760004960 @gharris1727, @yashmayya, @mimaison would any of you be able to take a look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

[jira] [Created] (KAFKA-15600) KIP-990: Capability to SUSPEND Tasks on DeserializationException

2023-10-12 Thread Nicholas Telford (Jira)
Nicholas Telford created KAFKA-15600: Summary: KIP-990: Capability to SUSPEND Tasks on DeserializationException Key: KAFKA-15600 URL: https://issues.apache.org/jira/browse/KAFKA-15600 Project: Kaf

[jira] [Commented] (KAFKA-15599) Move KafkaMetadataLog and RaftManager to metadata module

2023-10-12 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774620#comment-17774620 ] Ismael Juma commented on KAFKA-15599: - Open question: should `KafkaNetworkChannel` m

[jira] [Comment Edited] (KAFKA-15599) Move KafkaMetadataLog and RaftManager to metadata module

2023-10-12 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774620#comment-17774620 ] Ismael Juma edited comment on KAFKA-15599 at 10/12/23 4:58 PM: ---

Re: [PR] KAFKA-15597: Allow Connect DropHeaders SMT remove headers on a wildcard/regexp-basis [kafka]

2023-10-12 Thread via GitHub
Schm1tz1 commented on PR #14536: URL: https://github.com/apache/kafka/pull/14536#issuecomment-1760013569 Hi @mimaison , sure, will do that. Fair point, I was also thinking whether it makes sense to add a second config for header pattern to allow both options. -- This is an automated m

Re: [PR] KAFKA-15276: Implement partition assignment reconciliation [kafka]

2023-10-12 Thread via GitHub
kirktrue commented on code in PR #14357: URL: https://github.com/apache/kafka/pull/14357#discussion_r1357130345 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AssignmentReconciler.java: ## @@ -0,0 +1,328 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] KAFKA-15276: Implement partition assignment reconciliation [kafka]

2023-10-12 Thread via GitHub
kirktrue commented on code in PR #14357: URL: https://github.com/apache/kafka/pull/14357#discussion_r1357130345 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AssignmentReconciler.java: ## @@ -0,0 +1,328 @@ +/* + * Licensed to the Apache Software Foundation

[jira] [Comment Edited] (KAFKA-15599) Move KafkaMetadataLog and RaftManager to metadata module

2023-10-12 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774620#comment-17774620 ] Ismael Juma edited comment on KAFKA-15599 at 10/12/23 5:06 PM: ---

Re: [PR] KAFKA-15276: Implement partition assignment reconciliation [kafka]

2023-10-12 Thread via GitHub
lianetm commented on code in PR #14357: URL: https://github.com/apache/kafka/pull/14357#discussion_r1357145438 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AssignmentReconciler.java: ## @@ -0,0 +1,328 @@ +/* + * Licensed to the Apache Software Foundation

  1   2   3   >