[GitHub] [kafka] lanshiqin opened a new pull request, #13584: MINOR: Add log segment unit tests, If the maximum offset beyond index, appen…

2023-04-15 Thread via GitHub
lanshiqin opened a new pull request, #13584: URL: https://github.com/apache/kafka/pull/13584 Add log segment unit tests, If the maximum offset beyond index, appended to the log section, it throws LogSegmentOffsetOverflowException *More detailed description of change The unit tests

[jira] [Assigned] (KAFKA-12446) Define KGroupedTable#aggregate subtractor + adder order of execution

2023-04-15 Thread Farooq Qaiser (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Farooq Qaiser reassigned KAFKA-12446: - Assignee: Farooq Qaiser (was: Ben Ellis) > Define KGroupedTable#aggregate subtractor +

[GitHub] [kafka] fqaiser94 commented on pull request #10747: KAFKA-12446: Call subtractor before adder if key is the same

2023-04-15 Thread via GitHub
fqaiser94 commented on PR #10747: URL: https://github.com/apache/kafka/pull/10747#issuecomment-1510003420 Thanks for your support with this PR/KIP! > Hope we can get a system test earlier than later. If you are too busy, maybe somebody else can pick it up. Yea, sorry, I wont ha

[GitHub] [kafka] fqaiser94 commented on a diff in pull request #10747: KAFKA-12446: Call subtractor before adder if key is the same

2023-04-15 Thread via GitHub
fqaiser94 commented on code in PR #10747: URL: https://github.com/apache/kafka/pull/10747#discussion_r1167669456 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/ChangedSerializer.java: ## @@ -45,34 +49,87 @@ public void setIfUnset(final SerdeGetter getter) {

[jira] [Created] (KAFKA-14911) Add system tests for rolling upgrade path of KIP-904

2023-04-15 Thread Farooq Qaiser (Jira)
Farooq Qaiser created KAFKA-14911: - Summary: Add system tests for rolling upgrade path of KIP-904 Key: KAFKA-14911 URL: https://issues.apache.org/jira/browse/KAFKA-14911 Project: Kafka Issue

[GitHub] [kafka] fqaiser94 commented on a diff in pull request #13533: KAFKA-12446: update change encoding to use varint

2023-04-15 Thread via GitHub
fqaiser94 commented on code in PR #13533: URL: https://github.com/apache/kafka/pull/13533#discussion_r1167581045 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/ChangedSerializer.java: ## @@ -113,9 +113,9 @@ public byte[] serialize(final String topic, final H

[GitHub] [kafka] fqaiser94 commented on a diff in pull request #13533: KAFKA-12446: update change encoding to use varint

2023-04-15 Thread via GitHub
fqaiser94 commented on code in PR #13533: URL: https://github.com/apache/kafka/pull/13533#discussion_r1167560778 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/ChangedDeserializer.java: ## @@ -52,10 +52,10 @@ public Change deserialize(final String topic, fin

[GitHub] [kafka] aiven-anton opened a new pull request, #13583: docs: Correct type of record batch CRC field

2023-04-15 Thread via GitHub
aiven-anton opened a new pull request, #13583: URL: https://github.com/apache/kafka/pull/13583 The CRC field of Record Batch was incorrectly documented as int32 while in reality it's an [unsigned uint32 field](https://github.com/apache/kafka/blob/9c12e462106343fbc6af5873074d48f98687af39/cli

[GitHub] [kafka] ijuma commented on pull request #13312: KAFKA-14766: Improve performance of VarInt encoding and decoding

2023-04-15 Thread via GitHub
ijuma commented on PR #13312: URL: https://github.com/apache/kafka/pull/13312#issuecomment-1509978997 @divijvaidya Do we have any end to end test results? As stated previously, I'm not quite sure about the varint64 implementations - they seem to be the kind of thing that helps with microben

[GitHub] [kafka] ijuma commented on pull request #13579: KAFKA-14904: Flaky Test kafka.api.TransactionsBounceTest.testWithGroupId()

2023-04-15 Thread via GitHub
ijuma commented on PR #13579: URL: https://github.com/apache/kafka/pull/13579#issuecomment-1509969944 We should also cherry-pick this to 3.5. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

[GitHub] [kafka] ijuma commented on pull request #13575: KAFKA-14905: Intercept incrementalAlterConfigs in MM2 ForwardingAdminTest

2023-04-15 Thread via GitHub
ijuma commented on PR #13575: URL: https://github.com/apache/kafka/pull/13575#issuecomment-1509969810 We should cherry-pick this to 3.5 as well. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [kafka] ijuma commented on pull request #12830: KAFKA-8115: Reduce flakiness in Trogdor JsonRestServer shutdown

2023-04-15 Thread via GitHub
ijuma commented on PR #12830: URL: https://github.com/apache/kafka/pull/12830#issuecomment-1509969743 Merged to trunk and 3.5. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [kafka] ijuma commented on pull request #13567: MINOR: Update zstd-jni to 1.5.5

2023-04-15 Thread via GitHub
ijuma commented on PR #13567: URL: https://github.com/apache/kafka/pull/13567#issuecomment-1509969717 Merged to trunk and 3.5. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [kafka] ijuma commented on pull request #13543: MINOR: fix flaky KRaft-related tests

2023-04-15 Thread via GitHub
ijuma commented on PR #13543: URL: https://github.com/apache/kafka/pull/13543#issuecomment-1509969596 Cherry-picking this to 3.5. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

[GitHub] [kafka] jeqo commented on a diff in pull request #12637: KAFKA-14226 (KIP-821): [connect:transform] Introduce support for nested structures

2023-04-15 Thread via GitHub
jeqo commented on code in PR #12637: URL: https://github.com/apache/kafka/pull/12637#discussion_r1167636433 ## connect/transforms/src/main/java/org/apache/kafka/connect/transforms/field/SingleFieldPath.java: ## @@ -0,0 +1,585 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] jeqo commented on a diff in pull request #12637: KAFKA-14226 (KIP-821): [connect:transform] Introduce support for nested structures

2023-04-15 Thread via GitHub
jeqo commented on code in PR #12637: URL: https://github.com/apache/kafka/pull/12637#discussion_r1167635238 ## connect/transforms/src/main/java/org/apache/kafka/connect/transforms/field/SingleFieldPath.java: ## @@ -0,0 +1,585 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] ijuma opened a new pull request, #13582: MINOR: Fix lossy conversions flagged by Java 20

2023-04-15 Thread via GitHub
ijuma opened a new pull request, #13582: URL: https://github.com/apache/kafka/pull/13582 An example of the warning: > warning: [lossy-conversions] implicit cast from long to int in compound assignment is possibly lossy ### Committer Checklist (excluded from commit message) - [ ]

[GitHub] [kafka] jeqo commented on a diff in pull request #12637: KAFKA-14226 (KIP-821): [connect:transform] Introduce support for nested structures

2023-04-15 Thread via GitHub
jeqo commented on code in PR #12637: URL: https://github.com/apache/kafka/pull/12637#discussion_r1167632114 ## connect/transforms/src/main/java/org/apache/kafka/connect/transforms/field/MultiFieldPaths.java: ## @@ -0,0 +1,578 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] ijuma merged pull request #13567: MINOR: Update zstd-jni to 1.5.5

2023-04-15 Thread via GitHub
ijuma merged PR #13567: URL: https://github.com/apache/kafka/pull/13567 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

[GitHub] [kafka] ijuma commented on pull request #13567: MINOR: Update zstd-jni to 1.5.5

2023-04-15 Thread via GitHub
ijuma commented on PR #13567: URL: https://github.com/apache/kafka/pull/13567#issuecomment-1509937606 There are a bunch of mirrormaker related failures that are being addressed via a different PR and are unrelated to this change. -- This is an automated message from the Apache Git Service

[GitHub] [kafka] ijuma merged pull request #12830: KAFKA-8115: Reduce flakiness in Trogdor JsonRestServer shutdown

2023-04-15 Thread via GitHub
ijuma merged PR #12830: URL: https://github.com/apache/kafka/pull/12830 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

[GitHub] [kafka] ijuma commented on pull request #12830: KAFKA-8115: Reduce flakiness in Trogdor JsonRestServer shutdown

2023-04-15 Thread via GitHub
ijuma commented on PR #12830: URL: https://github.com/apache/kafka/pull/12830#issuecomment-1509935873 Failures seem to be related to `MirrorConnectors` only. There is a separate PR trying to fix that. > Build / JDK 17 and Scala 2.13 / org.apache.kafka.connect.mirror.integration.Mirro

[GitHub] [kafka] ijuma commented on pull request #13579: KAFKA-14904: Flaky Test kafka.api.TransactionsBounceTest.testWithGroupId()

2023-04-15 Thread via GitHub
ijuma commented on PR #13579: URL: https://github.com/apache/kafka/pull/13579#issuecomment-1509935179 @jolshan Thanks for the PR. Can you please share details regarding the frequency of failures before/after the change when you run this test locally in a loop? -- This is an automated mes

[GitHub] [kafka] ijuma commented on a diff in pull request #13579: KAFKA-14904: Flaky Test kafka.api.TransactionsBounceTest.testWithGroupId()

2023-04-15 Thread via GitHub
ijuma commented on code in PR #13579: URL: https://github.com/apache/kafka/pull/13579#discussion_r1167627192 ## core/src/main/scala/kafka/coordinator/transaction/TransactionCoordinator.scala: ## @@ -396,8 +397,14 @@ class TransactionCoordinator(txnConfig: TransactionConfig,

[GitHub] [kafka] jeqo commented on a diff in pull request #12637: KAFKA-14226 (KIP-821): [connect:transform] Introduce support for nested structures

2023-04-15 Thread via GitHub
jeqo commented on code in PR #12637: URL: https://github.com/apache/kafka/pull/12637#discussion_r1167626084 ## connect/transforms/src/main/java/org/apache/kafka/connect/transforms/field/FieldSyntaxVersion.java: ## @@ -0,0 +1,112 @@ +/* + * Licensed to the Apache Software Foundat

[GitHub] [kafka] bmscomp commented on pull request #13569: MINOR: Upgrade to Gradle 8.1

2023-04-15 Thread via GitHub
bmscomp commented on PR #13569: URL: https://github.com/apache/kafka/pull/13569#issuecomment-1509884313 I think it's enough to use a `bin` distribution type instead instead of `all` one for Gradle wrapper -- This is an automated message from the Apache Git Service. To respond to the me

[GitHub] [kafka] ijuma commented on a diff in pull request #12830: KAFKA-8115: Reduce flakiness in Trogdor JsonRestServer shutdown

2023-04-15 Thread via GitHub
ijuma commented on code in PR #12830: URL: https://github.com/apache/kafka/pull/12830#discussion_r1167565330 ## trogdor/src/main/java/org/apache/kafka/trogdor/rest/JsonRestServer.java: ## @@ -138,13 +138,13 @@ public void beginShutdown() { log.info("Stopping

[jira] [Commented] (KAFKA-14750) Sink connector fails if a topic matching its topics.regex gets deleted

2023-04-15 Thread Sagar Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17712649#comment-17712649 ] Sagar Rao commented on KAFKA-14750: --- Thanks for your response [~morozov] . I have crea

[jira] [Commented] (KAFKA-14881) Update UserScramCredentialRecord for SCRAM ZK to KRaft migration

2023-04-15 Thread Proven Provenzano (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17712647#comment-17712647 ] Proven Provenzano commented on KAFKA-14881: --- Please move it back to 3.5.0. The

[GitHub] [kafka] vamossagar12 commented on pull request #13557: KAFKA-14902: KafkaStatusBackingStore retries on a dedicated backgroun…

2023-04-15 Thread via GitHub
vamossagar12 commented on PR #13557: URL: https://github.com/apache/kafka/pull/13557#issuecomment-1509738740 Thanks for the PR @urbandan . This looks good. Now that you have introduced this method, I am thinking if other executors can use this method? I can see `herderExectuor`, `forwardReq

[jira] [Updated] (KAFKA-14323) KRaft broker time based snapshots

2023-04-15 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mickael Maison updated KAFKA-14323: --- Fix Version/s: 3.6.0 (was: 3.5.0) > KRaft broker time based snapshots

[jira] [Commented] (KAFKA-14323) KRaft broker time based snapshots

2023-04-15 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17712636#comment-17712636 ] Mickael Maison commented on KAFKA-14323: We're past feature freeze for 3.5.0 so

[jira] [Updated] (KAFKA-14852) Propagate Topic Ids to the Group Coordinator for Offset Fetch

2023-04-15 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mickael Maison updated KAFKA-14852: --- Fix Version/s: 3.6.0 (was: 3.5.0) > Propagate Topic Ids to the Group

[jira] [Commented] (KAFKA-14852) Propagate Topic Ids to the Group Coordinator for Offset Fetch

2023-04-15 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17712635#comment-17712635 ] Mickael Maison commented on KAFKA-14852: We're past feature freeze for 3.5.0, so

[jira] [Commented] (KAFKA-14881) Update UserScramCredentialRecord for SCRAM ZK to KRaft migration

2023-04-15 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17712633#comment-17712633 ] Mickael Maison commented on KAFKA-14881: 3.5.0 is in feature freeze, so I'm movi

[jira] [Updated] (KAFKA-14881) Update UserScramCredentialRecord for SCRAM ZK to KRaft migration

2023-04-15 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mickael Maison updated KAFKA-14881: --- Fix Version/s: 3.6.0 (was: 3.5.0) > Update UserScramCredentialRecord

[jira] [Updated] (KAFKA-14077) KRaft should support recovery from failed disk

2023-04-15 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mickael Maison updated KAFKA-14077: --- Fix Version/s: 3.6.0 (was: 3.5.0) > KRaft should support recovery fro

[jira] [Commented] (KAFKA-14077) KRaft should support recovery from failed disk

2023-04-15 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17712630#comment-17712630 ] Mickael Maison commented on KAFKA-14077: The associated KIP is not voted yet, so

[jira] [Updated] (KAFKA-14690) OffsetCommit API Version 9

2023-04-15 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mickael Maison updated KAFKA-14690: --- Fix Version/s: 3.6.0 (was: 3.5.0) > OffsetCommit API Version 9 >

[jira] [Commented] (KAFKA-14690) OffsetCommit API Version 9

2023-04-15 Thread Mickael Maison (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17712629#comment-17712629 ] Mickael Maison commented on KAFKA-14690: Since we're past feature freeze for 3.5

[GitHub] [kafka] sudeshwasnik commented on a diff in pull request #13361: KAFKA-14401: Fail the worker if any uncaught exception comes in WorkThread

2023-04-15 Thread via GitHub
sudeshwasnik commented on code in PR #13361: URL: https://github.com/apache/kafka/pull/13361#discussion_r1167474596 ## connect/runtime/src/main/java/org/apache/kafka/connect/storage/KafkaOffsetBackingStore.java: ## @@ -316,6 +316,9 @@ public Map convert(Void result) { /

[GitHub] [kafka] Scanteianu commented on pull request #13341: MINOR: add comment that timestamp unit is milliseconds

2023-04-15 Thread via GitHub
Scanteianu commented on PR #13341: URL: https://github.com/apache/kafka/pull/13341#issuecomment-1509668648 > @Scanteianu as a next step, we need to wait for a committer to look into this one so that it can be merged. > > Since it's related to consumers, perhaps @dajac would be interes

[GitHub] [kafka] yashmayya commented on a diff in pull request #13465: KAFKA-14368: Connect offset write REST API

2023-04-15 Thread via GitHub
yashmayya commented on code in PR #13465: URL: https://github.com/apache/kafka/pull/13465#discussion_r1167415541 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/Worker.java: ## @@ -1247,6 +1257,257 @@ void sourceConnectorOffsets(String connName, ConnectorOffse

[GitHub] [kafka] yashmayya commented on a diff in pull request #13465: KAFKA-14368: Connect offset write REST API

2023-04-15 Thread via GitHub
yashmayya commented on code in PR #13465: URL: https://github.com/apache/kafka/pull/13465#discussion_r1167415436 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/Worker.java: ## @@ -1247,6 +1257,257 @@ void sourceConnectorOffsets(String connName, ConnectorOffse