[GitHub] [kafka] philipnee commented on a diff in pull request #13477: KAFKA-7499: Handle serialization error in ProductionExceptionHandler

2023-04-11 Thread via GitHub
philipnee commented on code in PR #13477: URL: https://github.com/apache/kafka/pull/13477#discussion_r1163702789 ## streams/src/main/java/org/apache/kafka/streams/errors/ProductionExceptionHandler.java: ## @@ -34,6 +35,18 @@ public interface ProductionExceptionHandler extends C

[GitHub] [kafka] mjsax commented on a diff in pull request #13477: KAFKA-7499: Handle serialization error in ProductionExceptionHandler

2023-04-11 Thread via GitHub
mjsax commented on code in PR #13477: URL: https://github.com/apache/kafka/pull/13477#discussion_r1163702670 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/RecordCollectorImpl.java: ## @@ -213,6 +212,31 @@ public void send(final String topic,

[GitHub] [kafka] mjsax commented on a diff in pull request #13477: KAFKA-7499: Handle serialization error in ProductionExceptionHandler

2023-04-11 Thread via GitHub
mjsax commented on code in PR #13477: URL: https://github.com/apache/kafka/pull/13477#discussion_r1163683492 ## streams/src/main/java/org/apache/kafka/streams/errors/ProductionExceptionHandler.java: ## @@ -34,6 +35,18 @@ public interface ProductionExceptionHandler extends Confi

[GitHub] [kafka] artemlivshits commented on a diff in pull request #13391: KAFKA-14561: Improve transactions experience for older clients by ensuring ongoing transaction

2023-04-11 Thread via GitHub
artemlivshits commented on code in PR #13391: URL: https://github.com/apache/kafka/pull/13391#discussion_r1163676408 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -69,20 +108,52 @@ class KafkaRequestHandler(id: Int, completeShutdown()

[GitHub] [kafka] showuon merged pull request #13487: KAFKA-9550 Copying log segments to tiered storage in RemoteLogManager

2023-04-11 Thread via GitHub
showuon merged PR #13487: URL: https://github.com/apache/kafka/pull/13487 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

[GitHub] [kafka] gharris1727 commented on pull request #13465: KAFKA-14368: Connect offset write REST API

2023-04-11 Thread via GitHub
gharris1727 commented on PR #13465: URL: https://github.com/apache/kafka/pull/13465#issuecomment-1504680664 > jq is automatically converting floats like x.0 into x Thanks for catching that, you're completely correct. I should have cut jq out of my tests to verify that but I hadn't eve

[GitHub] [kafka] vcrfxia commented on pull request #13444: KAFKA-14491: [21/N] Docs updates for versioned state stores

2023-04-11 Thread via GitHub
vcrfxia commented on PR #13444: URL: https://github.com/apache/kafka/pull/13444#issuecomment-1504673370 Sorry for the delay in responding -- just updated the PR with your latest suggestion. -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [kafka] yashmayya commented on pull request #13465: KAFKA-14368: Connect offset write REST API

2023-04-11 Thread via GitHub
yashmayya commented on PR #13465: URL: https://github.com/apache/kafka/pull/13465#issuecomment-1504645043 I tried really hard to reproduce this via integration tests and I wasn't able to, so I tried doing a repro manually just as you outlined above. Turns out that this isn't actually a bug

[GitHub] [kafka] mjsax merged pull request #13497: KAFKA-14834: [2/N] Test coverage for out-of-order data in joins

2023-04-11 Thread via GitHub
mjsax merged PR #13497: URL: https://github.com/apache/kafka/pull/13497 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

[GitHub] [kafka] mjsax commented on a diff in pull request #13497: KAFKA-14834: [2/N] Test coverage for out-of-order data in joins

2023-04-11 Thread via GitHub
mjsax commented on code in PR #13497: URL: https://github.com/apache/kafka/pull/13497#discussion_r1163555907 ## streams/src/test/java/org/apache/kafka/streams/integration/AbstractJoinIntegrationTest.java: ## @@ -88,31 +88,35 @@ public static Collection data() { StreamsBuild

[GitHub] [kafka] mjsax merged pull request #13496: KAFKA-14834: [1/N] Add timestamped get to KTableValueGetter

2023-04-11 Thread via GitHub
mjsax merged PR #13496: URL: https://github.com/apache/kafka/pull/13496 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

[GitHub] [kafka] showuon commented on pull request #13487: KAFKA-9550 Copying log segments to tiered storage in RemoteLogManager

2023-04-11 Thread via GitHub
showuon commented on PR #13487: URL: https://github.com/apache/kafka/pull/13487#issuecomment-1504387988 Failed tests are unrelated: ``` Build / JDK 17 and Scala 2.13 / kafka.admin.TopicCommandIntegrationTest.testDescribeAndListTopicsWithoutInternalTopics(String).quorum=zk Bu

[GitHub] [kafka] philipnee commented on pull request #13490: KAFKA-14875: Implement wakeup

2023-04-11 Thread via GitHub
philipnee commented on PR #13490: URL: https://github.com/apache/kafka/pull/13490#issuecomment-1504343151 Thanks, @vvcephei - I addressed your comments there! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [kafka] philipnee commented on pull request #13477: KAFKA-7499: Handle serialization error in ProductionExceptionHandler

2023-04-11 Thread via GitHub
philipnee commented on PR #13477: URL: https://github.com/apache/kafka/pull/13477#issuecomment-1504326949 @bbejeck - Thanks for the comments, I pushed out a commit with to address the changes you suggested. -- This is an automated message from the Apache Git Service. To respond to the mes

[GitHub] [kafka] junrao commented on a diff in pull request #13391: KAFKA-14561: Improve transactions experience for older clients by ensuring ongoing transaction

2023-04-11 Thread via GitHub
junrao commented on code in PR #13391: URL: https://github.com/apache/kafka/pull/13391#discussion_r1163446164 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -69,20 +108,52 @@ class KafkaRequestHandler(id: Int, completeShutdown() return

[GitHub] [kafka] jolshan commented on pull request #13391: KAFKA-14561: Improve transactions experience for older clients by ensuring ongoing transaction

2023-04-11 Thread via GitHub
jolshan commented on PR #13391: URL: https://github.com/apache/kafka/pull/13391#issuecomment-1504315306 ^ Seems like this still may be initially caused by my change so I'm investigating. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [kafka] jolshan commented on pull request #13391: KAFKA-14561: Improve transactions experience for older clients by ensuring ongoing transaction

2023-04-11 Thread via GitHub
jolshan commented on PR #13391: URL: https://github.com/apache/kafka/pull/13391#issuecomment-1504291314 fyi I found this: https://issues.apache.org/jira/browse/KAFKA-14896 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[jira] [Created] (KAFKA-14896) TransactionsBounceTest causes a thread leak

2023-04-11 Thread Justine Olshan (Jira)
Justine Olshan created KAFKA-14896: -- Summary: TransactionsBounceTest causes a thread leak Key: KAFKA-14896 URL: https://issues.apache.org/jira/browse/KAFKA-14896 Project: Kafka Issue Type: B

[GitHub] [kafka] jolshan commented on a diff in pull request #13391: KAFKA-14561: Improve transactions experience for older clients by ensuring ongoing transaction

2023-04-11 Thread via GitHub
jolshan commented on code in PR #13391: URL: https://github.com/apache/kafka/pull/13391#discussion_r1163429337 ## core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala: ## @@ -2053,6 +2056,190 @@ class ReplicaManagerTest { assertEquals(Errors.NOT_LEADER_OR_FOLLOWER

[GitHub] [kafka] vcrfxia commented on a diff in pull request #13496: KAFKA-14834: [1/N] Add timestamped get to KTableValueGetter

2023-04-11 Thread via GitHub
vcrfxia commented on code in PR #13496: URL: https://github.com/apache/kafka/pull/13496#discussion_r1163422597 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/KTableValueGetter.java: ## @@ -25,5 +25,21 @@ ValueAndTimestamp get(K key); +/** +

[GitHub] [kafka] vcrfxia commented on a diff in pull request #13496: KAFKA-14834: [1/N] Add timestamped get to KTableValueGetter

2023-04-11 Thread via GitHub
vcrfxia commented on code in PR #13496: URL: https://github.com/apache/kafka/pull/13496#discussion_r1163422113 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/KTableKTableInnerJoin.java: ## @@ -153,11 +154,37 @@ public void init(final ProcessorContext context

[GitHub] [kafka] jolshan commented on a diff in pull request #13391: KAFKA-14561: Improve transactions experience for older clients by ensuring ongoing transaction

2023-04-11 Thread via GitHub
jolshan commented on code in PR #13391: URL: https://github.com/apache/kafka/pull/13391#discussion_r1163421979 ## core/src/test/scala/unit/kafka/server/AddPartitionsToTxnManagerTest.scala: ## @@ -0,0 +1,238 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one o

[GitHub] [kafka] jolshan commented on a diff in pull request #13391: KAFKA-14561: Improve transactions experience for older clients by ensuring ongoing transaction

2023-04-11 Thread via GitHub
jolshan commented on code in PR #13391: URL: https://github.com/apache/kafka/pull/13391#discussion_r1163421308 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -69,20 +108,52 @@ class KafkaRequestHandler(id: Int, completeShutdown() retur

[jira] [Created] (KAFKA-14895) Move AddPartitionsToTxnManager files to java

2023-04-11 Thread Justine Olshan (Jira)
Justine Olshan created KAFKA-14895: -- Summary: Move AddPartitionsToTxnManager files to java Key: KAFKA-14895 URL: https://issues.apache.org/jira/browse/KAFKA-14895 Project: Kafka Issue Type:

[GitHub] [kafka] jolshan commented on a diff in pull request #13391: KAFKA-14561: Improve transactions experience for older clients by ensuring ongoing transaction

2023-04-11 Thread via GitHub
jolshan commented on code in PR #13391: URL: https://github.com/apache/kafka/pull/13391#discussion_r1163420192 ## core/src/main/scala/kafka/server/AddPartitionsToTxnManager.scala: ## @@ -0,0 +1,180 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more +

[GitHub] [kafka] jolshan commented on a diff in pull request #13391: KAFKA-14561: Improve transactions experience for older clients by ensuring ongoing transaction

2023-04-11 Thread via GitHub
jolshan commented on code in PR #13391: URL: https://github.com/apache/kafka/pull/13391#discussion_r1163418845 ## core/src/test/scala/unit/kafka/server/AddPartitionsToTxnManagerTest.scala: ## @@ -0,0 +1,238 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one o

[GitHub] [kafka] vcrfxia commented on a diff in pull request #13497: KAFKA-14834: [2/N] Test coverage for out-of-order data in joins

2023-04-11 Thread via GitHub
vcrfxia commented on code in PR #13497: URL: https://github.com/apache/kafka/pull/13497#discussion_r1163410809 ## streams/src/test/java/org/apache/kafka/streams/integration/AbstractJoinIntegrationTest.java: ## @@ -88,31 +88,35 @@ public static Collection data() { StreamsBui

[GitHub] [kafka] vcrfxia commented on a diff in pull request #13497: KAFKA-14834: [2/N] Test coverage for out-of-order data in joins

2023-04-11 Thread via GitHub
vcrfxia commented on code in PR #13497: URL: https://github.com/apache/kafka/pull/13497#discussion_r1163410451 ## streams/src/test/java/org/apache/kafka/streams/integration/StreamStreamJoinIntegrationTest.java: ## @@ -384,7 +528,7 @@ public void testOuterRepartitioned() { p

[GitHub] [kafka] gharris1727 commented on pull request #13465: KAFKA-14368: Connect offset write REST API

2023-04-11 Thread via GitHub
gharris1727 commented on PR #13465: URL: https://github.com/apache/kafka/pull/13465#issuecomment-1504231021 So I was manually testing this feature and ran across a serialization problem. Here's the most concise repro case I can think of: ``` $ curl -sSX PATCH -H "Content-Type: applicat

[GitHub] [kafka] mjsax commented on a diff in pull request #13497: KAFKA-14834: [2/N] Test coverage for out-of-order data in joins

2023-04-11 Thread via GitHub
mjsax commented on code in PR #13497: URL: https://github.com/apache/kafka/pull/13497#discussion_r1163397015 ## streams/src/test/java/org/apache/kafka/streams/integration/AbstractJoinIntegrationTest.java: ## @@ -88,31 +88,35 @@ public static Collection data() { StreamsBuild

[GitHub] [kafka] mjsax commented on a diff in pull request #13497: KAFKA-14834: [2/N] Test coverage for out-of-order data in joins

2023-04-11 Thread via GitHub
mjsax commented on code in PR #13497: URL: https://github.com/apache/kafka/pull/13497#discussion_r1163394194 ## streams/src/test/java/org/apache/kafka/streams/integration/StreamStreamJoinIntegrationTest.java: ## @@ -384,7 +528,7 @@ public void testOuterRepartitioned() { pub

[GitHub] [kafka] bbejeck commented on a diff in pull request #13477: KAFKA-7499: Handle serialization error in ProductionExceptionHandler

2023-04-11 Thread via GitHub
bbejeck commented on code in PR #13477: URL: https://github.com/apache/kafka/pull/13477#discussion_r1163377252 ## streams/src/main/java/org/apache/kafka/streams/errors/ProductionExceptionHandler.java: ## @@ -34,6 +34,18 @@ public interface ProductionExceptionHandler extends Con

[GitHub] [kafka] cmccabe merged pull request #13541: KAFKA-14894: MetadataLoader must call finishSnapshot after loading a snapshot

2023-04-11 Thread via GitHub
cmccabe merged PR #13541: URL: https://github.com/apache/kafka/pull/13541 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

[GitHub] [kafka] bbejeck commented on a diff in pull request #13477: KAFKA-7499: Handle serialization error in ProductionExceptionHandler

2023-04-11 Thread via GitHub
bbejeck commented on code in PR #13477: URL: https://github.com/apache/kafka/pull/13477#discussion_r1163365163 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/RecordCollectorImpl.java: ## @@ -213,6 +212,30 @@ public void send(final String topic,

[GitHub] [kafka] cmccabe commented on pull request #13541: KAFKA-14894: MetadataLoader must call finishSnapshot after loading a snapshot

2023-04-11 Thread via GitHub
cmccabe commented on PR #13541: URL: https://github.com/apache/kafka/pull/13541#issuecomment-1504169270 Frustrating that there are so many unrelated test flakes in `:connect:mirror:integrationTest`. Anyway thanks, committing. -- This is an automated message from the Apache Git Service. To

[GitHub] [kafka] cmccabe commented on a diff in pull request #13407: KAFKA-14805 KRaft controller supports pre-migration mode

2023-04-11 Thread via GitHub
cmccabe commented on code in PR #13407: URL: https://github.com/apache/kafka/pull/13407#discussion_r1163358699 ## metadata/src/main/java/org/apache/kafka/controller/FeatureControlManager.java: ## @@ -284,6 +334,120 @@ FinalizedControllerFeatures finalizedFeatures(long epoch) {

[GitHub] [kafka] rreddy-22 commented on a diff in pull request #13524: MINOR: Refine `PartitionAssignor` server-side interface

2023-04-11 Thread via GitHub
rreddy-22 commented on code in PR #13524: URL: https://github.com/apache/kafka/pull/13524#discussion_r1163356477 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/AssignmentMemberSpec.java: ## @@ -89,7 +92,7 @@ public String toString() { ret

[GitHub] [kafka] cmccabe commented on a diff in pull request #13407: KAFKA-14805 KRaft controller supports pre-migration mode

2023-04-11 Thread via GitHub
cmccabe commented on code in PR #13407: URL: https://github.com/apache/kafka/pull/13407#discussion_r1163355497 ## metadata/src/main/java/org/apache/kafka/image/FeaturesImage.java: ## @@ -37,19 +40,30 @@ * This class is thread-safe. */ public final class FeaturesImage { -

[GitHub] [kafka] rreddy-22 commented on a diff in pull request #13524: MINOR: Refine `PartitionAssignor` server-side interface

2023-04-11 Thread via GitHub
rreddy-22 commented on code in PR #13524: URL: https://github.com/apache/kafka/pull/13524#discussion_r1163355154 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/AssignmentMemberSpec.java: ## @@ -37,29 +39,30 @@ public class AssignmentMemberSpec {

[GitHub] [kafka] cmccabe commented on a diff in pull request #13407: KAFKA-14805 KRaft controller supports pre-migration mode

2023-04-11 Thread via GitHub
cmccabe commented on code in PR #13407: URL: https://github.com/apache/kafka/pull/13407#discussion_r1163354759 ## metadata/src/main/java/org/apache/kafka/controller/metrics/ControllerMetadataMetrics.java: ## @@ -190,6 +200,14 @@ public int metadataErrorCount() { return

[GitHub] [kafka] rreddy-22 commented on a diff in pull request #13524: MINOR: Refine `PartitionAssignor` server-side interface

2023-04-11 Thread via GitHub
rreddy-22 commented on code in PR #13524: URL: https://github.com/apache/kafka/pull/13524#discussion_r1163354734 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/AssignmentMemberSpec.java: ## @@ -37,29 +39,30 @@ public class AssignmentMemberSpec {

[GitHub] [kafka] cmccabe commented on a diff in pull request #13407: KAFKA-14805 KRaft controller supports pre-migration mode

2023-04-11 Thread via GitHub
cmccabe commented on code in PR #13407: URL: https://github.com/apache/kafka/pull/13407#discussion_r1163354403 ## metadata/src/main/java/org/apache/kafka/controller/metrics/ControllerMetadataMetrics.java: ## @@ -108,6 +112,12 @@ public Integer value() { return m

[GitHub] [kafka] vvcephei commented on a diff in pull request #13490: KAFKA-14875: Implement wakeup

2023-04-11 Thread via GitHub
vvcephei commented on code in PR #13490: URL: https://github.com/apache/kafka/pull/13490#discussion_r1163354091 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/PrototypeAsyncConsumer.java: ## @@ -447,6 +482,19 @@ public void close() { @Override p

[GitHub] [kafka] cmccabe commented on a diff in pull request #13407: KAFKA-14805 KRaft controller supports pre-migration mode

2023-04-11 Thread via GitHub
cmccabe commented on code in PR #13407: URL: https://github.com/apache/kafka/pull/13407#discussion_r1163353132 ## metadata/src/main/java/org/apache/kafka/controller/QuorumController.java: ## @@ -1142,9 +1190,20 @@ public ControllerResult generateRecordsAndResult() throws Except

[GitHub] [kafka] cmccabe commented on a diff in pull request #13407: KAFKA-14805 KRaft controller supports pre-migration mode

2023-04-11 Thread via GitHub
cmccabe commented on code in PR #13407: URL: https://github.com/apache/kafka/pull/13407#discussion_r1163350977 ## metadata/src/main/java/org/apache/kafka/controller/QuorumController.java: ## @@ -1142,9 +1190,20 @@ public ControllerResult generateRecordsAndResult() throws Except

[GitHub] [kafka] cmccabe commented on a diff in pull request #13407: KAFKA-14805 KRaft controller supports pre-migration mode

2023-04-11 Thread via GitHub
cmccabe commented on code in PR #13407: URL: https://github.com/apache/kafka/pull/13407#discussion_r1163349040 ## metadata/src/main/java/org/apache/kafka/controller/QuorumController.java: ## @@ -1142,9 +1190,20 @@ public ControllerResult generateRecordsAndResult() throws Except

[GitHub] [kafka] junrao commented on a diff in pull request #13391: KAFKA-14561: Improve transactions experience for older clients by ensuring ongoing transaction

2023-04-11 Thread via GitHub
junrao commented on code in PR #13391: URL: https://github.com/apache/kafka/pull/13391#discussion_r1163287233 ## core/src/test/scala/unit/kafka/server/AddPartitionsToTxnManagerTest.scala: ## @@ -0,0 +1,238 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or

[GitHub] [kafka] gharris1727 commented on a diff in pull request #13465: KAFKA-14368: Connect offset write REST API

2023-04-11 Thread via GitHub
gharris1727 commented on code in PR #13465: URL: https://github.com/apache/kafka/pull/13465#discussion_r1163322562 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/Worker.java: ## @@ -1247,6 +1254,224 @@ void sourceConnectorOffsets(String connName, ConnectorOff

[GitHub] [kafka] gharris1727 commented on a diff in pull request #13465: KAFKA-14368: Connect offset write REST API

2023-04-11 Thread via GitHub
gharris1727 commented on code in PR #13465: URL: https://github.com/apache/kafka/pull/13465#discussion_r1163320472 ## connect/runtime/src/test/java/org/apache/kafka/connect/util/clusters/EmbeddedConnectCluster.java: ## @@ -19,24 +19,25 @@ import com.fasterxml.jackson.core.type.

[GitHub] [kafka] cmccabe commented on a diff in pull request #13407: KAFKA-14805 KRaft controller supports pre-migration mode

2023-04-11 Thread via GitHub
cmccabe commented on code in PR #13407: URL: https://github.com/apache/kafka/pull/13407#discussion_r1163311065 ## metadata/src/main/java/org/apache/kafka/controller/FeatureControlManager.java: ## @@ -284,6 +334,120 @@ FinalizedControllerFeatures finalizedFeatures(long epoch) {

[GitHub] [kafka] cmccabe commented on a diff in pull request #13407: KAFKA-14805 KRaft controller supports pre-migration mode

2023-04-11 Thread via GitHub
cmccabe commented on code in PR #13407: URL: https://github.com/apache/kafka/pull/13407#discussion_r1163305761 ## metadata/src/main/java/org/apache/kafka/controller/FeatureControlManager.java: ## @@ -266,6 +311,11 @@ private ApiError updateMetadataVersion( new Featu

[GitHub] [kafka] cmccabe commented on a diff in pull request #13407: KAFKA-14805 KRaft controller supports pre-migration mode

2023-04-11 Thread via GitHub
cmccabe commented on code in PR #13407: URL: https://github.com/apache/kafka/pull/13407#discussion_r1163303944 ## metadata/src/main/java/org/apache/kafka/controller/FeatureControlManager.java: ## @@ -110,23 +143,31 @@ public FeatureControlManager build() { */ private

[GitHub] [kafka] cmccabe commented on a diff in pull request #13407: KAFKA-14805 KRaft controller supports pre-migration mode

2023-04-11 Thread via GitHub
cmccabe commented on code in PR #13407: URL: https://github.com/apache/kafka/pull/13407#discussion_r1163299070 ## metadata/src/main/java/org/apache/kafka/controller/FeatureControlManager.java: ## @@ -304,6 +468,44 @@ public void replay(FeatureLevelRecord record) { }

[GitHub] [kafka] cmccabe commented on a diff in pull request #13407: KAFKA-14805 KRaft controller supports pre-migration mode

2023-04-11 Thread via GitHub
cmccabe commented on code in PR #13407: URL: https://github.com/apache/kafka/pull/13407#discussion_r1163295217 ## metadata/src/main/java/org/apache/kafka/controller/FeatureControlManager.java: ## @@ -284,6 +334,120 @@ FinalizedControllerFeatures finalizedFeatures(long epoch) {

[GitHub] [kafka] cmccabe commented on a diff in pull request #13407: KAFKA-14805 KRaft controller supports pre-migration mode

2023-04-11 Thread via GitHub
cmccabe commented on code in PR #13407: URL: https://github.com/apache/kafka/pull/13407#discussion_r1163293947 ## metadata/src/main/java/org/apache/kafka/controller/FeatureControlManager.java: ## @@ -284,6 +334,120 @@ FinalizedControllerFeatures finalizedFeatures(long epoch) {

[GitHub] [kafka] cmccabe commented on a diff in pull request #13407: KAFKA-14805 KRaft controller supports pre-migration mode

2023-04-11 Thread via GitHub
cmccabe commented on code in PR #13407: URL: https://github.com/apache/kafka/pull/13407#discussion_r1163287276 ## metadata/src/main/java/org/apache/kafka/controller/FeatureControlManager.java: ## @@ -266,6 +311,11 @@ private ApiError updateMetadataVersion( new Featu

[GitHub] [kafka] yashmayya commented on a diff in pull request #13465: KAFKA-14368: Connect offset write REST API

2023-04-11 Thread via GitHub
yashmayya commented on code in PR #13465: URL: https://github.com/apache/kafka/pull/13465#discussion_r1163225504 ## connect/runtime/src/test/java/org/apache/kafka/connect/util/clusters/EmbeddedConnectCluster.java: ## @@ -19,24 +19,25 @@ import com.fasterxml.jackson.core.type.Ty

[GitHub] [kafka] glasser commented on pull request #9720: KAFKA-10555: Improve client state machine

2023-04-11 Thread via GitHub
glasser commented on PR #9720: URL: https://github.com/apache/kafka/pull/9720#issuecomment-1503972837 Am I correct in believing this change was released in Kafka 2.8.0? I can't find it in [the release notes](https://archive.apache.org/dist/kafka/2.8.0/RELEASE_NOTES.html). We just had a min

[GitHub] [kafka] mjsax commented on a diff in pull request #13496: KAFKA-14834: [1/N] Add timestamped get to KTableValueGetter

2023-04-11 Thread via GitHub
mjsax commented on code in PR #13496: URL: https://github.com/apache/kafka/pull/13496#discussion_r1163228842 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/KTableKTableLeftJoin.java: ## @@ -159,11 +160,36 @@ public void init(final ProcessorContext context) {

[GitHub] [kafka] mjsax commented on a diff in pull request #13496: KAFKA-14834: [1/N] Add timestamped get to KTableValueGetter

2023-04-11 Thread via GitHub
mjsax commented on code in PR #13496: URL: https://github.com/apache/kafka/pull/13496#discussion_r1163228842 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/KTableKTableLeftJoin.java: ## @@ -159,11 +160,36 @@ public void init(final ProcessorContext context) {

[GitHub] [kafka] philipnee commented on a diff in pull request #13477: KAFKA-7499: Handle serialization error in ProductionExceptionHandler

2023-04-11 Thread via GitHub
philipnee commented on code in PR #13477: URL: https://github.com/apache/kafka/pull/13477#discussion_r1163228427 ## streams/src/main/java/org/apache/kafka/streams/errors/ProductionExceptionHandler.java: ## @@ -34,6 +34,18 @@ public interface ProductionExceptionHandler extends C

[GitHub] [kafka] mjsax commented on a diff in pull request #13496: KAFKA-14834: [1/N] Add timestamped get to KTableValueGetter

2023-04-11 Thread via GitHub
mjsax commented on code in PR #13496: URL: https://github.com/apache/kafka/pull/13496#discussion_r1163224319 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/KTableKTableInnerJoin.java: ## @@ -153,11 +154,37 @@ public void init(final ProcessorContext context)

[GitHub] [kafka] cmccabe commented on pull request #13541: KAFKA-14894: MetadataLoader must call finishSnapshot after loading a snapshot

2023-04-11 Thread via GitHub
cmccabe commented on PR #13541: URL: https://github.com/apache/kafka/pull/13541#issuecomment-1503951525 > Do we need a finishSnapshot call in initializeNewPublishers as well? Technically yes. Although finishSnapshot should be a no-op when the previous image is empty. Unfortunat

[GitHub] [kafka] jolshan commented on a diff in pull request #13493: KAFKA-14852: Propagate Topic Ids to the Group Coordinator for Offset Fetch

2023-04-11 Thread via GitHub
jolshan commented on code in PR #13493: URL: https://github.com/apache/kafka/pull/13493#discussion_r1163210151 ## core/src/main/scala/kafka/coordinator/group/GroupMetadata.scala: ## @@ -824,14 +823,14 @@ private[group] class GroupMetadata(val groupId: String, initialState: Grou

[GitHub] [kafka] gharris1727 commented on a diff in pull request #13465: KAFKA-14368: Connect offset write REST API

2023-04-11 Thread via GitHub
gharris1727 commented on code in PR #13465: URL: https://github.com/apache/kafka/pull/13465#discussion_r1163174096 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/Worker.java: ## @@ -1247,6 +1254,224 @@ void sourceConnectorOffsets(String connName, ConnectorOff

[GitHub] [kafka] jolshan commented on a diff in pull request #13505: KAFKA-14462; [5/N] Add EventAccumulator

2023-04-11 Thread via GitHub
jolshan commented on code in PR #13505: URL: https://github.com/apache/kafka/pull/13505#discussion_r1163208120 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/runtime/ConcurrentEventQueue.java: ## @@ -0,0 +1,258 @@ +/* + * Licensed to the Apache Software Fo

[GitHub] [kafka] jolshan commented on a diff in pull request #13505: KAFKA-14462; [5/N] Add EventAccumulator

2023-04-11 Thread via GitHub
jolshan commented on code in PR #13505: URL: https://github.com/apache/kafka/pull/13505#discussion_r1163207183 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/runtime/ConcurrentEventQueue.java: ## @@ -0,0 +1,258 @@ +/* + * Licensed to the Apache Software Fo

[GitHub] [kafka] jolshan commented on a diff in pull request #13505: KAFKA-14462; [5/N] Add EventAccumulator

2023-04-11 Thread via GitHub
jolshan commented on code in PR #13505: URL: https://github.com/apache/kafka/pull/13505#discussion_r1163201442 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/runtime/EventAccumulator.java: ## @@ -0,0 +1,256 @@ +/* + * Licensed to the Apache Software Founda

[GitHub] [kafka] mjsax commented on a diff in pull request #13496: KAFKA-14834: [1/N] Add timestamped get to KTableValueGetter

2023-04-11 Thread via GitHub
mjsax commented on code in PR #13496: URL: https://github.com/apache/kafka/pull/13496#discussion_r1163195888 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/KTableKTableInnerJoin.java: ## @@ -153,11 +154,37 @@ public void init(final ProcessorContext context)

[GitHub] [kafka] mjsax commented on a diff in pull request #13496: KAFKA-14834: [1/N] Add timestamped get to KTableValueGetter

2023-04-11 Thread via GitHub
mjsax commented on code in PR #13496: URL: https://github.com/apache/kafka/pull/13496#discussion_r1163195888 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/KTableKTableInnerJoin.java: ## @@ -153,11 +154,37 @@ public void init(final ProcessorContext context)

[GitHub] [kafka] gharris1727 commented on a diff in pull request #13165: KAFKA-14654: Connector classes should statically initialize with plugin classloader

2023-04-11 Thread via GitHub
gharris1727 commented on code in PR #13165: URL: https://github.com/apache/kafka/pull/13165#discussion_r1163148455 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/isolation/Plugins.java: ## @@ -159,13 +143,7 @@ public ClassLoader compareAndSwapWithDelegatingLoa

[GitHub] [kafka] cmccabe commented on pull request #13540: MINOR: improve QuorumController logging

2023-04-11 Thread via GitHub
cmccabe commented on PR #13540: URL: https://github.com/apache/kafka/pull/13540#issuecomment-1503834445 The test failure is a flake. I opened https://github.com/apache/kafka/pull/13543/files to fix it. -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [kafka] yashmayya commented on a diff in pull request #13465: KAFKA-14368: WIP: Connect offset write REST API

2023-04-11 Thread via GitHub
yashmayya commented on code in PR #13465: URL: https://github.com/apache/kafka/pull/13465#discussion_r1163144352 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/Worker.java: ## @@ -1247,6 +1261,229 @@ void sourceConnectorOffsets(String connName, ConnectorOffse

[GitHub] [kafka] cmccabe opened a new pull request, #13543: MINOR: fix some TOCTOU issues in SharedServer

2023-04-11 Thread via GitHub
cmccabe opened a new pull request, #13543: URL: https://github.com/apache/kafka/pull/13543 Fix some cases in SharedServer where we were assuming that a volatile variable could not be changed from under us. These issues tend to surface during shutdown, when we replace fields with nulls after

[GitHub] [kafka] ahuang98 opened a new pull request, #13542: [MINOR] getPartitionInfo fix

2023-04-11 Thread via GitHub
ahuang98 opened a new pull request, #13542: URL: https://github.com/apache/kafka/pull/13542 `getPartitionInfo` is an accessor of KRaftMetadataCache and I can only imagine it'll cause issues if it's not complete. Specifically, replica, observer, and offlineReplica information is missing from

[GitHub] [kafka] guozhangwang commented on pull request #13523: MINOR: Follow-up on failing streams test, and fix StoreChangelogReader

2023-04-11 Thread via GitHub
guozhangwang commented on PR #13523: URL: https://github.com/apache/kafka/pull/13523#issuecomment-1503820002 @mjsax could you take another look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[jira] [Created] (KAFKA-14894) MetadataLoader must call finishSnapshot after loading a snapshot

2023-04-11 Thread Colin McCabe (Jira)
Colin McCabe created KAFKA-14894: Summary: MetadataLoader must call finishSnapshot after loading a snapshot Key: KAFKA-14894 URL: https://issues.apache.org/jira/browse/KAFKA-14894 Project: Kafka

[GitHub] [kafka] ijuma commented on a diff in pull request #13456: KAFKA-14850: introduce InMemoryLeaderEpochCheckpoint

2023-04-11 Thread via GitHub
ijuma commented on code in PR #13456: URL: https://github.com/apache/kafka/pull/13456#discussion_r1163074411 ## core/src/test/scala/unit/kafka/server/checkpoints/InMemoryLeaderEpochCheckpointTest.scala: ## @@ -0,0 +1,58 @@ +/** + * Licensed to the Apache Software Foundation (ASF

[jira] [Created] (KAFKA-14893) Public API for reporting Yammer metrics

2023-04-11 Thread Mickael Maison (Jira)
Mickael Maison created KAFKA-14893: -- Summary: Public API for reporting Yammer metrics Key: KAFKA-14893 URL: https://issues.apache.org/jira/browse/KAFKA-14893 Project: Kafka Issue Type: Impro

[GitHub] [kafka] dajac commented on a diff in pull request #13524: KIP-848-Interface changes

2023-04-11 Thread via GitHub
dajac commented on code in PR #13524: URL: https://github.com/apache/kafka/pull/13524#discussion_r1163038733 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/common/TopicIdToPartition.java: ## @@ -0,0 +1,50 @@ +/* + * Licensed to the Apache Software Foundati

[GitHub] [kafka] rreddy-22 commented on a diff in pull request #13524: KIP-848-Interface changes

2023-04-11 Thread via GitHub
rreddy-22 commented on code in PR #13524: URL: https://github.com/apache/kafka/pull/13524#discussion_r1163037721 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/common/TopicIdToPartition.java: ## @@ -0,0 +1,76 @@ +/* + * Licensed to the Apache Software Foun

[GitHub] [kafka] guozhangwang commented on a diff in pull request #13534: KAFKA-14054: Handle TimeoutException gracefully

2023-04-11 Thread via GitHub
guozhangwang commented on code in PR #13534: URL: https://github.com/apache/kafka/pull/13534#discussion_r1163035279 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamTask.java: ## @@ -1195,13 +1195,8 @@ public boolean commitNeeded() {

[jira] [Created] (KAFKA-14892) Harmonize package names in storage module

2023-04-11 Thread Ismael Juma (Jira)
Ismael Juma created KAFKA-14892: --- Summary: Harmonize package names in storage module Key: KAFKA-14892 URL: https://issues.apache.org/jira/browse/KAFKA-14892 Project: Kafka Issue Type: Improveme

[jira] [Commented] (KAFKA-14892) Harmonize package names in storage module

2023-04-11 Thread Ismael Juma (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17711038#comment-17711038 ] Ismael Juma commented on KAFKA-14892: - [~satishd] [~junrao] [~showuon] Thoughts? >

[GitHub] [kafka] rreddy-22 commented on a diff in pull request #13524: KIP-848-Interface changes

2023-04-11 Thread via GitHub
rreddy-22 commented on code in PR #13524: URL: https://github.com/apache/kafka/pull/13524#discussion_r1163035038 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/common/TopicIdToPartition.java: ## @@ -0,0 +1,50 @@ +/* + * Licensed to the Apache Software Foun

[GitHub] [kafka] guozhangwang commented on pull request #13523: MINOR: Follow-up on failing streams test, and fix StoreChangelogReader

2023-04-11 Thread via GitHub
guozhangwang commented on PR #13523: URL: https://github.com/apache/kafka/pull/13523#issuecomment-1503662179 > Ideally, rebalancing is a temporary situation and if everything goes well we should leave that state as soon as possible. I totally agree with you @lucasbru . As I mentioned

[GitHub] [kafka] vvcephei commented on a diff in pull request #13534: KAFKA-14054: Handle TimeoutException gracefully

2023-04-11 Thread via GitHub
vvcephei commented on code in PR #13534: URL: https://github.com/apache/kafka/pull/13534#discussion_r1163018118 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamTask.java: ## @@ -1195,13 +1195,8 @@ public boolean commitNeeded() {

[GitHub] [kafka] ijuma commented on pull request #13280: KAFKA-14735: Improve KRaft metadata image change performance at high …

2023-04-11 Thread via GitHub
ijuma commented on PR #13280: URL: https://github.com/apache/kafka/pull/13280#issuecomment-1503626134 > I'm fine with the package/class/method names as they exist right now. +1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

[GitHub] [kafka] C0urante commented on a diff in pull request #13465: KAFKA-14368: WIP: Connect offset write REST API

2023-04-11 Thread via GitHub
C0urante commented on code in PR #13465: URL: https://github.com/apache/kafka/pull/13465#discussion_r1162903146 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/Worker.java: ## @@ -1247,6 +1259,217 @@ void sourceConnectorOffsets(String connName, ConnectorOffset

[GitHub] [kafka] divijvaidya commented on a diff in pull request #13135: KAFKA-14633: Reduce data copy & buffer allocation during decompression

2023-04-11 Thread via GitHub
divijvaidya commented on code in PR #13135: URL: https://github.com/apache/kafka/pull/13135#discussion_r1162952526 ## clients/src/main/java/org/apache/kafka/common/utils/ChunkedBytesStream.java: ## @@ -0,0 +1,211 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[jira] [Commented] (KAFKA-14750) Sink connector fails if a topic matching its topics.regex gets deleted

2023-04-11 Thread Sagar Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17710994#comment-17710994 ] Sagar Rao commented on KAFKA-14750: --- hi [~morozov] , even I am not an expert on the Ka

[GitHub] [kafka] divijvaidya commented on a diff in pull request #13135: KAFKA-14633: Reduce data copy & buffer allocation during decompression

2023-04-11 Thread via GitHub
divijvaidya commented on code in PR #13135: URL: https://github.com/apache/kafka/pull/13135#discussion_r1162924450 ## clients/src/main/java/org/apache/kafka/common/utils/BytesStreamBufferSource.java: ## @@ -0,0 +1,81 @@ +/* + * Licensed to the Apache Software Foundation (ASF) un

[GitHub] [kafka] divijvaidya commented on a diff in pull request #13135: KAFKA-14633: Reduce data copy & buffer allocation during decompression

2023-04-11 Thread via GitHub
divijvaidya commented on code in PR #13135: URL: https://github.com/apache/kafka/pull/13135#discussion_r1162921002 ## clients/src/main/java/org/apache/kafka/common/utils/ChunkedBytesStream.java: ## @@ -0,0 +1,211 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] [kafka] dajac commented on a diff in pull request #13505: KAFKA-14462; [5/N] Add EventDispatcher

2023-04-11 Thread via GitHub
dajac commented on code in PR #13505: URL: https://github.com/apache/kafka/pull/13505#discussion_r1162920767 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/runtime/ConcurrentEventQueue.java: ## @@ -0,0 +1,258 @@ +/* + * Licensed to the Apache Software Foun

[GitHub] [kafka] divijvaidya commented on a diff in pull request #13312: KAFKA-14766: Improve performance of VarInt encoding and decoding

2023-04-11 Thread via GitHub
divijvaidya commented on code in PR #13312: URL: https://github.com/apache/kafka/pull/13312#discussion_r1162915231 ## clients/src/main/java/org/apache/kafka/common/utils/ByteUtils.java: ## @@ -227,17 +268,73 @@ public static int readVarint(DataInput in) throws IOException {

[GitHub] [kafka] dajac commented on a diff in pull request #13505: KAFKA-14462; [5/N] Add EventDispatcher

2023-04-11 Thread via GitHub
dajac commented on code in PR #13505: URL: https://github.com/apache/kafka/pull/13505#discussion_r1162913636 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/runtime/ConcurrentEventQueue.java: ## @@ -0,0 +1,258 @@ +/* + * Licensed to the Apache Software Foun

[GitHub] [kafka] jeffkbkim commented on a diff in pull request #13520: KAFKA-14462; [4/N] Add Group, Record and Result

2023-04-11 Thread via GitHub
jeffkbkim commented on code in PR #13520: URL: https://github.com/apache/kafka/pull/13520#discussion_r1162909701 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/RecordTest.java: ## @@ -0,0 +1,59 @@ +/* + * Licensed to the Apache Software Foundation (ASF) un

[GitHub] [kafka] C0urante commented on a diff in pull request #13465: KAFKA-14368: WIP: Connect offset write REST API

2023-04-11 Thread via GitHub
C0urante commented on code in PR #13465: URL: https://github.com/apache/kafka/pull/13465#discussion_r1162903146 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/Worker.java: ## @@ -1247,6 +1259,217 @@ void sourceConnectorOffsets(String connName, ConnectorOffset

[GitHub] [kafka] C0urante commented on a diff in pull request #13465: KAFKA-14368: WIP: Connect offset write REST API

2023-04-11 Thread via GitHub
C0urante commented on code in PR #13465: URL: https://github.com/apache/kafka/pull/13465#discussion_r1162903146 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/Worker.java: ## @@ -1247,6 +1259,217 @@ void sourceConnectorOffsets(String connName, ConnectorOffset

[GitHub] [kafka] divijvaidya commented on a diff in pull request #13312: KAFKA-14766: Improve performance of VarInt encoding and decoding

2023-04-11 Thread via GitHub
divijvaidya commented on code in PR #13312: URL: https://github.com/apache/kafka/pull/13312#discussion_r1162901912 ## jmh-benchmarks/src/main/java/org/apache/kafka/jmh/util/ByteUtilsBenchmark.java: ## @@ -17,68 +17,241 @@ package org.apache.kafka.jmh.util; -import java.util

  1   2   >