[GitHub] [kafka] showuon commented on a diff in pull request #13525: KAFKA-14883: Expose `observer` state in KRaft metrics

2023-04-07 Thread via GitHub
showuon commented on code in PR #13525: URL: https://github.com/apache/kafka/pull/13525#discussion_r1161067012 ## raft/src/main/java/org/apache/kafka/raft/internals/KafkaRaftMetrics.java: ## @@ -63,9 +63,12 @@ public KafkaRaftMetrics(Metrics metrics, String metricGrpPrefix,

[GitHub] [kafka] showuon commented on a diff in pull request #13525: KAFKA-14883: Expose `observer` state in KRaft metrics

2023-04-07 Thread via GitHub
showuon commented on code in PR #13525: URL: https://github.com/apache/kafka/pull/13525#discussion_r1161067012 ## raft/src/main/java/org/apache/kafka/raft/internals/KafkaRaftMetrics.java: ## @@ -63,9 +63,12 @@ public KafkaRaftMetrics(Metrics metrics, String metricGrpPrefix,

[GitHub] [kafka] artemlivshits commented on a diff in pull request #13391: KAFKA-14561: Improve transactions experience for older clients by ensuring ongoing transaction

2023-04-07 Thread via GitHub
artemlivshits commented on code in PR #13391: URL: https://github.com/apache/kafka/pull/13391#discussion_r1161050689 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -35,6 +36,43 @@ trait ApiRequestHandler { def handle(request: RequestChannel.Request,

[GitHub] [kafka] rondagostino commented on pull request #13280: KAFKA-14735: Improve KRaft metadata image change performance at high …

2023-04-07 Thread via GitHub
rondagostino commented on PR #13280: URL: https://github.com/apache/kafka/pull/13280#issuecomment-1500734932 Thanks for the review @ijuma. I think you have suggested 2 name changes: one to rename the wrapper interfaces and one to Rena e the methods on those interfaces. Current

[GitHub] [kafka] kirktrue commented on pull request #13528: KAFKA-14879: Update system tests to use latest versions

2023-04-07 Thread via GitHub
kirktrue commented on PR #13528: URL: https://github.com/apache/kafka/pull/13528#issuecomment-1500714803 It's meant to be a draft for now. I first want to run through the system tests on a build server to make sure they behave. -- This is an automated message from the Apache Git Service.

[GitHub] [kafka] jolshan commented on pull request #13524: KIP-848-Interface changes

2023-04-07 Thread via GitHub
jolshan commented on PR #13524: URL: https://github.com/apache/kafka/pull/13524#issuecomment-1500714508 @rreddy-22 is there a jira for this change? If so can we use it in the title like KAFKA-XYZ: Name of the JIRA? -- This is an automated message from the Apache Git Service. To

[GitHub] [kafka] jolshan commented on pull request #13528: KAFKA-14879: Update system tests to use latest versions

2023-04-07 Thread via GitHub
jolshan commented on PR #13528: URL: https://github.com/apache/kafka/pull/13528#issuecomment-1500714144 Did you mean to make this a draft or is it ready for review @kirktrue ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [kafka] guozhangwang commented on a diff in pull request #13523: MINOR: Follow-up on failing streams test, and fix StoreChangelogReader

2023-04-07 Thread via GitHub
guozhangwang commented on code in PR #13523: URL: https://github.com/apache/kafka/pull/13523#discussion_r1161010646 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/StoreChangelogReader.java: ## @@ -456,15 +456,21 @@ public long restore(final Map tasks) {

[GitHub] [kafka] mjsax commented on a diff in pull request #13523: MINOR: Follow-up on failing streams test, and fix StoreChangelogReader

2023-04-07 Thread via GitHub
mjsax commented on code in PR #13523: URL: https://github.com/apache/kafka/pull/13523#discussion_r1161000136 ## streams/src/test/java/org/apache/kafka/streams/integration/PauseResumeIntegrationTest.java: ## @@ -188,12 +188,11 @@ public void

[GitHub] [kafka] rreddy-22 commented on a diff in pull request #13524: KIP-848-Interface changes

2023-04-07 Thread via GitHub
rreddy-22 commented on code in PR #13524: URL: https://github.com/apache/kafka/pull/13524#discussion_r116028 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/MemberAssignment.java: ## @@ -16,25 +16,28 @@ */ package

[GitHub] [kafka] mjsax commented on a diff in pull request #13523: MINOR: Follow-up on failing streams test, and fix StoreChangelogReader

2023-04-07 Thread via GitHub
mjsax commented on code in PR #13523: URL: https://github.com/apache/kafka/pull/13523#discussion_r1160999602 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/StoreChangelogReader.java: ## @@ -988,6 +1005,14 @@ private void prepareChangelogs(final Map

[GitHub] [kafka] rreddy-22 commented on a diff in pull request #13524: KIP-848-Interface changes

2023-04-07 Thread via GitHub
rreddy-22 commented on code in PR #13524: URL: https://github.com/apache/kafka/pull/13524#discussion_r1160999728 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/common/TopicIdToPartition.java: ## @@ -0,0 +1,50 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] mjsax commented on a diff in pull request #13523: MINOR: Follow-up on failing streams test, and fix StoreChangelogReader

2023-04-07 Thread via GitHub
mjsax commented on code in PR #13523: URL: https://github.com/apache/kafka/pull/13523#discussion_r1160998689 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/StoreChangelogReader.java: ## @@ -907,6 +920,9 @@ private void

[GitHub] [kafka] mjsax commented on a diff in pull request #13523: MINOR: Follow-up on failing streams test, and fix StoreChangelogReader

2023-04-07 Thread via GitHub
mjsax commented on code in PR #13523: URL: https://github.com/apache/kafka/pull/13523#discussion_r1160998595 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/StoreChangelogReader.java: ## @@ -907,6 +920,9 @@ private void

[GitHub] [kafka] mjsax commented on a diff in pull request #13523: MINOR: Follow-up on failing streams test, and fix StoreChangelogReader

2023-04-07 Thread via GitHub
mjsax commented on code in PR #13523: URL: https://github.com/apache/kafka/pull/13523#discussion_r1160998193 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/StoreChangelogReader.java: ## @@ -880,6 +896,9 @@ private void initializeChangelogs(final Map

[GitHub] [kafka] mjsax commented on a diff in pull request #13523: MINOR: Follow-up on failing streams test, and fix StoreChangelogReader

2023-04-07 Thread via GitHub
mjsax commented on code in PR #13523: URL: https://github.com/apache/kafka/pull/13523#discussion_r1160997049 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/StoreChangelogReader.java: ## @@ -456,15 +456,21 @@ public long restore(final Map tasks) {

[GitHub] [kafka] mjsax commented on a diff in pull request #13523: MINOR: Follow-up on failing streams test, and fix StoreChangelogReader

2023-04-07 Thread via GitHub
mjsax commented on code in PR #13523: URL: https://github.com/apache/kafka/pull/13523#discussion_r1160996434 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/StandbyTask.java: ## @@ -96,7 +96,11 @@ public boolean isActive() { } @Override -

[GitHub] [kafka] mjsax commented on a diff in pull request #13523: MINOR: Follow-up on failing streams test, and fix StoreChangelogReader

2023-04-07 Thread via GitHub
mjsax commented on code in PR #13523: URL: https://github.com/apache/kafka/pull/13523#discussion_r1160996173 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/StandbyTask.java: ## @@ -96,7 +96,10 @@ public boolean isActive() { } @Override -

[GitHub] [kafka] mjsax commented on a diff in pull request #13523: MINOR: Follow-up on failing streams test, and fix StoreChangelogReader

2023-04-07 Thread via GitHub
mjsax commented on code in PR #13523: URL: https://github.com/apache/kafka/pull/13523#discussion_r1160996049 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/StandbyTask.java: ## @@ -96,7 +96,10 @@ public boolean isActive() { } @Override -

[GitHub] [kafka] ijuma commented on a diff in pull request #13280: KAFKA-14735: Improve KRaft metadata image change performance at high …

2023-04-07 Thread via GitHub
ijuma commented on code in PR #13280: URL: https://github.com/apache/kafka/pull/13280#discussion_r1160986944 ## server-common/src/main/java/org/apache/kafka/pcoll/PHashMapWrapper.java: ## @@ -69,41 +48,4 @@ default Map asJava(Function valueMapping) { * @return a wrapped

[GitHub] [kafka] ijuma commented on a diff in pull request #13280: KAFKA-14735: Improve KRaft metadata image change performance at high …

2023-04-07 Thread via GitHub
ijuma commented on code in PR #13280: URL: https://github.com/apache/kafka/pull/13280#discussion_r1160986944 ## server-common/src/main/java/org/apache/kafka/pcoll/PHashMapWrapper.java: ## @@ -69,41 +48,4 @@ default Map asJava(Function valueMapping) { * @return a wrapped

[jira] [Updated] (KAFKA-14879) Update system tests to use latest versions

2023-04-07 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-14879: -- Description: Update the system tests to default to running the latest versions: * 2.6.3 * 2.7.2 *

[GitHub] [kafka] kirktrue opened a new pull request, #13528: KAFKA-14879: Update system tests to use latest versions

2023-04-07 Thread via GitHub
kirktrue opened a new pull request, #13528: URL: https://github.com/apache/kafka/pull/13528 Update the system tests to default to running the latest versions: - 2.6.3 - 2.7.2 - 3.3.2 ### Committer Checklist (excluded from commit message) - [ ] Verify design and

[jira] [Updated] (KAFKA-14879) Update system tests to use latest versions

2023-04-07 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-14879: -- Summary: Update system tests to use latest versions (was: Include newer versions in system tests) >

[GitHub] [kafka] ijuma commented on pull request #13280: KAFKA-14735: Improve KRaft metadata image change performance at high …

2023-04-07 Thread via GitHub
ijuma commented on PR #13280: URL: https://github.com/apache/kafka/pull/13280#issuecomment-1500671475 My suggestion is to introduce a new interface like `ImmutableMap` (`PMap` or `PersistentMap` are also possible, but they can be confused with a map that persists the data) and introduce

[GitHub] [kafka] jolshan commented on a diff in pull request #13391: KAFKA-14561: Improve transactions experience for older clients by ensuring ongoing transaction

2023-04-07 Thread via GitHub
jolshan commented on code in PR #13391: URL: https://github.com/apache/kafka/pull/13391#discussion_r1160980316 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -110,8 +110,23 @@ class KafkaRequestHandler(id: Int, case callback:

[GitHub] [kafka] guozhangwang commented on pull request #13523: [DO NOT MERGE] MINOR: Follow-up on failing streams test

2023-04-07 Thread via GitHub
guozhangwang commented on PR #13523: URL: https://github.com/apache/kafka/pull/13523#issuecomment-1500665780 And just to clarify some context here: currently we have a binary state for the main thread regarding `REBALANCING` and `RUNNING`: the thread would either be spending all its time

[GitHub] [kafka] artemlivshits commented on a diff in pull request #13391: KAFKA-14561: Improve transactions experience for older clients by ensuring ongoing transaction

2023-04-07 Thread via GitHub
artemlivshits commented on code in PR #13391: URL: https://github.com/apache/kafka/pull/13391#discussion_r1160975587 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -110,8 +110,23 @@ class KafkaRequestHandler(id: Int, case callback:

[GitHub] [kafka] guozhangwang commented on pull request #13523: [DO NOT MERGE] MINOR: Follow-up on failing streams test

2023-04-07 Thread via GitHub
guozhangwang commented on PR #13523: URL: https://github.com/apache/kafka/pull/13523#issuecomment-1500663315 Thanks @lihaosky @mjsax , I've refactored the PR to go with option 2), primarily: * Make sure state-updater enabled would also block the main thread from transiting to

[GitHub] [kafka] guozhangwang commented on a diff in pull request #13523: [DO NOT MERGE] MINOR: Follow-up on failing streams test

2023-04-07 Thread via GitHub
guozhangwang commented on code in PR #13523: URL: https://github.com/apache/kafka/pull/13523#discussion_r1160970925 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamTask.java: ## @@ -219,13 +219,13 @@ public boolean isActive() { }

[GitHub] [kafka] mjsax commented on a diff in pull request #10747: KAFKA-12446: Call subtractor before adder if key is the same

2023-04-07 Thread via GitHub
mjsax commented on code in PR #10747: URL: https://github.com/apache/kafka/pull/10747#discussion_r1160948695 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/ChangedSerializer.java: ## @@ -45,34 +49,87 @@ public void setIfUnset(final SerdeGetter getter) {

[GitHub] [kafka] pprovenzano commented on pull request #13513: KAFKA-14881: Update UserScramCredentialRecord for SCRAM ZK to KRaft migration

2023-04-07 Thread via GitHub
pprovenzano commented on PR #13513: URL: https://github.com/apache/kafka/pull/13513#issuecomment-1500649945 Should I move `ScramCredentialData.java` from `metadata/src/main/java/org/apache/kafka/image` to `metadata/src/main/java/org/apache/kafka/metadata `? -- This is an automated

[GitHub] [kafka] pprovenzano commented on pull request #13513: KAFKA-14881: Update UserScramCredentialRecord for SCRAM ZK to KRaft migration

2023-04-07 Thread via GitHub
pprovenzano commented on PR #13513: URL: https://github.com/apache/kafka/pull/13513#issuecomment-1500649607 > Thanks for this, @pprovenzano . Seems like a relatively straightforward change. However, you need to give a reason why we should do this. I think the reason here is for

[GitHub] [kafka] cmccabe commented on pull request #13280: KAFKA-14735: Improve KRaft metadata image change performance at high …

2023-04-07 Thread via GitHub
cmccabe commented on PR #13280: URL: https://github.com/apache/kafka/pull/13280#issuecomment-1500625439 Type safety is a tricky thing. There's no hard and fast rule about where to stop :) I do genuinely think encoding "this is persistent and therefore immutable" in the type system

[GitHub] [kafka] rondagostino commented on pull request #13280: KAFKA-14735: Improve KRaft metadata image change performance at high …

2023-04-07 Thread via GitHub
rondagostino commented on PR #13280: URL: https://github.com/apache/kafka/pull/13280#issuecomment-1500610824 > Also, as you yourself argued, converting back to java.util.Map loses type safety, since now we don't know if we have a persistent map or not. It is quite useful to know that your

[GitHub] [kafka] rondagostino commented on pull request #13280: KAFKA-14735: Improve KRaft metadata image change performance at high …

2023-04-07 Thread via GitHub
rondagostino commented on PR #13280: URL: https://github.com/apache/kafka/pull/13280#issuecomment-1500606628 We can implement the `java.util` interfaces in the wrapper classes. It is trivial to do so via delegation when the underlying library classes already implement those interfaces --

[GitHub] [kafka] cmccabe commented on pull request #13280: KAFKA-14735: Improve KRaft metadata image change performance at high …

2023-04-07 Thread via GitHub
cmccabe commented on PR #13280: URL: https://github.com/apache/kafka/pull/13280#issuecomment-1500597366 It's messy to have to go through a conversion function to use standard interfaces. The fact that we have to do it in Scala is a big negative for Scala. I'm greatly looking forward to no

[GitHub] [kafka] mjsax commented on pull request #13382: KAFKA-14722: Make BooleanSerde public

2023-04-07 Thread via GitHub
mjsax commented on PR #13382: URL: https://github.com/apache/kafka/pull/13382#issuecomment-1500595771 @SpacRocket -- what @divijvaidya says. -- Can you do a follow up PR? This is a smaller feature though, and I don't think it should go into `docs/upgrade.html` but only into

[jira] [Resolved] (KAFKA-14318) KIP-878: Autoscaling for Statically Partitioned Streams

2023-04-07 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-14318. - Resolution: Fixed > KIP-878: Autoscaling for Statically Partitioned Streams >

[jira] [Reopened] (KAFKA-14318) KIP-878: Autoscaling for Statically Partitioned Streams

2023-04-07 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reopened KAFKA-14318: - > KIP-878: Autoscaling for Statically Partitioned Streams >

[GitHub] [kafka] mjsax opened a new pull request, #13527: Revert "KAFKA-14318: KIP-878, Introduce partition autoscaling configs (#12962)"

2023-04-07 Thread via GitHub
mjsax opened a new pull request, #13527: URL: https://github.com/apache/kafka/pull/13527 This reverts commit d9b139220ee253da673af44d58dc87bd184188f1. KIP-878 implementation did not make any progress, so we need to revert the public API changes which are not functional right now.

[jira] [Assigned] (KAFKA-14318) KIP-878: Autoscaling for Statically Partitioned Streams

2023-04-07 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax reassigned KAFKA-14318: --- Assignee: (was: A. Sophie Blee-Goldman) > KIP-878: Autoscaling for Statically

[jira] [Updated] (KAFKA-14318) KIP-878: Autoscaling for Statically Partitioned Streams

2023-04-07 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-14318: Fix Version/s: (was: 3.5.0) > KIP-878: Autoscaling for Statically Partitioned Streams

[jira] [Updated] (KAFKA-14879) Include newer versions in system tests

2023-04-07 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-14879: -- Description: Update the system tests to default to running the latest versions in the different

[jira] [Updated] (KAFKA-14879) Include newer versions in system tests

2023-04-07 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-14879: -- Description: Update the system tests to default to running the latest versions in the different

[jira] [Updated] (KAFKA-14879) Include newer versions in system tests

2023-04-07 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-14879: -- Description: Update the system tests to default to running the latest versions in the different

[GitHub] [kafka] vcrfxia commented on pull request #13523: [DO NOT MERGE] MINOR: Follow-up on failing streams test

2023-04-07 Thread via GitHub
vcrfxia commented on PR #13523: URL: https://github.com/apache/kafka/pull/13523#issuecomment-1500571126 I have a similar clarification question as Hao: what is the significance of the instance state transitioning to `RUNNING`? Is it the case that until the instance state transitions to

[GitHub] [kafka] mjsax commented on a diff in pull request #13496: KAFKA-14834: [1/N] Add timestamped get to KTableValueGetter

2023-04-07 Thread via GitHub
mjsax commented on code in PR #13496: URL: https://github.com/apache/kafka/pull/13496#discussion_r1160906740 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/KTableKTableInnerJoin.java: ## @@ -153,11 +154,37 @@ public void init(final ProcessorContext context)

[GitHub] [kafka] jolshan commented on a diff in pull request #13391: KAFKA-14561: Improve transactions experience for older clients by ensuring ongoing transaction

2023-04-07 Thread via GitHub
jolshan commented on code in PR #13391: URL: https://github.com/apache/kafka/pull/13391#discussion_r1160896272 ## core/src/main/scala/kafka/server/KafkaApis.scala: ## @@ -2334,6 +2334,10 @@ class KafkaApis(val requestChannel: RequestChannel, } } + // If

[GitHub] [kafka] jolshan commented on a diff in pull request #13391: KAFKA-14561: Improve transactions experience for older clients by ensuring ongoing transaction

2023-04-07 Thread via GitHub
jolshan commented on code in PR #13391: URL: https://github.com/apache/kafka/pull/13391#discussion_r1160895598 ## core/src/main/scala/kafka/network/RequestChannel.scala: ## @@ -227,8 +233,9 @@ object RequestChannel extends Logging { } val requestQueueTimeMs =

[GitHub] [kafka] artemlivshits commented on a diff in pull request #13391: KAFKA-14561: Improve transactions experience for older clients by ensuring ongoing transaction

2023-04-07 Thread via GitHub
artemlivshits commented on code in PR #13391: URL: https://github.com/apache/kafka/pull/13391#discussion_r1160892559 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -69,10 +108,23 @@ class KafkaRequestHandler(id: Int, completeShutdown()

[GitHub] [kafka] rondagostino commented on pull request #13280: KAFKA-14735: Improve KRaft metadata image change performance at high …

2023-04-07 Thread via GitHub
rondagostino commented on PR #13280: URL: https://github.com/apache/kafka/pull/13280#issuecomment-1500542906 > I think we have to implement Set, Map, and TreeMap. Using custom types just hurts programmer productivity and ability to understand the code too much... [it] is a lot of tricky

[GitHub] [kafka] cmccabe commented on pull request #13280: KAFKA-14735: Improve KRaft metadata image change performance at high …

2023-04-07 Thread via GitHub
cmccabe commented on PR #13280: URL: https://github.com/apache/kafka/pull/13280#issuecomment-1500536745 > i think you are referring to having to implement that if we don't go with the current PR implementation and instead go with something like what you proposed -- since the current PR

[GitHub] [kafka] artemlivshits commented on a diff in pull request #13391: KAFKA-14561: Improve transactions experience for older clients by ensuring ongoing transaction

2023-04-07 Thread via GitHub
artemlivshits commented on code in PR #13391: URL: https://github.com/apache/kafka/pull/13391#discussion_r1160875392 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -108,11 +108,10 @@ class KafkaRequestHandler(id: Int, completeShutdown()

[GitHub] [kafka] muralibasani commented on pull request #13417: KAFKA-14585: Moving StorageTool from core to tools module

2023-04-07 Thread via GitHub
muralibasani commented on PR #13417: URL: https://github.com/apache/kafka/pull/13417#issuecomment-1500525012 > * RaftConfig @mimaison while continuing on this, moving constants to RaftConfig, creating classes like MetaProperties, RawMetaProperties looks okish. But this one

[GitHub] [kafka] rondagostino commented on pull request #13280: KAFKA-14735: Improve KRaft metadata image change performance at high …

2023-04-07 Thread via GitHub
rondagostino commented on PR #13280: URL: https://github.com/apache/kafka/pull/13280#issuecomment-1500522402 > Probably the bigger issue is the extra complexity of implementing our own Set / Map / TreeMap / etc. It's all doable (and perhaps we'll end up doing it for Vavr) but it feels kind

[GitHub] [kafka] mjsax merged pull request #13449: KAFKA-14491: [22/N] Add test for manual upgrade to versioned store

2023-04-07 Thread via GitHub
mjsax merged PR #13449: URL: https://github.com/apache/kafka/pull/13449 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] jolshan commented on a diff in pull request #13391: KAFKA-14561: Improve transactions experience for older clients by ensuring ongoing transaction

2023-04-07 Thread via GitHub
jolshan commented on code in PR #13391: URL: https://github.com/apache/kafka/pull/13391#discussion_r1160872423 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -69,10 +108,23 @@ class KafkaRequestHandler(id: Int, completeShutdown()

[GitHub] [kafka] jolshan commented on a diff in pull request #13391: KAFKA-14561: Improve transactions experience for older clients by ensuring ongoing transaction

2023-04-07 Thread via GitHub
jolshan commented on code in PR #13391: URL: https://github.com/apache/kafka/pull/13391#discussion_r1160872423 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -69,10 +108,23 @@ class KafkaRequestHandler(id: Int, completeShutdown()

[GitHub] [kafka] artemlivshits commented on a diff in pull request #13391: KAFKA-14561: Improve transactions experience for older clients by ensuring ongoing transaction

2023-04-07 Thread via GitHub
artemlivshits commented on code in PR #13391: URL: https://github.com/apache/kafka/pull/13391#discussion_r1160849589 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -69,10 +108,23 @@ class KafkaRequestHandler(id: Int, completeShutdown()

[GitHub] [kafka] dajac commented on a diff in pull request #13505: KAFKA-14462; [5/N] Add EventDispatcher

2023-04-07 Thread via GitHub
dajac commented on code in PR #13505: URL: https://github.com/apache/kafka/pull/13505#discussion_r1160571805 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/runtime/EventDispatcher.java: ## @@ -0,0 +1,258 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] lihaosky commented on pull request #13523: [DO NOT MERGE] MINOR: Follow-up on failing streams test

2023-04-07 Thread via GitHub
lihaosky commented on PR #13523: URL: https://github.com/apache/kafka/pull/13523#issuecomment-1500494292 I feel option 2 makes more sense. To clarify: option 1 means that the task is transitioned to `RUNNING` state while it's paused and restoring? Somehow this kind of doesn't make sense if

[GitHub] [kafka] cmccabe commented on pull request #13280: KAFKA-14735: Improve KRaft metadata image change performance at high …

2023-04-07 Thread via GitHub
cmccabe commented on PR #13280: URL: https://github.com/apache/kafka/pull/13280#issuecomment-1500464493 > It is true that the wrapper implementation in the PR as it stands right now always creates a new object when creating a new persistent collection (either by creating an empty/singleton

[GitHub] [kafka] jolshan commented on a diff in pull request #13391: KAFKA-14561: Improve transactions experience for older clients by ensuring ongoing transaction

2023-04-07 Thread via GitHub
jolshan commented on code in PR #13391: URL: https://github.com/apache/kafka/pull/13391#discussion_r1160831069 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -69,10 +108,23 @@ class KafkaRequestHandler(id: Int, completeShutdown()

[GitHub] [kafka] jolshan commented on a diff in pull request #13391: KAFKA-14561: Improve transactions experience for older clients by ensuring ongoing transaction

2023-04-07 Thread via GitHub
jolshan commented on code in PR #13391: URL: https://github.com/apache/kafka/pull/13391#discussion_r1160830029 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -69,10 +108,23 @@ class KafkaRequestHandler(id: Int, completeShutdown()

[GitHub] [kafka] vcrfxia commented on a diff in pull request #13496: KAFKA-14834: [1/N] Add timestamped get to KTableValueGetter

2023-04-07 Thread via GitHub
vcrfxia commented on code in PR #13496: URL: https://github.com/apache/kafka/pull/13496#discussion_r1160829732 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/KTableKTableInnerJoin.java: ## @@ -153,11 +154,37 @@ public void init(final ProcessorContext

[GitHub] [kafka] jeffkbkim opened a new pull request, #13526: KAFKA-14869: bump coordinator Value records to flexible versions

2023-04-07 Thread via GitHub
jeffkbkim opened a new pull request, #13526: URL: https://github.com/apache/kafka/pull/13526 KIP-915: Part 2 of 2 Bump all Value records in __transaction_state and __consumer_offsets topics to flexible versions. Serialize value records to highest known non-flexible version to allow

[jira] [Updated] (KAFKA-14869) txn and group coordinator downgrade foundation

2023-04-07 Thread Jeff Kim (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Kim updated KAFKA-14869: - Description: Implement proposed changes in

[GitHub] [kafka] jolshan commented on a diff in pull request #13391: KAFKA-14561: Improve transactions experience for older clients by ensuring ongoing transaction

2023-04-07 Thread via GitHub
jolshan commented on code in PR #13391: URL: https://github.com/apache/kafka/pull/13391#discussion_r1160819448 ## core/src/main/scala/kafka/server/AddPartitionsToTxnManager.scala: ## @@ -0,0 +1,170 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more +

[jira] [Created] (KAFKA-14884) Include check transaction is still ongoing right before append

2023-04-07 Thread Justine Olshan (Jira)
Justine Olshan created KAFKA-14884: -- Summary: Include check transaction is still ongoing right before append Key: KAFKA-14884 URL: https://issues.apache.org/jira/browse/KAFKA-14884 Project: Kafka

[GitHub] [kafka] jolshan commented on a diff in pull request #13391: KAFKA-14561: Improve transactions experience for older clients by ensuring ongoing transaction

2023-04-07 Thread via GitHub
jolshan commented on code in PR #13391: URL: https://github.com/apache/kafka/pull/13391#discussion_r1160817791 ## core/src/main/scala/kafka/server/AddPartitionsToTxnManager.scala: ## @@ -0,0 +1,173 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more +

[GitHub] [kafka] jsancio merged pull request #13345: KAFKA-13020; Implement reading Snapshot log append timestamp

2023-04-07 Thread via GitHub
jsancio merged PR #13345: URL: https://github.com/apache/kafka/pull/13345 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] jolshan commented on a diff in pull request #13391: KAFKA-14561: Improve transactions experience for older clients by ensuring ongoing transaction

2023-04-07 Thread via GitHub
jolshan commented on code in PR #13391: URL: https://github.com/apache/kafka/pull/13391#discussion_r1160807017 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -69,10 +108,23 @@ class KafkaRequestHandler(id: Int, completeShutdown()

[jira] [Resolved] (KAFKA-14617) Replicas with stale broker epoch should not be allowed to join the ISR

2023-04-07 Thread Jun Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-14617. - Fix Version/s: 3.5.0 Resolution: Fixed merged all PRs to trunk. > Replicas with stale broker

[GitHub] [kafka] junrao merged pull request #13489: KAFKA-14617: Fill broker epochs to the AlterPartitionRequest

2023-04-07 Thread via GitHub
junrao merged PR #13489: URL: https://github.com/apache/kafka/pull/13489 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] jolshan commented on a diff in pull request #13391: KAFKA-14561: Improve transactions experience for older clients by ensuring ongoing transaction

2023-04-07 Thread via GitHub
jolshan commented on code in PR #13391: URL: https://github.com/apache/kafka/pull/13391#discussion_r1160803600 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -69,10 +108,23 @@ class KafkaRequestHandler(id: Int, completeShutdown()

[GitHub] [kafka] jolshan commented on a diff in pull request #13391: KAFKA-14561: Improve transactions experience for older clients by ensuring ongoing transaction

2023-04-07 Thread via GitHub
jolshan commented on code in PR #13391: URL: https://github.com/apache/kafka/pull/13391#discussion_r1160803600 ## core/src/main/scala/kafka/server/KafkaRequestHandler.scala: ## @@ -69,10 +108,23 @@ class KafkaRequestHandler(id: Int, completeShutdown()

[GitHub] [kafka] rondagostino commented on pull request #13280: KAFKA-14735: Improve KRaft metadata image change performance at high …

2023-04-07 Thread via GitHub
rondagostino commented on PR #13280: URL: https://github.com/apache/kafka/pull/13280#issuecomment-1500342559 Thanks for the review, @cmccabe. No problem with the package change and the more precise import controls. Regarding the wrapper suggestion: It is true that the wrapper

[GitHub] [kafka] showuon commented on a diff in pull request #13525: KAFKA-14883: Expose `observer` state in KRaft metrics

2023-04-07 Thread via GitHub
showuon commented on code in PR #13525: URL: https://github.com/apache/kafka/pull/13525#discussion_r1160682237 ## raft/src/test/java/org/apache/kafka/raft/internals/KafkaRaftMetricsTest.java: ## @@ -137,14 +137,14 @@ public void shouldRecordNonVoterQuorumState() throws

[GitHub] [kafka] ppatierno commented on pull request #13525: KAFKA-14883: Expose `observer` state in KRaft metrics

2023-04-07 Thread via GitHub
ppatierno commented on PR #13525: URL: https://github.com/apache/kafka/pull/13525#issuecomment-1500214479 @showuon thanks! just updated :-) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [kafka] ppatierno commented on pull request #13525: KAFKA-14883: Expose `observer` state in KRaft metrics

2023-04-07 Thread via GitHub
ppatierno commented on PR #13525: URL: https://github.com/apache/kafka/pull/13525#issuecomment-1500137557 @hachikuji maybe you can take a look to this PR? Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [kafka] ppatierno opened a new pull request, #13525: KAFKA-14883: Expose `observer` state in KRaft metrics

2023-04-07 Thread via GitHub
ppatierno opened a new pull request, #13525: URL: https://github.com/apache/kafka/pull/13525 Currently, the `current-state` KRaft related metric reports `follower` state for a broker while technically it should be reported as an `observer` as the `kafka-metadata-quorum` tool does.

[jira] [Created] (KAFKA-14883) Broker state should be "observer" in KRaft quorum

2023-04-07 Thread Paolo Patierno (Jira)
Paolo Patierno created KAFKA-14883: -- Summary: Broker state should be "observer" in KRaft quorum Key: KAFKA-14883 URL: https://issues.apache.org/jira/browse/KAFKA-14883 Project: Kafka Issue

[GitHub] [kafka] dajac commented on a diff in pull request #13524: KIP-848-Interface changes

2023-04-07 Thread via GitHub
dajac commented on code in PR #13524: URL: https://github.com/apache/kafka/pull/13524#discussion_r1160578628 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/common/TopicIdToPartition.java: ## @@ -0,0 +1,50 @@ +/* + * Licensed to the Apache Software

[jira] [Resolved] (KAFKA-12634) Should checkpoint after restore finished

2023-04-07 Thread Bruno Cadonna (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Cadonna resolved KAFKA-12634. --- Resolution: Fixed > Should checkpoint after restore finished >

[jira] [Updated] (KAFKA-12634) Should checkpoint after restore finished

2023-04-07 Thread Bruno Cadonna (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Cadonna updated KAFKA-12634: -- Fix Version/s: 3.5.0 > Should checkpoint after restore finished >

[GitHub] [kafka] cadonna merged pull request #13269: KAFKA-12634 enforce checkpoint after restoration

2023-04-07 Thread via GitHub
cadonna merged PR #13269: URL: https://github.com/apache/kafka/pull/13269 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] dajac commented on a diff in pull request #13505: KAFKA-14462; [5/N] Add EventDispatcher

2023-04-07 Thread via GitHub
dajac commented on code in PR #13505: URL: https://github.com/apache/kafka/pull/13505#discussion_r1160571805 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/runtime/EventDispatcher.java: ## @@ -0,0 +1,258 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] cadonna commented on pull request #13269: KAFKA-12634 enforce checkpoint after restoration

2023-04-07 Thread via GitHub
cadonna commented on PR #13269: URL: https://github.com/apache/kafka/pull/13269#issuecomment-1500105174 Build failures are unrelated: ``` Build / JDK 8 and Scala 2.12 / kafka.security.authorizer.AuthorizerTest.testAllowAccessWithCustomPrincipal(String).quorum= Build / JDK 11 and