Jenkins build is unstable: Kafka » Kafka Branch Builder » trunk #2198

See

Re: Apache Kafka 3.6.0 release

Thanks David for the quick resolution. ~Satish. On Tue, 12 Sept 2023 at 22:51, David Arthur wrote: > > Satish, > > KAFKA-15450 is merged to 3.6 (as well as trunk, 3.5, and 3.4) > > Thanks! > David > > On Tue, Sep 12, 2023 at 11:44 AM Ismael Juma wrote: > > > Justine, > > > > Probably best to

Re: [DISCUSS] KIP-966: Eligible Leader Replicas

Hi Colin Thanks for the comments! Rewrite the Additional High Watermark advancement requirement There was feedback on this section that some readers may not be familiar with HWM and Ack=0,1,all requests. This can help them understand the proposal. I will rewrite this part for more readability.

Re: [DISCUSS] KIP-966: Eligible Leader Replicas

Hi Jun, I have a second thought on the kafka.replication.electable_replicas_count. I don't think it is a useful metric and it seems wasteful to have a per partition metric to track the sum(ISR, ELR). I will remove this metric. On Tue, Sep 12, 2023 at 3:35 PM Calvin Liu wrote: > Hi Jun, > Thanks

Re: Apache Kafka 3.6.0 release

Hi Justine, Thanks for the update. From JIRA[1] it looks like it affects only 3.6.0 but not for the earlier releases. Is that right? https://issues.apache.org/jira/browse/KAFKA-15459 ~Satish. On Wed, 13 Sept 2023 at 00:42, Justine Olshan wrote: > > It's me again.  > While reviewing the

Re: [DISCUSS] KIP-966: Eligible Leader Replicas

Hi Jun, Thanks for the follow-up. Adjusted the KIP as following. 16. The controller does not know whether a broker is live or not when it is fenced. Updated the behavior. In Balanced mode, the URM will start recovery when all the members in the LastKnownELR are all unfenced. In the Proactive

Re: [VOTE] KIP-942: Add Power(ppc64le) support

I just disagree with the idea of running it for every commit. If we can't compromise on this condition then I just have to cast a -1. It's tough enough to get stuff committed since our tests take quite a long time. The fact that we only manage 5-6 commits a day is a bad thing, and not

Re: [DISCUSS] KIP-966: Eligible Leader Replicas

Also, we should have metrics that show what is going on with regard to the eligible replica set. I'm not sure exactly what to suggest, but something that could identify when things are going wrong in the clsuter. For example, maybe a metric for partitions containing replicas that are

Re: [VOTE] KIP-858: Handle JBOD broker disk failure in KRaft

Thanks, Igor. Regarding the last point, I want to drill in on something. The KIP says this: "Replicas will also be considered offline if the replica references a log directory UUID (in the new field partitionRecord.Assignment.Directory) that is not present in the hosting Broker's latest

Re: [DISCUSS] KIP-966: Eligible Leader Replicas

Hi Calvin, Thanks for the KIP. I think this is a great improvement. > Additional High Watermark advance requirement Typo: change "advance" to "advancement" > A bit recap of some key concepts. Typo: change "bit" to "quick" > Ack=1/all produce request. It defines when the Kafka server should

Jenkins build is unstable: Kafka » Kafka Branch Builder » 3.4 #165

See

Build failed in Jenkins: Kafka » Kafka Branch Builder » 3.5 #72

See Changes: -- [...truncated 281886 lines...] Gradle Test Run :streams:integrationTest > Gradle Test Executor 180 > VersionedKeyValueStoreIntegrationTest > shouldRestore

Jenkins build is still unstable: Kafka » Kafka Branch Builder » 3.6 #37

See

Re: Apache Kafka 3.6.0 release

It's me again.  While reviewing the previous PR, it was discovered we had a potential breaking return code for non-java clients. This unfortunately seems like a blocker. https://issues.apache.org/jira/browse/KAFKA-15459 I will be able to get a PR open today. Apologies for all the noise in the

[jira] [Created] (KAFKA-15459) Convert coordinator retriable errors to a known producer response error.

Justine Olshan created KAFKA-15459: -- Summary: Convert coordinator retriable errors to a known producer response error. Key: KAFKA-15459 URL: https://issues.apache.org/jira/browse/KAFKA-15459

Re: [DISCUSS] KIP-966: Eligible Leader Replicas

Hi, Calvin, Thanks for the reply and the updated KIP. A few more comments. 16. "The URM will query all the replicas including the fenced replicas." Currently, could we tell whether a fenced broker is alive or not? Ideally, we only want to query the live brokers. Otherwise URM will be forced to

[jira] [Resolved] (KAFKA-15450) Disable ZK migration when JBOD configured

[ https://issues.apache.org/jira/browse/KAFKA-15450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Arthur resolved KAFKA-15450. -- Resolution: Fixed > Disable ZK migration when JBOD configured >

Re: Apache Kafka 3.6.0 release

Satish, KAFKA-15450 is merged to 3.6 (as well as trunk, 3.5, and 3.4) Thanks! David On Tue, Sep 12, 2023 at 11:44 AM Ismael Juma wrote: > Justine, > > Probably best to have the conversation in the JIRA ticket vs the release > thread. Generally, we want to only include low risk bug fixes that

Jenkins build is still unstable: Kafka » Kafka Branch Builder » 3.6 #36

See

[jira] [Created] (KAFKA-15458) Fully resolve endpoint information before registering controllers

Colin McCabe created KAFKA-15458: Summary: Fully resolve endpoint information before registering controllers Key: KAFKA-15458 URL: https://issues.apache.org/jira/browse/KAFKA-15458 Project: Kafka

Re: Apache Kafka 3.6.0 release

Justine, Probably best to have the conversation in the JIRA ticket vs the release thread. Generally, we want to only include low risk bug fixes that are fully compatible in patch releases. Ismael On Tue, Sep 12, 2023 at 7:16 AM Justine Olshan wrote: > Thanks Satish. I understand. > Just

Re: [DISCUSS] KIP-977: Partition-Level Throughput Metrics

Oh, and does metrics.partition.level.reporting.topics allow for regex? > On Sep 12, 2023, at 8:26 AM, Kirk True wrote: > > Hi Qichao, > > Thanks for the KIP! > > Divij—questions/comments inline... > >> On Sep 11, 2023, at 4:32 AM, Divij Vaidya wrote: >> >> Thank you for the proposal

Re: [DISCUSS] KIP-977: Partition-Level Throughput Metrics

Hi Qichao, Thanks for the KIP! Divij—questions/comments inline... > On Sep 11, 2023, at 4:32 AM, Divij Vaidya wrote: > > Thank you for the proposal Qichao. > > I agree with the motivation here and understand the tradeoff here > between observability vs. increased metric dimensions (metric

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #2196

See

[jira] [Created] (KAFKA-15457) Add support for OffsetFetch version 9 in admin

David Jacot created KAFKA-15457: --- Summary: Add support for OffsetFetch version 9 in admin Key: KAFKA-15457 URL: https://issues.apache.org/jira/browse/KAFKA-15457 Project: Kafka Issue Type:

[jira] [Created] (KAFKA-15456) Add support for OffsetFetch version 9 in consumer

David Jacot created KAFKA-15456: --- Summary: Add support for OffsetFetch version 9 in consumer Key: KAFKA-15456 URL: https://issues.apache.org/jira/browse/KAFKA-15456 Project: Kafka Issue Type:

Re: [VOTE] KIP-858: Handle JBOD broker disk failure in KRaft

Hi Ron, Thank you for having a look a this KIP. Indeed, the log directory UUID should always be generated and loaded. I've have corrected the wording in the KIP to clarify. It is a bit of a pain to replace the field, but I agree that is the best approach for the same reason you pointed out. I

[jira] [Created] (KAFKA-15455) Add support for OffsetCommit version 9 in consumer

David Jacot created KAFKA-15455: --- Summary: Add support for OffsetCommit version 9 in consumer Key: KAFKA-15455 URL: https://issues.apache.org/jira/browse/KAFKA-15455 Project: Kafka Issue Type:

[jira] [Created] (KAFKA-15454) Add support for OffsetCommit version 9 in admin client

David Jacot created KAFKA-15454: --- Summary: Add support for OffsetCommit version 9 in admin client Key: KAFKA-15454 URL: https://issues.apache.org/jira/browse/KAFKA-15454 Project: Kafka Issue

Re: Apache Kafka 3.6.0 release

Thanks Satish. I understand. Just curious, is this something that could be added to 3.6.1? It would be nice to say that hanging transactions are fully covered in a 3.6 release. I'm not as familiar with the rules around minor releases, but adding it there would give more time to ensure stability.

Re: [VOTE] KIP-714: Client metrics and observability

Hi Philip, Thanks for your vote and interest in the KIP. KIP-714 does not introduce any new client metrics, and that’s intentional. It does tell how that all of the client metrics can have their names transformed into equivalent "telemetry metric names”, and then potentially used in metrics

Re: Apache Kafka 3.6.0 release

Hi Justine, We can skip this change into 3.6 now as it is not a blocker or regression and it involves changes to the API implementation. Let us plan to add the gap in the release notes as you mentioned. Thanks, Satish. On Tue, 12 Sept 2023 at 04:44, Justine Olshan wrote: > > Hey Satish, > > We

Re: Apache Kafka 3.6.0 release

Hi David, It looks like the PR is already accepted with a minor config change validation, and you mentioned that the failed tests are not related. Please merge it to 3.6 by today. Thanks, Satish. On Tue, 12 Sept 2023 at 00:59, David Arthur wrote: > > Another (small) ZK migration issue was

[jira] [Resolved] (KAFKA-14993) Improve TransactionIndex instance handling while copying to and fetching from RSM.

[ https://issues.apache.org/jira/browse/KAFKA-14993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Satish Duggana resolved KAFKA-14993. Resolution: Fixed > Improve TransactionIndex instance handling while copying to and

Re: [VOTE] KIP-858: Handle JBOD broker disk failure in KRaft

Hi Ziming, Thank you for having a look and taking the time to vote. I have already opened some PRs, see: https://issues.apache.org/jira/browse/KAFKA-14127 Best, -- Igor

[jira] [Resolved] (KAFKA-14067) Sink connector override.consumer.group.id can conflict with worker group.id

[ https://issues.apache.org/jira/browse/KAFKA-14067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yash Mayya resolved KAFKA-14067. Resolution: Fixed > Sink connector override.consumer.group.id can conflict with worker group.id >

Jenkins build is still unstable: Kafka » Kafka Branch Builder » 3.6 #35

See

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #2195

See

Re: Complete Kafka replication protocol description

Hi Jack, thanks for providing this material. I'll spend some time digging into it. On Mon, Sep 11, 2023 at 6:56 PM Jack Vanlightly wrote: > > Hi all, > > I agree that we should have the protocol description and specification in > the Kafka repository. I have two other specifications to