[jira] [Created] (KAFKA-15680) Partition-Count is not getting updated Correctly in the Incremental Co-operative Rebalancing(ICR) Mode of Rebalancing

Pritam Kumar created KAFKA-15680: Summary: Partition-Count is not getting updated Correctly in the Incremental Co-operative Rebalancing(ICR) Mode of Rebalancing Key: KAFKA-15680 URL:

Re: [DISCUSS] KIP-975 Docker Image for Apache Kafka

The reference I meant to include: https://docs.oracle.com/en/java/javase/17/vm/class-data-sharing.html On Tue, Oct 24, 2023, 10:25 PM Ismael Juma wrote: > Hi Krishna, > > One last question from me, did we confuse using AppCDS or Dynamic CDS? > > Thanks, > Ismael > > On Tue, Oct 24, 2023, 9:54 

Re: [DISCUSS] KIP-975 Docker Image for Apache Kafka

Hi Krishna, One last question from me, did we confuse using AppCDS or Dynamic CDS? Thanks, Ismael On Tue, Oct 24, 2023, 9:54 PM Krishna Agarwal wrote: > Hi, > Thanks for the insightful feedback on this KIP. If there are no further > questions, I'm considering wrapping up this discussion

Re: [DISCUSS] KIP-975 Docker Image for Apache Kafka

Hi, Thanks for the insightful feedback on this KIP. If there are no further questions, I'm considering wrapping up this discussion thread. We'll be moving into the voting process in the next couple of days. Your continued input is greatly appreciated! Regards, Krishna On Fri, Sep 8, 2023 at 1:27 

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #2326

See Changes: -- [...truncated 320910 lines...] Gradle Test Run :core:test > Gradle Test Executor 88 > ZkMigrationClientTest > testEmptyWrite() STARTED Gradle Test Run

Jenkins build is still unstable: Kafka » Kafka Branch Builder » 3.5 #88

See

Jenkins build is unstable: Kafka » Kafka Branch Builder » trunk #2325

See

[jira] [Resolved] (KAFKA-15304) CompletableApplicationEvents aren't being completed when the consumer is closing

[ https://issues.apache.org/jira/browse/KAFKA-15304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True resolved KAFKA-15304. --- Fix Version/s: 3.7.0 Reviewer: Jun Rao Resolution: Fixed Fixed as part of PR

Re: UncleanLeaderElectionsPerSec metric and Raft

Hey folks, Thanks for replying. If we could file a JIRA to track this work, that would be great. Thanks, Justine On Tue, Oct 24, 2023 at 11:55 AM Colin McCabe wrote: > Hi Neil, > > Yes, I think we should probably report the UncleanLeaderElectionsPerSec > metric in KRaft. We don't have it

Re: [DISCUSS] How to detect (and prevent) complex bugs in Kafka?

Hey Colin, For context on this specific issue, we have opened a JIRA to consider thread safety in the future. Another option is documentation or to make thread local. Don't want to detract too much from this conversation, but did want to say there is a JIRA to discuss the buffer specific problem.

[VOTE] KIP-978: Allow dynamic reloading of certificates with different DN / SANs

Hi all, I would like to start a vote for the KIP-978: Allow dynamic reloading of certificates with different DN / SANs . Thanks & Regards Jakub

Re: [DISCUSS] KIP-968: Support single-key_multi-timestamp interactive queries (IQv2) for versioned state stores

Thank you, Matthias, Bruno, and Guozhang for keeping the discussion going. Here is the list of changes I made: 1. I enriched the "Example" section as Bruno suggested. Do you please have a look at that section? I think I devised an interesting one ;-) 2. As Matthias and Guozhang

Re: [DISCUSS] How to detect (and prevent) complex bugs in Kafka?

Hi Divij, I've worked on several projects that had a "debug mode." It was something that a lot of old-fashioned C and C++ projects would do. Usually implemented through an ASSERT macro or similar that was defined away when in "production mode" I didn't like this back then, and still don't like

Re: UncleanLeaderElectionsPerSec metric and Raft

Hi Neil, Yes, I think we should probably report the UncleanLeaderElectionsPerSec metric in KRaft. We don't have it currently. We do have the concept of unclean leader election in KRaft, but it has to be triggered by the leader election tool currently. We've been talking about adding

Re: UncleanLeaderElectionsPerSec metric and Raft

Hi Neil, You are correct. I don't think we have implemented the unclean election configuration. The kafka-leader-election CLI works with KRaft. Have you looked at that option for performing unclean leader elections with the CLI? Does it meet your requirements? In general using the unclean

[jira] [Created] (KAFKA-15679) Client support new consumer configs

Lianet Magrans created KAFKA-15679: -- Summary: Client support new consumer configs Key: KAFKA-15679 URL: https://issues.apache.org/jira/browse/KAFKA-15679 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-15678) [Tiered Storage] Stall remote reads with long-spanning transactions

Alexandre Dupriez created KAFKA-15678: - Summary: [Tiered Storage] Stall remote reads with long-spanning transactions Key: KAFKA-15678 URL: https://issues.apache.org/jira/browse/KAFKA-15678

Re: [DISCUSS] How to detect (and prevent) complex bugs in Kafka?

Hi Divij, I have limited experience tackling bugs in Kafka, but for the issues I've fixed are usually found through..: 1. flaky unit/integration/system tests 2. Insufficient test coverage: When trying to add tests... 3. Related to #2 - Found through production at scale I have heard Kraft

[jira] [Created] (KAFKA-15677) Add support for re2/j regex parsing for patterns

Apoorv Mittal created KAFKA-15677: - Summary: Add support for re2/j regex parsing for patterns Key: KAFKA-15677 URL: https://issues.apache.org/jira/browse/KAFKA-15677 Project: Kafka Issue

[jira] [Created] (KAFKA-15676) Scheduled rebalance delay for Connect is unnecessarily triggered when Kafka cluster bounces

Chris Egerton created KAFKA-15676: - Summary: Scheduled rebalance delay for Connect is unnecessarily triggered when Kafka cluster bounces Key: KAFKA-15676 URL: https://issues.apache.org/jira/browse/KAFKA-15676

Re: [DISCUSS] KIP-990: Capability to SUSPEND Tasks on DeserializationException

Hi Sophie, Thanks for the review! 1-3. I had a feeling this was the case. I'm thinking of adding a PAUSED state with the following valid transitions: - RUNNING -> PAUSED - PAUSED -> RUNNING - PAUSED -> SUSPENDED The advantage of a dedicated State is it should make testing easier and

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #2324

See Changes: -- [...truncated 320421 lines...] > Task :connect:json:compileTestJava UP-TO-DATE > Task :connect:api:generateMetadataFileForMavenJavaPublication > Task

Re: [DISCUSS] KIP-989: RocksDB Iterator Metrics

I don't really have a problem with adding such a metric, I'm just not entirely sure how it would work. If we used "iterator-duration-max", for example, would it not be confusing that it includes Iterators that are still open, and therefore the duration is not yet known? When graphing that over

Re: [DISCUSS] How to detect (and prevent) complex bugs in Kafka?

I think it is a great technique and I've used local invariants when doing system modelling in Jepsen Maelstrom which has no global view of state for checking global invariants. Sometimes the kind of assertions you want could be too costly for inclusion in a production system so the idea of gating

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #2323

See Changes: -- [...truncated 212516 lines...] Gradle Test Run :streams:test > Gradle Test Executor 90 > DefaultStateUpdaterTest >

[DISCUSS] How to detect (and prevent) complex bugs in Kafka?

Hey folks We recently came across a bug [1] which was very hard to detect during testing and easy to introduce during development. I would like to kick start a discussion on potential ways which could avoid this category of bugs in Apache Kafka. I think we might want to start working towards a

Re: [DISCUSS] KIP-963: Upload and delete lag metrics in Tiered Storage

Hello all, Now that 3.6 has been released, I would like to bring back attention to the following KIP for adding metrics to tiered storage targeting 3.7 - https://cwiki.apache.org/confluence/display/KAFKA/KIP-963%3A+Add+more+metrics+to+Tiered+Storage . Let me know your thoughts about the list of

[DISCUSS] Change website "blog" to "news"

(cc: PMC) Hey folks We require a forum to provide users of Kafka with information such as release announcements. Traditionally, we have been announcing through users@ mailing list and blogs hosted by Apache. More recently, we have started sharing release announcements via blog on Apache Kafka

Re: [DISCUSS] KIP-992 Proposal to introduce IQv2 Query Types: TimestampedKeyQuery and TimestampedRangeQuery

Hi Hanyu, reading the KIP, I was wondering the same thing as Bill. Other than that, this looks good to me. Thanks for KIP. nit: you have method names `LowerBound` and `UpperBound`, where you probably want to fix the capitalization. Cheers, Lucas On Mon, Oct 23, 2023 at 5:46 PM Bill Bejeck

Re: UncleanLeaderElectionsPerSec metric and Raft

Hi Justine, Thanks for the response. I also agree that even if after KIP-966 the unclean leader election might be changed, we should still figure out if it's a missing feature or not, and what's our plan for it. Thanks. Luke On Mon, Oct 23, 2023 at 11:48 PM Justine Olshan wrote: > Hey Neil, >

Re: [DISCUSS] KIP-990: Capability to SUSPEND Tasks on DeserializationException

Hey Nick, A few high-level thoughts: 1. We definitely don't want to piggyback on the SUSPENDED task state, as this is currently more like an intermediate state that a task passes through as it's being closed/migrated elsewhere, it doesn't really mean that a task is "suspended" and there's no

Re: [DISCUSS] KIP-989: RocksDB Iterator Metrics

I actually think we could implement Lucas' suggestion pretty easily and without too much additional effort. We have full control over the iterator that is returned by the various range queries, so it would be easy to register a gauge metric for how long it has been since the iterator was created.