[jira] [Created] (KAFKA-13653) Proactively discover alive brokers from bootstrap server lists when all nodes are down

Luke Chen created KAFKA-13653: - Summary: Proactively discover alive brokers from bootstrap server lists when all nodes are down Key: KAFKA-13653 URL: https://issues.apache.org/jira/browse/KAFKA-13653

Re: [DISCUSS] KIP-792: Add "generation" field into consumer protocol

Hi David, Thanks for your comments. I've updated the KIP to add changes in Subscription class. Thank you. Luke On Fri, Feb 4, 2022 at 11:43 PM David Jacot wrote: > Hi Luke, > > Thanks for updating the KIP. I just have a minor request. > Could you fully describe the changes to the Subscription

Re: KAFKA-12495 - kafka connect unbalanced task distribution

Hi Eric, Actually, there are also other users waiting for this patch to get merged. The committer has some concern with my fix, and has a better solution for it. I'm also waiting for his suggestion to continue this PR. I'll ping him again in JIRA and PR. Hope this issue can be fixed soon. Thank

Re: [VOTE] KIP-704: Send a hint to partition leader to recover the partition

I'm +1 (non-binding) Thanks for the KIP! Luke On Tue, Feb 8, 2022 at 7:42 AM Raman Verma wrote: > +1, thanks Jose. > > On Mon, Feb 7, 2022 at 8:56 AM David Jacot wrote: > > > > +1, thanks! > > > > Le lun. 7 févr. 2022 à 17:41, José Armando García Sancio > > a écrit : > > > > > Thanks for

[jira] [Created] (KAFKA-13652) encrypt the passwords in kafka jaas conf file

Sunil created KAFKA-13652: - Summary: encrypt the passwords in kafka jaas conf file Key: KAFKA-13652 URL: https://issues.apache.org/jira/browse/KAFKA-13652 Project: Kafka Issue Type: Test

[jira] [Resolved] (KAFKA-8410) Strengthen the types of Processors, at least in the DSL, maybe in the PAPI as well

[ https://issues.apache.org/jira/browse/KAFKA-8410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-8410. Resolution: Fixed This feature (KIP–478:

Re: [DISCUSS] KIP-794: Strictly Uniform Sticky Partitioner

Hello, After trying a few prototypes, I've made some changes to the public interface. Please see the updated document https://cwiki.apache.org/confluence/display/KAFKA/KIP-794%3A+Strictly+Uniform+Sticky+Partitioner . -Artem On Thu, Nov 4, 2021 at 10:37 AM Artem Livshits wrote: > Hello, > >

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #668

See Changes: -- [...truncated 441001 lines...] [2022-02-08T01:37:43.129Z] > Task :streams:generateMetadataFileForMavenJavaPublication [2022-02-08T01:37:45.644Z]

Re: [DISCUSS] KIP-704: Send a hint to broker if it is an unclean leader

You are correct Raman. I updated the KIP to reflect your observations and corrections. Raman wrote: > I think the line #2 here is wrong with regard to AlterPartitionRequest. Yes. Bullet 2. Now reads: 2. The LeaderRecoveryState is changing from RECOVERED to RECOVERING. Raman wrote: > Also, in

[jira] [Created] (KAFKA-13651) StandardAuthorizer should implement all the required log4j stuff

Colin McCabe created KAFKA-13651: Summary: StandardAuthorizer should implement all the required log4j stuff Key: KAFKA-13651 URL: https://issues.apache.org/jira/browse/KAFKA-13651 Project: Kafka

Re: [VOTE] KIP-799: Align behaviour for producer callbacks with documented behaviour

Hi Seamus, Is there a conflict between KIP-799 and KAFKA-12841 ? A related fix for the latter issue was just merged a few days ago (https://github.com/apache/kafka/pull/11689) and it seems like there's some overlap or discrepancy between the

[jira] [Created] (KAFKA-13650) Add StandardAuthorizer JMH benchmark

Colin McCabe created KAFKA-13650: Summary: Add StandardAuthorizer JMH benchmark Key: KAFKA-13650 URL: https://issues.apache.org/jira/browse/KAFKA-13650 Project: Kafka Issue Type: Improvement

[jira] [Created] (KAFKA-13649) StandardAuthorizer should not finish loading until it reads up to the high water mark

Colin McCabe created KAFKA-13649: Summary: StandardAuthorizer should not finish loading until it reads up to the high water mark Key: KAFKA-13649 URL: https://issues.apache.org/jira/browse/KAFKA-13649

Re: [DISCUSS] KIP-816: Topology changes without local state reset

Hello Nick, I think I'm on the same page of the scope of your KIP, and what I was trying to get is that, there are some other efforts going on in parallel that tries to identify if two topologies, or some part of them, are isomorphic in structure, and hence their corresponding persistent states

Re: [VOTE] KIP-704: Send a hint to partition leader to recover the partition

+1, thanks Jose. On Mon, Feb 7, 2022 at 8:56 AM David Jacot wrote: > > +1, thanks! > > Le lun. 7 févr. 2022 à 17:41, José Armando García Sancio > a écrit : > > > Thanks for the vote Jason. > > > > This is a friendly reminder to vote on this KIP. > > > > -- > > -José > > -- Best Regards,

Re: [DISCUSS] KIP-704: Send a hint to broker if it is an unclean leader

Also, in the response schema, I think the last line is a bit misleading. As we discussed offline, the field can be either Recovering or Recovered, depending on when the leader sent out an AlterPartition request to the controller (i.e. during recovery or after recovery has completed). But the

Re: [DISCUSS] KIP-704: Send a hint to broker if it is an unclean leader

Jose, I think the line #2 here is wrong with regard to AlterPartitionRequest. ``` Request Handling The controller will validate the LeaderRecoveryState field and return an INVALID_REQUEST error when: 1. The size of the ISR is greater than 1 and the LeaderRecoveryState is RECOVERING. 2. The

Re: Kafka <= 3.1 upgrade RocksDB to v6.27.3?

Thanks for the clarification Jonathan, I think timing wise, waiting to get 3.2.0 v.s. get 3.1.1 / 3.0.1 would not be much longer. On Mon, Feb 7, 2022 at 12:16 PM Jonathan Albrecht wrote: > > > > Hi Bruno and Guozhang, > > Thanks for your thoughtful replies and I hope I can clarify. I work on >

Re: [VOTE] KIP-799: Align behaviour for producer callbacks with documented behaviour

Hi all, Bumping this thread again. Kind reminder that* this KIP addresses a* *bug in the Kafka producer client*, where the implemented behaviour of the producer client is not the same as the documented behaviour in the case of callbacks. This has caused production incidents 'in the wild'.

Re: [DISCUSS] KIP-817: Fix inconsistency in dynamic application log levels

Hi David, Thanks for picking up this issue. 1. Why does this proposal regard OFF only, not ALL? The reason is simple; As you can see, both of ALL and OFF were excluded in KIP-412. However, ALL level is equivalent to TRACE without a custom logging level. (please refer to this discussion

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #667

See

Re: Kafka <= 3.1 upgrade RocksDB to v6.27.3?

Hi Bruno and Guozhang, Thanks for your thoughtful replies and I hope I can clarify. I work on a team that helps port open source software to s390x. The example I used about users needing to be on a specific minor release is a general one that I see and not specific to kafka and usually its not

Jenkins build is unstable: Kafka » Kafka Branch Builder » 3.0 #173

See

[jira] [Created] (KAFKA-13648) KRaft ClusterInstance does not allow for deferred start

David Arthur created KAFKA-13648: Summary: KRaft ClusterInstance does not allow for deferred start Key: KAFKA-13648 URL: https://issues.apache.org/jira/browse/KAFKA-13648 Project: Kafka

Re: [DISCUSS] Apache Kafka 3.0.1

+1, thanks Mickael! On Mon, Feb 7, 2022 at 9:27 AM Bruno Cadonna wrote: > +1 Thank you! > > Best, > Bruno > > On 07.02.22 18:24, Ismael Juma wrote: > > Thanks Mickael, +1. > > > > Ismael > > > > On Mon, Feb 7, 2022 at 9:17 AM Mickael Maison > wrote: > > > >> Hi, > >> > >> I'd like to volunteer

Re: [DISCUSS] Apache Kafka 3.0.1

+1 Thank you! Best, Bruno On 07.02.22 18:24, Ismael Juma wrote: Thanks Mickael, +1. Ismael On Mon, Feb 7, 2022 at 9:17 AM Mickael Maison wrote: Hi, I'd like to volunteer to be the release manager for the next bugfix release, 3.0.1. Thanks, Mickael

Re: [DISCUSS] Apache Kafka 3.0.1

Thanks Mickael, +1. Ismael On Mon, Feb 7, 2022 at 9:17 AM Mickael Maison wrote: > Hi, > > I'd like to volunteer to be the release manager for the next bugfix > release, 3.0.1. > > Thanks, > Mickael >

[DISCUSS] Apache Kafka 3.0.1

Hi, I'd like to volunteer to be the release manager for the next bugfix release, 3.0.1. Thanks, Mickael

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #666

See

Re: [DISCUSS] Should we automatically close stale PRs?

Hi David, I agree with you, I think we should close stale PRs. Overall, I think we should also see if there are other Github actions that may ease the work for reviewers and/or give more visibility of the process to PR authors. I'm thinking things like: - code coverage changes - better view on

Re: [DISCUSS] KIP-816: Topology changes without local state reset

Thanks, Nick, It sounds like we're on the same page. I didn't think (A) would be fundamentally "hard", just that it might be a pain in practice. Reading your response, if you're up for it, it sounds like a prototype of (A) would be the tie-breaker between the two approaches. To be honest, I've

Re: MM2 setup using Kafka 2.4 compatibility with Kafka Broker built using 2.0.0 ver.

Ranga, yes that will work. You are correct that mm2's compatibility matrix is essentially the same as the client's. I've tested 2.4 mm2 against 0.10.2 brokers and it works fine. Anything older than that may still generally work, but you may see errors for unsupported apis. Ryanne On Mon, Feb 7,

Re: [VOTE] KIP-704: Send a hint to partition leader to recover the partition

+1, thanks! Le lun. 7 févr. 2022 à 17:41, José Armando García Sancio a écrit : > Thanks for the vote Jason. > > This is a friendly reminder to vote on this KIP. > > -- > -José >

Re: [VOTE] KIP-704: Send a hint to partition leader to recover the partition

Thanks for the vote Jason. This is a friendly reminder to vote on this KIP. -- -José

MM2 setup using Kafka 2.4 compatibility with Kafka Broker built using 2.0.0 ver.

Hi Kakfa Experts Trying to understand compatibility of MM2 libraries packaged as part of core kafka release 2.4 with Kafka Cluster running with Kafka core version 2.0.0. The major improvements to MM2 were implemented as part of the KIP-382

KAFKA-12495 - kafka connect unbalanced task distribution

Hi, I have balancing issues with kafka connect when scaling in/out workers. The PR from KAFKA-12495 seems to fix it. Is there a plan to merge it ? Regards, Eric

Re: [DISCUSS] KIP-787 - MM2 Interface to manage Kafka resources

Hi Mickael. Thanks for the feedback. I address some of your points below. *> This seems to address a relatively advanced and specific use case* The main point of the KIP is that MM2 is making a massive assumption that it has the right to alter/create resources. This assumption isn't valid in the

Re: [DISCUSS] Apache Kafka 3.2.0 release

+1, thanks Bruno! On Sat, 5 Feb 2022 at 07:17, Luke Chen wrote: > Thanks Bruno! > > +1 > > Luke > > On Sat, Feb 5, 2022 at 10:50 AM Guozhang Wang wrote: > > > Thanks Bruno! +1 > > > > On Fri, Feb 4, 2022 at 4:14 PM Ismael Juma wrote: > > > > > Thanks for volunteering, Bruno. +1! > > > > > >

[jira] [Created] (KAFKA-13647) RocksDb metrics 'number-open-files' is not correct

Sylvain Le Gouellec created KAFKA-13647: --- Summary: RocksDb metrics 'number-open-files' is not correct Key: KAFKA-13647 URL: https://issues.apache.org/jira/browse/KAFKA-13647 Project: Kafka

Re: [DISCUSS] Should we automatically close stale PRs?

Hi, It seems like a great idea. I agree with you that we should use this as a means to notify contributors and reviewers that there is some work to be done. Regarding labels, a couple of things, first one is that PR participants won't get notified when a label is applied. So probably it would be

Re: [DISCUSS] KIP-816: Topology changes without local state reset

Hi everyone, Guozhang, the scope of my KIP is specifically about deploying structural changes to existing applications, i.e. "upgrades". Sharing state between different applications was not in the scope of my original proposal. John's email has it exactly right, and I think this points to my KIP

Re: [DISCUSS] Should we automatically close stale PRs?

> We do not have a separate list of PRs that need pre-reviews. Ok. What should I do if PR need to be to closed found? Who can I tag to do actual close? > 7 февр. 2022 г., в 13:53, Bruno Cadonna написал(а): > > Hi, > > Thank you David for bringing this up! > > I am in favour of automatically

Re: [DISCUSS] Should we automatically close stale PRs?

Hi, Thank you David for bringing this up! I am in favour of automatically closing stale PRs. I agree with Guozhang that notifications of staleness to authors would be better than silently closing them. I assume the notification happens automatically when the label "Stale" is added to the PR.

Re: Kafka <= 3.1 upgrade RocksDB to v6.27.3?

Hi Jonathan and Guozhang, I am pretty sure that upgrading RocksDB in 3.0.x and 3.1.x should not be an issue, since we recently upgraded it to 6.27.3 on trunk and judging from the compatibility report in the ticket [1] and the code the API does not break backward-compatibility for the AK 3.x