Re: [DISCUSS] KIP-1056: Remove `default.` prefix for exception handler StreamsConfig

I think we should pause for a minute and have an honest conversation about whether the benefits of making this change outweigh the negatives. Here's my quick roundup of the positives and negatives, feel free to add to this list if there's else you think should be considered but then let's evaluate

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #3012

See

Re: [DISCUSS] Apache Kafka 3.8.0 release

I agree that we should keep 4.0 time-based. My question is based on which release. If I understand you, you would like to keep it based on 3.8. This means that 4.0 would be released in October. It would be helpful to fix the dates so we can plan accordingly. I will start a separate thread on Monday

Jenkins build is unstable: Kafka » Kafka Branch Builder » 3.7 #181

See

Re: [DISCUSS] Apache Kafka 3.9.0 release

+1, thank you Colin Given the July freeze deadlines, I take it we are going with the "short 3.9.0 release" option and that the existence of this release will impact the 4.0.0 deadlines which will still follow the usual schedule -- in other words, this is an "additional release" outside of the regu

Jenkins build is still unstable: Kafka » Kafka Branch Builder » 3.8 #46

See

Re: [VOTE] 3.7.1 RC1

The import fix is in. (As well as the integration tag) I did notice a large number of failing tests on the PR build though. Not sure if some of these were fixed in trunk and if we want to pick up those fixes. Justine On Fri, Jun 14, 2024 at 6:19 PM Matthias J. Sax wrote: > Replied on https://is

Jenkins build is unstable: Kafka » Kafka Branch Builder » 3.4 #177

See

Re: [VOTE] 3.7.1 RC1

Replied on https://issues.apache.org/jira/browse/KAFKA-16960 On 6/14/24 6:05 AM, Luke Chen wrote: Hi all, After running system tests, here are failing tests: 1. kafkatest.tests.core.upgrade_test failed at lz4 and snappy tests: KAFKA-16962 2.

[jira] [Created] (KAFKA-16968) Make 3.8-IV0 a stable MetadataVersion and create 3.9-IV0

Colin McCabe created KAFKA-16968: Summary: Make 3.8-IV0 a stable MetadataVersion and create 3.9-IV0 Key: KAFKA-16968 URL: https://issues.apache.org/jira/browse/KAFKA-16968 Project: Kafka Issu

Re: [DISCUSS] Apache Kafka 3.9.0 release

+1 thanks Colin!

[jira] [Created] (KAFKA-16967) NioEchoServer fails to register connection and causes flaky failure

Greg Harris created KAFKA-16967: --- Summary: NioEchoServer fails to register connection and causes flaky failure Key: KAFKA-16967 URL: https://issues.apache.org/jira/browse/KAFKA-16967 Project: Kafka

Jenkins build is still unstable: Kafka » Kafka Branch Builder » 3.8 #45

See

Re: [DISCUSS] Apache Kafka 3.9.0 release

Thanks, Matthias. I did a quick sweep just now and moved a bunch of things that have already been implemented in trunk into 3.9. Interestingly, more than half of the list seems to be "remove deprecated thing X", which is genuinely a 4.0 issue :) I suppose we will be grooming the 3.8 JIRA backlo

Re: [DISCUSS] Apache Kafka 3.9.0 release

+1 Btw: I just looked into Jira, and we have 90 tickets with "fixed version" set to 4.0.0. I would assume that most of them need to be updated to 3.9.0? https://issues.apache.org/jira/browse/KAFKA-16924?jql=project%20%3D%20KAFKA%20AND%20fixVersion%20%3D%204.0.0 For anybody who has context on

Re: [DISCUSS] Apache Kafka 3.8.0 release

+1. I think it would be good to keep 4.0 time-based. Most of the refactors we want to do are optional in some sense and can be descoped if time runs out. For example, we can drop support for JDK 8 without immediately refactoring everything that could benefit from the improvements in JDK9+. best

Re: [DISCUSS] KIP-1056: Remove `default.` prefix for exception handler StreamsConfig

*`DEFAULT_PRODUCTION_EXCEPTION_HANDLER_CLASS_DOC` is private and does notneed to be covered in the KIP technically -- it's ok to just leave it inof course.* Yea, technically it has no implications, but as we are anyways touching code around this, could it be better to rename ? Yes, we should st

Re: [DISCUSS] Apache Kafka 3.8.0 release

That's my understanding, and I would advocate strongly to keep the 4.0 release schedule as planed originally. The 3.9 one should really be an additional "out of schedule" release not impacting any other releases. -Matthias On 6/14/24 1:29 PM, David Jacot wrote: The plan sounds good to me.

Re: [QUESTION] What to do about conflicted KIP numbers?

I don't think that there is an official guideline. Personally, I would suggest that the corresponding KIP owners agree who is keeping the conflicting number, and how is changing it. For the ones changing the number, I would propose to restart a new DISCUSS thread using the new number to separ

Build failed in Jenkins: Kafka » Kafka Branch Builder » 3.7 #180

See Changes: -- [...truncated 2871 lines...] [2024-06-14T22:12:47.275Z] [Warn] /home/jenkins/workspace/Kafka_kafka_3.7/streams/streams-scala/src/main/scala/org/apache/kafka/s

[QUESTION] What to do about conflicted KIP numbers?

Hi Kafka devs, I submitted a KIP last week and encountered a KIP-process race condition where my KIP number was consumed by another dev without updating the wiki page containing KIPs: Kafka Improvement Proposals - Apache Kafka - Apache Software Foundation

Re: [DISCUSS] KIP-1056: Remove `default.` prefix for exception handler StreamsConfig

Thanks Matthias and Nick. Regarding the prefix "uncaught." is fine with me too, but any other opinions on this one ? *101 follow-up:* *Given that `DESERIALIZATION_EXCEPTION_HAN**DLER_CLASS_DOC` is public we* *cannot just rename it, but must follow the same deprecation pattern as* *for the config

Re: [DISCUSS] Apache Kafka 3.8.0 release

The plan sounds good to me. I suppose that we will follow our regular cadence for 4.0 and release it four months after 3.9 (in November?). Is this correct? Best, David Le ven. 14 juin 2024 à 21:57, José Armando García Sancio a écrit : > +1 on the proposed release plan for 3.8. > > Thanks! > > O

[jira] [Resolved] (KAFKA-16946) Utils.getHost/getPort cannot parse SASL_PLAINTEXT://host:port

[ https://issues.apache.org/jira/browse/KAFKA-16946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-16946. -- Resolution: Fixed > Utils.getHost/getPort cannot parse SASL_PLAINTEXT://host:port > --

Re: [DISCUSS] Apache Kafka 3.8.0 release

+1 on the proposed release plan for 3.8. Thanks! On Fri, Jun 14, 2024 at 3:33 PM Ismael Juma wrote: > > +1 to the plan we converged on in this thread. > > Ismael > > On Fri, Jun 14, 2024 at 10:46 AM Josep Prat > wrote: > > > Hi all, > > > > Thanks Colin, yes go ahead. > > > > As we are now past

Re: [DISCUSS] Apache Kafka 3.9.0 release

+1, thanks Colin. Ismael On Fri, Jun 14, 2024 at 10:55 AM Colin McCabe wrote: > Hi all, > > Based on the discussion in the 3.8 release thread, we will need a 3.9 > release. I guess I should create a new email thread to discuss this. > > I'd like to volunteer as release manager for the Apache Ka

Re: [DISCUSS] Apache Kafka 3.8.0 release

+1 to the plan we converged on in this thread. Ismael On Fri, Jun 14, 2024 at 10:46 AM Josep Prat wrote: > Hi all, > > Thanks Colin, yes go ahead. > > As we are now past code freeze I would like to ask everyone involved in a > KIP that is not yet complete, to verify if what landed on the 3.8 br

Re: [VOTE] KIP-1017: A health check endpoint for Kafka Connect

Hey Chris, Thanks for the KIP. I think demonstrating the health of the tick thread through a longer request is going to be more reliable than caching the health and status in order to respond quickly to requests. It should also be a natural negative feedback loop, so that when the herder tick thr

[jira] [Created] (KAFKA-16966) Allow offset commit fetch to reuse previous request if partitions are a subset

Kirk True created KAFKA-16966: - Summary: Allow offset commit fetch to reuse previous request if partitions are a subset Key: KAFKA-16966 URL: https://issues.apache.org/jira/browse/KAFKA-16966 Project: Kaf

Jenkins build is unstable: Kafka » Kafka Branch Builder » 3.8 #44

See

Re: [DISCUSS] Apache Kafka 3.9.0 release

+1 from me. Thanks Colin! Best, -- Josep Prat Open Source Engineering Director, Aiven josep.p...@aiven.io | +491715557497 | aiven.io Aiven Deutschland GmbH Alexanderufer 3-7, 10117 Berlin Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen Amtsgericht Charlottenburg, HRB 209739

Re: [DISCUSS] Apache Kafka 3.9.0 release

Thanks Colin, +1 from me. On Fri, Jun 14, 2024 at 10:55 AM Colin McCabe wrote: > Hi all, > > Based on the discussion in the 3.8 release thread, we will need a 3.9 > release. I guess I should create a new email thread to discuss this. > > I'd like to volunteer as release manager for the Apache Ka

[DISCUSS] Apache Kafka 3.9.0 release

Hi all, Based on the discussion in the 3.8 release thread, we will need a 3.9 release. I guess I should create a new email thread to discuss this. I'd like to volunteer as release manager for the Apache Kafka 3.9.0 release. There's a release plan here: https://cwiki.apache.org/confluence/displ

Re: [DISCUSS] Apache Kafka 3.8.0 release

Hi all, Thanks Colin, yes go ahead. As we are now past code freeze I would like to ask everyone involved in a KIP that is not yet complete, to verify if what landed on the 3.8 branch needs to be reverted or if it can stay. Additionally, I'll be pinging KIPs and Jira reporters asking for their sta

Re: [DISCUSS] Apache Kafka 3.8.0 release

Hi all, We have had many delays with releases this year. We should try to get back on schedule. I agree with the idea that was proposed a few times in this thread of drawing a line under 3.8 now, and doing a short 3.9 release. I posted a 3.9 release plan here: https://cwiki.apache.org/confluen

[jira] [Created] (KAFKA-16965) Add a "root cause" exception as a nested exception to TimeoutException for Producer

Alieh Saeedi created KAFKA-16965: Summary: Add a "root cause" exception as a nested exception to TimeoutException for Producer Key: KAFKA-16965 URL: https://issues.apache.org/jira/browse/KAFKA-16965 P

Re: [DISCUSS] KIP-1017: A health check endpoint for Kafka Connect

Chris, I was about to vote too but have one trivial question. You designed this health check for a completely generic connect runtime, regardless of which connectors it's running, right? Because if I run a well-known to me set of connectors, i.e. mm2 :-) , then checking the connectors and tasks s

[jira] [Resolved] (KAFKA-16917) DescribeTopicsResult should use mutable map in order to keep compatibility

[ https://issues.apache.org/jira/browse/KAFKA-16917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-16917. Fix Version/s: 3.9.0 Resolution: Fixed > DescribeTopicsResult should use mutable ma

Re: [VOTE] 3.7.1 RC1

thanks Justine ! On Fri, 14 Jun 2024 at 16:10, Bill Bejeck wrote: > > Thanks for the quick action Justine, I just reviewed and approved. > > On Fri, Jun 14, 2024 at 11:08 AM Justine Olshan > wrote: > > > The PR to fix is: https://github.com/apache/kafka/pull/16342/files > > > > On Fri, Jun 14, 2

Re: [VOTE] KIP-1017: A health check endpoint for Kafka Connect

Thanks Chris, +1 (non-binding) From: dev@kafka.apache.org At: 06/14/24 11:43:02 UTC-4:00To: dev@kafka.apache.org Subject: Re: [VOTE] KIP-1017: A health check endpoint for Kafka Connect Hi Chris, +1 (binding) Thanks, Mickael On Fri, Jun 14, 2024 at 5:01 PM Andrew Schofield wrote: > > Hi Ch

Re: [VOTE] KIP-1017: A health check endpoint for Kafka Connect

Hi Chris, +1 (binding) Thanks, Mickael On Fri, Jun 14, 2024 at 5:01 PM Andrew Schofield wrote: > > Hi Chris, > Thanks for the KIP. > > +1 (non-binding) > > Thanks, > Andrew > > > On 14 Jun 2024, at 15:48, Chris Egerton wrote: > > > > Hi all, > > > > Happy Friday! I'd like to kick off a vote on

Jenkins build is unstable: Kafka » Kafka Branch Builder » trunk #3009

See

[jira] [Created] (KAFKA-16964) Add and remove voter integration tests

José Armando García Sancio created KAFKA-16964: -- Summary: Add and remove voter integration tests Key: KAFKA-16964 URL: https://issues.apache.org/jira/browse/KAFKA-16964 Project: Kafka

[jira] [Created] (KAFKA-16963) Add and remove voter system tests

José Armando García Sancio created KAFKA-16963: -- Summary: Add and remove voter system tests Key: KAFKA-16963 URL: https://issues.apache.org/jira/browse/KAFKA-16963 Project: Kafka

Re: [VOTE] 3.7.1 RC1

Thanks for the quick action Justine, I just reviewed and approved. On Fri, Jun 14, 2024 at 11:08 AM Justine Olshan wrote: > The PR to fix is: https://github.com/apache/kafka/pull/16342/files > > On Fri, Jun 14, 2024 at 8:02 AM Justine Olshan > wrote: > > > Looks like the previous cherrypick of

Re: [VOTE] 3.7.1 RC1

The PR to fix is: https://github.com/apache/kafka/pull/16342/files On Fri, Jun 14, 2024 at 8:02 AM Justine Olshan wrote: > Looks like the previous cherrypick of Edoardo's feature broke the build. I > have a fix coming shortly. > > I may also want to look at the system tests again. > > Justine >

Re: [VOTE] 3.7.1 RC1

Looks like the previous cherrypick of Edoardo's feature broke the build. I have a fix coming shortly. I may also want to look at the system tests again. Justine On Fri, Jun 14, 2024 at 6:06 AM Luke Chen wrote: > Hi all, > > After running system tests, here are failing tests: > > 1. kafkatest.t

Re: [VOTE] KIP-1017: A health check endpoint for Kafka Connect

Hi Chris, Thanks for the KIP. +1 (non-binding) Thanks, Andrew > On 14 Jun 2024, at 15:48, Chris Egerton wrote: > > Hi all, > > Happy Friday! I'd like to kick off a vote on KIP-1017. > > Design doc: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1017%3A+Health+check+endpoint+for+Kafka+C

[VOTE] KIP-1017: A health check endpoint for Kafka Connect

Hi all, Happy Friday! I'd like to kick off a vote on KIP-1017. Design doc: https://cwiki.apache.org/confluence/display/KAFKA/KIP-1017%3A+Health+check+endpoint+for+Kafka+Connect Discussion thread: https://lists.apache.org/thread/95nto84wtogdgk3v97rxcwxr258wnjnt Cheers, Chris

Re: [DISCUSS] KIP-1017: A health check endpoint for Kafka Connect

Hi Chris, Both points make sense to me. 1) I'm good with your original solution where it would return after 10 seconds for both 5xx endpoints. The second solution seems complicated and maybe we shouldn't overcomplicate it. 2) Unfortunately I can't commit to that hypothetical KIP but I'll make a m

Re: [DISCUSS] KIP-1017: A health check endpoint for Kafka Connect

Thanks Viktor! Seems as good a time as any to kick off the vote thread. On Fri, Jun 14, 2024, 10:43 Viktor Somogyi-Vass wrote: > Hi Chris, > > Both points make sense to me. > > 1) I'm good with your original solution where it would return after 10 > seconds for both 5xx endpoints. The second sol

Re: [VOTE] 3.7.1 RC1

Hi all, After running system tests, here are failing tests: 1. kafkatest.tests.core.upgrade_test failed at lz4 and snappy tests: KAFKA-16962 2. StreamsUpgradeTest failed at "fromVersion"=2.2 ~ 2.5, but passed for the rest of the versions: KAFKA

[jira] [Created] (KAFKA-16962) kafkatest.tests.core.upgrade_test system tests failed in v3.7.1 RC1

Luke Chen created KAFKA-16962: - Summary: kafkatest.tests.core.upgrade_test system tests failed in v3.7.1 RC1 Key: KAFKA-16962 URL: https://issues.apache.org/jira/browse/KAFKA-16962 Project: Kafka

[jira] [Created] (KAFKA-16961) TestKRaftUpgrade system tests fail in v3.7.1 RC1

Luke Chen created KAFKA-16961: - Summary: TestKRaftUpgrade system tests fail in v3.7.1 RC1 Key: KAFKA-16961 URL: https://issues.apache.org/jira/browse/KAFKA-16961 Project: Kafka Issue Type: Test

[jira] [Created] (KAFKA-16960) StreamsUpgradeTest system tests fail in v3.7.1 RC1

Luke Chen created KAFKA-16960: - Summary: StreamsUpgradeTest system tests fail in v3.7.1 RC1 Key: KAFKA-16960 URL: https://issues.apache.org/jira/browse/KAFKA-16960 Project: Kafka Issue Type: Impr

[jira] [Resolved] (KAFKA-16752) Implement acquire functionality in SharePartition

[ https://issues.apache.org/jira/browse/KAFKA-16752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apoorv Mittal resolved KAFKA-16752. --- Resolution: Done > Implement acquire functionality in SharePartition > -

Build failed in Jenkins: Kafka » Kafka Branch Builder » 3.8 #43

See Changes: -- [...truncated 440526 lines...] [2024-06-14T11:45:33.732Z] Gradle Test Run :core:test > Gradle Test Executor 51 > KafkaZkClientTest > testControllerEpochMethods

[jira] [Resolved] (KAFKA-16747) Implement share sessions and context

[ https://issues.apache.org/jira/browse/KAFKA-16747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhinav Dixit resolved KAFKA-16747. --- Resolution: Fixed > Implement share sessions and context > -

[jira] [Created] (KAFKA-16959) ConfigCommand should not allow to define both `entity-default` and `entity-name`

Chia-Ping Tsai created KAFKA-16959: -- Summary: ConfigCommand should not allow to define both `entity-default` and `entity-name` Key: KAFKA-16959 URL: https://issues.apache.org/jira/browse/KAFKA-16959

Re: [DISCUSS] Apache Kafka 3.8.0 release

+1 on going with 3.8 release with the existing plan and targeting the required features in 3.9 timelines. 4.0 will be targeted in the usual cycle(4 months) after 3.9 is released. On Fri, 14 Jun 2024 at 15:19, Edoardo Comar wrote: > > Josep, > past the deadline sorry but I can't see reasons not t

Re: [DISCUSS] Apache Kafka 3.8.0 release

Josep, past the deadline sorry but I can't see reasons not to cherry-pick this https://github.com/apache/kafka/pull/16326 On Wed, 12 Jun 2024 at 17:14, Josep Prat wrote: > > Hi Edoardo, > > Correct, you can still cherry-pick this one. > > I'll send an email tomorrow morning (CEST) asking maintain

[jira] [Created] (KAFKA-16958) add `STRICT_STUBS` to `EndToEndLatencyTest`, `OffsetCommitCallbackInvokerTest`, `ProducerPerformanceTest`, and `TopologyTest`

Chia-Ping Tsai created KAFKA-16958: -- Summary: add `STRICT_STUBS` to `EndToEndLatencyTest`, `OffsetCommitCallbackInvokerTest`, `ProducerPerformanceTest`, and `TopologyTest` Key: KAFKA-16958 URL: https://issues.apa

[jira] [Created] (KAFKA-16957) Enable KafkaConsumerTest#configurableObjectsShouldSeeGeneratedClientId to work with CLASSIC and CONSUMER

Chia-Ping Tsai created KAFKA-16957: -- Summary: Enable KafkaConsumerTest#configurableObjectsShouldSeeGeneratedClientId to work with CLASSIC and CONSUMER Key: KAFKA-16957 URL: https://issues.apache.org/jira/browse

Re: [VOTE] 3.7.1 RC1

well if you still need to build RC2, I'd cherry pick this that I missed from the other PR https://github.com/apache/kafka/pull/16326 On Thu, 13 Jun 2024 at 13:20, Igor Soarez wrote: > > Hi Edoardo, > > It is late, but not too late. I have cherry-picked your change > to the 3.7 branch and I'll bui

[jira] [Resolved] (KAFKA-16317) Add event rate in GroupCoordinatorRuntimeMetrics

[ https://issues.apache.org/jira/browse/KAFKA-16317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-16317. - Resolution: Won't Do > Add event rate in GroupCoordinatorRuntimeMetrics > --

[jira] [Resolved] (KAFKA-16674) Adjust classicGroupJoinToConsumerGroup to add subscription model

[ https://issues.apache.org/jira/browse/KAFKA-16674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-16674. - Resolution: Fixed This was done in https://github.com/apache/kafka/pull/15785. > Adjust classic

[jira] [Reopened] (KAFKA-16673) Optimize toTopicPartitions with ConsumerProtocolSubscription

[ https://issues.apache.org/jira/browse/KAFKA-16673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot reopened KAFKA-16673: - > Optimize toTopicPartitions with ConsumerProtocolSubscription > ---

[jira] [Resolved] (KAFKA-16673) Optimize toTopicPartitions with ConsumerProtocolSubscription

[ https://issues.apache.org/jira/browse/KAFKA-16673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-16673. - Fix Version/s: 3.8.0 Resolution: Duplicate This was done as part of https://github.com/ap

[jira] [Resolved] (KAFKA-12708) Rewrite org.apache.kafka.test.Microbenchmarks by JMH

[ https://issues.apache.org/jira/browse/KAFKA-12708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-12708. Fix Version/s: 3.9.0 Resolution: Fixed > Rewrite org.apache.kafka.test.Microbenchma

[jira] [Resolved] (KAFKA-16942) Use ConcurrentHashMap in RecordAccumulator#nodeStats

[ https://issues.apache.org/jira/browse/KAFKA-16942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuan Po Tseng resolved KAFKA-16942. --- Resolution: Won't Do The performance in get method is not obvious between ConcurrentHashMap

Re: [VOTE] KIP-1040: Improve handling of nullable values in InsertField, ExtractField, and other transformations

Hi All, We have 4 binding +1 votes. Do you think the vote can be closed and mark the KIP as approved? Thanks, Mario. On Fri, Jun 14, 2024 at 8:44 AM Punsak Incham wrote: > Thanks! Chris. > Waiting @Mario Fiore Vitale to close the voting and mark it as approved. > > Cheers. > > On 2024/06/14 06

[jira] [Created] (KAFKA-16956) Broker-side ability to subscribe to record delete events

Luke Chen created KAFKA-16956: - Summary: Broker-side ability to subscribe to record delete events Key: KAFKA-16956 URL: https://issues.apache.org/jira/browse/KAFKA-16956 Project: Kafka Issue Type