Build failed in Jenkins: Kafka » Kafka Branch Builder » 3.1 #16

See Changes: -- [...truncated 501283 lines...] [2021-11-17T03:51:32.611Z] > Task :streams:test-utils:compileJava UP-TO-DATE [2021-11-17T03:51:32.611Z] > Task :streams:jar

Re: [DISCUSS] KIP-796: Interactive Query v2

Thanks John! Some more thoughts inlined below. On Mon, Nov 15, 2021 at 10:07 PM John Roesler wrote: > Thanks for the review, Guozhang! > > 1. This is a great point. I fell into the age-old trap of > only considering the simplest store type and forgot about > this extra wrinkle of the "key

Build failed in Jenkins: Kafka » Kafka Branch Builder » 3.1 #15

See Changes: -- [...truncated 500803 lines...] [2021-11-16T23:49:00.671Z] EndToEndClusterIdTest > testEndToEnd() STARTED [2021-11-16T23:49:00.873Z] [2021-11-16T23:49:00.873Z]

Re: [DISCUSS] KIP-778 KRaft Upgrades

On Tue, Nov 16, 2021, at 15:13, Jun Rao wrote: > Hi, David, Colin, > > Thanks for the reply. > > 16. Discussed with David offline a bit. We have 3 cases. > a. We upgrade from an old version where the metadata.version has already > been finalized. In this case it makes sense to stay with that

Re: [DISCUSS] KIP-778 KRaft Upgrades

Hi, David, Colin, Thanks for the reply. 16. Discussed with David offline a bit. We have 3 cases. a. We upgrade from an old version where the metadata.version has already been finalized. In this case it makes sense to stay with that feature version after the upgrade. b. We upgrade from an old

Re: [DISCUSS] KIP-778 KRaft Upgrades

Yeah I agree that checking a majority of voters support the metadata.version is sufficient. What I was originally considering is whether (in the future) we could consider encoding the metadata.version value in the vote request as well, so that the elected leader is supposed to have a version

[DISCUSS] Brokers disconnect intermittently with TLS1.3

Hi Luke, Sorry about the miscommunication, I was not talking about making TLS1.2 default. My assumption is that if JDK version < 11, TLS 1.2 will be used. So wanted to come up with a solution that worked for both cases. To provide more details about the issue, given below is the error

Re: [DISCUSS] KIP-778 KRaft Upgrades

On Tue, Nov 16, 2021, at 13:36, Guozhang Wang wrote: > Hi Colin, > > If we allow downgrades which would be appended in metadata.version, then > the length of the __cluster_medata log may not be safe to indicate higher > versions, since older version records could still be appended later than a >

Build failed in Jenkins: Kafka » Kafka Branch Builder » 3.1 #14

See Changes: -- [...truncated 501092 lines...] [2021-11-16T21:43:05.949Z] [2021-11-16T21:43:05.949Z] DeleteConsumerGroupsTest > testDeleteCmdWithMixOfSuccessAndError()

Re: [DISCUSS] KIP-778 KRaft Upgrades

Hi Colin, If we allow downgrades which would be appended in metadata.version, then the length of the __cluster_medata log may not be safe to indicate higher versions, since older version records could still be appended later than a new version record right? On Tue, Nov 16, 2021 at 1:16 PM Colin

Re: [DISCUSS] KIP-778 KRaft Upgrades

Thanks, I think having the leader election to consider the metadata.version would be a good idea moving forward, but we do not need to include in this KIP. On Tue, Nov 16, 2021 at 6:37 AM David Arthur wrote: > Guozhang, > > 1. By requiring a majority of all controller nodes to support the

Re: Track topic deletion state without ZK

Hi Omnia, Topic deletion doesn't get stuck if a broker is down, when using KRaft. There is no "deleting" state, only deleted or not deleted. best, Colin On Tue, Nov 2, 2021, at 09:24, Omnia Ibrahim wrote: > Hi Colin, thanks for your response. > Regards your point that the topic gets deleted

Re: [DISCUSS] KIP-778 KRaft Upgrades

On Tue, Nov 16, 2021, at 06:36, David Arthur wrote: > An interesting case here is how to handle a version update if a majority of > the quorum supports it, but the leader doesn't. For example, if C1 was the > leader and an upgrade to version 4 was requested. Maybe this would trigger > C1 to resign

Re: [DISCUSS] KIP-778 KRaft Upgrades

Hi David, On Mon, Nov 15, 2021, at 08:13, David Arthur wrote: > This (probably) means the quorum peers must continue to rely on ApiVersions > to negotiate compatible RPC versions and these versions cannot depend on > metadata.version. I agree. Can we explicitly spell out in KIP that quorum peers

Re: [DISCUSS] KIP-779: Allow Source Tasks to Handle Producer Exceptions

Thank you all for the feedback, the KIP has been updated. On Tue, Nov 16, 2021 at 10:46 AM Arjun Satish wrote: > One more nit: the RetryWithToleranceOperator class is not a public > interface. So we do not have to call the changes in them out in the Public > Interfaces section. > > > On Tue,

Re: [DISCUSS] KIP-778 KRaft Upgrades

On Fri, Nov 5, 2021, at 08:10, David Arthur wrote: > Colin and Jun, thanks for the additional comments! > > Colin: > >> We've been talking about having an automated RPC compatibility checker > > Do we have a way to mark fields in schemas as deprecated? It can stay in > the RPC, it just complicates

Re: [VOTE] KIP-779: Allow Source Tasks to Handle Producer Exceptions

+1 (non-binding). Thanks Knowles! On Tue, Nov 16, 2021 at 10:48 AM Arjun Satish wrote: > +1 (non-binding). Thanks for the KIP, Knowles! and appreciate the > follow-ups! > > On Thu, Nov 11, 2021 at 2:55 PM John Roesler wrote: > > > Thanks, Knowles! > > > > I'm +1 (binding) > > > > -John > > > >

Re: [DISCUSS] KIP-778 KRaft Upgrades

On Fri, Nov 5, 2021, at 15:18, Jun Rao wrote: > Hi, David, > > Thanks for the reply. > > 16. My first concern is that the KIP picks up meta.version inconsistently > during the deployment. If a new cluster is started, we pick up the highest > version. If we upgrade, we leave the feature version

Re: KIP-769: Connect API to retrieve connector configuration definitions

Hi, I'm +1 for adding a GET endpoint for obtaining config definitions. It always felt odd to me that one has to issue a PUT for that purpose. If nothing else, it'd be better in terms of discoverability of the KC REST API. One additional feature request I'd have is to expose the valid enum

[jira] [Resolved] (KAFKA-13071) Deprecate and remove --authorizer option in kafka-acls.sh

[ https://issues.apache.org/jira/browse/KAFKA-13071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson resolved KAFKA-13071. - Fix Version/s: 3.1.0 Resolution: Fixed > Deprecate and remove --authorizer

Build failed in Jenkins: Kafka » Kafka Branch Builder » 3.1 #13

See Changes: -- [...truncated 502245 lines...] [2021-11-16T18:37:20.230Z] > Task :raft:testClasses UP-TO-DATE [2021-11-16T18:37:20.230Z] > Task :connect:json:testJar

Build failed in Jenkins: Kafka » Kafka Branch Builder » 2.8 #91

See Changes: -- [...truncated 466165 lines...] [Pipeline] // timestamps [Pipeline] } [Pipeline] // timeout [Pipeline] } [Pipeline] // stage [Pipeline] }

[jira] [Resolved] (KAFKA-13426) Add recordMetadata to StateStoreContext

[ https://issues.apache.org/jira/browse/KAFKA-13426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Roesler resolved KAFKA-13426. -- Resolution: Fixed > Add recordMetadata to StateStoreContext >

Re: [VOTE] KIP-779: Allow Source Tasks to Handle Producer Exceptions

+1 (non-binding). Thanks for the KIP, Knowles! and appreciate the follow-ups! On Thu, Nov 11, 2021 at 2:55 PM John Roesler wrote: > Thanks, Knowles! > > I'm +1 (binding) > > -John > > On Wed, 2021-11-10 at 12:42 -0500, Christopher Shannon > wrote: > > +1 (non-binding). This looks good to me and

Re: [DISCUSS] KIP-779: Allow Source Tasks to Handle Producer Exceptions

One more nit: the RetryWithToleranceOperator class is not a public interface. So we do not have to call the changes in them out in the Public Interfaces section. On Tue, Nov 16, 2021 at 10:42 AM Arjun Satish wrote: > Chris' point about upgrades is valid. An existing configuration will now >

Re: [DISCUSS] KIP-779: Allow Source Tasks to Handle Producer Exceptions

Chris' point about upgrades is valid. An existing configuration will now have additional behavior. We should clearly call this out in the kip, and whenever they are prepared -- the release notes. It's a bit crummy when upgrading, but I do think it's better than introducing a new configuration in

Re: KIP-769: Connect API to retrieve connector configuration definitions

Hi Viktor, It sounds like there are three major points here in favor of a new GET endpoint for connector config defs. 1. You cannot issue a blank ("dummy") request for sink connectors because a topic list/topic regex has to be supplied (otherwise the PUT endpoint returns a 500 response) 2. A

Re: KIP-769: Connect API to retrieve connector configuration definitions

Hi Folks, I too think this would be a very useful feature. Some of our management applications would provide a wizard for creating connectors. In this scenario the user basically would fill out a sample configuration generated by the UI which would send it back to Connect for validation and

Re: [DISCUSS] KIP-778 KRaft Upgrades

Guozhang, 1. By requiring a majority of all controller nodes to support the version selected by the leader, we increase the likelihood that the next leader will also support it. We can't guarantee that all nodes definitely support the selected metadata.version because there could always be an

Re: [DISCUSS] KIP-797 Accept duplicate listener on port for IPv4/IPv6

Since no one has commented on either this thread or the original one I will summon a vote by the end of this week. Regards On Wed, Nov 10, 2021 at 5:28 PM Matthew de Detrich < matthew.dedetr...@aiven.io> wrote: > Hello everyone, > > I would like to start a discussion for KIP-797 which is about

[Vote] KIP-787 - MM2 Interface to manage Kafka resources Kafka/KIPs

Hi, I'd like to start a vote on KIP-787 https://cwiki.apache.org/confluence/display/KAFKA/KIP-787%3A+MM2+Interface+to+manage+Kafka+resources Thanks Omnia

Jenkins build is unstable: Kafka » Kafka Branch Builder » trunk #573

See

[jira] [Resolved] (KAFKA-13449) Comment optimization for parameter log.cleaner.delete.retention.ms

[ https://issues.apache.org/jira/browse/KAFKA-13449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-13449. - Fix Version/s: 3.2.0 Resolution: Fixed > Comment optimization for parameter

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #572

See Changes: -- [...truncated 499591 lines...] [2021-11-16T11:54:56.504Z] [2021-11-16T11:54:56.504Z] PlaintextConsumerTest > testConsumeMessagesWithCreateTime() PASSED

Jenkins build is unstable: Kafka » Kafka Branch Builder » 3.1 #12

See

Jenkins build is unstable: Kafka » Kafka Branch Builder » 3.0 #155

See